Critiquing and Designing test Items

Create plausible options.

• Which of the following substances is the hardest?a) cardboard b)glass c) paper d) steel

BetterWhich of the following substances is the hardest?b) concrete b)diamond c)iron d)steel

There is no advantage in writing distractors no one will select, so each alternative should be plausible.

Better item

• The type of test used to measure academic learing is called

• a) an achievement test• b) a case study• c) a special aptitude test• d) a test of intelligence

Define the task in the stem• Poor Americaa) Contains the world´s largest populationb) Is the site of the world´s oldest culturec) Is an unilingual cultured) Was named after Vespucci

BetterAfter which person was America named?e) Columbusf) Magellang) Ponce de Leonh) Vespucci

Avoid ambigous and confusing wording and sentence structure

• Maine is not the only state that does not have a border with a neighboring state T F

BETTERMaine borders another state. T F

Keep questions short and to the pointWhich of the following statements about geography of Switzerland is true?a) It is located in Asiab) It is a flat, arid plainc) It has not direct access to the oceand) It has a tropical climate.

To make an apple pie Billy´s mother needed 8 apples. If apples cost 30 cents for two, how much will 8 apples cost?a) $.30 b) $.90 c) $1.20 d) $2.40

Good multiple choice items

• Multiple choice items consist of two parts: a stem and a number of options, or alternatives. The stem is a question or statement that is answered or completed by one of the alternatives. All incorrect or less appropriate alternatives are called distractors, or foils, and the students task is to select the correct or best alternative from all options.

Advantages of multiple choice items

• It measures objectives from the rote knowledge to the most complex level.

• - It can sample a substantial amount of course material in a relatively short time.

• Scoring is objective, little interpretation is needed.

• Guessing is reduced. (3 to 5 options present)• You can do item analysis to detect areas of

student weaknes.

Disadvantages of MCIs

• They are time consuming to write and students complain that more than one defensible answer can be given. This can be remedied by editing and pretesting items.

Advantages of Matching Items

• It is simple to construct and score.• It reduces the effect of guessing.• The teacher can sample a variety of

substantial amount of course material in short time.

Constructing Matching items• Limit the number of items within each set to 5 or

6 per set. A longer list is difficult for students to handle. A 6 item subtest will probably have about 9 or 10 options. Any more options will make the search for the correct answer overly difficult and time- consuming.

• Place the shorter response in column B.• Provide complete directions.• Place options on the same page

Completion and essay tests

• The completion test asks students to complete a sentence with a word or phrase. The short answer test poses a question that can be answered with a word or phrase.• The essay test allow greater freedom of

response to questions and require more writing.

Advantages and disadvantages• Construction is relatively easy.• Guessing is eleminated• Item sampling is improved• What is the square root of 25?• Who discovered radium?• Disadvantages• They are difficult to score.• They are susceptible to an may encourage bluffing• They typically measure rote objectives

Advantages and disadvantages of Essay Tests

• Advantages

They allow free responseThey eliminate guessingThey are practicalThey reduce assembling time (for the teacher)They can measure divergent

thinking

Disadvantages- They are difficult to score- They measure limited knowledge- They are time consuming.- They are subject to bluffing- They place a premiumon writing.

Checklist for writing better tests Have you taken enough samples of behavior? Have you allowed students too much freedom? Is the item unambiguous? Have you provided clear and explicit instructions? Is the test well laid out and perfectly legible? Are students familiar with format and testing technique? Does the item permit scoring which is as objective as

possible? Have you agreed on acceptable responses at outset of

scoring? Is there only one correct answer?

Key Terms in evaluation

A test: is a task or series of tasks used to obtainsystematic observations, usually paper andpencil procedure, about sts´s performance.

Measurement: is the process that assignsnumbers to attributes or characteristics ofpersons, objects, or events according to explicitformulations or rules. More quantitative

Assessment vrs Evaluation• Assessment: the process of collecting, synthesizing,

and interpreting information to aid in decision making. More qualitative

• Evaluation: the process of determining the value or worth of a program, course, or other initiative, toward the ultimate goal of making decisions about adopting, rejecting, or revising the innovation. Evaluation is the more inclusive term, often making use of assessment data in addition to many other data sources.

Discrete point testing vrs integrative testing

• Discrete point testing refers to measuring one element at a time, item by item: a series of items, each testing particular grammatical structures.

• Integrative requires the candidate to combine many language elements in the completion of a task.

• Discrete point testing will be indirect and Integrative will be direct.

Direct and indirect test items

• It is direct if it asks candidates to perform the communicative skill which is being tested ( through a composition, oral interview)

• Indirect items: measure a students´ knowledge and ability by getting at what lies beneath their receptive and productive skills ( controlled items: multiple choice type for example).