Using Common Testing Techniques Effectively

Writing Multiple-choice, Gap-filling, and Short-answer Items

Hanh Thi Nguyen, Ph.D.

TESL Programs

Learning Assessment Committee member

What is a ‘good’ test?

An effective test measures what it is supposed to measure

(has validity) measures consistently (has reliability) is practical to make, administer, and score

(has practicality) has positive effects on learning

(has positive washback)

How to meet those criteria?

Validity: Write test items that correspond to course objectives Make sure only these objectives are tested

Reliability: Include not too few items, but not too long a test either Eliminate distracting factors in the environment Make sure scoring is done consistently Make sure candidates are familiar with testing format Make sure instructions are non-ambiguous

Practicality Consider time and cost Sometimes practicality conflicts with validity and reliability, so test what

needs to be tested, do not just test what is easy to test Positive washback

Make sure test can improve teaching Make sure test has positive impact on learning

Common testing techniques

Multiple-choice items Gap-filling items Short-answer items Extended response (essay) items – not

covered today

Multiple-choice items

1. What is the shape of the earth? Stem Options

A. round key

B. square distractor

C. oval distractor

D. triangular distractor

Advantages of MC items

Objective testing Scoring is easy Less dependent on students’ writing

skills Can cover a broad range of contents Very effective when eliciting a best

answer

Disadvantages of MC items

Testing only recognition knowledge, no production knowledge

Candidates have a 25% chance to guess right (if 4 options) – hence, “multiple-guess” test

Not always possible to have 3-4 plausible alternatives to correct answer

Examples

If I ___ him last year, I ___ him to help me.A. know, askB. knew, askedC. have known, askedD. had known, would have asked

Key is the longest string – facilitates guessing

Option A not likely to distract

Examples

She started to ____, thinking about his death.A. giggleB. complainC. shoutD. weep

All 4 options are possible answers.Need further context to restrict answer.

Suggestions for constructing MC items

For the stem: Use direct questions rather than

incomplete sentences. Present a single question or problem in

stem Make sure to include in the stem the words

that are repeated in all options (avoid redundancy)

Suggestions for constructing MC items

For the options Use errors commonly made by students in class as

basis for distractors Use options that say “none of the above”, “all of the

above”, “A & B only” sparingly Avoid having the key being the longest or shortest

item When possible, present options in some logical

order (chronological, least to most, alphabetical…) Use similar format for all options (all are phrases,

all are single words, etc.)

True/False items

Essentially, T/F items are MC items with only 2 options

Advantages: Good for testing misconceptions,

comprehension Can be scored quickly

Disadvantages: Students have a 50-50 chance to guess

the right answer

Suggestions for constructing T/F items Make sure there is a single idea in the stem Make sure the statement is either absolutely

true or false, no exceptions Avoid using extreme qualifiers (never, always,

etc.) whenever possible Avoid using negative statements Avoid recycling the exact statement from the

text Avoid using unfamiliar vocabulary

Matching items

Directions: On the line next to each children’s book inColumn A print the letter of the animal or insect in ColumnB that is a main character in that book. Each animal or

insectin Column B can be used only once.Column A Column B____1. Charlotte’s Web A. Bear____2. Winnie the Pooh B. Chimpanzee____3. Black Beauty C. Cricket____4. Tarzan D. Deer____5. Pinocchio E. Horse____6. Bambi F. Pig

Matching items: advantages & disadvantages Essentially are MC items, in which the answer of one item

also depends on the answers of the other items in the matching set

Advantages: Good for testing Definitions Classifications Cause and effects Problems and solutions Relationship between 2 entities (e.g., event-date, book-

character) Disadvantages

Items are inter-related Time consuming for test-takers Not suitable for all knowledge and learning skills

Suggestions for constructing matching items

Make sure all items are of the same topic Directions: Match the following.

1. Water A. NaCl2. Discovered Radium B. Fermi3. Salt C. NH34. Ammonia D. 19425. Year of the first Nuclear Fission E. H20

F. CurieG. 1957

Do not mix matching with sentence completion, e.g.,__ 1. A ____ orbits a planet. A. sun etc.

Gap-filling items

The colors of the United States flag are ____, ____, and ____.

Gap-filling items are open-ended, requiring students to produce language.

Gap-filling items: advantages & disadvantages Advantages:

Reducing guessing Students must know answers, no hint given More or less objective

Disadvantages: Answers limited to a few restricted words, not

assessing higher level learning Difficult to restrict answers, it is easy to have more

than one correct answer Scoring takes more time than MC items

Examples

1. The smallest phonetic ____ in a language that is capable of conveying a distinction in meaning is called a phoneme.

2. The ______ were to Egypt as the ______were to Persia and as ______ were to the early tribes of Israel.

3. Some men don’t have ____ on their face.

4. An ____ is a part or particle considered to be an irreducible constituent of a specified system.

Suggestions for constructing gap-filling items The gap should contain the key

information being tested. Make sure the meaning of the statement

is not lost after words have been taken out

Make sure only one correct answer is possible.

Avoid grammatical clue that gives away the answer

Short-answer items

What are the colors of the United States flag?

Essay-test items: Extended response: paragraphs Restricted-response: short-answer items

How short is short? A word, a phrase, a sentence (not a

paragraph)

Short-answer itemsAdvantages:

student needs to produce language, less guessing

can test some higher learning level easier to write: no need for distractors or strict

answer restriction avoid the problems of extended response items

Disadvantages: scoring is less objective more time consuming for test-takers and scorers

References

Hughes, A. (2003). Testing for language teachers (2nd ed). Cambridge University Press.

Clay, Ben. (2001). Is this a trick question? A short guide to writing effective test questions. Kansas Curriculum Center.