Writing Meaningful Writing Meaningful Test QuestionsTest Questions

Kurt Gilliland, Ph.D.Dept. of Cell Biology and Physiology

Ryan Madanick, M.D.Division of Gastroenterology and Hepatology

Dept. of Medicine

Writing Meaningful Test QuestionsWriting Meaningful Test Questions

• Format of stems, lead-ins, and options» What are common flaws to avoid?

• Content of stems, lead-ins, and options» How can your items reflect learning

objectives?» How can you develop items that test

application instead of recall?

1. The primary purpose of the stam is 1. The primary purpose of the stam is to remove which of the following? to remove which of the following?

A. CarmA. CarmB. DentonB. DentonC. MeniceC. MeniceD. Stam barD. Stam bar

1. The primary purpose of the 1. The primary purpose of the stamstam is is to remove which of the following? to remove which of the following?

A. CarmA. CarmB. DentonB. DentonC. MeniceC. MeniceD. D. Stam barStam bar

Word repeat:A word or phrase is included in the stem and in the correct answer.

Item FlawItem Flaw

✔

2. Which of the following pairs has 2. Which of the following pairs has won the greatest number of Abby won the greatest number of Abby awards?awards?

A. Jones and SmithA. Jones and SmithB. Smith and TaylorB. Smith and TaylorC. Smith and WhiteC. Smith and WhiteD. White and AllenD. White and Allen

2. Which of the following pairs has 2. Which of the following pairs has won the greatest number of Abby won the greatest number of Abby awards?awards?

A. Jones and A. Jones and SmithSmithB. B. SmithSmith and Taylor and TaylorC. C. SmithSmith and and WhiteWhiteD. D. WhiteWhite and Allen and Allen

Convergence strategy:The correct answer included the most elements in common with other options.

Item FlawItem Flaw

✔

3. How many pounds of pressure are 3. How many pounds of pressure are exerted by a callam?exerted by a callam?

A. 260A. 260B. 150B. 150C. 2.6C. 2.6D. 2600D. 2600

3. How many pounds of pressure are 3. How many pounds of pressure are exerted by a callam?exerted by a callam?

A. A. 260260B. 150B. 150C. C. 2.62.6D. D. 26002600

Language in the options is not parallel:Options are in an illogical order, in this case.

Item FlawsItem Flaws

✔

Logical cues:A subset of the options are collectively exhaustible. Often, the correct answer is in the middle of the range.

4. The stanon is aided by a4. The stanon is aided by a

A. anstelA. anstelB. immonB. immonC. octalC. octalD. portD. port

4. The stanon is aided by 4. The stanon is aided by aa

A. anstelA. anstelB. immonB. immonC. octalC. octalD. D. pportort

Item FlawItem Flaw

✔

Grammatical cues:One or more distractors don’t follow grammatically from the stem.

5. Why does the stanon frequently 5. Why does the stanon frequently overheat?overheat?

A. All grestels are belious.A. All grestels are belious.B. No immons are directly fectitious.B. No immons are directly fectitious.C. Octals are usually casable.C. Octals are usually casable.D. Ports are always actial.D. Ports are always actial.

5. Why does the stanon 5. Why does the stanon frequentlyfrequently overheat?overheat?

A. A. AllAll grestels are belious. grestels are belious.B. B. NoNo immons are directly fectitious. immons are directly fectitious.C. Octals are usually casable.C. Octals are usually casable.D. Ports are D. Ports are alwaysalways actial. actial.

Item FlawsItem Flaws

✔

Absolute terms:Terms such as “always” or “never” are used in options.

Frequency terms in the topics are vague:Terms such as “rarely” or “usually” not consistently defined or interpreted, even by experts.

6. Stamation normally occurs when 6. Stamation normally occurs when the…the…

A. anstels rupture.A. anstels rupture.B. immon falls and the denton is in B. immon falls and the denton is in

place.place.C. octal rotates easily.C. octal rotates easily.D. ports pass over the carm.D. ports pass over the carm.

6. Stamation normally occurs when 6. Stamation normally occurs when the…the…

A. anstels rupture.A. anstels rupture.B. B. immon falls and the denton is in immon falls and the denton is in

place.place.C. octal rotates easily.C. octal rotates easily.D. ports pass over the carm.D. ports pass over the carm.

Item FlawItem Flaw

✔

Long correct answer:Correct answer is longer, more specific, or more complete than other options.

BONUS. Find 4 flaws. BONUS. Find 4 flaws.

Regarding syphilis, all are true except…Regarding syphilis, all are true except…

A. The skin rash of secondary syphilis may be macular or papular and spirochetes A. The skin rash of secondary syphilis may be macular or papular and spirochetes may be visualized by the Warthin-Starry stain.may be visualized by the Warthin-Starry stain.

B. Nontreponemal serologic tests for syphilis include the FTA and MHA tests; these B. Nontreponemal serologic tests for syphilis include the FTA and MHA tests; these are positive early in are positive early in T. pallidum T. pallidum infections but become negative in infections but become negative in tertiary tertiary

syphilis.syphilis.C. Gummas are granulomatous lesions of tertiary syphilis and show necrotic C. Gummas are granulomatous lesions of tertiary syphilis and show necrotic

centers and a surrounding rim of plasma cells.centers and a surrounding rim of plasma cells.D. Cardiovascular tertiary syphylis is seen in the ascending and or thoracic aorta D. Cardiovascular tertiary syphylis is seen in the ascending and or thoracic aorta

with aortitis and aneurysm.with aortitis and aneurysm.E. None of the aboveE. None of the above

BONUS. Find 4 flaws. BONUS. Find 4 flaws.

Regarding syphilis, all are true except…Regarding syphilis, all are true except…

A. The skin rash of secondary syphilis may be macular or papular and spirochetes A. The skin rash of secondary syphilis may be macular or papular and spirochetes may be visualized by the Warthin-Starry stain.may be visualized by the Warthin-Starry stain.

B. Nontreponemal serologic tests for syphilis include the FTA and MHA tests; these B. Nontreponemal serologic tests for syphilis include the FTA and MHA tests; these are positive early in are positive early in T. pallidum T. pallidum infections but become negative in infections but become negative in tertiary tertiary

syphilis.syphilis.C. Gummas are granulomatous lesions of tertiary syphilis and show necrotic C. Gummas are granulomatous lesions of tertiary syphilis and show necrotic

centers and a surrounding rim of plasma cells.centers and a surrounding rim of plasma cells.D. Cardiovascular tertiary syphylis is seen in the ascending and or thoracic aorta D. Cardiovascular tertiary syphylis is seen in the ascending and or thoracic aorta

with aortitis and aneurysm.with aortitis and aneurysm.E. None of the aboveE. None of the above

Item FlawsItem Flaws

1. Short stem but long, heterogeneous options.2. A negative term in the stem.3. Unfocused stem.4. Use of “none of the above” as option.

Frequency TermsFrequency Terms

What do terms like “usually” or “frequently” mean to you?

Frequency TermsFrequency Terms

The most important prognostic factor The most important prognostic factor in AIDS is…in AIDS is…

A. CD4:CD8 ratioA. CD4:CD8 ratioB. presence of Kaposi’s sarcomaB. presence of Kaposi’s sarcomaC. absolute leukocyte countC. absolute leukocyte countD. absolute CD4 countD. absolute CD4 countE. level of GP120 antibodyE. level of GP120 antibody

The most important prognostic factor The most important prognostic factor in AIDS is…in AIDS is…

A. CD4:CD8 ratioA. CD4:CD8 ratioB. presence of Kaposi’s sarcomaB. presence of Kaposi’s sarcomaC. absolute leukocyte countC. absolute leukocyte countD. absolute CD4 countD. absolute CD4 countE. level of GP120 antibodyE. level of GP120 antibody

Unfocused stem

Use of absolute terms such as “most”

What is the cause of muscle What is the cause of muscle soreness that is present the day or soreness that is present the day or two after exercise?two after exercise?

A. Accumulation of lactic acidA. Accumulation of lactic acidB. Microscopic muscle damage leading to an B. Microscopic muscle damage leading to an inflammatory response including the release of inflammatory response including the release of bradykinin, PGE2, and histamine which sensitive bradykinin, PGE2, and histamine which sensitive nerve endingsnerve endingsC. Muscle stiffnessC. Muscle stiffnessD. Use of anabolic steroidsD. Use of anabolic steroids

What is the cause of muscle What is the cause of muscle soreness that is present the day or soreness that is present the day or two after exercise?two after exercise?

A. Accumulation of lactic acidA. Accumulation of lactic acidB. Microscopic muscle damage leading to an B. Microscopic muscle damage leading to an inflammatory response including the release of inflammatory response including the release of bradykinin, PGE2, and histamine which sensitive bradykinin, PGE2, and histamine which sensitive nerve endingsnerve endingsC. Muscle stiffnessC. Muscle stiffnessD. Use of anabolic steroidsD. Use of anabolic steroids

Long distractor

Heterogeneous distractors

The tophi of gouty arthritis:The tophi of gouty arthritis:

A. are a form of erythema nodosumA. are a form of erythema nodosumB. have a fibrinoid center surrounded by peripheral B. have a fibrinoid center surrounded by peripheral

histiocyteshistiocytesC. are composed of prominent osteophytes at the C. are composed of prominent osteophytes at the DIP joints of fingersDIP joints of fingersD. none of the aboveD. none of the above

The tophi of gouty arthritis:The tophi of gouty arthritis:

A. are a form of erythema nodosumA. are a form of erythema nodosumB. have a fibrinoid center surrounded by peripheral B. have a fibrinoid center surrounded by peripheral

histiocyteshistiocytesC. are composed of prominent osteophytes at the C. are composed of prominent osteophytes at the DIP joints of fingersDIP joints of fingersD. none of the aboveD. none of the above

Unfocused stem

Short stem, long distractors

“None of the above”

Sigmund Freud’s Sigmund Freud’s Interpretation of Interpretation of DreamsDreams was published in what year? was published in what year?

A. 1911A. 1911B. 1912B. 1912C. 1913C. 1913D. 1914D. 1914E. 1915E. 1915

Sigmund Freud’s Sigmund Freud’s Interpretation of Interpretation of DreamsDreams was published in what year? was published in what year?

A. 1911A. 1911B. 1912B. 1912C. 1913C. 1913D. 1914D. 1914E. 1915E. 1915

Correct answer (1913) in middle of range

Question is recall of minutia and not application

Aspirin should NOT be prescribed Aspirin should NOT be prescribed when the patient has which of the when the patient has which of the following conditions?following conditions?

A. Peptic ulcers A. Peptic ulcers B. HeadacheB. HeadacheC. FeverC. FeverD. PainD. PainE. InflammationE. Inflammation

Aspirin should NOT be prescribed Aspirin should NOT be prescribed when the patient has which of the when the patient has which of the following conditions?following conditions?

A. Peptic ulcers A. Peptic ulcers B. HeadacheB. HeadacheC. FeverC. FeverD. PainD. PainE. InflammationE. Inflammation

Use of “not” is acceptable in this case since the teaching point was in fact a contraindication.

Which of the following is true Which of the following is true regarding lysosomal storage regarding lysosomal storage disease?disease?

A.A. 会意字 会意字 / / 會意字會意字B. B. ربي/عربىربي/عربىC.C. 日本語日本語D. D. में� जन्मेंज�त स्वतन्त्रत� और में� जन्मेंज�त स्वतन्त्रत� और

Which of the following is true Which of the following is true regarding lysosomal storage regarding lysosomal storage disease?disease?

A.A. 会意字 会意字 / / 會意字會意字B. B. ربي/عربىربي/عربىC.C. 日本語日本語D. D. में� जन्मेंज�त स्वतन्त्रत� और में� जन्मेंज�त स्वतन्त्रत� और

Unfocused stem