Guidelines for preparing Multiple Choice Questions

(MCQ’s)

How?

References available: Case and Swanson, NBME “Constructing

Questions for the Basic and Clinical Sciences” http://www.nbme.org/about/itemwriting.asp -

publications, item writing manual

Traditional MCQ’s

Multiple types

Do not use:True / False Negative StemsR typeK type

Good MCQ’s *

Assess application of knowledge

Assess reasoning skills (how do you think)

DO NOT assess simple recall of isolated facts

Good MCQ’s *

“One Best Answer” type

The Basic Rules for One-Best-Answer Items

1. Each item should focus on an important conceptCommon, serious or potentially

catastrophic clinical problemAssesses an Objective or Competency of

the curriculumAvoid :

Trivial, Overly complex, or Tricky

Case and Swanson, 2001

The Basic Rules for One-Best-Answer Items

2. Each item should assess the application of knowledge NOT recall of an isolated fact.

Case and Swanson, 2001

How?

Present a Stem:Present a clinical case scenario as a good

basis for a questionPresent a question

Provide Options for answers (generally 5) “One Best” correct answer several “distractors”

How? An old exampleAcute intermittent porphyria is the result of

a defect in the biosynthetic pathway for:A. collagenB. corticosteroidC. fatty acidD. glucoseE. heme

How to do it better?A better question is one that uses a clinical scenario in the stem: An otherwise healthy 33 year-old man has mild weakness and

occasional episodes of steady, severe abdominal pain with some cramping and no diarrhea. One aunt and a cousin have had similar episodes. During an episode, his abdomen is distended, and bowel sounds are decreased. Neurological examination shows mild weakness in the upper arms. These findings suggest a defect in the biosynthetic pathway for

A. collagen B. corticosteroid C. fatty acid D. glucose E. heme

How? Another example

What area is supplied with blood by the posterior inferior cerebellar artery?

or

A 63-year old man develops left-sided limb ataxia, Horner’s syndrome, nystagmus, and loss of appreciation of facial pain and temperature sensations. What artery is most likely to be occluded?

Case and Swanson, 2001

Basic Principle

Application of knowledge

rather than

Isolated Fact

Basic Principle

• Not only the Memory (memorization)• Avoid questions of simple fact that can

be found on a single page in a textbook• Engage the Mind (cognitive –reasoning,

problem solve)

The Basic Rules for One-Best-Answer Items

3. The stem of the item must pose a clear question

Based on the scenario, pose the question. It should be possible to arrive at an answer without seeing the options

Case and Swanson, 2001

The Basic Rules for One-Best-Answer Items

4. Write the answer and distractors• All options should be homogeneous• Correct and incorrect options (distractors)

should fall into the same category (all diagnoses, tests, treatments, etc)

Case and Swanson, 2001

Distractors

Should be: Plausible Grammatically consistent Logically compatible Same relative length as the correct answer

Place the options in logical order (numeric or alphabetical)

Case and Swanson, 2001

The Basic Rules for One-Best-Answer Items

5. Avoid “technical item flaws” that provide special benefit to test-wise examinees or that pose irrelevant or unnecessary difficulty

(please refer to manual pg 26 and 33)

Case and Swanson, 2001

ExampleTechnical Item Flaws

In patients with advance dementia, Alzheimer’s type, the memory defect A. can be treated adequately with phosphatidylcholine

(lecithin) B. could be a sequela of early parkinsonism C. is never seen in patients with neurofibrillary tangles at

autopsy D. is never severe E. possibly involves the cholinergic system

Possible changes

A 87 yrs female has advanced dementia of the Alzheimer’s type. Her memory and communications with her family are improved following a new medication. The drug type most likely responsible for her improvement is: A. cholinergic agonists B. dopaminergic agonists C. GABA agonists D. nicotinic agonists E. serotonergic agonists

Advanced Question Writing

K1 Questions – designed to test Competency of Application of Knowledge

K2 Questions – Designed to Assess Competency of Reasoning and Clinical Problem Solving

Advanced Question Writing

• K2 Questions – Designed to Assess reasoning and Clinical Problem Solving

- the question goes beyond application

- the question skips the step of application and requires the candidate to engage in Clinical Reasoning

- the answer usually involves a diagnosis, differential diagnosis or a management option

Advanced Question Writing

• K2 Questions

Ref: Ware, J. Torstein V. Quality Assurance of Item Writing. Medical Teacher, 2009, Vol. 31, No. 3 : Pages 238-243

Review the Basic Rules

Subject each question you write to the above 5 “tests”

If a question passes all 5, it is probably well-written and a good question

Does your question assess application of Knowledge or Clinical Reasoning

Case and Swanson, 2001

Final Check?

Please remember to confirm: The question is testing a learning objective

or competencyThe case scenario is appropriateThe question and the answer are both

clear

What to do now?

Each lecturer or tutor is asked to construct at least two questions for the block.

Each question is based on one of the weekly case objectives

Your questions can be reviewed with your writing team providing feedback to you.