Upload
julian-hubbard
View
220
Download
1
Embed Size (px)
Citation preview
Writing MCQsWriting MCQs
Susan ChamberlainSusan Chamberlain
Dept of OBS/GYNDept of OBS/GYN
Queen’s UniversityQueen’s University
Writing MCQsWriting MCQs
At the end of this session, the participant will At the end of this session, the participant will be able to be able to
Identify different types of MCQ itemsIdentify different types of MCQ items Describe the structure of an MCQDescribe the structure of an MCQ List the technical flaws common to MCQ’sList the technical flaws common to MCQ’s write one well-structured MCQ testing write one well-structured MCQ testing
appropriate contentappropriate content
Writing MCQsWriting MCQs
At the end of this session, the participant will At the end of this session, the participant will be able to be able to
Identify different types of MCQ itemsIdentify different types of MCQ items Describe the structure of an MCQDescribe the structure of an MCQ List the technical flaws common to MCQ’sList the technical flaws common to MCQ’s write one well-structured MCQ testing write one well-structured MCQ testing
appropriate contentappropriate content All of the aboveAll of the above
MCQ Check-listMCQ Check-list
Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it
Why do we test?Why do we test?
Communicate to the students what material Communicate to the students what material is important (formative vs summative)is important (formative vs summative)
Motivate students to studyMotivate students to study Identify areas of deficiency in need of Identify areas of deficiency in need of
remediation or further learningremediation or further learning Determine final grades or make promotion Determine final grades or make promotion
decisionsdecisions Identify areas where the course/curriculum Identify areas where the course/curriculum
is weakis weak
What Should Be Tested?What Should Be Tested?
Exam content should match course Exam content should match course objectivesobjectives
Important topics should be weighted more Important topics should be weighted more heavily than less important topicsheavily than less important topics
The testing time devoted to each topic The testing time devoted to each topic should reflect the relative importance of the should reflect the relative importance of the topicstopics
The sample of items should be The sample of items should be representative of the instructional goalsrepresentative of the instructional goals
What Should Be Tested?What Should Be Tested?
SamplingSampling““Performance on the sample provides a basis Performance on the sample provides a basis for estimating achievement in the broader for estimating achievement in the broader domain that is actually of interest.”domain that is actually of interest.”
Questions sample: Questions sample: – TOPICSTOPICS– SKILLS ie determining diagnosis, next step in SKILLS ie determining diagnosis, next step in
management, etc.management, etc.
MCQ ‘Items’MCQ ‘Items’
Should test important information – either Should test important information – either common or life-threateningcommon or life-threatening
Should be well-structuredShould be well-structured– Flaws benefit test-wise examineesFlaws benefit test-wise examinees– Irrelevant difficulty does not test examinees’ Irrelevant difficulty does not test examinees’
specific knowledgespecific knowledge
Anatomy of an MCQ ItemAnatomy of an MCQ Item
StemStem Lead-inLead-in OptionsOptions
– One correct answerOne correct answer– DistractorsDistractors
Types of MCQ’sTypes of MCQ’s
True/False familyTrue/False family Assess recall of isolated factsAssess recall of isolated facts
– Stems must be clear and unambiguousStems must be clear and unambiguous– Options must be absolutely true or false or Options must be absolutely true or false or
examinee must decide examinee must decide how truehow true options are options are
Generally not recommended – usually have Generally not recommended – usually have subtle flawssubtle flaws
Types of MCQ’s: True/FalseTypes of MCQ’s: True/False
Which of the following statements is true regarding Which of the following statements is true regarding contraception?contraception?
A)A) oral contraceptives should not be initiated until six weeks oral contraceptives should not be initiated until six weeks after therapeutic abortion to avoid the risk of after therapeutic abortion to avoid the risk of hypercoagulabilityhypercoagulability
B)B) an intrauterine device can provide good contraceptive an intrauterine device can provide good contraceptive protection 5 years after insertionprotection 5 years after insertion
C)C) oral contraceptives should not be prescribed to individuals oral contraceptives should not be prescribed to individuals under the age of 15 without parental consentunder the age of 15 without parental consent
D)D) lesbian women do not require counseling about lesbian women do not require counseling about contraception or sexually transmitted diseasescontraception or sexually transmitted diseases
E)E) oral contraceptives should not be prescribed for women overoral contraceptives should not be prescribed for women over the age of 35the age of 35
Types of MCQ’sTypes of MCQ’s
One-Best-Answer family (A type)One-Best-Answer family (A type) Options may be diagrammed:Options may be diagrammed:
D C A E BD C A E B
Correct answer is the Correct answer is the most likelymost likely correct correct Better application of knowledge, integration, Better application of knowledge, integration,
synthesis and judgmentsynthesis and judgment
Least correct Most correct
Types of MCQ’s: Types of MCQ’s: One-Best-Answer familyOne-Best-Answer family
A 24 year old university student came to see you because she is A 24 year old university student came to see you because she is missing several days of school each month because of severe missing several days of school each month because of severe dysmenorrhea unresponsive to the OCP and NSAIDS. She also dysmenorrhea unresponsive to the OCP and NSAIDS. She also complains of deep dyspareunia and has a tender uterosacral complains of deep dyspareunia and has a tender uterosacral nodule on the pelvic examination. Which one of the following nodule on the pelvic examination. Which one of the following conditions is most likely responsible for her symptoms?conditions is most likely responsible for her symptoms?
A)A) Chronic pelvic inflammatory diseaseChronic pelvic inflammatory diseaseB)B) EndometriosisEndometriosisC)C) AdenomyosisAdenomyosisD)D) FibroidsFibroids
MCQ Check-listMCQ Check-list
Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it
MCQ Technical FlawsMCQ Technical Flaws
Issues Related to Test-Wiseness Issues Related to Test-Wiseness Issues Related to Irrelevant DifficultyIssues Related to Irrelevant Difficulty
MCQ Technical FlawsMCQ Technical Flaws
Issues Related to Test-WisenessIssues Related to Test-Wiseness– provide clues to the right answer provide clues to the right answer – examinee can eliminate options based on flaws examinee can eliminate options based on flaws
and not on knowledgeand not on knowledge
Issues Related to Irrelevant DifficultyIssues Related to Irrelevant Difficulty
Item Flaws: Test-wisenessItem Flaws: Test-wiseness
Grammatical cuesGrammatical cues Logical cuesLogical cues Absolute termsAbsolute terms Long correct answerLong correct answer Word repeatsWord repeats Convergence strategyConvergence strategy
Item Flaws: Test-wisenessItem Flaws: Test-wiseness
Grammatical cuesGrammatical cues One or more distractors don’t follow One or more distractors don’t follow
grammatically from the stem grammatically from the stem
Item Flaws: Test-wisenessItem Flaws: Test-wiseness
Grammatical cuesGrammatical cuesA 24 year old patient presents with a one year A 24 year old patient presents with a one year
history of dysmenorrhea, mid-cycle pain, history of dysmenorrhea, mid-cycle pain, dyspareunia, and painful bowel movements with dyspareunia, and painful bowel movements with her periods. The most likely diagnosis is:her periods. The most likely diagnosis is:
A)A) Fibroid uterusFibroid uterus
B)B) AdenomyosisAdenomyosis
C)C) EndometriosisEndometriosis
D)D) Psychosomatic conditionPsychosomatic condition
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Logical cuesLogical cues A subset of the options are collectively A subset of the options are collectively
exhaustiveexhaustive
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Logical cuesLogical cues Arterial blood gases in a normal pregnant Arterial blood gases in a normal pregnant
woman, compared to her non-pregnant state may woman, compared to her non-pregnant state may show which of the following?show which of the following?
A)A) Decreased pODecreased pO22
B)B) Increased pCOIncreased pCO22
C)C) Decreased pCODecreased pCO22
D)D) Decreased pHDecreased pH
E)E) No difference in pCONo difference in pCO22
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Logical cuesLogical cues One or more distractors don’t follow One or more distractors don’t follow
logicallylogically from the stem from the stem
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Logical cuesLogical cuesAn important tool for assessing fetal well-being is An important tool for assessing fetal well-being is
the Non Stress test. A well baby demonstrates the Non Stress test. A well baby demonstrates the following fetal heart rate pattern:the following fetal heart rate pattern:
A)A) A baseline heart rate above 120 bpmA baseline heart rate above 120 bpm
B)B) Evidence of fetal movementEvidence of fetal movement
C)C) Baseline variabilityBaseline variability
D)D) Two fetal heart rate accelerations in 20 minutesTwo fetal heart rate accelerations in 20 minutes
E)E) Fetal breathing movementsFetal breathing movements
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Absolute termsAbsolute terms Always, neverAlways, never Not a problem if stem is long and lead-in Not a problem if stem is long and lead-in
asks a precise questionasks a precise question
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Absolute termsAbsolute terms
In patients with a previous Cesarean section:In patients with a previous Cesarean section:A)A) Vaginal birth is NEVER an optionVaginal birth is NEVER an option
B)B) Repeat Cesarean section is less dangerous to the Repeat Cesarean section is less dangerous to the mothermother
C)C) The uterine scar may rupture during labourThe uterine scar may rupture during labour
D)D) Care should ALWAYS be supervised by an Care should ALWAYS be supervised by an obstetricianobstetrician
E)E) No further pregnancies should EVER be consideredNo further pregnancies should EVER be considered
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Long correct answerLong correct answer Correct answer is longer, more specific, or Correct answer is longer, more specific, or
more complete than other optionsmore complete than other options
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Long correct answerLong correct answer Which of the following problems in vaginal Which of the following problems in vaginal
breech delivery is most likely to cause serious breech delivery is most likely to cause serious long-term complications?long-term complications?
A)A) Failure to descendFailure to descendB)B) Cord prolapseCord prolapseC)C) Nuchal armsNuchal armsD)D) Entrapment of the after coming head with an Entrapment of the after coming head with an
incompletely dilated cervixincompletely dilated cervixE)E) 1 minute Apgar score of 41 minute Apgar score of 4
Item Flaws: Test-wisenessItem Flaws: Test-wiseness
Word repeatsWord repeats A word or phase is included in the stem and A word or phase is included in the stem and
in the correct answerin the correct answer
Item Flaws: Test-wisenessItem Flaws: Test-wiseness
Word repeatsWord repeats
A 24 year old with a previous history of chronic A 24 year old with a previous history of chronic PID presents with primary infertility. Select the PID presents with primary infertility. Select the procedure that would be most appropriate to procedure that would be most appropriate to evaluate the status of her evaluate the status of her tubestubes and pelvic and pelvic anatomyanatomy
A)A) Post-coital testPost-coital test
B)B) Endometrial biopsyEndometrial biopsy
C)C) Laparoscopy with dye insufflation of the Laparoscopy with dye insufflation of the fallopian fallopian tubestubes
D)D) HysterosalpingogramHysterosalpingogram
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Convergence strategyConvergence strategy The correct answer includes the most The correct answer includes the most
elements in common with the other optionselements in common with the other options
Item Flaws: Test-wiseness Item Flaws: Test-wiseness
Convergence strategyConvergence strategyNormal bladder neck mobility is measured by:Normal bladder neck mobility is measured by:
A)A) A Q-tip angle measurement under 40 degreesA Q-tip angle measurement under 40 degrees
B)B) A Q-tip angle measurement under 30 degreesA Q-tip angle measurement under 30 degrees
C)C) A bladder neck ultrasound angle less than 30 A bladder neck ultrasound angle less than 30 degreesdegrees
D)D) A Q-tip angle measurement under 20 degreesA Q-tip angle measurement under 20 degrees
MCQ Technical FlawsMCQ Technical Flaws
Issues Related to Test-Wiseness Issues Related to Test-Wiseness
Issues Related to Irrelevant DifficultyIssues Related to Irrelevant Difficulty– do not assess knowledge of intended topicdo not assess knowledge of intended topic
Irrelevant DifficultyIrrelevant Difficulty
Options are long, complicated or doubleOptions are long, complicated or double Extraneous details in stemExtraneous details in stem Numeric data not stated consistentlyNumeric data not stated consistently Vague frequency termsVague frequency terms Non-parallel language in optionsNon-parallel language in options Stems are tricky or unnecessarily complicatedStems are tricky or unnecessarily complicated ‘‘None of the above’None of the above’
Irrelevant DifficultyIrrelevant Difficulty
Options are long, complicated or doubleOptions are long, complicated or double
Extraneous details in stemExtraneous details in stem Shifts difficulty from content knowledge to Shifts difficulty from content knowledge to
speed readingspeed reading
Irrelevant DifficultyIrrelevant Difficulty
Options are long, complicated or doubleOptions are long, complicated or double Mrs. Brown is a 70 year old woman who had a stroke 6 months Mrs. Brown is a 70 year old woman who had a stroke 6 months
ago and is still impaired. Her daughter tells you that her ago and is still impaired. Her daughter tells you that her mother’s recent urinary incontinence makes her care mother’s recent urinary incontinence makes her care unmanageable. She is seeking permanent placement. Mrs. unmanageable. She is seeking permanent placement. Mrs. Brown complains of urinary frequency, nocturia, and urge Brown complains of urinary frequency, nocturia, and urge incontinence. Which of the following statements is true?incontinence. Which of the following statements is true?
A)A) Neurological conditions are involved in the majority of women with Neurological conditions are involved in the majority of women with symptoms like those presented by Mrs. Brownsymptoms like those presented by Mrs. Brown
B)B) Diabetes has a greater impact on quality of life than Mrs. Brown’s Diabetes has a greater impact on quality of life than Mrs. Brown’s condition.condition.
C)C) The pharmacological blockage of muscarinic receptors in the detrusor The pharmacological blockage of muscarinic receptors in the detrusor muscle controls symptoms compatible with Mrs. Brown’s conditionmuscle controls symptoms compatible with Mrs. Brown’s condition
D)D) Overactive bladder is defined as urinary incontinence accompanied by Overactive bladder is defined as urinary incontinence accompanied by or immediately preceded by urgencyor immediately preceded by urgency
E)E) Alpha stimulants are effective in treating this conditionAlpha stimulants are effective in treating this condition
Irrelevant DifficultyIrrelevant Difficulty
Extraneous details in stemExtraneous details in stem You see in your office a 24 year old woman for her routine postpartum You see in your office a 24 year old woman for her routine postpartum
visit, at 6 weeks. Her baby is irritable and cries “all the time’ visit, at 6 weeks. Her baby is irritable and cries “all the time’ (including during her visit.) As she is changing the diaper during her (including during her visit.) As she is changing the diaper during her visit, you notice that the child has a very red bum but upon visit, you notice that the child has a very red bum but upon questioning, she tells you she did not notice anything abnormal. Her questioning, she tells you she did not notice anything abnormal. Her partner is away in Afghanistan and she does not have a lot of friends, partner is away in Afghanistan and she does not have a lot of friends, as they just recently were posted in Kingston. She states that she has a as they just recently were posted in Kingston. She states that she has a hard time getting dressed in the morning, cries easily and feels like she hard time getting dressed in the morning, cries easily and feels like she does not love her child. She denies any medical or psychiatric history. does not love her child. She denies any medical or psychiatric history. Her exam is essentially normal. She weighs 2 kg less than at her first Her exam is essentially normal. She weighs 2 kg less than at her first antenatal visit at 10 weeks. You most likely clinical diagnosis is:antenatal visit at 10 weeks. You most likely clinical diagnosis is:
A)A) Postpartum hypothyroidismPostpartum hypothyroidism
B)B) HyperthyroidismHyperthyroidism
C)C) Postpartum bluesPostpartum blues
D)D) Postpartum depressionPostpartum depression
E)E) Psychotic depressionPsychotic depression
Irrelevant Difficulty Irrelevant Difficulty
Numeric DataNumeric Data Not stated consistentlyNot stated consistently Mixed formatMixed format Illogical orderIllogical order
Irrelevant Difficulty Irrelevant Difficulty
Numeric Data not stated consistentlyNumeric Data not stated consistently Following a second episode of salpingitis, what Following a second episode of salpingitis, what
is the likelihood that a woman is infertile?is the likelihood that a woman is infertile?A)A) Less than 20%Less than 20%
B)B) 20 – 30 %20 – 30 %
C)C) Greater than 50 %Greater than 50 %
D)D) 90%90%
E)E) 75%75%
Irrelevant Difficulty Irrelevant Difficulty
Vague frequency termsVague frequency terms Often, usually, rarely, frequently Often, usually, rarely, frequently Not consistently definedNot consistently defined
*may be used in stem if otherwise carefully *may be used in stem if otherwise carefully constructed unambiguous questionconstructed unambiguous question
Irrelevant DifficultyIrrelevant Difficulty
Vague frequency termsVague frequency terms
Gonorrhea isGonorrhea is
A) sometimes asymptomaticA) sometimes asymptomatic
B) treated with cefixime 450 mg orallyB) treated with cefixime 450 mg orally
C) caused by a gram positive intracellular C) caused by a gram positive intracellular diplococcusdiplococcus
D) diagnosed by a DNA testD) diagnosed by a DNA test
E) usually associated with vulvar ulcersE) usually associated with vulvar ulcers
Irrelevant DifficultyIrrelevant Difficulty
Non-parallel language in optionsNon-parallel language in options Non-logical orderNon-logical order
Irrelevant DifficultyIrrelevant Difficulty
Non-parallel language in optionsNon-parallel language in optionsIn a vaccine trial, 200 2-year-old boys were given a vaccine In a vaccine trial, 200 2-year-old boys were given a vaccine
against a certain disease and then monitored for five against a certain disease and then monitored for five years for occurrence of the disease. Of this group, 85% years for occurrence of the disease. Of this group, 85% never contracted the disease. Which of the following never contracted the disease. Which of the following statements concerning these results is correct?statements concerning these results is correct?
A)A) No conclusion can be drawn, since no follow-up was No conclusion can be drawn, since no follow-up was made of non-vaccinated childrenmade of non-vaccinated children
B)B) The number of cases (ie 30 cases over five year) is The number of cases (ie 30 cases over five year) is too small for statistically meaningful conclusionstoo small for statistically meaningful conclusions
C)C) No conclusions can be drawn because the trial No conclusions can be drawn because the trial involved only boysinvolved only boys
D)D) Vaccine efficacy (%) is calculated as 85-15/100Vaccine efficacy (%) is calculated as 85-15/100
Irrelevant DifficultyIrrelevant Difficulty
““None of the above”None of the above” Problematic where judgment is involved or Problematic where judgment is involved or
if options not absolutely true or falseif options not absolutely true or false Turns question into true/false questionTurns question into true/false question Confuses very knowledgeable studentsConfuses very knowledgeable students
Irrelevant DifficultyIrrelevant Difficulty
““None of the above”None of the above”
The main reason that fertility is reduced in a The main reason that fertility is reduced in a woman in her early 40’s is:woman in her early 40’s is:
A)A) Irregular cycles make it hard to time intercourseIrregular cycles make it hard to time intercourse
B)B) Chromosomal damage adversely affects oocytes’ Chromosomal damage adversely affects oocytes’ genetic complementgenetic complement
C)C) Male erectile difficulties increaseMale erectile difficulties increase
D)D) Hormone imbalance leads to poor endometrial Hormone imbalance leads to poor endometrial developmentdevelopment
E)E) None of the aboveNone of the above
Irrelevant DifficultyIrrelevant Difficulty
Stems are tricky or unnecessarily complicatedStems are tricky or unnecessarily complicated Arrange the parents of the following children with Down’s syndrome Arrange the parents of the following children with Down’s syndrome
in order of the highest to lowest risk of recurrence. Assume that the in order of the highest to lowest risk of recurrence. Assume that the maternal age in all cases is 22 years and that a subsequent pregnancy maternal age in all cases is 22 years and that a subsequent pregnancy occurs within 5 years. The karyotypes of the daughters are:occurs within 5 years. The karyotypes of the daughters are:
I: 46, XX, –14,+T (14q21q) patI: 46, XX, –14,+T (14q21q) patII: 46, XX, –14,+T (14q21q) de novoII: 46, XX, –14,+T (14q21q) de novoIII: 46, XX, –14,+T (14q21q) matIII: 46, XX, –14,+T (14q21q) matIV: 46, XX, –21,+T (14q21q) patIV: 46, XX, –21,+T (14q21q) patV: 46, XX, –21,+T (21q21q) (parents not typed)V: 46, XX, –21,+T (21q21q) (parents not typed)
A)A) III, IV, I, V, IIIII, IV, I, V, IIB)B) IV, III, V, 1, IIIV, III, V, 1, IIC)C) III, I, IV, V, IIIII, I, IV, V, IID)D) I, V, III, I, V, III, V, III, I, V, IIE)E) III, IV, I, II, VIII, IV, I, II, V
MCQ Check-listMCQ Check-list
Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it
What to test?What to test?
Items classified by cognitive processes required to Items classified by cognitive processes required to answer the question:answer the question:
Recall (Memory)– tests knowledge of isolated factsRecall (Memory)– tests knowledge of isolated facts Interpretation (Comprehension)– review information Interpretation (Comprehension)– review information
to reach conclusion, eg diagnosisto reach conclusion, eg diagnosis Problem solving (Reasoning) – situation requires Problem solving (Reasoning) – situation requires
action, eg next step in managementaction, eg next step in management
– Process depends on background of traineeProcess depends on background of trainee
What to test?What to test?
Simpler classification based on task:Simpler classification based on task: Application of knowledge item Application of knowledge item ****
– Reach conclusionsReach conclusions– Make predictionsMake predictions– Select course of actionSelect course of action
Recall itemRecall item– Test knowledge of isolated factsTest knowledge of isolated facts
What to test?What to test?
Test application of knowledge using clinical Test application of knowledge using clinical vignettesvignettes– Adds ‘face-validity’Adds ‘face-validity’– Usually selects more important, less trivialUsually selects more important, less trivial– Identifies student who’ve memorized but do not Identifies student who’ve memorized but do not
understandunderstand
BUT experience in patient care should not BUT experience in patient care should not be necessarybe necessary
What to test - summaryWhat to test - summary
Focus item on key concepts and principles that are Focus item on key concepts and principles that are essential information (without access to essential information (without access to references) for all examinees to understandreferences) for all examinees to understand
Test material that is relevant to learning in clinical Test material that is relevant to learning in clinical clerkships and beyondclerkships and beyond
Avoid items that Avoid items that onlyonly require recall of isolated require recall of isolated factsfacts
Avoid esoteric or interesting topics that are not Avoid esoteric or interesting topics that are not essentialessential
MCQ Check-listMCQ Check-list
Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it
Writing the QuestionsWriting the Questions
Construct stem –include all necessary Construct stem –include all necessary information to arrive at the right answerinformation to arrive at the right answer
Choose lead-in – pose a clear questionChoose lead-in – pose a clear question Choose distractorsChoose distractors
Stem*Stem*
Usually a clinical scenario Usually a clinical scenario Clear, unambiguousClear, unambiguous Should be long relative to optionsShould be long relative to options Includes all pertinent informationIncludes all pertinent information
– Patient’s age/genderPatient’s age/gender– Clinical settingClinical setting– ComplaintComplaint– Other important info – history, physical Other important info – history, physical
findings, test resultsfindings, test results
Lead-in*Lead-in*
Follows stemFollows stem In the form of a questionIn the form of a question Should relate to the stemShould relate to the stem Should be answerable without looking at Should be answerable without looking at
optionsoptions– ““Cover the Options test”Cover the Options test”
DistractorsDistractors
Each should be selected by some, therefore Each should be selected by some, therefore all plausible, none obviously incorrectall plausible, none obviously incorrect
3 or 3 or 44 choices choices– Common misconceptionsCommon misconceptions– Faulty reasoningFaulty reasoning
DistractorsDistractors
Homogeneous in content to the correct answerHomogeneous in content to the correct answer Should be in same category as correct answer– Should be in same category as correct answer–
diagnosis, treatment, diagnostic testdiagnosis, treatment, diagnostic test Clearly incorrect or inferior to the correct answerClearly incorrect or inferior to the correct answer Plausible and attractive to the uninformedPlausible and attractive to the uninformed Similar to the correct answer in construction and Similar to the correct answer in construction and
length length Grammatically consistent and logically compatible Grammatically consistent and logically compatible
with the stemwith the stem
General Guidelines for Item General Guidelines for Item ConstructionConstruction Include as much of the item as possible in the Include as much of the item as possible in the
stems: Long stems, short optionsstems: Long stems, short options Avoid negatively phrased items (Avoid negatively phrased items (exceptexcept or or notnot in in
the lead-in). If you must, use only with short the lead-in). If you must, use only with short optionsoptions
Avoid writing question of the form:Avoid writing question of the form:– Which of the following statements is correct?Which of the following statements is correct?
– Each of the following statements is correct EXCEPTEach of the following statements is correct EXCEPT» Unfocussed Unfocussed
» Heterogeneous optionsHeterogeneous options
““Top-down” versus “Bottom-up”Top-down” versus “Bottom-up”
Top-DownTop-Down Knowledge or recall question Knowledge or recall question Cite a disease and then ask what patient Cite a disease and then ask what patient
findings are expectedfindings are expected Structured like textbooksStructured like textbooks
Clinically backwardClinically backward
““Top-down” versus “Bottom-up”Top-down” versus “Bottom-up”
Bottom-upBottom-up Application of knowledgeApplication of knowledge Gives findings and asks examinee to Gives findings and asks examinee to
indicate the diseaseindicate the disease Examinees need to be able to synthesize Examinees need to be able to synthesize
information from several pages of a information from several pages of a textbook (or notes) to answer these textbook (or notes) to answer these questionsquestions
Clinically realisticClinically realistic
Guidelines for Clinical SciencesGuidelines for Clinical Sciencesie Clerkship and beyond…ie Clerkship and beyond… Test application of knowledge using clinical Test application of knowledge using clinical
vignettes to pose medical decisions in vignettes to pose medical decisions in patient care situationspatient care situations
Focus items on common or potentially Focus items on common or potentially catastrophic problemscatastrophic problems
Pose clinical decisions that would be Pose clinical decisions that would be expected of a successful examineeexpected of a successful examinee
Avoid clinical situations that would be Avoid clinical situations that would be handled by a (sub)specialisthandled by a (sub)specialist
MCQ Check-listMCQ Check-list
Why we write themWhy we write them What type is bestWhat type is best What flaws are to be avoidedWhat flaws are to be avoided What content to testWhat content to test How to construct itHow to construct it
Basic Rules for MCQ’s (A type)*Basic Rules for MCQ’s (A type)*
Focus item on important conceptFocus item on important concept Item should assess application of Item should assess application of
knowledge, not recall of an isolated factknowledge, not recall of an isolated fact Stem must pose clear question – ‘Cover the Stem must pose clear question – ‘Cover the
options’ testoptions’ test All distractors should be homogeneousAll distractors should be homogeneous Avoid technical item flawsAvoid technical item flaws
ReferenceReference
Constructing Written Test Questions For the Constructing Written Test Questions For the Basic and Clinical SciencesBasic and Clinical Sciences– Case & Swanson, NBME 1996Case & Swanson, NBME 1996