Upload
hatuyen
View
223
Download
4
Embed Size (px)
Citation preview
Scenario-based multiple-choice
that assess knowledge at higher
Bloom’s levels
Janet Lindsley, PhD Associate Professor of Biochemistry,
Assistant Dean for Curriculum
Member of the National Board of Medical Examiners USLME Step 1 Integrated Review Committee
Past Step 1 Item Writing Committee
Workshop objectives
By the end of this workshop, participants will be able to:
1. identify the Bloom's taxonomy levels at which well written multiple choice questions can assess knowledge,
2. distinguish between well written and poorly written multiple choice questions,
3. create well written multiple choice questions which assess knowledge higher Bloom's levels.
Bloom’s taxonomy for the
cognitive domain
Handbook I: Cognitive (Bloom et al. 1956)
Image accessed on 9/21/15 http://www.maggiehosmcgrane.com/2014/09/flipping-grade-4-and-flipping-blooms.html
Analyze the following NBME
questions to identify key
elements/similarities
A 64-year-old man comes to the physician because of swelling in his feet for the past 2 years. He says that his skin is dry and itchy and his feet "feel heavy." One of his legs is shown. Which of the following is the most likely cause of his condition?
(A) Arteriolar constriction and arteriolar hypertension (B) Arteriolar dilation and venous hypertension (C) Venous constriction and arteriolar constriction (D) Venous hypertension and incompetent valves (E) Venous hypertension and venous constriction
2015samples_step1.pdf
A 12-year-old girl with a 1-year history of systemic lupus erythematosus is brought to the physician for a routine follow-up examination. During the past year she has done well with the exception of occasional mild frontal headaches, fatigue, and arthralgias; the results of regular laboratory evaluations have been stable. She has missed 20 days of school in the past 4 months. Her parents express concern that sending her to school might cause her to be exposed to children with contagious diseases that might exacerbate their daughter's condition. The girl tells the physician that she misses seeing her many friends at school. Passive motion of the elbows, wrists, and knees produces mild discomfort. Physical examination shows no other abnormalities. Which of the following is the most likely cause of this patient's excessive school absences?
(A) Lupus arthritis
(B) Lupus cerebritis
(C) Malingering
(D) School phobia
(E) Vulnerable child syndrome
2015samples_step1.pdf
A placebo-controlled clinical trial is conducted to assess whether a new antihypertensive drug is more effective than standard therapy. A total of 5000 patients with essential hypertension are enrolled and randomly assigned to one of two groups: 2500 patients receive the new drug and 2500 patients receive placebo. If the alpha is set at 0.01 instead of 0.05, which of the following is the most likely result?
(A) Significant findings can be reported with greater confidence (B) The study will have more power (C) There is a decreased likelihood of a Type II error (D) There is an increased likelihood of statistically significant findings (E) There is an increased likelihood of a Type I error
2015samples_step1.pdf
A 32-year-old woman with type 1 diabetes mellitus has had progressive
renal failure over the past 2 years. She has not yet started dialysis.
Examination shows no abnormalities. Her hemoglobin concentration is
9 g/dL, hematocrit is 28%, and mean corpuscular volume is 94 μm3. A
blood smear shows normochromic, normocytic cells. Which of the
following is the most likely cause of her anemia?
(A) Acute blood loss
(B) Chronic lymphocytic leukemia
(C) Erythrocyte enzyme deficiency
(D) Erythropoietin deficiency
(E) Immunohemolysis
(F) Microangiopathic hemolysis
(G) Polycythemia vera
(H) Sickle cell disease
(I) Sideroblastic anemia
(J) β-Thalassemia trait
2015samples_step1.pdf
Key elements/similarities?
Appropriately constructed MCQs can
be effective at assessing higher level
cognitive skills
• MCQs with clinical or research vignettes can assess ability to:
– Analyze information (data interpretation)
– Synthesize information
– Apply knowledge to solving novel problems
Of course, the MCQs should be aligned with curricular learning objectives.
UUSOM current practice of tagging
questions by “Bloom’s level”
• Recall of factual knowledge
• Applied knowledge
• Data Interpretation
– Including interpretation of images
• Scientific (Research) scenario
– Complex interpretation of research data
• Clinical Scenario
– Complex interpretation of clinical data
A 7-month-old infant is brought to the physician's office because of poor
weight gain despite large food intake. He has had two episodes of
pneumonia and has frequent bulky stools. He coughs frequently. X-rays
of the lungs show increased markings and hyperinflation. Trypsin is
absent in a fresh stool sample, and the fat content is increased. Which of
the following is the most likely cause of this infant's disorder?
(A) Autoimmune disorder
(B) Defective ion transport at epithelial surfaces
(C) Disaccharidase deficiency
(D) Inability to synthesize apolipoprotein B
(E) Villous atrophy of the jejunum
2015samples_step1.pdf
The stem poses a clear question; it is usually possible to arrive at the answer with the options covered.
Answer choices are homogeneous.
Anatomy of a USMLE-style MCQ
A 7-month-old infant is brought to the physician's office because of poor
weight gain despite large food intake. He has had two episodes of
pneumonia and has frequent bulky stools. He coughs frequently. X-rays
of the lungs show increased markings and hyperinflation. Trypsin is
absent in a fresh stool sample, and the fat content is increased. Which of
the following is the most likely cause of this infant's disorder?
(A) Autoimmune disorder
(B) Defective ion transport at epithelial surfaces
(C) Disaccharidase deficiency
(D) Inability to synthesize apolipoprotein B
(E) Villous atrophy of the jejunum
2015samples_step1.pdf
The stem poses a clear question; it is usually possible to arrive at the answer with the options covered.
Answer choices are homogeneous.
Anatomy of a USMLE-style MCQ
A 4-year-old boy is brought to the physician because of slow growth during the
past year. He has had recurrent urinary tract infections since the age of 1 year.
He is at the 10th percentile for height and 25th percentile for weight. Physical
examination shows pallor. Laboratory studies show a normochromic, normocytic
anemia and increased serum concentrations of urea nitrogen and creatinine.
Urinalysis shows a low specific gravity. Which of the following sets of additional
serum findings is most likely in this patient?
2015samples_step1.pdf
Calcium Inorganic
Phosphorous
1,25-Dihydroxycholecalciferol Erythropoietin
A
B
C
D
E
F
Tables of answer choices can be
helpful
A 4-year-old boy is brought to the physician because of slow growth during the
past year. He has had recurrent urinary tract infections since the age of 1 year.
He is at the 10th percentile for height and 25th percentile for weight. Physical
examination shows pallor. Laboratory studies show a normochromic, normocytic
anemia and increased serum concentrations of urea nitrogen and creatinine.
Urinalysis shows a low specific gravity. Which of the following sets of additional
serum findings is most likely in this patient?
2015samples_step1.pdf
Calcium Inorganic
Phosphorous
1,25-Dihydroxycholecalciferol Erythropoietin
A
B
C
D
E
F
Balancing answer choices prevents
cuing to the correct answer
Avoid giving away the answer by
cuing
Good questions are not
purposefully tricky or deceptive
Summary: A baker’s dozen of “rules” for writing
effective, NBME-style MCQs 1. MCQs with vignettes can test higher levels of knowledge than simple MCQs.
2. Avoid pseudo-vignettes, in which the vignette is not needed.
3. Question stems should be long and answer choices short and homogeneous (e.g. all mechanisms described in similar depth).
4. Don’t be negative. (No “which of the following is NOT...”)
5. Tables of answers can be useful.
6. Use as many distracters as necessary to avoid giving away the answer.
7. Questions should never be purposefully tricky or deceptive.
8. Order answers alphabetically or numerically.
9. Avoid giving away the answer. (No cuing by having the right answer longest or more detailed, spelling or grammatical errors in distractors, having 2 choices similar and one being correct.)
10.Students should be able to answer the question before reading the options (No “which of the following is true?”)
11.The lead-in question should include “most likely” (or similar).
12.No questions that would result in malpractice.
13. Avoid “buzz” words.
Gather ideas throughout the year
Application of these concepts
How could this question be improved?
A 68-year-old man with no known cardiovascular disease has normal adult hemoglobin that displays cooperative interactions between its polypeptide subunits. Which of the following functional characteristics of hemoglobin does this cooperativity most strongly affect? a) Fraction of hemoglobin present as methemoglobin in the peripheral circulation b) Fraction of hemoglobin present in tetrameric form in erythrocytes c) Fraction of hemoglobin subunits bound to a heme group d) Fraction of hemoglobin's bound oxygen transferred from erythrocytes to tissues
Better? A female newborn was referred for hemoglobin analysis after a newborn screen detected a HbF or HbA variant. DNA sequence analysis revealed a heterozygous alpha-globin Asp126Tyr mutation. Functional analysis of this mutant protein gave the following data. What is the most likely consequence of this mutation?
A. Decreased fraction of globin chains with heme attached
B. Decreased methemoglobin fraction
C. Decreased oxygen transfer from erythrocytes to tissues
D. Increased fraction of globin chains with heme attached
E. Increased methemoglobin fraction
F. Increased oxygen transfer from erythrocytes to tissues
Patient
Reference
Better? A 68-year-old man with chronic obstructive pulmonary disease (COPD) is brought to the ED because of confusion and lethargy. His pulse is 96/min, respirations 22/min, blood pressure 148/82 mm Hg. Physical exam shows erythema of the skin and mucosal membranes, as well as cyanosis of the nail beds. There is no splenomegaly. Lab studies show hemoglobin of 17 g/dL and hematocrit of 49%. Arterial blood gas on room air reveals the following:
pH = 7.33
pCO2 = 58 mm Hg
pO2 = 61 mm Hg
HCO3- = 36 mEq/L
This patient most likely has a decrease in which of the following relative to a healthy patient?
A. affinity of hemoglobin for oxygen
B. ferritin concentration
C. heme synthesis
D. percent HbA1c
E. red blood cell volume
A 26-year-old woman presents with pustular skin lesion on the right arm for 2 days and is found to have acute arthritis of the left knee and right wrist. Gram stain of the pustule is shown in the attached image. Which of the following is the most likely initial portal of entry for this pathogen in this patient? a) Cutaneous b) Gastrointestinal c) Genital d) Pulmonary e) Urinary
Good question?
A 25-year-old woman with cystic fibrosis is receiving care for her condition in a patient-centered medical home. Which of the following practices best exemplifies the patient-centered aspect of care in a medical home model? a) Delivering outpatient and emergency room care b) Facilitating access to the emergency room when symptoms are exacerbated c) Following national guidelines for care d) Reducing cost of prescription medications *e) Teaching self-management of symptoms based on condition severity
How could this question be
improved?
A 25-year-old woman with cystic fibrosis is receiving care for her condition in a patient-centered medical home. Her care is most likely to differ from traditional services by including: a) Coverage for both outpatient and emergency room care. *b) Instruction on how to manage her GI and respiratory symptoms. c) Only protocols that follow US Preventative Services guidelines for care of patients with cystic fibrosis. d) Rapid access to emergency room care when she experiences exacerbated shortness of breath. e) Reduced cost of Pancreatic enzymes, Fluticasone, Albuterol, nebulized DNase and Montelukast.
Better?
A 5-year-old boy was brought to the emergency room, unconscious, and in shock after he accidentally overdosed on his mother's iron pills, mistaking them for sweet-sour candy. He is hyperglycemic with leukocytosis and the empty container recovered from the scene suggests that he had ingested about 33 ferrous sulfate tablets. How would you treat this patient? a) Chelation therapy with dimercaprol b) Detoxification with activated charcoal *c) Gastric lavage with deferoxamine d) IV infusion with British Anti-Lewisite e) Oral succimer, then pennicillamine
Good question?
How could this question be improved?
A 20-year-old man presents to the ER after an automobile accident. Radiography shows extensive fractures of the skull, including a fracture and partial closure of the foramen ovale. This fracture would damage the nerve that supplies all of the following muscles, EXCEPT: *a) Buccinator b) Lateral pterygoid c) Masseter d) Medial pterygoid e) Temporalis
Better? A 20-year-old man presents to the ER after an automobile accident. Vital signs are T 97.3 F, RR 22/min, HR 105/min, BP 110/75 mm Hg. Radiography shows extensive damage to his skull, including a fracture and partial closure of the foramen ovale. This fracture most likely results in nerve damage to which of the following muscles?
Buccinator Lateral pterygoid
Masseter Medial pterygoid
Temporalis
A Yes Yes Yes Yes No
B Yes Yes Yes No Yes
C Yes Yes No Yes Yes
D Yes No Yes Yes Yes
E No Yes Yes Yes Yes
What is one thing that you are
going to start, stop or continue
doing as a result of this workshop?
References
• NBME item writing guide (see: http://www.nbme.org/publications/item-writing-manual-download.html)
Questions?