Upload
examsoft
View
94
Download
1
Embed Size (px)
Citation preview
AssessmentConsistency:IsEveryoneontheSamePage?
DepartmentofPharmaceuticalSciencesWegmansSchoolofPharmacy
St.JohnFisherCollege,Rochester,NY
MindyLull,Ph.D.AssociateProfessor
JenniferMathews,Ph.D.AssociateProfessor
Participantswilllearn:
1. Howtousefacultydevelopmentworkshopstoprovidethefoundationofwritingqualityexams.
2. Howtoeffectivelyusethepeer-reviewprocess.3. Howtocreateconsensusandguidelinesfor
creatingandlaunchingexams.
Whyisconsistencyimportant?
• Consistencycontributestoexamquality• Examsshouldreflectcontentlearning• Reducesstudentconfusionorfrustration• Examsarehigh-stakes• Dataareusedformanypurposes
FacultyDevelopmentforConsistency
• Topreparefacultyforwritinghighqualityexams,weconductedaseriesofworkshops:– learningoutcomes– examblueprints– codingtoBloom’sTaxonomy– peer-reviewprocess
• Eachoftheseworkshopshelpedtoformthefoundationofconstructingexamswhichtrulyreflectedstudentlearning.
Developclearlywrittenstatementsofkeylearning
outcomes.
Useresultsofassessmentsto
improveteachingandlearning.
Designexperiencesthatprovide
opportunitiestoachieveoutcomes.
Assessstudentachievementof
outcomes
Cycleas described byMiddlesStatesCommission on HigherEducation
CycleofAssessment
Clearlywrittenoutcomesandexammaps
• WemadethedecisiontouseExamSoft®asourtoolforcourseassessments.
• Beforeanyexamswerecreatedusingthetechnologywehadseveralfacultydevelopmentworkshops.
• DevelopmentsessionsledbyNABP– Revisionofcourselearningoutcomesbyfacultyinthe
course– Techniquesforwritinggoodexamquestions– UseofexamMaps– PropercodingtoBloomsTaxonomy
• Allfacultyarewritingandcodingtheirownexamquestions.
Exammap:ismyexamreflectiveofwhatItaught?
Course learningoutcome:Basedonthesiteandmechanismofaction,predictthetherapeuticandsideeffectsassociatedwithpharmacologicalagentsusedtotreatvariousdisease states. CourseActivity LectureLO Knowledge Application Synthesis
Lecture2(9.11.12– 9.13.12)APPElectures
Lecture,TP,patientscenario
EPOREMS· pg.;Bank question 1420(MC)
AnemiaOutcome4 1
Iron+VitaminC· pg.9;Bank question 1525(FITB)
AnemiaOutcome4 1
AnemiafromRBCdestruction• pg.; Bankquestion 1527(SA)
AnemiaOutcome1,2,3
1
Continuedupdatesandquicktips
• Usetimeduringfacultyretreatsandmeetingstopracticecodingsamplequestions,toaddresscodinginconsistencies,holesinourcurricularcoverage,questions/concerns
• Emailreminderswith“howto”– Ex.HowtowriteagoodFITBquestion
• Bindersforallfacultywithsamplematerials– AppendixBtopics,WSOPprogramoutcomes,Bloom’sTaxonomyverbs
Thepeerreviewprocess
• “Peer-reviewedarticlesprovideatrusted formofscientificcommunication.”
• “Evenifyouareunfamiliarwiththetopicorthescientistswhoauthoredaparticularstudy,youcantrustpeer-reviewedworktomeetcertainstandardsofscientificquality.”
Diagramandquotesfrom:http://undsci.berkeley.edu/article/howscienceworks_16
Peerreview
• Asacademicsweplacevalueinthepeerreviewprocess.
• Weshouldalsoplacevalueinwritinghighquality,peer-reviewedassessments.
OurProcess
§ Thisisavoluntaryprocessforfacultywhowanttoparticipate
§ 2-3reviewers/exam§ Facultyareinvestedintheprocess§ ReviewersDONOTmakechangestoexams,anditemwriterscanchoosenottomakesuggestedchanges
§ Studentdataismuchmorereflectiveofwhatwaslearnedversuscouldmytestitembedeciphered
Qualityofquestions:Peerreviewedtestitem
YES NO Comments/Suggestions
1 Isthequestion cleartothereader?2. Arethere typographicalerrors?3. Isthequestion codedtothe
appropriatelevelofBloom’sTaxonomy?
4. Isthequestion codedappropriatelytothe learningoutcome(s)?
5. If thequestionhasahistory,diditperformwellinthepast?
6. Is thecontent levelappropriateforthecourse?
7. Otherconsiderations:“positive”language,jargonfree,limitedabbreviations,double-jeopardy,examtime
Howtolearnfromothers
• Problemareas:1. FITBquestions2. Shortanswer/essay(characterlimits)3. EliminatingK-type4. Selectallthatapply
CreatingFormattingGuidelinesforConsistency
• Ensuresaconsistentlooktoallexamsandexamreports
• Considerhavinga“QuickTips”documentthatoutlinestheseforallfaculty
• Guidelines– Questionformat– Examcreationandpostings– Examadministration– Examreports
ConsistencyinExamCreation
• Preventlogisticalerrorsinexamwithachecklist:– Correct#ofpoints– Checkquestiongroupings– Overridefontandfontsize– Checksettingsonpartialcredit,select-all-that-applyandk-typequestion
– Exportcopyofexamforlast-minutespellingerrors– Previewexamsbeforeposting
Conclusions
• Goodexamconsistencyinvolvesquestionwriting,examcreation,examposting,examadministration,andexamreports.
• Trainingisrequiredtogetallfacultyonthesamepage,andtoemphasizetheimportanceofconsistency.
• A“Quick-Tips”guideisusefulforlogisticalrecommendationsforconsistency.