Upload
vokhuong
View
214
Download
0
Embed Size (px)
Citation preview
GuidetoAssessmentatGreenvilleUniversity Rev.January2017EricWatterson,DeanofAssessmentandInstitutionalResearch
NationalInstituteforLearningOutcomesAssessment.(2011).TransparencyFramework.Urbana,IL:UniversityofIllinoisandIndianaUniversity,NationalInstituteforLearningOutcomesAssessment(NILOA).Retrievedfrom:http://www.learningoutcomesassessment.org/TransparencyFramework.htm
AssessmentasaProcessofImprovementTheprocessofassessmentatGreenvilleUniversityisfacilitative—itpromotesacontinuousprocessimprovementthroughreview,revision,andtheimplementationofbestpracticeswithinacademicdepartmentsandotherfacetsofcampusoperation.Assessmentisthesystematicprocessofcollectingandanalyzingdataforthepurposeofimprovinglearning.Asa“communityoflearners”thisprocesstouchesthelivesofpersonsinallsegmentsofinstitutionaloperations.Assessmentisnottheevaluationofindividualprograms,individualfaculty,staff,orstudents.ItisaprocessthatprovidesinformationastohowwellprogramswithintheUniversitycarryouttheirpurposes.Faculty,administrators,staff,andstudentsparticipateinthisprocessofongoingimprovementinhowtheUniversityservesthepublicatlarge.Theassessmentprocessisviewedasformofscholarship,awayofinquiringastowhethertheprogramsoftheUniversityareactuallyproducingthedesiredoutcomesandgoals.Thecontentofeachassessmentplanisuniquetoeachprogramanddepartmentbuttheprocesswithineachareainvolvessimilarstepsinacycleofmeasurement,analysisandreporting,developingimprovements,implementingimprovements,andevaluatingtoseeifthedesiredchangehastakenplace.GreenvilleUniversityadoptedtheNILOATransparencyFrameworkin2014toassistoureffortsofcollectingandcommunicatingevidenceofourprocessofinstitutionalimprovementandtransformation.Ourwebsitedisplaysthismodelwithlinkstothecontentineachcomponent.http://www.greenville.edu/about/institutional_assessment/index.html
GuidetoAssessmentatGreenvilleUniversity
InstructionsforDevelopingaProgramAssessmentPlan
Theassessmentplanguidestheselectionandcreationofstudent-focusedassessmentactivitiesacrossthecurriculuminordertoimproveoureducationalprograms.Thisplan,incoordinationwithinstitutionalassessmentpolicies,shouldprovideguidancetodepartmentandprogramfacultyforselectingappropriatecourse-basedassessmentactivitiesthatalignwithlearningoutcomes.Overall,theassessmentplanshouldbecenteredon: A.outcomestobeachieved B.performancelevelstobeattainedrelatedtothoseoutcomes C.methodsforanalyzingtheperformanceresults D.describinghowtheresultswillbeusedforimprovement
Step1.Identifyprogramlearninggoals
“Learningiseasierwhenlearnersunderstandwhatgoaltheyaretryingtoachieve,thepurposeofachievingthegoal,andthespecificattributesofsuccess.”Chappuis,S.,&Stiggins,R.(2002).
Goalsaregeneralstatementsaboutwhatyouhopeyourstudentswillgainfromyourprogramormajor.Theyserveasthepurposetowardwhichspecificclassroomandotheractivitiesaredirectedanddonotnecessarilyneedtobemeasurable,perse.Programlearninggoalsaregenerallyaccomplishedduringalonger-termtimeframe.
• Whatwouldasuccessfulgraduateoftheprogramlookliketodayandinthefuture?• Whatarethemajoracademicgoalsstudentsshouldachieveuponcompletionoftheprogram?• Whatwouldasuccessfulstudentknowandbeabletodobytheendoftheprogram?
Eachprogramshoulddevelopapproximately3to5learninggoals.Departmentswithmultipleprogramsormajorswillneedtodeveloplearninggoalsthatcharacterizetheuniquenessofeachprogram.
Step2:Identifyprogramlearningobjectives
Objectivesarespecificstatementswithanactionverbandcontentreference.Thesespecificclassroom-basedandotherlearningactivitiesareintendedtohelpyoureachyourprogramlearninggoals.Theymustbemeasurableandhaveashort-tomedium-termtimeframeduringwhichtheyshouldbedemonstrated.Determinetheactivities,thendeterminewheretolocatetheminthecurriculum(seestep3).
ThethreemaintypesofobjectivesinuseatGreenvilleareknowledge,skill,andvalue.
GuidetoAssessmentatGreenvilleUniversity
• Whenwritingknowledgeobjectives,youaretryingtodefinethemainconcepts(e.g.,theoreticalprinciples;disciplineknowledge;etc.)thatstudentsmustknowwhentheygraduate.
• Whenwritingskillobjectives,youaretryingtodescribethelargerskills(e.g.,problemsolving;analysis;etc.)thatstudentshavegainedbythetimetheygraduate.
• Finally,valueobjectivesusuallydescribebeliefsaboutthenatureofafieldofstudyorsubjectmatter,perceptionsaboutinterdisciplinaryconnections,ethics,orexpressionofone’sChristianfaith.
LevelsofObjectivesObjectivescanalsoreflectdifferentlevelsoflearning
• MasteryObjectivesreflectminimumcompetencieswithinthediscipline–thoseskillsthatmustbemasteredbythestudentbeforemovingontothenextlevelofinstruction.Masteryobjectivestendtobeveryspecificandlimitedinscopeand,therefore,canoftenbearticulatedwithgreatspecificity(Palomba,et.al.,1999).
o Forexample,AllAccountingmajorsshouldbeableto:Balanceafinancialstatement,PrepareanExcelspreadsheet,Trackaccountsreceivable.
• DevelopmentalObjectivesreflectmorecomplex(orhigherorder)learning
outcomes–thoselearningtasksonwhichstudentscanbeexpectedtodemonstratevaryingdegreesofprogress.Notethatthesedevelopmentalobjectivesareoftenwritteninatwo-stageprocessinwhichthegeneralobjectiveisstatedalongwithasampleofspecificlearningoutcomes.
o Forexample,AccountingstudentsareexpectedtounderstandGenerallyAcceptedAccountingPractices(GAAP):ExplainGAAPinlayman’sterms,Discussdifferencesbetweenacceptedandnon-acceptedpractices,GiveexamplesofwhentouseandreferenceGAAP.
Inthiscase,theobjectiveforthestudentistounderstandGAAP.Whilesomestudentsmaydemonstrateeachofthelearningoutcomesassociatedwiththisobjective,somemayonlydemonstratetwo,andsomeonlyone.
AlistofsampleactionverbsbasedonBloom’staxonomyoflearningmaybefoundintheAppendixA.
Step3:AlignCourseswithProgramGoalsandObjectives
Onceaprogramhasclearlystatedlearningoutcomes,each courseinthecurriculumshouldbelinkedtoatleastoneprogramobjective.Somecourseswillbeassociatedwithmorethanoneobjective.
• UsetheLearningOutcomeAlignmentWorksheettodisplaytheselinkages.Examinetheworksheetforgaps.Objectivesthatarenotlinkedmayneedtobere-examinedforrelevance.Objectivesthatareover-usedmayneedtobere-examinedanddividedintosmaller,morespecificobjectives.SeeAppendixB.
GuidetoAssessmentatGreenvilleUniversity
• CreateaCurriculumMaptodisplayhowprogramobjectivesarestagedacrossthecurriculum.Programobjectivesneedatleast3touchpointsinthecurriculum,e.g.introduced,reinforced,demonstrated.SeeAppendixC.
Step4:Documentevidence
Programfacultyandstaffshouldmakedecisionstogetheraboutwhatcourseswillprovideevidencetodeterminehowwellstudentsaremeetinglearninggoalsandprogramobjectives.Iftheprogramincludesacapstonecourse,useitasoneofthecourses/learningexperiencestoselectasampleofstudentworkforassessment.Goodassessmentincludesamixofdirectandindirectevidence.
Direct Evidence - ClearandCompellingEvidenceofWhatStudentsAreLearning• Ratingsofstudentskillsbyfieldexperiencesupervisors• Scoresandpassratesonappropriatelicensure/certificationexamsorotherpublishedtests(e.g.,MajorFieldTests)thatassesskeylearningoutcomes• “Capstone”experiencessuchasresearchprojects,presentations,theses,dissertations,oraldefenses,exhibitions,orperformances,scoredusingarubric• Portfoliosofstudentwork• Scoregainsbetweenpre-andposttests(publishedorlocal)orwritingsamples• Studentreflectionsontheirvalues,attitudesandbeliefs,ifdevelopingthoseareintendedoutcomesoftheprogram
Indirect Evidence - EvidencethatStudentsAre“Probably”Learning,ButExactly WhatorHowMuchisLessClear
• Coursegrades*(seenoteonGrades,below)• Assignmentgrades,ifnotaccompaniedbyarubricorscoringguide• Admissionratesintograduateprogramsandgraduationrates• Placementratesofgraduatesintoappropriatecareerpositionsandstartingsalaries• Studentratingsoftheirknowledge,skillsandreflectionsonwhattheyhavelearnedintheprogram• Student/alumnisatisfactionwithlearning,collectedthroughsurveys,exitinterviews,orfocusgroups• Studentparticipationratesinfacultyresearch,publicationsandconferencepresentations• Honors,awards,andscholarshipsearnedbystudentsandalumni
*Grades&ProgramAssessmenta.WhyarecoursegradesinsufficientforProgramAssessment?•Coursegradesreflectwhatstudentshaveachievedinasinglecourse•Gradesareacompositeofastudent’sachievementofcourseoutcomes•Gradesreflecttheevaluationpractices,
policies,andcriteriaofindividualinstructors•Facultyteachingthesamecoursemayteachdifferentmaterial•Facultyteachingthesamecoursemayemphasizedifferentcourseobjectives
GuidetoAssessmentatGreenvilleUniversity
b.WhatgradesmightbeuseableforProgramAssessment?•Gradesonasingleitemorseriesofitemsfromanexamorquizdirectlylinkedtoaprogramobjectiveandthatallstudentsanswer•Gradesonanassignmentdirectlylinkedtoaprogramgoalandcompletedbyallstudents
•Gradesbasedonthesamegradingstandardsandcriteriaacrossallfacultyandcourseofferings(withconfirmedinter-raterreliability)•Gradessupportedbyorverifiedusingotherevidence•Gradesthatreflecttheconsensusofmultiplefaculty
*Suskie,L.(2009).Assessing student learning: A common sense guide (2nded.).SanFrancisco:Jossey-Bass
Step5:Conductassessmentsandusetheresultsforimprovement
Specificmetricstobeevaluated,aswellasbenchmarklevelsofperformancetobeachieved,shouldbeexpresslyidentifiedwithintheassessmentplan.Theperformanceresultsshouldbereviewed,analyzed,andsharedwithstakeholders.Keypointsforimprovementbasedontheseresultsshouldalsobeidentifiedandimprovementactionsshouldberecommended.Thoseactionscanthenbereviewed,approved,andimplemented.Thiscontinuousimprovementcyclecanthenberepeatedovertimecreatingahistoryofcontinuousimprovement.
Theevidencegathered(data)willbeusedtoimprovecourseofferingsintermsofcontent,deliverystrategies,learningexperiencesprovided,assignments,andlearningoutcomes.Theevidencecollectedfromindividualcoursesoragroupofcoursesinamajorwillbesharedwithprogramanddepartmentfacultytomakecourseimprovementsastheyofferthecoursesinsubsequentsemesters.Inaddition,datacanalsobeusedmid-semestertomakechangesand/orimprovementstocourseofferings.Asindicatedearlier,thegoalofthiseffortistoenhancethecurriculumtoimprovestudentlearning.
Step6:ClosetheLoop
Themissinglinkinmostassessmentplanningisadescriptionofhowtheresultingevidencewillbeusedforimprovement,whowillusetheevidence,andwhentheimprovementswillbeimplemented.Attheendofeachsemester,yourdepartmentsandprogramsshoulddiscussyourassessmentresultsandtheirimplicationsforpossibleimprovements.Describetheinstructionalimprovementsyouintenttomake,whenandhowtheywillbeimplemented,andthemethodforknowingiftheinitiativeissuccessful.SeetheAnnualAssessmentReport(AppendixD)forinstructionsondocumentingthisfinalstepofyourplan.
GuidetoAssessmentatGreenvilleUniversity
AppendixA:SampleActionVerbsSampleactionverbsforeachtypeoflearningobjective.AdaptedfromBloomsTaxonomy.Knowledge Comprehension Application Analysis Synthesis Evaluation(Remember) (Understand) (Apply) (Analyze) (Evaluate) (Create)CountDefineDescribeDrawLabelListMatchNameOutlinePointQuoteReadRecallReciteRecognizeRecordRepeatReproduceSelectStateWriteMemorizeArrangeDuplicateOrderRelateTabulate
AssociateClassifyComputeContrastConvertDefendDescribeDifferentiateDiscussDistinguishEstimateExplainExtendExtrapolateGeneralizeGiveexamplesInferIdentifyIndicateInterpretLocateParaphrasePredictReportRestateReviewRewriteTranslate
AddApplyCalculateChangeChooseClassifyCompleteComputeDemonstrateDiscoverDivideEmployExamineExperimentGraphInterpolateManipulateModifyOperatePerformPracticePrepareProduceRelateResearchOrganizeScheduleServiceShowSketchSolveSubtractTranslateTroubleshootUseUtilizeWrite
AnalyzeApplicationAppraiseBreakdownCalculateCategorizeCombineCompareConnectContrastCriticizeDesignDetectDiagramDifferentiateDiscriminateDistinguishExamineExperimentExplainInferOutlinePointoutQuestionRelateSelectSeparateSubdivideTestUtilize
ArrangeAssembleCategorizeCollectCombineCompileComposeConstructCreateDesignDevelopDeviseDriveExplainFormulateGeneralizeGenerateGroupIntegrateInventFormulateManageModifyOrderOrganizePlanPreparePrescribeProposeRearrangeReconstructRelateReorganizeReviseRewriteSetupSpecifySubstituteSummarizeTransform
AppraiseArbitrateArgueAssessAttachAwardChooseCompareConcludeContrastConvinceCoreCriticizeCritiqueDecideDefendDetermineDiscriminateEvaluateExplainGradeInterpretJudgeJustifyMeasurePredictPrioritizeRankRateRecommendRefereeRejectSelectSummarizeSupportTestValue
AppendixB.LearningOutcomeAlignmentWorksheetGreenvilleUniversity,Greenville,IL62246 Date:________________________
7
Department__________________________________________________Program______________________________________________ChairofDepartment________________________________________ProgramCoordinator___________________________________________DepartmentMissionStatement ProgramMissionStatement(includeamissionstatementforeachprograminthedepartment)DepartmentObjectives:QualitiesandcompetenciesexpectedingraduatesofthisdepartmentAtthecloseoftheirdegreestudentswillbeableto:1.2.3.4.5.expandlistasneeded
Program/MajorObjectives:Qualitiesandcompetenciesexpectedingraduatesfromthisprogram/majorAtthecloseoftheirdegreestudentswillbeableto:1.2.3.4.5.expandlistasneeded
AppendixB.LearningOutcomeAlignmentWorksheetGreenvilleUniversity,Greenville,IL62246 Date:________________________
8
StudentLearningOutcomes DepartmentObjective#
ProgramObjective#
CourseNumber
CourseObjective#
AssignmentinCourse AssessmentMethod
1.SeekTruthThroughCriticalInquiryandResearch1.1Practicecriticalself-awareness
1.2Understandourworldandcomprehendquantitativeandconceptualrelationships
1.3Thinkintegrativelytosolveproblems
1.4Applyskillsandsystematicreasoning
2.CollaborateandCommunicate2.1CommunicateandCooperate
2.2Valueothers
3.EngageCultureandBeCreative3.1Demonstrateculturalawareness
3.2Demonstratecreativityandappreciationforarts,beauty,andideas
4.DemonstrateFaithandLearninginAction4.1Recognizeworldviews
4.2ApplyChristianvalues
4.3RespondtoGod
4.4Maintainhealthyself-regardandagrowth-focusedlifestyle
Note:makesuresyllabiareloadedintoD2Lforallcourses.Usethecurrentsyllabustemplate.
Department&ProgramAssessmentPlans
AppendixC:CurriculumMapProgram:______________________________________________________________ Year:_________________
ProgramLearningOutcomesCourses/LearningOpportunities
(programoutcome1) (programoutcome2) (programoutcome3) (programoutcome4) (programoutcome5) (programoutcome6) Key:I=Introduced,R=Reinforced,D=Demonstrated*Thekeycanbemodifiedaccordingtoparticulardisciplinaryfeaturesandcurricularofferings.Forexample,programscanindicatewhethertheoutcomeisintendedtobemetatabasic(B),intermediate(I),oradvanced(A)level.Stepstofollowindevelopingacurriculummap:1.Listprogramlearningoutcomes.2.Listrequired/coreprogramcoursesandotherlearningopportunities(e.g.seniorresearch,seniorrecitals,etc.)notnecessarilyconnectedtoacourse3.Reviewsyllabitodeterminethealignmentbetweencoursecontent(assignments),courseobjectives,andprogramlearningoutcomes.4.Makeajudgmentregardingthelevelatwhichoutcomesareaddressedineachcourseandlearningopportunity(e.g.,Introduced,Reinforced,Demonstrated;Sample:
ProgramLearningOutcomesCourses/LearningOpportunities
101 202 210 212 304 310 315 412 4851 I R R R D D 2 I R R D 3 I R R D D4 I I R D D D5 I R R R R D
Key:I=IntroducedR=ReinforcedD=Demonstrated
Department&ProgramAssessmentPlans
AppendixD:AnnualReportFormInstructionsforDepartmentChairsandProgramCoordinators: Attheendofeachacademicyear,completethisworksheetandsendittothe appropriateyeardropboxinD2L.Keepacopyforyourownrecords.Developingtheyearendsummaryreport:1.BeginwithreviewingwhatyouhavediscussedinDepartmentaland/orProgrammeetingsorconversationswithyourdepartment/programfaculty. a.WhatdiscoveriesdidyoumakefromyourIDEAfeedback,orfrom comments/complaintsmadebystudentsthatcausedyoutomakechangesinhow youdeliveredorsequencedcontentinacourse? b.Werethereinnovationsinyourfieldthatinspiredyoutore-organizecontent withinacourseinordertoincludethenewinformation? c.Didyoudevelopanewwayoftestingstudentunderstandingofanareaof content?Howdidtheyrespond? d.Anyofthesetypesofresponsestostudent’sneedsandtheiractualperformance couldbeabeginningpointforyourreport.2.Outlinewhatyouchangedincurriculumdesign,testingstrategies,generatingdiscussion,developingpeerreviewtechniques,orwhateveradaptationyoumadetoimprovestudentlearning.3.Reportonwhatyoulearnedfromtheexperience.Ifyouhavenewstudentoutcomedata,summarizeit.Ifyouareintheprocessofgatheringthesedata,indicatethetimeframeinwhichitwillbeaccomplished.4.Describewhereyouwillgofromhere...whataretheimplicationsofwhatyouhavelearnedintermsoffuturechangesincurriculumand/orcoursesequencing?
ProgramAssessmentAnnualReportForm:ProgramImprovements
Program__________________________________________Semester_______________Coordinator__________________________________Date______________________FacultyParticipating:AreaforImprovementidentifiedthroughStudentOutcomeData:Facultyobservationsonthisarea:ProposedchangesinresponsetoStudentOutcomeData:TimelineforResults:CriteriaforEvaluation:
ProgramAssessmentAnnualReportForm:CourseImprovements
Program__________________________________________Semester_______________Coordinator__________________________________Date______________________FacultyParticipating:AreaforImprovementidentifiedthroughStudentOutcomeData:CourseNumber
CourseComponent
Whatdidyouchangewithinthecoursetorespondtotheproblem?
Whatresultswereobtained?Howdidstudentcompetenciesimproveasaresultofthechange?Willthiscomponentbecomeapermanentpartofthecurriculumofferingswithinthiscourse?WhatimplicationsdoesthisimprovementhavefortheProgramasawhole?