Resource-Based Learning and Course Design: A Brief

Georgia State University College of LawReading Room

Faculty Publications By Year Faculty Publications

1-1-2012

Resource-Based Learning and Course Design: ABrief Theoretical Overview and PracticalSuggestionsMeg ButlerGeorgia State University College of Law, [email protected]

Follow this and additional works at: https://readingroom.law.gsu.edu/faculty_pub

Part of the Legal Education Commons, and the Legal Ethics and Professional ResponsibilityCommons

This Article is brought to you for free and open access by the Faculty Publications at Reading Room. It has been accepted for inclusion in FacultyPublications By Year by an authorized administrator of Reading Room. For more information, please contact [email protected].

Recommended CitationMeg Butler, Resource-Based Learning and Course Design: A Brief Theoretical Overview and Practical Suggestions, 104 L. Libr. J. 219(2012).

https://readingroom.law.gsu.edu?utm_source=readingroom.law.gsu.edu%2Ffaculty_pub%2F1209&utm_medium=PDF&utm_campaign=PDFCoverPages

https://readingroom.law.gsu.edu/faculty_pub?utm_source=readingroom.law.gsu.edu%2Ffaculty_pub%2F1209&utm_medium=PDF&utm_campaign=PDFCoverPages

https://readingroom.law.gsu.edu/faculty?utm_source=readingroom.law.gsu.edu%2Ffaculty_pub%2F1209&utm_medium=PDF&utm_campaign=PDFCoverPages

https://readingroom.law.gsu.edu/faculty_pub?utm_source=readingroom.law.gsu.edu%2Ffaculty_pub%2F1209&utm_medium=PDF&utm_campaign=PDFCoverPages

http://network.bepress.com/hgg/discipline/857?utm_source=readingroom.law.gsu.edu%2Ffaculty_pub%2F1209&utm_medium=PDF&utm_campaign=PDFCoverPages



mailto:[email protected]

219

LAW LIBRARY JOURNAL Vol. 104:2 [2012-19]

Resource-Based Learning and Course Design: A Brief Theoretical Overview and Practical Suggestions*

Margaret Butler**

Ms. Butler argues that librarians teaching legal research should follow resource-based learning pedagogical strategies. Her article provides a background in constructivist educational theory and resource-based learning before identifying useful instruc-tional strategies regarding course design decisions related to goal setting, assignments, rubrics, and assessment.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219Resource-BasedLearning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221Problem-BasedLearning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223DevelopingMetacognitiveSkills. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224InstructionalStrategies:QuestioningStudentsandScaffolding . . . . . . . . . . . . . 225BuildingSchematatoMaximizeWorkingMemory . . . . . . . . . . . . . . . . . . . . . . . 229BenefitsofResource-BasedLearning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232CourseDesignDecisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

SettingGoals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234InstructionalStrategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

Assignments,Rubrics,andAssessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240Assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240Rubrics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

Introduction

¶1Thebestmethodology for teaching students legal research is a subjectofdebatewithinthelawlibrariancommunity.1Thoughthedebateexistedbeforethecurrent push in legal education to improve law students’ practical and ethical

* ©MargaretButler,2012.Thisisarevisedversionofthewinningentryinthenewmemberdivisionofthe2011AALL/LexisNexisCallforPaperscompetition.Thearticlewaspresentedatthe2010BoulderConferenceonLegalInformation:ScholarshipandTeaching,andIwouldliketothanktheconferenceparticipantsfortheirfeedbackandsupportindraftingit. ** Associate Director for Public Services, Georgia State University College of Law Library,Atlanta,Georgia. 1. Forahistoryofthedebate,seePaulD.Callister,Beyond Training: Law Librarianship’s Quest for the Pedagogy of Legal Research Education,95lAW libr. J.7,8–9,2003lAW libr. J.1,¶4.

220 LAW LIBRARY JOURNAL Vol. 104:2 [2012-19]

understandingsofthelaw,2lawlibrarians’analysisofthebestwaystoteachlegalresearchisseeingmoreprominence.3Likelawprofessors,however,lawlibrariansdo not generally have pedagogical training.4 Most pedagogical training for lawlibrarianscomesintheformofon-the-jobtraining,presentationsatprofessionalconferences,5andprofessionalliterature.

¶2Althoughsomepeoplemaybedescribedas“natural teachers,” thatgift israre.Butteacherscanbetrainedinthemechanicsofteaching,ultimatelyimprov-ing the education delivered.6 Through training, teachers may learn to considerinstructional,orpedagogical,theoryastheydeveloptheircourses.KristinGerdyhassuggestedthatadultlearningtheory—thelearningtheoryrelevanttolawstu-dents,ratherthanelementaryschoolstudents—shouldbeconsideredinthedevel-opmentoflegalresearchcourses.7

¶3Adultlearnerssharesometraitsthatshouldbeconsideredwhendesigningacourse.First,adultlearnersareabletochooseoptionsthatbestsuittheirlearningneeds.8Whenpreparinglessonsforadultlearners,instructorsshouldbrieflypro-videanoverviewandcontext,andsummarizethe“bigpicture”forstudents—this

2. The MacCrate Report and the Carnegie Report both represent efforts to improve legaleducation.Am. bAr Ass’n, seCtion of legAl eduC. & Admissions to the bAr, legAl eduCAtion And professionAl deVelopment—An eduCAtionAl Continuum: report of the tAsK forCe on lAW sChools And the profession: nArroWing the gAp(1992)[hereinaftermACCrAte report];WilliAm m. sulliVAn et Al., eduCAting lAWyers: prepArAtion for the profession of lAW(2007)(CarnegieReport).Thoughnotasadirect resultof those reports, the law librariancommunityhasengagedin conversation about the best way to teach legal research, through debates around bibliographicinstructionandprocess(theBerringandWrendebate)aswellasothertopics.Recently,theBerringandWrendebatewasrevisitedbyBerring:“Almost20yearslater,onemightwonderwhatallthefusswasabout.Inhindsight,theWrensespousedamoreimportantroleforlegalresearchtrainingandtheyfeltitwasbestdoneinanenvironmentwherethestudentwaslearninghowtousetheresearchtools.”RobertC.(Bob)BerringJr.,Twenty Years On: The Debate over Legal Research Instruction,17perspeCtiVes: teAChing legAl reseArCh And Writing1,3(2008). 3. The annual Conference on Legal Information: Scholarship and Teaching, which beganin 2009, is the product of a discussion among librarians about legal information scholarship andinstruction,lookingtowardthedevelopmentofa“theoreticalfoundationofasignaturepedagogyforlegalresearcheducation.”ConferenceonLegalInformation:ScholarshipandTeaching,TheBoulderStatement on Legal Research Education (June 22, 2009) (unpublished manuscript), available athttp://www.colorado.edu/law/events/legalResearchEducation.pdf. The conversation was broadenedbeyondBoulderConferenceparticipantsthroughprogramsattheAnnualMeetingoftheAmericanAssociationofLawLibraries(AALL)in2009and2010. 4. Jobpostingsforacademicreferencelibrarians(whoaregenerallythelibrariansinvolvedinteachinglegalresearch)typicallyrequireaJ.D.degreeaswellasadegreeinlibraryorinformationsci-ence.Degreesineducation(foreitherchildoradultlearners)arenotmentionedinthesejobpostings,andarenotgenerallyrequiredoflibrarianinstructors.See generallyEmployment Opportunities,lAW librAriAn blog, http://lawprofessors.typepad.com/law_librarian_blog/employment_opportunties/(lastvisitedDec.21,2011). 5. TheAALLAnnualMeetingusuallyprovidesseveraltracksofprograms,basedoncompeten-cies,oneofwhichisteaching.See, e.g.,Program Track: Teaching,2011 AAll AnnuAl meeting And ConferenCe,http://aall11.sched.org/subject/Teaching(lastvisitedJan.30,2012). 6. ElizabethGreen,Can Good Teaching Be Learned?,n.y. times,Mar.7,2010,§MM(Magazine),at30. 7. Kristin B. Gerdy, Making the Connection: Learning Style Theory and Legal Research Curriculum,19legAl referenCe serViCes Q.nos.3/4,2001,at71,73–77. 8. Id.at74.

221ReSOURCe-BASed LeARNINg ANd COURSe deSIgNVol. 104:2 [2012-19]

enablesstudents to learnexperientially.Many law librarianshavenotedthatstu-dent interest in “real world” questions is very high;9 this interest is importantbecausestudentslearnbestwhentheyseetherelevanceoftheresearchtothetaskstheyknowtheywillbeexpectedtoperform,whetherassummerassociates,interns,orpracticingattorneys.Accordingly,agoodlegalresearchinstructorshouldcon-textualize legal research and allow students to learn by using legal researchresources—whetherelectronicorprint—toanswerquestions,sostudentscandrawtheirownconclusionsabouttherelevanceorutilityoftheinformationpresentedbytheteacher.

¶4Inotherwords,legalresearchstudentswillbenefitfromaresource-basedora problem-based approach to teaching. These approaches, which are discussedmorefullybelow,requirestudentstoengagewithresources,suchasprimaryandsecondarylegalsources,andproblemstolearntoconductlegalresearch.Buttheseapproachestoteaching,bythemselves,arenotallthatinstructorsshouldconsiderwhen seeking to improve their teaching skills. Teaching strategies; course designdecisions;andassignments,rubrics,andassessmentplansmustbeconsideredwhenonehopestoimproveone’steaching.Thisarticleaddressesthepedagogicalbenefitsofresource-basedandproblem-basedlearninginthelegalresearchclassroomandoffers theoreticalandpractical suggestions forcoursedesigndecisions, includingthe use of teaching strategies, the development of assignments, the benefits ofrubrics,andassessmenttechniques.

Resource-Based Learning

¶5 In the resource-based learning model described in British academic legalliterature,teachersmustpay“carefulattentiontopedagogy,includinglearningout-comestobeachievedbystudentsfromtheprojectandmethodsoffeedback.”10Inthismodel,“studentslearnbyusingresources,”11withinformationandcommuni-cationstechnology“usedtosupportlearninginmoreflexibleways.”12Thelanguageof“resource-basedlearning”andresource-basedlearningasapedagogicalapproacharealsousedintheUnitedStates,thoughnotusuallyinlaw-specificcontexts.13

¶6Resource-basedlearningapproacheshavegreatpotentialtobehelpfulnotonlyinresearchcoursesinwhichstudentsareaskedtoconsiderchallengingprob-lems, but also in clinical work and other project-based law school coursework.“Resource-basedlearninginvolvesestablishingcontextsfor,toolsforactingonandwith,andscaffoldstoguidethedifferentiatedinterpretation,use,andunderstand-ingofresourcesinwaysthatareconsistentwiththeepistemology,foundations,andassumptions of a given learning model.”14“[R]esource-based learning is a peda-

9. Id.at76–77. 10. PaulMaharg&AbdulPaliwala,Negotiating the Learning Process with Electronic Resources,ineffeCtiVe leArning And teAChing in lAW81,84(R.Burridgeetal.eds.,2002). 11. Id.at82. 12. Id.at83. 13. Forexample,a search inHeinOnline’sLawJournalLibrary for“resource-based learning” OR “resource based learning”returnedonlyfiveresults. 14. MichaelJ.Hannafin&JanetteR.Hill,Resource-Based Learning,inhAndbooK of reseArCh on eduCAtionAl CommuniCAtions And teChnology525,528(J.MichaelSpectoretal.eds.,3ded.2008).


gogical approach associated with inquiry- and project-based learning in which[students work with]‘a wide range of learning resources rather than from classexposition.’”15

¶7 Resource-based learning presents an attractive pedagogical approach forteaching legal research for several reasons. First, resource-based learning lendsitselftovirtuallearning,anditisoftenassociatedwithdistanceorvirtuallearningineducationalliterature.16Althoughlawschoolaccreditationruleslimitthewaysin which law schools may implement distance education,17 many law schoolcoursescontainsomevirtualcomponentsiftheyuseTWEN,BlackBoard,orotherwebcoursetechnology.18Resource-basedlearningalsomaybeusedwithavarietyofepistemologicalmodels,ormodelsofpeoples’waysofknowing.19Inparticular,those who oppose the“banking model” of education, in which an all-knowingteacherstandsatthefrontoftheroomand“datadumps”knowledgeintoawaiting(empty) student minds,20 may find resource-based learning appealing, as it “isunderlain by the philosophical assumption that allowing the learner to achievelearningoutcomesinamoreflexibleandindependentmannerisinherentlybetterthanthetraditionallearningmethodology,epitomizedbythe‘banking’conceptofeducationcriticizedby[Paolo]Freire.”21

15. BarbaraA.Greene&SusanM.Land,A Qualitative Analysis of Scaffolding Use in a Resource-Based Learning Environment Involving the World Wide Web, 23 J. eduC. Computing res. 151, 152(2000)(quotingG.C.Rakes,Using the Internet as a Tool in a Resource-Based Learning Environment,36eduC. teCh.52,52(1996)). 16. See steVe ryAn et Al., the VirtuAl uniVersity: the internet And resourCe-bAsed leArning(2000).Becauseofthecorrespondencebetweenresource-basedlearningmaterialsandvir-tualoronlinelearningmaterials,someoftheteachingapproachessuggestedhereareadaptedfrommaterialsthataddressthedevelopmentorteachingofonlineorvirtualcourses. 17. SeeAm. bAr Ass’n, stAndArds And rules of proCedure for ApproVAl of lAW sChools27–28(2011–2012)(Standard306)(requiringdistanceeducationcourses tobeapprovedusing thesameprocessastraditionalcoursesandlimitingstudentstonomorethanfourcredithoursperterm,foramaximumof twelvecredithours,withdistance learning functionallyprohibited in the first-yearcurriculum). 18. FeaturesavailablewithinWestlaw’sTWENsystemincludepolling,onlinediscussionforums,wikipages, anelectronicgradebook,and theability topost filesandexternal links, allowing stu-dentstoaccesspodcastsandexternalvideocontent.AdministrAtor’s guide to tWen,(Aug.2011),available at http://lscontent.westlaw.com/images/content/documentation/2011/adminiguide2011.pdf. The features available through LexisNexis Web Courses include online discussion forums,chat,anonymousgrading,andtheabilitytoaddfilesandexternallinks,allowingstudentstoaccesspodcasts and external video content. instruCtor’s QuiCKguide: lexisnexis Web Courses (2009),available athttp://www.lexisnexis.com/documents/LawSchoolTutorials/20090824041047_small.pdf. 19. Hannafin&Hill,supranote14,at528. 20. A less inflammatory description of this type of teaching would be direct instruction.Teachingmayinvolveavarietyofapproaches,soaconstructivistmightspendfivetosevenminutesofdirectinstructionteachingamini-lessononanarrowtopic,possiblyinresponsetoastudentques-tion;butthebankingmodelofeducationsuggeststhatthebulkoflearningshouldbedonebydirectinstruction. 21. Bernard Lisewski & Chris Settle, Integrating Multimedia Resource-Based Learning into the Curriculum,inresourCe-bAsed leArning109(SallyBrown&BrendaSmitheds.,1996).PaoloFreireisperhapsbestknownforPedagogy of the Oppressed,abriefbutcompellingworkaddressingtheroleofpowerintheclassroom.pAolo freire, pedAgogy of the oppressed(30thanniv.ed.2007).


¶8Resource-basedlearningisonetypeofconstructivistpedagogicaltheory.22Constructivismhas,atitsbase,theassumptionthat“Knowledgeisnottransmitted:itisconstructed.”23Withinconstructionistschools,thereareindividualconstruc-tivists and social constructivists. The individual constructivists generally believethat“Learning results fromapersonal interpretationofknowledge,”while socialconstructivistsgenerallyholdthat“Learningiscollaborativewithmeaningnegoti-atedfrommultipleperspectives.”24Someconstructivistswouldaddanelementofcontextualismtotheirphilosophy,recommending“presentingproblemsinsitua-tionsthatarerealistictolearnersandcommontoeverydayapplicationsofknowl-edge,”thusprovidingstudentswithopportunitiesfor“authenticlearning.”25

¶9Understandingconstructivist theory, itsunderlyingprinciples, andhow itrelatestoresource-basedlearningmayhelpinstructorsincreating,planning,andteaching a course. Resource-based learning may be described as a constructivistapproachincorporatingvaluableinstructionalstrategiesthatshouldbeconsideredintheprofessionaldiscussionofthedevelopmentofapedagogyoflegalresearch.26

Problem-Based Learning

¶10Problem-basedlearningissimilartoresource-basedlearning.Inproblem-basedlearning,“studentsworkinsmallcollaborativegroupsandlearnwhattheyneedtoknowinordertosolveaproblem.Theteacheractsasafacilitatortoguidestudentsthroughthelearningcycle.”27Problem-basedlearningoriginatedinmedi-cal education, though ithasbeenadoptedbyother fields.28Both resource-based

22. Thisarticleis,at least inpart,aresponsetoPaulCallister’scallforincreaseddiscussionofpedagogical theories in lawlibrarianprofessional literature.SeeCallister,supranote1.AsnotedbyNolanWright,however,“fewhavetakenuphiscallandrespondedinscholarlywritingsoftheirown....illustrat[ing]thebasisforthisauthor’sconcernaboutthelackofpubliclyairedscholarlydialoguewithin the profession, let alone between the profession and other disciplines.” Nolan L. Wright,Standing at the Gates: A New Law Librarian Wonders About the Future Role of the Profession in Legal Research Education,27legAl referenCe serViCes Q.305,322–23(2008).Perhapspartofthereasonthattheliteratureismoreheavilyweightedtowardscholarshipdescribingparticularteachingchoicesatparticularinstitutions,ratherthantowardadiscussionbasedinpedagogicaltheory,isthatlibrar-ians—evenlawlibrarians—arenotgenerallytrainedinpedagogicaltheory.SeePaulD.Callister,Time to Blossom: An Inquiry into Bloom’s Taxonomy as a Means to Ordered Legal Research Skills,102lAW libr. J.191,194–95,2010lAW libr. J.12,¶¶8–9. 23. pAtriCiA l. smith & tillmAn J. rAgAn, instruCtionAl design15(2ded.1999). 24. Id. 25. Id.at16. 26. Indescribingapedagogicalmodelforlegalresearch,Callistersuggests

thatacompletemodelrequires(1)anidentifiableandfullyunderstoodobjectiveinteachinglegalresearch(whichobjectivemustdistinguishbetweenthekindsofresearchdonebyattorneys,schol-ars,andlibrarians);(2)atheoryandunderstandingofthenatureoflegalsourcematerials(whichcontemplates changes in volume, accessibility,“gestalt,” etc.); (3) a theory of mathetics, or thenatureofstudentsandhowtheylearn(withemphasisontheprovisionofconceptualmodelsforinternalizingresearchtechniques);and(4)amethodologyconsistentwiththepreviouselements.

Callister,supranote1,at8–9,¶4. 27. CindyE.Hmelo-Silver,Problem-Based Learning: What and How Do Students Learn?,16eduC. psyChol. reV.235,236(2004). 28. Whenstudentsenter themedical school theyaredivided intogroupsof fiveandeachgroup is

assignedafacilitator.Thestudentsarethenpresentedaproblemintheformofapatiententeringwithpresentingsymptoms.Thestudents’taskistodiagnosethepatientandbeabletoprovidearationaleforthatdiagnosisandrecommendedtreatment.

John r. sAVery & thomAs m. duffy, problem-bAsed leArning: An instruCtionAl model And its


andproblem-based learningrelyonstudentexperienceas the locusof learning,treating the teacherasa facilitator, thoughproblem-based learningoftenhasanadditionalexpectationthatstudentsareworkingcollaboratively,ratherthanindi-vidually.29Problem-basedlearningfocusesonthedevelopmentofcriticalthinkingskills,30makingitatemptingpedagogicalapproachinthelegalresearchcontext.However,itisextremelytimeintensiveanddoesnotlenditselftoeasyuseinafirst-yearlegalresearchcourse.31

¶11 For problem-based learning to be effective, the problems generated andusedininstructionshouldmeetseveralcriteria:problemsshouldbecomplexandpresentopen-endedquestions,andtheyshould“berealisticandresonatewiththestudents’experiences”whilealsopresentingstudentswithopportunitiestoevalu-atetheirknowledgeandtheirapproachtotheproblem.32Bydefinition,aneffectiveproblemraisesstudentinterestinthesubjectmatterandengagesstudentswiththeinformationnecessarytosolvetheproblemaswellaswithproblem-solvingstrate-gies. The problem-based learning approach may be particularly successful in anadulteducationcontextbecausetherealisticnatureoftheproblemsservestomoti-vatestudents.

Developing Metacognitive Skills

¶12TheMacCrateandCarnegieReportsbothcallforthedevelopmentoflaw-yeringskillsandvalues.33Resource-basedlearning,aswellasproblem-basedlearn-ing,totheextentthattheycanbeimplementedinalawschoolsetting,canbeusedto advance students’ ability to become effective problem solvers, employing thetoolsthattheywillultimatelyworkwithinpracticeastheydeveloptheskillsneces-sarytoapproachaclient’sproblemfromalegalperspective.Usingresourcesandhypothetical problems can provide instructors with the opportunity, as well, toengagestudentsindialogueabouttheirprofessionalresponsibilitiestoclients.Forexample,aninstructormaymakeanethicalquestionabouttherepresentationofaclient’sintereststhebasisforbotharesearchproblemaboutthestate’sadministra-

ConstruCtiVist frAmeWorK7(Ctr.forRes.onLearningandTech.,Tech.Rep.No.16-01,2001). 29. Hmelo-Silver,supranote27,at239. 30. Id.Aninstructionalstrategythatmightbehelpfulwouldbe“discussingproblemsina[prob-lem-basedlearning]group(beforebeginningtoresearchlearningissues)[to]activate[]relevantpriorknowledgeandfacilitate[]theprocessingofnewinformation.Studentsarebetterabletoconstructnewknowledgewhentheycanrelateittowhattheyalreadyknow.”Id.(citationsomitted). 31. Problemsinthemedicaleducationalcontextmaylastfromonetothreeweeks,andstudentsaretaughtusingproblem-basedlearningfortwoyears.sAVery & duffy,supranote28,at10.Legalresearchcourses,whentheyareseparatefromlegalwritingcourses,areoftenlimitedtoonecredit,whichmeanslawstudentshaveonly700minutesofclasstime,asrequiredunderABAaccreditationstandards.Am. bAr Ass’n,supranote17,at23(Interpretation304-4). 32. Hmelo-Silver,supranote27,at244. 33. TheMacCrateReport identifiesparticular skills andvalues that are integral to lawyering.mACCrAte report,supranote2,at138–41.And,astheCarnegieReportindicates,“‘Studentscannotbecomeeffectivelegalproblem-solversunlesstheyhaveopportunitiestoengageinproblem-solvingactivities inhypotheticalor real legal contexts.’”sulliVAn et Al., supranote2, at95 (quotingroy stuCKey et Al., best prACtiCes for legAl eduCAtion109(2007)).


tivelawandauthorityregulatinglawyers,andatheoreticalquestionaboutlawyers’professionalresponsibilitiestozealouslyrepresenttheirclient’sinterests.

¶13Bothproblem-andresearch-basedlearningmotivatestudentsbyprovidingthemwithreal-life,oratleastrealistic,problems.34Legalresearchinstructionmustpreparestudentstocontinuelearning,evenaftertherequiredlegalresearchcourseiscompleted.35Thedevelopmentofmetacognitiveskills,definedas“executivecon-trol process of planning one’s problem solving, monitoring one’s progress, andevaluatingwhetherone’sgoalshavebeenmet,”36 is a critical function fora legalresearcher.37 Callister notes that“the final skill is meta-cognition—the ability toassess,notonlytheresult,buttheschemata,includingtheprocessesleadingtotheresult. It is a kind of self-awareness and reflection of the research experience.”38Bothproblem-basedandresource-basedlearningencouragestudentstodevelopanawarenessoftheresearchprocessastheymayencounteritinprofessionalpractice.Theabilityof a researcher toexplainhowananswerwas reached—forexample,whyoneresourcewaspreferable—ratherthansimplystatingtheanswerisacriticalmetacognitivetaskdevelopedinresource-basedandproblem-basedlearning.

Instructional Strategies: Questioning Students and Scaffolding

¶14Inboththeresource-basedandtheproblem-basedlearningenvironments,theteacherplaystheroleoffacilitator,modelingappropriatebehaviorforstudentsand guiding students to use learning or instructional strategies such as thinkingaloudwhengeneratinga listof indexorsearchtermsrelatedtoaresearchprob-lem.39 This process of thinking aloud develops students’ metacognition when

34. Hmelo-Silver,supranote27,at236(discussingproblem-basedlearning).ShawnG.NeversandDavidArmondhavedescribedthevaluetheyhavefoundincreatingaPractitioners’Council,asitconnects“realworld”researchingpractitionerswithlegalresearchinstructors,allowingforbettermotivation of students. Shawn G. Nevers & DavidArmond, The Practitioners’ Council: Connecting Legal Research Instruction and Current Legal Research Practice,103lAW libr. J.575,593–94,2011lAW libr. J.36,¶¶68–70. 35. When researching, whether as students or attorneys, motivation to address a researchquestionmaybeeither internal—curiosityor self-interest—orexternal—aclientquestion,aboss’sdemand for an answer, an ethical obligation, etc. Most important is that the researcher performadequatelyregardlessofmotivation.Studentsinlegalresearchclassesmaybemotivatedbylearningoftherisksofmalpracticeforfailuretoperformadequatelegalresearch. 36. Hmelo-Silver,supranote27,at240. 37. SeeKristinaL.Niedringhaus,Teaching Better Research Skills by Teaching Metacognitive Ability,18perspeCtiVes: teAChing legAl res. & Writing113,115(2010)(“Astudentwhoismetacognitivelyawarewillbebetterabletoassesswhatknowledgetheyhavenotlearnedthoroughly.Thesestudentswillbeabletodevelopaplanforrelearningthematerialusingtechniquesthatspeaktotheirpreferredmethodsoflearning.Thesestudents,byreflectingonwhattheyhavelearnedandfillingthegaps,willnotonlybebetterstudentsbutwillbeabletocontributemorefullytotheclassroomexperience.”). 38. Callister,supranote22,at210,¶39.See alsoKristinB.Gerdy,Teacher, Coach, Cheerleader, and Judge: Promoting Learning Through Learner-Centered Assessment,94lAW libr. J.59,64,2002lAW libr. J.4,¶21(notingthattocompletethelearningcycle,“learnersandteachersmustassessandevaluatethe learningthathasoccurred”;withoutthismetacognitivestep, learnersarenotas likelytoretaintheirlearning). 39. Greene&Land,supranote15,at153.TheCarnegieReportstatesthatexpertteachersmay“advancedialogue”intheirclassrooms“bymakingcognitionvisible”throughmodeling.sulliVAn et Al.,supranote2,at61.


addressingtheresearchproblem.Withininstructionalliterature,scaffoldingisusedtodescribe“instructionalproceduresdesignedtosupport learningsothatastu-dentcanimprovebeyondhisorhercurrentlevelofunderstandingwithguidancefromapeer,teacher,orinstructionalaid.”40Aninstructormayprovideproceduralscaffoldsintheformofguidingquestionsforstudentstoconsiderastheyapproachaproblem.41Inalegalresearchcourse,suchquestionsmayencouragestudentstoreflectonwhytheychosetoconsultaprimaryresource,ratherthanasecondaryresource.Anotherexampleofscaffoldingintheclassroomwouldbewhenaclasssolves a problem as a whole group, perhaps with the instructor thinking aloudthroughtheproblem;theclassthengoesontosolveanewproblemwithasimilarstructure.Thesimilarityoftheproblemsandtheopportunitiesforcollaborationarescaffolding—opportunitiesforstudentstoapplytheirknowledgeabouthowtosolveoneproblemtoanotherproblem.

¶15Instructorsmayusedifferentstrategiesofquestioningstudentstoscaffoldstudentlearning.42Ahistoricalreviewofquestioningintheclassroomnotedtheimportanceofquestioninginteaching.Theauthorexplainedthattheteacherhasbeencalled“‘aprofessionalquestionmaker’andclaimedthattheaskingofques-tionsis‘oneofthebasicwaysbywhichtheteachersimulatesstudentthinkingandlearning.’”43

¶16 Many in the law librarian community are familiar with the questioningformatknownas theSocraticmethod, inwhich“the teacherasks students forapositiononanissue,thenasksappropriatefollow-upquestionstoprobethestu-dent’sposition.”44Ofcourse, in theSocraticmethod,“the teacherhas the‘right’answeranditisthestudent’stasktoguess/deducethroughlogicalquestioningthatcorrectanswer.”45Thenotionthattheteacherhasthe“right”answerandisquery-ingstudentstoguidethemlogically46tothatrightanswerisinconsistentwiththe“teacherasfacilitator”modelofbothresource-basedandproblem-basedlearning.Underthosetheories,instructorquestioningshouldpushstudentstothe“leadingedge”oftheirthinking.47However,asCallisterhassuggested,Socraticquestioningmayhaveaplaceinthelegalresearchclassroom,becauseitmayforcethelearnerto examine her own frameworks for how she understands and solves problems.“[T]heSocraticmethodisanappropriateandperhapsevennecessarytooltofacili-tate the learning experience of law students studying legal research.”48 In otherwords,theSocraticmethodmaybeusedtohelpstudentsengageinmetacognition,

40. Greene&Land,supranote15,at153. 41. Id.at159. 42. Hmelo-Silver,supranote27,at246. 43. MeredithD.Gall,The Use of Questions in Teaching,40reV. eduC. res.707,707(1970)(quot-ingM.J.McCueAschner,Asking Questions to Trigger Thinking,NEAJ.,Sept.1961,at44,44). 44. Id.at711. 45. sAVery & duffy,supranote28,at5. 46. ForadescriptionoftheSocraticmethodinalegaleducationcontext,includingadiscussionofitsstrengthsandweaknesses,seePeggyCooperDavis&ElizabethEhrenfestSteinglass,A Dialogue About Socratic Teaching,23n.y.u. reV. l. & soC. ChAnge249(1997). 47. sAVery & duffy,supranote28,at5. 48. Callister,supranote1,at33–34,¶59.


thinkingaboutandunderstandingtheresearchprocess thatwillbestaddress theresearchquestionathand.

¶17TheSocraticmethodmaynotimmediatelycometomindasastrategyonewoulduseinthecontextofresource-basedlearning,asitusuallycaststheinstruc-torasexpertandchallengesthelearner’sgraspofthematerial.However,“thefacili-tatorscaffoldsstudentlearningthroughmodelingandcoaching,primarily through the use of questioning strategies.”49A“goodquestion,”onethatencouragesstudentstolearn,“isalwaysontheedgeofwhatanindividualknows—ontheedgeofone’sconstruct(orschema)ofreality.Tobeabletoseethatedge—torecognizewhenoneisapproachingit—isthebeginningofallinquiryandanecessaryskill.”50Forfirst-yearstudentsinalegalresearchcourse,theedgeoftheirknowledgeonthefirstdayofclassmaybethatGoogleisthebestwaytofindtheanswertoaquestion.AgooddemonstrationofscaffoldingwouldbetotakestudentstothatedgeandteachthemtoseetheresourcesthatexistinadditiontoGoogle,showingthemthatthetheirfamiliarity with using Google may help them learn how to use other researchtools.51

¶18 Teachers can also be trained to improve the questions that they ask stu-dents.52Ofcourse,questionsshouldbealignedwithlearninggoals,andtheyshouldideally enable students to achieve these learning goals.53 Less helpful questionsmight require only that students recall facts, rather than encouraging them toengagemoredeeplywiththematerial.54Gallnotedthatelementaryschoolteacherswho went through a training program had“many highly significant changes in[their]questioningbehavior.”55Someofthepositivechangesincludedanincreasedfrequency of questions“designed to have a number of students respond to onestudent’s original question,” “thought questions,” and “questions which requirestudentstoimproveorelaborateontheiroriginalresponse.”56Teacherscanalsobetaughttominimize“poorquestioninghabits,”suchasrepeatingquestions,repeat-ingstudentanswers,answeringtheirownquestions,andinterruptingstudentsastheyanswerquestions.57

¶19 Teacher questioning may take many forms. One of the most commonlydiscussedmodelsforquestioningisbasedonBloom’staxonomy.BenjaminBloompublishedahandbookin1956classifyingeducationalgoalsandobjectivesinthreeways,cognitive,affective,andpsychomotor.58Inthisarticle,IfocusonthecognitiveskillsdescribedbyBloom,leavingotherstoaddresshiscategorizationofaffective

49. Hmelo-Silver,supranote27,at245(emphasisadded). 50. Callister,supranote22,at200,¶20. 51. AsdescribedintheCarnegieReport,scaffolding“provid[es]supportforstudentswhohavenotyetreachedthepointofmastery.”sulliVAn et Al.,supranote2,at61. 52. Gall,supranote43,at717–18. 53. Id.at711. 54. “About60%ofteachers’questionsrequirestudentstorecallfacts;about20%requirestudentstothink;andtheremaining20%areprocedural.”Id.at713. 55. Id.at717. 56. Id. 57. Id. 58. tAxonomy of eduCAtionAl obJeCtiVes: the ClAssifiCAtion of eduCAtionAl goAls(BenjaminS.Bloomed.,1956).


andpsychomotorskills.SinceBloom’shandbookwasfirstpublished,ithasbeensubjected to discussion, study, and refinement.59 Based on the most recent andwidelyacceptedrefinement,thecognitiveskills,fromthelowestlevelofthinkingto thehighest, are remembering,understanding,applying,analyzing,evaluating,andcreating.60ThefollowinglistshowsBloom’soriginalcognitiveskillsandtheirrevisedcounterparts:61

Original Version Revised VersionEvaluation CreatingSynthesis EvaluatingAnalysis AnalyzingApplication ApplyingComprehension UnderstandingKnowledge Remembering

¶20EducatorsfindBloom’staxonomy(originalandrevised)62usefulforbothquestioningandgoalsetting.63Bloom’staxonomyhelpsteacherstodevelopappro-priate questions for students—questions that will help deepen student under-standingofsubjectmaterial.Inthecontextofalegalresearchcourse,thedeepenedunderstandingmayreflectthedifferencebetweensimplyknowingthatthereisaservice to help legal researchers identify whether a legal opinion remains“goodlaw,”andunderstandingthesignificanceofayellowflaginKeyCiteorShepard’s.

¶21EachlevelofcognitiveskillinBloom’staxonomyisassociatedwithverbsthatmaybeusefulwhenposingstudentquestions.Forexample,thelowest-levelcognitiveskill,remembering(whetherthestudentcanrecallorrememberinfor-mation),canbeassociatedwiththefollowingverbs:define,duplicate,list,memo-rize, recall, repeat, reproduce, and state.64 Higher-order cognition, such asevaluating, may be associated with verbs such as appraise, argue, defend, judge,select,support,value,andevaluate.65

¶22Althoughtheseverbsmaybeusedinquestioningstudents,forexamplebyaskingastudenttodefendadecisiontorelyonacaseforwhichacitatorshowsayellowwarningsignal,studentanswersmaynotrisetothehigherlevelofcognitionsoughtbytheinstructor.66Itisatthispointthatateacher’sabilitytoaskfollow-up

59. SeeMaryForehand,Bloom’s Taxonomy,inemerging perspeCtiVes on leArning, teAChing, And teChnology(M.Oreyed.,2005),http://www.coe.uga.edu/epltt/bloom.htm.CallisterhascalledforaprofessionaldiscussiontorefineBloom’staxonomyforlegalresearchpedagogy.Callister,supranote22. 60. Richard C. Overbaugh & Lynn Schultz, Bloom’s Taxonomy, http://www.odu.edu/educ/roverbau/Bloom/bloomstaxonomy.htm(lastvisitedDec.22,2011). 61. Adaptedfromid. 62. Fromthispointforward,unlessotherwisenoted,thediscussionofBloom’staxonomyrelatestotherevisedtaxonomyofcognitiveskills. 63. TheroleofBloom’staxonomyingoalsettingisdiscussedinfra¶45.Consideringquestioningbeforeconsideringgoalsmaybeputtingthecartbeforethehorse,intermsofcurriculumdesign.Thebestpractice ininstructionaldesignistofirst identifytheeducationalobjectivesandthendevelop“questionswhichenablethestudenttoreacheachobjective.”Gall,supranote43,at711. 64. Overbaugh&Schultz,supranote60. 65. Id. 66. “Aweaknessofthecognitive-processapproachtoquestionclassificationisthatthesepro-cessesareinferentialconstructs.Therefore,theycannotbeobserveddirectly.”Gall,supranote43,at


questionsbecomescritical.A follow-upquestionmaychallengea studentwhoseresponseisatthelevelofrecalltoengagewiththematerialandansweratamorecriticallevel.QuestionclassificationsystemssuchasBloom’staxonomydonotspe-cifically takequestion sequence intoaccount.67Though it is tempting toassumethat an instructor would begin asking questions at the lowest (recall) level andmovethroughthecognitivestagestothehigher-orderthinkinglevels,thelevelsofBloom’staxonomydonotsimplypresentalinearprogressionforinstruction.68Asin research, an instructor may need to loop back and ask simpler questions toensure students all move toward the ultimate goal of full engagement andunderstanding.

Building Schemata to Maximize Working Memory

¶23Adifficultyforinstructorsoflegalresearcharisesfromthelargeamountofinformationthatstudentsmustbeabletorecallinordertolearnhowtoresearcheffectively.69Whenplanningacourse,aninstructormustbalancetheneedtogivestudentsinformationaboutresourceswiththeneedtoteachstudentshowtocon-ductresearch(think,analyze,refinethequery,etc.).70Aresearcherneedsanade-quatetoolboxofresourcesthatmaybeconsultedtoaddressaresearchquestion,butinstructionthatfocusestoocloselyonresourcesmayresembletheworstformofbibliographicinstruction:datadumping.Ontheotherhand,aresearcherfamiliarwiththeresearchprocessisstymiedifshedoesnotknowwhatresourcestoconsult.Legalinformationischanging,anditiscriticalthatstudentsunderstandnotonlythevalueof the information,buthow the resources areused.71BobBerringhasdescribedtheapproachheandKathleenVandenHeuveltaketoteachingadvancedlegalresearchasa“functionalapproach.”72Astudentwhounderstandsthepurposeofacitatorandhowacitatorworks,forexample,willbeabletofigureouthowtouseacitatorthatbecomesavailableinanewformat.73

710.QuestionsdevelopedwithBloom’s taxonomy inmindmaybedesignedashigher-orderques-tions, suchasoneasking students tocompare theLexisNexisandWestlawcitators,buta student’sanswermaydemonstrateonlyrecall(ofmaterialfromatextbookoraclassdiscussion).Id.Inotherwords,thebestlaidlessonplansmaygoawry. 67. Id.at712. 68. Justasresearchdoesnotalwaysfollowalinearpath,sodoesinstructiondeviate. 69. For example, AALL’s Core Legal Research Competencies is 113 pages, not an insignificantvolume of information. reseArCh instruCtion CAuCus, Am. Ass’n of lAW librAries, Core legAl reseArCh CompetenCies: A Compendium of sKills And VAlues As defined in the AbA’s mACCrAte report (Ellen M. Callinan ed., 1997), available at http://www.aallnet.org/sis/ripssis/PDFs/core.pdf[hereinafterCore legAl reseArCh CompetenCies]. 70. Callisternotesthatresearchers’needsmaydiffer,dependingontheirstatus.Studentsresearchdifferentquestions,withdifferentconstraints,thandolawyers,clerks,judges,orlibrarians.Hesug-gests that legal research instruction should prepare students to research effectively in a variety ofcontexts.Callister,supranote1,at23–24,¶¶37–38. 71. Berring,supranote2,at3. 72. Id.“Thoughwecouldnotforeseethefuture,wecouldguessthatnewformatsandnewtoolswerecoming.” Id.Byemphasizing the functionof resources,BerringandVandenHeuvelhoped topreparestudentstocontinuetouseandevaluatenewresourcesandaccessmethodsastheybecameavailable. 73. Id.TheintroductionofBloombergLawtothelegalmarketisjustsuchanexample.AstheyexploretheoptionsavailableonBloombergLaw,studentswillhavetousetheirexistingknowledgeof


¶24 From a learning theory perspective, the challenge of designing a legalresearchcoursethatconveysall that informationisdauntingbecauseofthewaythat knowledge develops in the human brain. According to one explanation ofhumancognitivearchitecture,apersoncangenerallyholdnomorethansevennewpiecesofinformationinworkingmemory.74“[B]ecauseworkingmemoryismostcommonly used to process information in the sense of organizing, contrasting,comparing,orworkingonthatinformationinsomemanner,humansareprobablyonly able to deal with two or three items of information simultaneously whenrequiredtoprocessratherthanmerelyholdinformation.”75

¶25Notonlyistheworkingmemorylimitedinthenumberofpiecesofinfor-mationitcanhold,itisalsolimitedinitsduration.Studiessuggestthatthebrainisabletoholdinformationinworkingmemoryforonlytentotwentyseconds.76Toholdinformationforlonger,theinformationmustmovefromworkingmem-orytolong-termmemory.Thistransferofinformationis“themostcriticalprocessofall the informationprocessing to thosewhoare interested in learning.”77Theprocessofmakingmeaningfrominformationhelpslearnerstoretaininformation.“[T]he more ‘deeply’ information is processed, the more likely it is to beremembered.”78

¶26 How is information processed deeply? According to schema theory, thelong-termmemorystoresknowledgeintheformofaschemathat“categorizesele-mentsof informationaccordingto themanner inwhichtheywillbeused.”79 Inotherwords,forinformationtomovefromworkingmemorytolong-termmem-ory,thestudentneedstodevelopaschemainwhichtostoretheinformation.Theschemamaybenewlycreated,oritmayrelatetoanexistingschema.Thisislikelywhy encouraging students to relate new information to information that theyalreadyknowisaneffectiveteachingstrategy.80Themorecomfortableapersonisusingaschema,themoreautomaticusingthatschemamaybe,themoreworkingmemorymaybeavailablefornewinformationandlearning.81“Fromaninstruc-

citatorsontheLexisNexisandWestlawplatformstolearnandevaluatetheefficacyoftheBloombergLawcitator. 74. JohnSwelleretal.,Cognitive Architecture and Instructional Design,10eduC. psyChol. reV.251,252(1998). 75. Id. 76. smith & rAgAn,supranote23,at21. 77. Id. 78. Id.Somemightsaythatdeeperprocessingofinformationisassociatedwiththehigher-ordercognitiveskillsofBloom’staxonomy. 79. Swelleretal.,supranote74,at255. 80. Theuseofaschemacanbedistinguishedfromthestrategyofscaffolding.Scaffoldinggener-allyreferstothesupportthatalearnermayreceivefromateacherorafellowstudentinlearning.A“student’spartnercouldalsoprovideacoachingandscaffoldingrole....Theteacherdidnottakeanintentionalroleinprovidingconceptualormetacognitivescaffolding,butprovidedsupportwhenit was requested.” Jan Herrington & Ron Oliver, An Instructional Design Framework for Authentic Learning Environments,eduC. teCh. res. & deV.,Sept.2000,at23,40.“Studentsbenefit fromtheopportunitytoarticulate,reflectandscaffoldwithapartner,andtheywillseektheseopportunitiescovertlyiftheyarenotavailablebydesign.”Id.at42. 81. “With automation, familiar tasks are performed accurately and fluidly, whereas unfamil-iar tasks—thatpartially require theautomatedprocess—canbe learnedwithmaximumefficiencybecausetheworkingmemoryisavailable.”Swelleretal.,supranote74,at258.


tional design perspective, it follows that designs should not only encourage theconstruction of schemas, but also the automation of schemas that steer thoseaspectsofataskthatareconsistentfromproblemtoproblem.”82

¶27Another theory isbasedonmentalmodels,whichare like schemata,butwhichalso“containinformationabouttaskdemandsandtaskperformancesthatareusedforproblemsolving.”83Inshort,informationstoredinlong-termmemoryisorganized,andgoodteachingcreatesopportunitiesforstudentstoundertaketheorganizationalprocessofmovinginformationfromworkingmemorytolong-termmemory.Toputthattheoryinthecontextoflegalresearchpedagogy,asstudentsencounterandinteractwithnewresources,whichmayincludelearningnotonlythenamefortheresourcebutalsohowthesourceiscreated,itsauthority,howtoaccessanduse the source,andhowtoproperlycite the source, theyarecreatingschemataormentalmodelsforthenewinformation.

¶28 Creating opportunities for students to build schemata in which relatedsources are explicitly compared may help students more quickly learn resourcesand move that knowledge from working to long-term memory. Drill problems,thoughgenerallyoutoffavor,84mayalsoallowstudentstopracticeresearchskillssothataspectsoftheuseofparticularresourcesbecomeautomatic,increasingtheavailabilityofworkingmemorytoconsideraresearchproblem.Forexample,afterdevelopingaschemaandabitofresearchpractice,studentsmayautomaticallyseekthe“current-as-of” information for a statute or regulation, while a student justlearning about statutes will more likely have to stop and consider the question:“Whatnext?”beforerememberingtocheckthecurrentnessofastatute.

¶29What does all of this mean for the development of a pedagogy for legalresearch?85Keepinmindthatpedagogyhastwodefinitions.Onerefersto“theartor profession of teaching,” while the second refers to “preparatory training orinstruction.”86 A more complete definition of pedagogy as an art or professiondescribesitasthe“studyofteachingmethods,includingtheaimsofeducationand

82. Id. 83. smith & rAgAn,supranote23,at21. 84. “Allevidence,fromthelaboratoryandfromextensivecasestudiesofprofessionals,indicatesthat real competence only comes with extensive practice. . . . The instructional task is not to‘kill’motivationbydemandingdrill,but to find tasks thatprovidepracticewhileat the sametimesus-taininginterest.”JohnR.Andersonetal.,Applications and Misapplications of Cognitive Psychology to Mathematics Education,tex. eduC. reV.,Summer2000,at21–22. 85. Considerationoflearningtheoryinthelawlibrarianliteratureistypicallydiscussedintermsofstudentlearningstylesorpedagogy,thoughKristinGerdyproperlyusesthetermandragogytoreferspecificallytoadultlearners.Gerdy,supranote7,at73.Lawlibrariansareconcernedaboutwhetherstudentsareabletosuccessfullyintegrateresearchskills.Inherhistoricalreviewofthedevelopmentofthetheoryofandragogy,SharanMerriamexplains:

The fiveassumptionsunderlyingandragogydescribe theadult learneras someonewho(1)hasan independent self-concept and who can direct his or her own learning, (2) has accumulateda reservoirof life experiences that is a rich resource for learning, (3)has learningneedscloselyrelatedtochangingsocialroles,(4)isproblem-centeredandinterestedinimmediateapplicationofknowledge,and(5)ismotivatedtolearnbyinternalratherthanexternalfactors.

SharanMerriam,Andragogy and Self-Directed Learning: Pillars of Adult Learning Theory,inthe neW updAte on Adult leArning theory5(2001). 86. AmeriCAn heritAge diCtionAry of the english lAnguAge1299(5thed.2011).


thewaysinwhichsuchgoalsmaybeachieved.Thefieldreliesheavilyoneduca-tionalpsychology,ortheoriesaboutthewayinwhichlearningtakesplace.”87Thedevelopmentofapedagogyoflegalresearch,then,referstomorethansimplytheteachingoflegalresearch.Itincludesthestudyofteachingmethodsaswellastheinstructionalgoalsthataresetforlawstudents,anditencouragestheconsiderationoflearningtheoryfromotherfields,suchaseducationalpsychology,toensurethatstudentshave thebest learningexperiencespossible.PaulCallisterhascalled foropen dialogue and scholarly engagement within the law librarian professionalcommunity regarding the “underlying pedagogy at the heart of legal researchinstruction.”88

Benefits of Resource-Based Learning

¶30Lawstudentswillbenefitfromtheconstructivist,resource-basedlearningapproach, particularly if elements of problem-based learning are included.Unfortunately, the limited time available for basic or first-year legal researchinstructiondoesnotprovideenoughopportunityforstudentstobeexposedtothenumberandvarietyofproblemsthatwouldbenecessarytomeetfirst-yearlegalresearchrequirements.89However,aresource-basedapproachmayincorporatetheuseoflimitedrealorrealisticproblemstoincreasestudentinterestandthesensethatresearchskillsarerelevanttotheirfutureneeds.Providingstudentswithamixoftasksthatallowsforthepracticenecessarytoautomateresearchskillsandalsoencouragesthedevelopmentofschemataormentalmodelsregardingresearchiscritical.Thosetasksshouldincludeavarietyofinstructionalformatsandtypes—ranging from drill exercises90 to computer-assisted legal instruction to in-classgroupassignmentstoindividualproblems.Themixoftasksshouldoptimallypro-mote“notrotelearningbutlearningwithunderstanding.”91Studentsshouldhaveadequate opportunities to engage with resources, such that the use of thoseresourcesbecomesautomatic.

87. EdwinA.Peel,Pedagogy,enCyClopAediA britAnniCA,http://www.britannica.com/EBchecked/topic/448410/pedagogy(lastvisitedFeb.13,2012). 88. Callister,supranote22,at192,¶4.Generaleducationalpedagogicaltheories,suchascon-structivismorbehaviorism,mayunderliealegalresearchinstructor’sdecisiontoemphasizeabiblio-graphicorprocess-basedapproachtolegalresearch(thoughshemaynotrealizeitifshehasnotstud-iededucationalphilosophy).Aconstructivistapproachwouldlenditselftoprocess-basedteaching. 89. SeeNancyP.Johnson,Best Practices: What First-Year Law Students Should Learn in a Legal Research Class,28legAl referenCe serViCes Q.77(2009). 90. So-calledtreasurehuntresearchexercises,inwhichthestudentresearcherisgivenaproblemwithaclearandcorrectanswer, enabling the researcher to self-check theaccuracyof the researchprocess,mayalsobedescribedas“drillandkill.”Forexample,studentresearcherscouldbeaskedtofindparticularcasesfromparticularcourtsanddecidedonparticulardates,tofamiliarizethemwiththedigest system.Proponentsof the treasurehuntpoint to students’ ability togainconfidence intheirskillsaswellastheautomationofresearchskills.Thetreasurehuntexerciseiscomplementedbytheprocess-typeproblem,whichoftendoesnothaveaclear-cutanswer.Proponentsoftheprocessproblembelievethattheproblemspresentstudentswithrealisticresearchexperiences,particularlylearningtoaddresstheindeterminacyoflegalresearch. 91. Andersonetal.,supranote84,at31.


¶31Afulldiscussionoflegalpedagogyshouldincludeamajordeterminantinthesuccessofanypedagogy:themotivationofthelearner.Resource-basedlearningtechniques are particularly appropriate for adult learners, who benefit from themotivationalaspectsofthepedagogy.Studentmotivationisincreased“whentheybelievethattheoutcomeoflearningisundertheircontrol.”92Lawstudentsshould“learn most effectively when new information is connected to and built upon astudent’spriorknowledgeandreal-lifeexperiences,”andstudents“tendtodowellwhenallowed tohave somecontrolover the learningenvironment,andrespondbesttocollaborativelearningenvironments.”93Forapedagogyoflegalresearchtobesuccessful, itmustatahigh levelactivatestudent interest in learning.Interestmaybegeneratedanumberofways,rangingfromexplicitapplicationofproblemsandlearningtoreallifetoinvolvingstudentsinthecreationoftheirowneduca-tionalgoals.94“Studentsencouragedtoaskquestions[whenintroducedtoanewtopic of study] will learn more than a group of students deprived of thisopportunity.”95Mostimportant,“thereisalmostuniversalconsensusthatonlytheactivelearnerisasuccessfullearner.”96

¶32 Resource-based learning reflects elements of both process and biblio-graphic methods of teaching legal research. For example, students may be givenproblem-basedprojects(whichrequirethemtoreadthroughfactsanddeterminewhich resources to consult to answer the question, akin to a process-orientedapproach),97butinstructorsarealsoencouragedto“maketheresourcespartoftheculture of [their] teaching and learning,” advice which would be expected toaccompanyabibliographicapproachtoteaching.98

¶33Resource-basedlearningisapedagogyparticularlysuitedtolegalresearchcourses.Althoughlegalresearchcoursesmaybetaughtwithanemphasisonbiblio-graphic instruction or with an emphasis on the research process, in both casesstudents need to develop and build skills using resources to become successfulresearchers.Theneedtobeconversantwithbasicresourcesexistsforallresearchers,whether they are planning on print or electronic research. Additionally, legal

92. Hmelo-Silver,supranote27,at241. 93. GershonTenenbaumetal.,Constructivist Pedagogy in Conventional On-Campus and Distance Learning Practice: An Exploratory Investigation,11leArning & instruCtion87,90(2001). 94. Goal setting, like asking studentsquestions at thebeginningof a learningexperience, canraisestudentinterest.Unfortunately,instructionaldesigners,“especiallythosewhoholddeterministicbeliefsandsetgoalsaboutlearning,”haveadifficulttimeallowingstudentstogenerategoals.Id.at108.Perhapsthediscomfortexperiencedbyinstructionaldesignersarisesfromdistrustthatstudentswillgenerateadequategoalsandfearthatstudentswillnotbeabletoreevaluateandamendgoalsastheymaybefoundwanting.Thisiscontrarytotheresource-basedlearninggoalofdevelopingstu-dents’metacognitiveskills. 95. Gall,supranote43,at716. 96. Anderson et al., supra note 84, at 32. Though it is tempting for instructors to take theapproachthat“youcanleadahorsetowater,butyoucan’tmakeitdrink,”suchanattitudeisself-defeating and overlooks an instructor’s responsibility to create an educational environment thatmotivatesstudents. 97.Maharg&Paliwala,supranote10,at100. 98. Id.at102.


researchclassesalsoprovideinstructorswithanopportunitytoteachstudentshowtoapproachbothnew,unknownproblemsandnew,unknownresources.99

Course Design Decisions

¶34Whencreatingandteachingaclass,ateacher’spathisfilledwithchoices.Although some of those choices may be dictated—consciously or not—by thepedagogical theory espoused by the instructor, other choices may stem frominstructionalorinstitutionalmandates.Instructionalorpedagogicalchoicesmaybeasbasicaswhethertobeginwithelectronicorpaperresources,ortheymaybemorecomplicated,suchashowtoimplementanelectronicwebcourse.Underlyingteachingdecisionsarechoicesaboutcontent—whatmustbe included,andwhattheteacherbelievesstudentsshould“know”oncompletingthecourse.

¶35Formanycourses,includingresearchcourses,therearetextbooksreadilyavailable.A“good”textbookmaybechosenbasedonpopularity,theinstitutionalaffiliationofitsauthor,ortheinstructionalbiasesunderlyingthetextbook.100Forexample,aprofessormaybetemptedtouseThe Process of Legal Researchbecausethe title suggests a process emphasis, rather than a bibliographic instructionemphasis.Selectinga textbookwithout first consideringcoursedesign,however,mayleadtheinstructortoinvestinabookthatdoesn’tsupporthisinstructionalchoices.

¶36 The following sections describe some of these choices, first addressingtheoreticalconcernsregardingthedevelopmentofinstructionalgoals,theimple-mentationofinstructionalstrategiesthatprovidestudentsguidanceandsupportin their work, and course evaluation, and then offering suggestions regardingpraxis. Suggestions cover syllabus design, assignments, and student assessment.Theanalysisthatfollowspresumesalooselyconstructivistpedagogy.

Setting goals

¶37Thefirststepinplanningacourse,whetheritisdoctrinalorfocusesonaskillsuchasresearchordrafting,istoidentifylearninggoals.Thiscriticalstepisnotasobviousasitsounds.Tobeginwith,whatarelearninggoals?Learninggoalsarethegoalsthatastudentshouldhaveachievedonsuccessfulcompletionofthecourse.Soundslikenonsense,right?Rephrasingthatdefinitionmakesthemeaningabitmoreapparent,andmuchmorehelpful:Astudentshouldbeabletoperform

99. Berring’sdiscussionof the“functionalapproach to legal information,” inwhichresearch-ersunderstandthenatureoftheinformationitself,notthespecificformatinwhichitisdelivered,isgermanetothechangingnatureofthedeliveryofinformationtoday.Berring,supranote2,at3.Studentresearchers—andlawlibrarians—areconstantlyadjustingtochangingformatsandchanginginterfaces.TheWestlawNextplatformandconcomitantdebateareoneexampleoftheongoingnatureofchange.See RonaldE.Wheeler,Does WestlawNext Really Change Everything? The Implications of WestlawNext on Legal Research,103lAW libr. J.359,2011lAW libr. J.23. 100. The commonly used textbooks are identified in the literature. Ann Hemmens, Advanced Legal Research Courses: A Survey of ABA-Accredited Law Schools,94lAW libr. J.209,228–29,2002lAW libr. J.17,¶¶49–50.See alsoNancyP.Johnson,Should You Use a Textbook to Teach Legal Research?,103 lAW libr. J. 415, 428–35, 2011 lAW libr. J. 26,¶¶ 48–85 (reviewing a number of recent legalresearchtexts).


the course learning goals, meeting certain performance standards, on successfulcompletionofacourse.Aninstructorwhoishopefulthat,bytheendoftheclass,studentswillbeabletorecognizethatafederalregulationisthepropersourcetoconsulttoansweraresearchquestionandtolocatetheregulationonpointfortheresearch problem, may have as a learning goal that students will understand theauthority of federal regulations and be able to navigate the Code of Federal Regulations, includingthestepsnecessaryforupdating.Setting learninggoals forstudentsbeforecommencingthecoursewillmorelikelyensurethattheinstructionwillmeetthegoals.Ideally,aninstructor’soverallcoursegoalsaremetbythesub-sidiarygoalsassociatedwithunitsandindividuallessons.

¶38Recognizingtheimportanceofcoursegoalsisrelativelyeasy,buthowdoesonegeneratethosecoursegoals?Itdepends.Theguidingquestioniswhatthestu-dentshouldbeabletodo(orknow)attheendofthecourse.Doesthecoursepre-pare the student for a subsequent course?Are there several sectionsof the samecoursetaught,suchthatthestudentsacrossallsectionsshouldhaveacorecommonexperienceorknowledge?Shouldthestudentswhocompletethecoursebeabletomeetskills/knowledgelevelsofpeersatotherinstitutions?Afirst-yearcriminallawcourse,forexample,raisesallofthesequestions—thecoursemaypreparestudents,atabasiclevel,forasubsequentcriminalprocedurecourse.Theremaybemultiplesectionsofthecourseinoneinstitution,andallthestudentsshouldlikelybeabletodefinemens reaandactus reus,regardlessofthetheoreticalbiasesoftheinstruc-tors.Atleastonemeetingofallthecourseinstructorswillhelpensurethattheyalladdress the basic issues.Additionally, the students are likely paying their tuitionwiththeexpectationthattheywill learnwhattheyneedtoknowtopassthebarexamandsuccessfullypracticelaw;thisexpectationstemsfromareasonablebeliefthatallaccreditedlawschoolswillteachcertaincorematerials.101Thesesameques-tionsariseinthecontextofresearchcourses.

¶39Oneoftheeasiestwaystoidentifylearninggoalsistoconsidereducationalstandards.Althoughstateshavedevelopededucationalstandardsandgoalsforstu-dentsinelementaryandsecondaryeducation,102suchstandardsandgoalshavenotbeendevelopedforlawstudents.103Inthelawschoolcontext,educationalstandards

101. Student expectations are complex. A professor may reasonably guess that students willexpecttolearnmaterialsnecessaryforpassingthebarexamandsuccessfullypracticinglaw.Butotherfactorsmayaffectstudents’expectationsaswell.Astudent’sreasonsandmotivationsforenrollinginacourse—atasbasicalevelaswhetherthecoursewasrequiredoranelective—mayaffectthestudent’senthusiasmandgoals.Forexample,astudenttakingaone-creditweekendresearchcoursebecausethatonecreditwillenablethestudenttograduatethatsemestermaysimplywantto“getthroughit,”whileastudentelectingtotakeanadvancedresearchcoursetoprepareforasummerjoborexternshipmaybringdifferentexpectations. 102. State standards exist for many subjects and for every grade. In New York State, forexample,standardsaremeanttohelpteachersidentifywhattheirstudentsneedtoknowandbeabletodo inorder to succeedonmandatory state testing.New York State Learning Standards and Core Curriculum,nysed.goV(lastupdatedJan.23,2012),http://www.emsc.nysed.gov/ciai/cores.html. 103. One could argue that the minimum standards for law students have been set by themultistatebarexamination,asthattestisthemostcommonsharedexperienceoflawstudentsacrossthecountry.Thatexam,however,doesnotaddressresearchskills.


areintheirinfancy.104Absentexternalstandardsthathavebeenratifiedbyaninsti-tutionsuchastheAmericanBarAssociationortheAssociationofAmericanLawSchools,aninstructorinaresearchcoursemaybeginbyidentifyinganycommu-nitystandardsthatmayexist.105Thankfully,toolsexisttoassistinthedevelopmentofcoursegoalsforresearchcourses.

¶40Professionalassociations,suchasAALL,mayprovideguidance.Followingthereleaseof theMacCrateReport, theAALLResearchInstructionCaucuspro-duced the Core Legal Research Competencies, setting forth the information thatstudents should know about researching by the time they graduate from lawschool.106Anothersourcetoconsiderisprofessionalliteratureonthesubject.Forexample, Nancy Johnson memorialized her view of what first-year law studentsshouldlearn,basedonhertwenty-fiveyearsofteaching.107Onthepremisethatasyllabus will reflect learning goals, one might also consult syllabi for researchcourses.ThesemaybeaccessedbysearchingtheWorldWideWeb,108byconferringwithcolleaguesinperson,orbysolicitingsyllabionalistserv.

¶41Generatinggoalsforafirst-yearlegalresearchcoursedoesnothavetobecomplicated.109Goalsarewrittenwithstudentperformanceinmind.Whengener-atingasetofgoals,startwithanideaofwhatstudentsshouldknow,orwhattheyshouldbeabletodo,andthenconsiderBloom’staxonomyandtheverbsassociatedwiththedifferentcognitivethinkinglevels.Forexample,foralessonaboutusingannotated statutes, a lower-level goal might be that students will be able to“describethetypesofannotationsonemayfindinanannotatedstatute.”Ahigher-levelgoalmightbe that studentswillbeable to“evaluatewhetheranannotatedstatuteoranofficialcodewouldbeamoreappropriateresourcetoconsult,givenavarietyofcircumstances.”

¶42Whendevelopingaclass,aninstructormayfinditeasiertodevelopmicro-levelgoals(e.g.,atthelessonlevel,asintheexampleabove)andthenbuildthemup tomacro-level goals, suchas“studentswillbeable to consult a state statute,usingsearch/indextermstoidentifytherelevantsection(s),andevaluatethestat-utetodetermineitsapplicabilitytoaresearchquestionandthecurrentnessofthestatute.”Othersmayfinditeasiertobeginwithbroadlearninggoalsandbreakthebroad goals down into component goals. However developed, learning goals

104. The American Bar Association sets forth standards and Rules of Procedure forApprovalofLawSchools,whicharerelevantforlawschoolaccreditation,butthosestandardsarenotveryhelpfulfordesigninglearninggoals.SeeAm. bAr Ass’n,supranote17. 105. This discussion presupposes an instructor who is either new to teaching research orwhoisteachinganewcourse.Aveteranteacherwithsignificantexperiencemaybenefitfromconsid-eringinstructionalgoals,butmaynotneedtodomuchworktoidentifycommunitystandards. 106. Core legAl reseArCh CompetenCies, supra note 69. Because the competencies expressidealstudentknowledgeongraduation,theymayhavelimitedvaluefordeterminingwhatshouldbeincludedinafirst-yearlegalresearchcourse.Presumablysomeoftheknowledgeorskillscapturedbythecompetencieswouldbelearnedinlaterlawschoolcoursesoractivities. 107. Johnson,supranote89. 108. E.g., AALL’s Foreign, Comparative, and International Special Interest Section makessyllabiavailable.2011SyllabiandCourseMaterialsDatabase,available athttp://www.aallnet.org/sis/fcilsis/syllabi.html(lastvisitedMar.22,2012). 109. Goals for an advanced legal research course would be different, because students in anadvancedcoursearepresumablymoreexperiencedthanfirst-yearlawstudents.


shouldbemadeexplicittostudents,sotheyknowwhattoexpecttolearn.110Itmaybe thatoneclass sessionhas severalgoals—oreven several separate lessons.Thegoals(andlessons)presentedinonedayofinstructionmayaddressseveraltopics.Ideally,nosingle lessonshouldbebrokenupintotwoclasssessions,buta largergoalmayhavesubsidiarylessonsthatspantwoclasssessions.Insuchacase,spend-ingaminuteortwotoreviewthepriorlessonisagooduseofclasstime.

¶43 Once the learning goals are set, the instructor may engage in backwardplanning, identifying the intermediate steps necessary to reach the educationalgoal.111Forexample,ifstudentsshouldlearnhowtolocatecasesusingadigest,112theinstructorneedstoplanwheninthecoursetointroducethesubjectofdigests.Tounderstandoruseadigest,astudentneedstounderstandtheelementsofacaseandtheworkingofthereportersystems.Consequently,theintroductionofdigestsshouldhappenaftertheintroductionofcases.113

¶44Thelogicalsequencingoflearninggoalsforacoursewillideallybereflectedinacourse syllabus.114Thoughgoalsmaybe inferred fromthe syllabus,abetterpracticeistostatethemexplicitly.Asyllabusmaycontainasectioncalled“CourseGoals,”inwhichtheinstructordescribeswhatstudentswilllearnintheclass.Thesyllabus may then be broken down into units (e.g., cases, statutes, secondarysources), and each unit and individual lesson should have its own objectives.Consistently generating learning goals and expectations regarding what studentswilltakeawayfromaunitorlessonensuresthatstudentsknowhowtomeetthecoursegoals.115

110. When learning goals are made explicit to learners, learners are better able to evaluatetheirprogresstowardreachingthegoalsandmaybeabletoadjustaccordingly. 111. See, e.g., Bay Area School Reform Collaborative, Inquiry in Curriculum Design 3 (Oct.5,1999 rev.),http://www.sfsu.edu/~teachers/download/Inquiryframework.pdf.This isoneexampleoftheabundantpedagogicalmaterialsprovidedforK–12educatorsthatarefreelyavailableontheweb.Thoughsomeadaptationofthematerialsmaybenecessaryforadults,manyofthecoreinstruc-tionalstrategiesorplanningideasaresound. 112. “Unfortunately, most students do not share the professors’ passions for the West keynumbersystem.Somestudentsneverreallyunderstanddigests,whichisunfortunatebecausedigestsprovideaneffectiveandefficientmethodfor findingcases.”Johnson,supranote89,at85.Thoughstudentsmaynotunderstandthedigestsystemasitappearsinprint,studentsusingtheonlineinter-facefortheLexisNexisandWestlawcasedigestsystemsmaystumbleacrossthevalueofthesesystemsbyclickingonthehyperlinks.HowthenewWestlawNextinterfacewillaffectstudentsearchers’useoftheWestdigestsystemremainstobestudied. 113. Students come to research class familiar with the idea of cases, at least, even if they haveneverseenawrittenjudicialopinionpriortotheirfirstdayoflawschool.Theyaremuchlesslikelytohaveinteractedwithadigestsystem.Keepinginmindthatitiseasierforstudentstolearnwhenbuildingonexistingknowledge,beginningwithcasesandfollowingwithdigestsisappropriate. 114. In his interesting discussion of Bloom’s taxonomy, Callister includes a table relatinglearningtypes(Bloom’staxonomylevels)toresearchcompetenciesandactivities.Heexplainsthat“itisthebeginningofasyllabus.”Callister,supranote22,at218,¶43.Thoughhistableisquiteuseful,particularlywithregardtoassessmentideas,othersmightfinditmorehelpfultodevelopasyllabusbeginningwithlearninggoals(closelyrelatedtohisstudentcompetencies),ratherthanwithlearningtypes.Hischart suggests that learning isa linearprocess, inwhichstudentsbeginwith lower-levelthinkingandmoveultimatelytohigher-levelthinkingtasks.Infact,learningofteninvolvesrevisitingpriorknowledgetobuildnewknowledgeandskills. 115. Ontheotherhand,toomuchemphasisoncoursegoalscanbedetrimental.

Publicationofpre-specifiedlearningoutcomesincoursematerialsmayinadvertentlystiflecreativ-ityandoriginalityinbothstaffandstudents.Usedrigidly,thereisadangerthatlearningoutcomes


¶45Bloom’staxonomycanbeusefulwhengeneratinglearninggoalsbecausethecognitiveskills,fromthelowestlevelofthinkingtothehighest,areassociatedwithverbsdescribingstudentlearningbehaviors.116Associatinglearninggoalswithparticular student behaviors will enable both the student and the instructor toevaluate whether the learning goal has been met. For example, the lowest levelcognitiveskillofrememberingcanbeassociatedwiththeverb“recall,”so thatastudentwhohasparticipatedinalessonaboutcaselawresearchoughttobeabletorecallthecomponentpartsofajudicialopinionbytheendofthelesson.Asec-ondary benefit of clearly stated learning goals is that they encourage studentengagementwiththematerial.Learninggoalsmaybothmakecleartostudentsthelacunae in their knowledge and provide students with the ability to assess theirownprogresstowardfillinginthegaps.

Instructional Strategies

¶46Educationaltrainingmaterialssuggestthat,afteridentifyinglearninggoals,aninstructor’snextstepindesigningacourseistheselectionofaninstructionalstrategyorstrategies(alsocalledinstructionalmethods).117Instructionalstrategiesaredescribedinavarietyofways.Abriefandsimpledefinitionis“decisionsaboutteachingsequencesandtactics.”118JohnsonandAragon,whodevelopedanonlinemaster’sdegreeprograminhumanresources,identifiedthefollowingstrategiesasnecessaryincreatinganeffectivelearningenvironment:(1)addressindividualdif-ferences, (2) motivate the student, (3) avoid information overload, (4) create areal-lifecontext,(5)encouragesocialinteraction,(6)providehands-onactivities,and(7)encouragestudentreflection.119Forpurposesofthisdiscussion, instruc-tionalstrategiesaretheapproachesanddecisionsmadebyaninstructortoensurethatstudentsareabletoengagewith,comprehend,andlearnmaterial.

¶47Strategiesused in theclassroommayvarydependingon thegoalof thelesson.120 Although constructivist theory places a premium on the preexistingknowledgeofthelearnerandplacestheinstructorinthepositionoffacilitator,theinstructormaychoosetousedirectinstruction.Anotherstrategymightbetoaskstudentsto“think,pair,share.”Inthistypeofexercise,studentsaregivenaresearch

becomethedriverofclassroominteractionsandpreventdiscussionofideasorquestionsthatdonotclearlyrelatetothesetoutcomesforthecourse/module.

Angela Maher, Learning Outcomes in Higher Education: Implications for Curriculum Design and Student Learning,3J. hospitAlity, leisure, sport & tourism eduC.42,49(2004). 116. Overbaugh&Schultz,supranote60. 117. Seesmith & rAgAn,supranote23,at6. 118. ryAn et Al.,supranote16,at47. 119. Scott D. Johnson & Steven R. Aragon, An Instructional Strategy Framework for Online Learning Environments,100neW direCtions for Adult And Continuing eduCAtion31,34(2003). 120. Searching Google for instructional strategies provides over a million results thatmayinspireinstructorsastheyplantheirclasses.Althoughmanyonlinecoursedesignmaterialsarehostedbyschooldistricts, someuniversitiesandcollegesofeducation,not tomentionothernon-profitsites,makecoursedesignmaterials, includinggoal-settingandinstructionalstrategies, freelyavailable.See generallyGlossaryofInstructionalStrategies,http://glossary.plasmalink.com/glossary.html(lastupdatedAug.28,2010)(containing988instructionalstrategies);SaskatoonPublicSchools,InstructionalStrategiesOnline,http://olc.spsd.sk.ca/de/pd/instr/index.html(lastvisitedJan.3,2011)(linksfromthispagedescribedirect instruction, interactive instruction, indirect instruction, inde-pendentstudy,experientiallearning,andinstructionalskills).


problem and a set amount of time to think about the problem independently.Studentsthenpairuptodiscusstheproblem,andfinallyacoupleofstudentpairsareinvitedtosharetheiranswerswiththeclass.Inaresearchcontext,theanswermight be a research process, such as, “We started with the United States Code Annotated,butwerealizedthatweneededaregulation,soweturnedtotheCode of Federal Regulations,whichhadouranswer.Last,weupdatedtheregulationontheInternet.”

¶48Encouragingstudentstoputtheirunderstandingofasubject intoactionusingagraphicorganizerisanothervaluableinstructionalstrategy.Graphicorga-nizersare“visualdisplays teachersuse toorganize information inamanner thatmakestheinformationeasiertounderstandandlearn.”121AnexampleofagraphicorganizerisaTchart(achartwithtwocolumnsandaheadingorquestionontop),which may be used for comparisons.122 For example, students often wonderwhether LexisNexis or Westlaw is “better.” Asking students to test the services,evaluate theircitatorproducts(ahigher-orderskill),andchart theresults inaTchartallowsthemtomoredeeplyprocessinformation.Assigningstudentstocreateor use a graphic organizer encourages them to develop metacognitive skills—“help[ing]studentsworkthroughtheideasandconnections.”123

¶49Strategiesofteninvolvethecreationofaproduct.Thelearningproductmaybeintangible,suchasthethink-pair-shareresponse,oritmaybetangible,suchasachartcomparingShepard’stoKeyCite.Theselearningproductsmaybeusedbyboththeinstructorandthestudenttoevaluate—orassess—learning.124Thelearn-ingproduct,suchasananswertoaquestionorfollow-upquestion,maybeinfor-mallyassessed.Ananswertoawrittenexammaybeformallyassessed.

¶50 One of the more difficult tasks in teaching is evaluating student under-standing.Amajorsourceofthisdifficultyisthattheprocessofevaluationisnevercomplete. While teaching, whether acting as a facilitator or providing directinstruction, an instructor must continually assess student comprehension andinteractionwiththematerial.Whenthestudentsarequiet,doesthatreflectdeepcontemplationofahigher-orderquestion,ordoesitmeantheyareintentlyreadingthelatestcelebrityanticsonFacebook?Howdoesaninstructorfindtherightbal-ancewhenpartoftheclassunderstandsthelessonandwouldbeabletoperform

121. Gloria A. Dye, Graphic Organizers to the Rescue! Helping Students Link—and Remember—Information, 32 teAChing exCeptionAl Children 72, 72 (2000) (quoting e.l. meyen et Al., strAtegies for teAChing exCeptionAl Children in inClusiVe settings132(1996)).Graphicorganizersmaybeespeciallyusefulforteachingstudentsrelationalknowledge.SeeVonnieM.DiCecco&MaryM.Gleason,Using Graphic Organizers to Attain Relational Knowledge from Expository Text,35J. leArning disAbilities306(2002). 122. Graphic organizers are often used in elementary and secondary education, and manyare freely available on the web. See Houghton Mifflin Harcourt, Classroom Resources: GraphicOrganizers,http://www.eduplace.com/graphicorganizer/(lastvisitedNov.30,2011). 123. Niedringhaus,supranote37,at117. 124. The Carnegie Report uses different language; rather than providing interim assess-ment,theinstructorcoachesstudents,“providingguidanceandfeedback.”sulliVAn et Al.,supranote2, at61.Whichever language isused, thepedagogicalpurpose is that students receive feedbackontheirperformanceastheyarelearning,sotheycanengageinthemetacognitiveanalysisnecessarytoimprovetheirperformance.


thelearninggoalandtherestoftheclassdoesnotandwouldnot?Ifaninstructoris not evaluating student success, both informally and formally, she is not evengoingtobegintoaskthesecriticalquestions.

Assignments, Rubrics, and Assessment

¶51Assignments,rubrics,andassessmentare integrallyrelated.Theymaybeimaginedasthreesidesofatriangle:eachsideisnecessaryforthetriangletoexist,andthoughthesidesmaylookalotalike,theyaredistinguishable.

Assignments

¶52 For purposes of this discussion, an assignment is a task assigned by aninstructor,thefunctionofwhichistoreinforcethelearningobjectiveforaparticu-larlessonorunit.Afewexampleassignmentsincludeansweringaresearchques-tion, writing a description of a research process, or participating in an onlinecoursediscussion.Theassignmentshouldreflecttheinstructor’slearninggoals;anassignmentthatrelatestoasubjectorissueunrelatedtothelearninggoalsislikelyawasteoftime.125Assignmentsaretypicallylistedinasyllabus,andtheportionofthecoursegradethatisattributabletoaparticularassignmentisalsomadeclearinthesyllabus.

Rubrics

¶53Instructionalrubricsarerarelyseeninlawschool;126theyare,however,veryhelpful in making clear to students an instructor’s expectations about perfor-mance.Aninstructionalrubricisashortdocument—ideallyoneortwopages—that“giv[es]studentsinformativefeedbackabouttheirworksinprogressand...give[s]detailedevaluationsoftheirfinalproducts.”127Generally,arubricisorga-nizedasatable,withassignmentqualityalongoneaxisandparticularcriteriafortheassignmentalongtheother.Therubricshouldbegeneratedbytheinstructoranddistributedtothestudentsatoraboutthesametimeastheassignment.

¶54Rubricshaveseveralinstructionalbenefits.Thefirstisclarity.Studentsandinstructor alike should see the alignment of the learning goals with the criteriadescribedintherubric.Studentsundertakeassignmentswithaclearerunderstand-ingoftheirinstructor’sexpectations,andtherubricencouragestheinstructortoconsiderwhetherthequestionsaskedbytheassignmentare,infact,thequestionstheinstructorintendsthestudentstoanswer.Studentsappreciateunderstandinginadvancetheissuesofconcernforaparticularassignment.128If,forexample,an

125. In addition to wasting students’ time completing the task and the instructor’s timegradingorreviewingstudents’work,anassignmentunrelatedtocoursegoalsrunstheriskofmakingstudentsthinkofalltheassignmentsinacourseasawasteoftime—eventhosethatareintegraltothecompletionofthecoursegoals. 126. Students in doctrinal courses are more likely to be given model answers or old examsforpractice,ratherthanrubrics. 127. Heidi Goodrich Andrade, What Do We Mean by Results? Using Rubrics to Promote Thinking and Learning,eduC. leAdership,Feb.2000,at13,13. 128. Gerdynotesthat“legalresearchteachersmustnotonlycreate learningoutcomesbutalsopublicizethembyprovidingtheirstudentswithalistofimportantconceptsandskillsthattheywill


instructor isnotconcernedwithcitationstyle for in-classassignments,butcaresdeeplyaboutitonatake-homeassignment(andgradesaccordingly),therubricsforin-classassignmentswouldmakethatcleartostudents,whocouldthenfocustheirlearningenergyappropriately.

¶55Rubricshavealsobeenshowntosupportstudentlearning.Withinthecon-textofaconstructivistpedagogy,arubricencouragesstudentstodevelopmetacog-nitiveskills.Asnotedearlier,studentsmayhavedifficultyrealizingthattheyhavefound“theanswer,”orsometimesevenananswer,toaparticularresearchquestion.Whenusingarubric,studentscanstopandevaluatetheirprogresstowardcomplet-inganassignment,129encouragingthemtomonitortheirownthinkingabouttheassignments130andtheirprogresstowardachievinglearninggoals.131Inadditiontodevelopingmetacognition,rubricshavebeenshowntoimproveboththedevelop-mentofcontentlearningandcriticalthinking,andthedevelopmentofskills.132

Assessment

¶56 The third leg of the triangle is assessment.133 As discussed earlier, aninstructor may assess student progress or understanding formally, with assign-mentsandexaminations,orinformally,throughquestioning.Likeassignmentsandrubrics,assessmentshouldreflectlearninggoals,toensurethatinstructionaltimeandstudentout-of-classworktimearebothbeingusedtopromotestudentlearn-ing.Assessments thatarenotalignedwithgoalsareneither fairnorequitable.134

beresponsibleforandthatwillbemeasuredinanassessment.Presentingthisinformation‘upfront’iskey.”Gerdy,supranote38,at73–74,¶55. 129. Students who have not used rubrics in their prior education will benefit from a brieflessoninhowtoreadandusearubric.Ihaveusedrubricsinanupper-divisionlegalresearchcourse,withoutexplicitlydescribingtostudentshowtheycouldusetherubricstotheiradvantage.Duringanofficevisitregardinganassignment,astudentindicatedthatitwouldhaveaffectedhisperformanceifhehadactuallyreadtherubricinadvanceofcompletingtheassignment.Inalawschoolsetting,studentsmayalsobenefitfromparticipatinginthecreationofarubric. 130. SeeAndrade,supranote127,at15. 131. James W. Pellegrino, Rethinking and Redesigning Curriculum, Instruction andAssessment:WhatContemporaryResearchandTheorySuggests6 (Nov.2006),available athttp://www.skillscommission.org/wp-content/uploads/2010/05/Rethinking-and-Redesigning.pdf. 132. Andrade, supra note 127, at 16. Although the studies involved middle school students,thereisnoreasontobelievethatrubricswouldnotprovidesimilarvaluetolawstudents. 133. Gerdy describes assessment as answering two questions: “What have my studentslearnedandhowwellhave they learned it?Howsuccessfulhave Ibeenataccomplishing thegoalsandobjectivesIhaveset(forasingleclassperiod,aparticularskillsset,oranentirecourse)?”Gerdy,supranote38,at65,¶25. IdisagreewithGerdy’scharacterizationofbothquestionsas relating toassessment.Inmyview,thequestionofstudentlearningisassessment.Thesecondquestion,abouttheinstructor’ssuccessataccomplishinggoalsandobjectives,iscourseevaluation.Gerdy’sdiscussionoflearner-centeredassessment,however,isenlightening.Id.at68–78,¶¶38–68.

Assessment of learning goals should not be confused with assessment of teaching goals(i.e., course evaluation). If one’s teaching is to be observed and evaluated, whether for an annualevaluation,promotion,ortenure,itisadvisabletoreviewtheevaluationformpriortothescheduledobservation. An example of a form used in the teaching development program at the UniversityofMissouri–KansasCitySchoolofLaw(UMKC) is instructive.UMKCLawTeachingObservationEvaluationForm,http://law2.umkc.edu/faculty/profiles/glesnerfines/Classroom%20Observation%20Form.pdf(lastvisitedMar.22,2012). 134. Lisewski&Settle,supranote21,at109;Pellegrino,supranote131,at9.


Additionalfactorstoconsiderwhenplanningassessmentinalegalresearchcourseincludewhetherthemeasurementiseffective(doesitmeasurewhatitpurportstomeasure), whether the assessment may be used to improve both teaching andlearning,andwhethertheassessmentprovidesasnapshotoracontinuingpictureofstudentdevelopmentovertime.135

¶57 Assessment may be used in a legal research class to both excite studentinterestandevaluatestudents’priorknowledgeandunderstanding.Apreliminaryassessment,giventostudentsbeforeclassbegins136oronthefirstdayofclass,offersseveralbenefits.Theassessmentresultscanhelpaninstructorplantheamountoftimenecessarytoadequatelyaddressrequiredtopics.Itmayalsohelpaninstructoridentify students who would be able to explain research process concepts orresearchresourcestootherstudents.137Students,uponrealizingthedepthoftheirignorance,maybemoremotivatedtoactivelyparticipateinacourse.138Aprelimi-naryassessmentmaycomplementacourse’s finalassessment.Bycomparing thetwo assessments for a particular student, it is possible to evaluate the degree ofimprovement—thestudent’ssuccessatachievingthecourse’slearninggoals.139

¶58 Assessment can be used to facilitate individualized instruction. Ideally,studentsshouldreceivefeedbackonall theassessmentstheycomplete.Feedbackcan be verbal correction of a misunderstanding demonstrated by a student’sanswer to an in-class question.140 Alternatively, it can take the form of detailedcomments on a research exercise, perhaps combined with a model answer or arubric.Instructionaltechnologiescanbeespeciallyusefulinthisrespect.141Coursemanagementsystemsenableinstructorstoprovideimmediatefeedbackonassess-mentsbyincludingspecificexplanationsaboutanswersandwhytheyare(orarenot)correct.AnexampleofthisistheexercisesavailableonlineattheCenterforComputer-Assisted Legal Instruction (CALI).142 Additionally, instructors maydirectstudentstoparticularresourcesthatwouldimprovetheirunderstandinginanareainwhichtheyfailedtoachievelearninggoals.

135. Pellegrino,supranote131,at8. 136. By giving an assessment as an assignment prior to the first class session, the instructorcanbettertargettheinitialclasssessiontothestudents.Theassessmentmayalsoincludesomeques-tionsthatwillhelptheinstructorrememberstudents’names. 137. Encouraging students to explain difficult concepts to each other is an effectiveinstructional strategy. Sometimes students are more attentive to and better able to understand anexplanationdeliveredbyapeer,ratherthanbyaninstructor.Conductingapre-assessmentmayhelptargetstudentswhobringvaluable(andaccurate)priorknowledgetoaclassroom.Alternatively,apre-assessmentmayhelpaninstructorcreatelearninggroups.Studentsmaybegroupedandassigneddifferenttasks,dependingonthepriorknowledgetheybringtothecourse. 138. The preliminary assessment may both gain students’ attention and help them to seetherelevanceoftheinstructionalgoals.Theteachinginresponsemaybuildconfidenceandsatisfac-tion.SeeNiedringhaus,supranote37,at115–16. 139. According to Ann Hemmens’s survey, only 26.8% of advanced legal research coursesusearesearchexamtoevaluatestudents.Hemmens,supranote100,at234,¶58.Hemmens’ssurveyisfrom2000though;assessmentstrategiesmayhavechangedsincethen. 140. Green,supranote6. 141. Pellegrino,supranote131,at11–12. 142. CALI makes interactive, online lessons on a variety of topics available to law students.Anumber of research skills lessons are available, some of which are targeted to specific subjects orjurisdictions.SeeCALI,http://www.cali.org(lastvisitedJan.4,2012).


¶59Finalexaminationsareatypicalformoflawschoolassessment.Doctrinallawschoolcourses,particularlythoseinthefirstyear,assessstudentsprimarilybyacomprehensive finalexamat theendof the semester.143Some first-yearcourseprofessorsmayofferstudents in first-yearcoursesanopportunity totakeamid-termexam,thusgivingthemexposuretothehigh-stakestestingthatistypicalinlawschool.

¶60Otheroptionsmayexist forassessingstudents ina legal researchcourse.Nancy Armstrong advocates that instructors of legal research courses considerimplementinganoralfinalexam.Sheexplainsthatthegoalofsuchanexamistohavestudentstalkaboutresearchtechniquesoractuallydemonstratetheirresearchstrategiesandskills.144Sheadvisesinstructorswhowishtotrythismethodthattheyshouldestimatetheamountoftimetheythinkisneededtocompletetheexamandthendoubleit.Whenproctoringherexams,sheusuallyschedulesstudentsforonehour, with forty-five minutes spent working in the library and fifteen minutesdebriefingtheexamtogetherintheoffice.145Suchanexammaypleaselearnerswhohaveavarietyof learning styles,but itmaybemore time-consuming toproctorthanamoretypicaltake-homeresearchproblemsetorpathfinder.

¶61Apedagogicalquestionnotyetdiscussed,butraisedbyassessment,iswhatconstitutesa“right”answer.AssumeaninstructordesignedaquestionthatwouldrequireastudenttoidentifyasectionintheCode of Federal Regulations,readthesection,andprovideananswertoalegalquestion.Iftheinstructor’spedagogicalgoalissimplythatstudentscanidentifyappropriateresourcesandnavigatethoseresourceswhen facedwitha researchproblem, the studentmight earncompletecreditforidentifyingaproperresourceandlocatingtherelevantsection(s)inthesource, regardless of the accuracy of the answer to the legal question. Anotherinstructor,havingdesignedthesamequestion,mightonlygivepartialcreditforthesamestudentanswer,onthebasisthatthestudentfailedtocorrectlyreadandana-lyzethesourcewhenansweringthelegalquestion.Thebetterpracticewouldbetoconsider the accuracy of the analysis, at least in part because the use of legalresourcestoanswerquestionsrequiresanalysisandevaluationatmanystagesintheresearch process (developing the initial research query, reviewing results forresponsivenesstotheproblem,revisingthequery).146Proponentsoftheopposingviewmightarguethatthelegalresearchinstructor’sjobistoteachresearch,ratherthanwritingandanalysis.Regardlessofthepedagogicalperspectiveoftheinstruc-tor,theassessmentisnotcompleteifitdoesnotincludeadequatefeedback.147

143. According to Hemmens, advanced legal research courses are remarkably standardizedintheirmethodsofassessment.Thoughthereareavarietyofassessmentoptionsusedinadvancedlegalresearchcourses,88.7%ofthecoursesuselibraryexercisesorresearchassignments,while69%ofthecoursesrequirestudentstocreatepathfinders.Hemmens,supranote100,at234tbl.15. 144. Nancy A. Armstrong, “Tell Me More About That . . .”: Using an Oral Exam as a Final Assessment Tool,25legAl referenCe serViCes Q.,nos.2/3,2006,at117,119. 145. Id.at119–20. 146. In the interest of transparency, students should understand—from instructions or arubric—whetherornottheaccuracyofanalysiswillbeafactorinthegradingofthequestion. 147. Richard Higgins et al., The Conscientious Consumer: Reconsidering the Role of Assess-ment Feedback in Student Learning,27stud. in higher eduC.53,54(2002).


¶62 Legal research courses are not required to follow the doctrinal coursemodelofformalassessment,inwhichstudentsaregradedbasedsolelyonasum-mative examination at the end of a course; they have a panoply of assessmentoptionsavailable.148Astudentcouldbeassessedbasedonhisperformanceinrela-tiontoinstructorquestioning—thestudent’sanswerscouldbethebasisofahighergradeattheendofthesemester.Theinstructormightalsogiveadditionalassign-mentsthatarethebasisofthestudent’sfinalgrade.Additionalassignmentscouldincludetreasurehuntquestions,process-basedresearchquestions,completionofCALI lessons, or required “lab time” in which students are taught computer-assisted legal research skills. The doctrinal course model provides students withlittleornofeedbackabouttheirprogresstowardachievinglearninggoals;awell-designed legal research course should provide students with ongoing feedback,encouragingthedevelopmentofschemataandmetacognitiveskills.

Conclusion

¶63Thepedagogyoflegalresearchisanimportantissueforlawlibrarianstoconsider, in no small part because law librarians are experts in legal research,includingtheresourcesandstrategiesthatmaybestbeusedtoansweraresearchquestion. Even without formal pedagogical training, law librarians can improvetheir teachingbyreadingprofessional literatureandengaging in theburgeoningconversationaboutteaching.Byconsideringboththetheoryofteachingstrategies,suchastheuseofscaffolding,schematheory,andtheroleofquestioning,aswellasthepracticalapplicationofteachingstrategies,suchasthethink-pair-sharetech-niqueandrelatedquestioningstrategies,lawlibrarianscanimprovetheireffective-nessinthelegalresearchclassroom.Further,byarticulatingcoursedesigndecisionsthrough learning goals and the use of rubrics and assessments, legal researchinstructorscanprovidestudentswithhelpful tools fordevelopingmetacognitiveskills,enablingstudentstocontinuetoimprovetheir legalresearchskills later inlawschool.

148. The options implemented may be limited by the type of class offered; an advancedlegalresearchcoursewithanenrollmentoffourteenstudentslendsitselftodifferentassessmenttoolsthanafirst-yearbasiclegalresearchcoursewithanenrollmentofsixty(ormore)students.

Documents

Resource-Based Learning and Course Design: A Brief