Slide 1

A Framework for Evidence-based Teaching in Developmental Biology Scott Freeman, Department of Biology University of Washington srf991@u.washington.edu

Slide 2

Why are we still lecturing?

Slide 3

I dont believe that active learning can work in a large lecture. (UW professor, 8/12) I just know that students.... (UW professor, 3/09) Although it did not occur to us.... to collect data, we consistently observed (Barzilai 2000) we feel that our junior-senior cell biology course... works extraordinarily well (Lodish et al. 2005) We think that our objective of teaching the students to think was well-accomplished. (Miller & Cheetham 1990) We strongly believe that they lead to deeper understanding.... (Rosenthal 1995)

Slide 4

I dont believe that active learning can work in a large lecture. (UW professor, 8/12) I just know that students.... (UW professor, 3/09) Although it did not occur to us.... to collect data, we consistently observed (Barzilai 2000) we feel that our junior-senior cell biology course... works extraordinarily well (Lodish et al. 2005) We think that our objective of teaching the students to think was well-accomplished. (Miller & Cheetham 1990) We strongly believe that they lead to deeper understanding.... (Rosenthal 1995)

Slide 5

Slide 6

Other changes to our mindset, as faculty: Id like to change my lectures, but I dont have time. (or dont know how) If a new technique is sweeping my research field, do I require release time and other special support to learn it? Oh, I tried active learning (or clickers, or group exercises)it doesnt work. The first PCR I ever tried didnt work. Should I conclude that PCR doesnt work?

Slide 7

Why be concerned about the failure rate? Predicted grade Average % EOP students in Bio180 Previous work on Biology 180 How can we lower failure ratesand help capable but underprepared studentsin introductory biology courses?

Slide 8

Spring 2002-2003 Course design Spr 02 < 1.518.2% < 2.544.8% 2002: Modified Socratic style Student performance (does not include drops): Spr 02Spr 03 < 1.518.2%15.8% < 2.544.8%42.3% ; 2003: + ungraded active learning

Slide 9

Spring 2005, Fall 2005 Course design Spr 02Spr 03 < 1.518.2%15.8% < 2.544.8%42.3% Socratic lecturing; Cards or clickers (daily multiple-choice questions in class); weekly, peer-graded practice exam (short- answer) Spr 02Spr 03Spr 05Fall 05 < 1.518.2%15.8%10.9%11.7% < 2.544.8%42.3%37.9%39.3%

Slide 10

Slide 11

Low structureMedium structureHigh structure Fall 2007, 2009 Course design Spr 02Spr 03Spr 05Fall 05Fall 07Fall 09 < 1.518.2%15.8%10.9%11.7%7.4%6.3% < 2.544.8%42.3%37.9%39.3%33.9%28.3% Lecture-free; clickers in peer instruction format; weekly, peer-graded practice exam; daily reading quiz; random-call ~15 students/class %As has increased from 14.5% to 24.3% Spr 02Spr 03Spr 05Fall 05 < 1.518.2%15.8%10.9%11.7% < 2.544.8%42.3%37.9%39.3%

Slide 12

Are exams equivalent across quarters? Approach #1: Predicted exam score Spr 02Spr 03Spr 05Fall 05Fall 07Fall 09 Course Average PES (100pt exam) 70.670.270.970.568.067.5 Approach #2: Weighted Blooms Index Spr 02Spr 03Spr 05Fall 05Fall 07Fall 09 Course Average (weighted Blooms index) 45.852.146.952.252.153.5

Slide 13

Are students equivalent across quarters? Spring 2002 Spring 2003 Spring 2005 Autumn 2005 Autumn 2007 Autumn 2009 Predicted grade (mean) 2.462.572.642.672.852.70 n327338334328339691 Create a general linear model to explain actual grade, based on predicted grade and degree of structure in course. Regression model with UW GPA (at time of entering) and SAT-V; R 2 0.63

Slide 14

2002, 03 2005 2007,09 Course structure

Slide 15

Did we reduce the achievement gap? without spending a lot more money? or maybe even less money? 2003-2008 (Aut/Win/Spr) averages: EOP v non-EOP final grade differences in UW gateway STEM courses

Slide 16

General linear mixed-effects modeling and MMI: Best models include EOP as a fixed effect; likelihood-ratio test, p = 0.0027). Bio180: lecturing vs. high-structure UW Regents Low structure High structure

Slide 17

What could cause a disproportionate increase in performance by disadvantaged students? The Carnegie Hall hypothesis: How do you get to Carnegie Hall? and how you practice matters (deliberate practice): 1)high-level questionsnew contexts/applications); 2)group workteach others/explain yourself, challenge and be challengedwith instructor feedback; 3)daily/weekly basis. PRACTICE!

Slide 18

Dave Parichys questions: Can PIs do this and still run their labs? How do we balance the explosion of detail in developmental biology with big-picture concepts, and help students integrate facts into a cohesive framework? Does this approach transfer to upper-division courses?

Slide 19

Broadening the research focus: From course design in introductory biology to all of the STEM disciplines A meta-analysis of 642 papers from across the STEM disciplines: studies that compare any active-learning intervention to traditional lecturing. 1.Exam/concept inventory/quiz performance: controlling for instructor, student, and assessment equivalence; n = 158 2.DFW (failure) rates; n = 67

Slide 20

Exam performance data: Overall effect size = 0.47 In intro STEM, 6% increase in exam scores; 0.3 increase in average grade. Course levelnHedgess gs.e.95% C.I.: lower limit 95% C.I.: upper limit Introductory1160.4890.0650.3610.616 Upper division380.4800.1200.2450.715

Slide 21

Failure rate data: Overall odds ratio = 1.94 Biomed RCTs stopped for benefit: mean relative risk of 0.53 (0.22-0.66) and/or p < 0.001. Course levelnOdds ratio95% C.I.: lower limit 95% C.I.: upper limit Introductory441.9941.7322.296 Upper division171.7621.3722.263

Slide 22

Daves Second Question: The content problem

Slide 23

Apply: Can I use these ideas in a new situation? Understand: Can I explain these ideas to someone else? Remember: Can I recall key terms and ideas? Analyze: Can I recognize underlying patterns and structure? Synthesize: Can I put ideas and information together to create something new? Evaluate: Can I make judgments on the relative value of ideas and information? Lower order thinking Higher order thinking Blooms taxonomy as a conceptual framework: and hierarchical

Slide 24

Coping strategies: State learning objectives; use backward course design Reading quizzes or other flipping strategies

Slide 25

Daves Third Question: The 6-jobs problem Breaking the Research vs. Teaching dichotomy with RICs Find a colleague/mentor to help with new techniques Recruit grad students/post-docs who want to teach Start small and expect to fail (the first time)

Slide 26

My all-time favorite line from a course evaluation: Keep pushing uswe can do it!

Slide 27

Bill Hoese Anne Casper Kelly Hogan Clarissa Dirks Carol Pollack Megan Rector Pam Pape-Lindstrom Ross Nehm Brian Casper Jenny Knight Joan Sharp Michelle Smith Peter Shaffer Paula Heron Lillian McDermott David Hodge Ferric Fang Emile Pitre Robert Harrington Kevin Mihata Cathy Beyer Deb McGhee Michael Griego Mercedes Converse Michael Fleming Iggy Chau Mikhail Koval Dozie Okoroafor Roddy Theobald David Haak Micah Horwith Chris Gast Riley Brazil Eunice Lau Hannah Jordt Eliza Heery Alan Sunada Chelsea Mann Dave Hays Elli Jenkins Sara Brownell Sarah Eddy Jen Nemhauser Dave Hurley Matt Cunningham Tom Daniel Alison Crowe Barbara Wakimoto Janneke Hille Ris Lambers Eileen OConnor John Parks Mary Pat Wenderoth Toby Bradshaw Ben Wiggins Mandy Schivell