Pathways to Scientific Teaching

Diane Ebert-MayDepartment of Plant Biology

Michigan State University

[email protected]://first2.org

Pathways to Scientific Teaching

mailto:[email protected]

http://first2.org/

QuickTime™ and a Graphics decompressor are needed to see this picture.

Team at MSU

Rett Weber - Plant Biology (postdoctoral researcher)Deb Linton - Plant Biology (postdoctoral researcher)Duncan Sibley - GeologyDoug Luckie - PhysiologyScott Harrison - Microbiology (graduate student)Tammy Long - Plant BiologyHeejun Lim - Chemistry Education Rob Pennock - PhilosophyCharles Ofria - EngineeringRich Lenski - MicrobiolgyJanet Batzli - Plant Biology [U of Wisconsin]

The trouble with our times is that the future is not what it used to be.

-Paul Valery, The Art of Poetry

Engage

Question 1

Scientific teaching involves active learning strategies to engage students in the process of science.

Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly

disagree

Question 2

Students learn science best by doing science.


disagree

Question 3

How important is it to use multiple kinds of data to assess student learning?

Please respond on a scale of 0-100 in increments of 10:

Question 4

How often do you use multiple kinds of data to make instructional decisions?


Question 5

In my department, excellence/scholarship in teaching is rewarded at a level comparable to excellence/scholarship in research.


disagree

Question 1

Scientific teaching involves active learning strategies to engage students in the process of science.


disagree

Question 2

Students learn science best by doing science.


disagree

Large Class Meeting

Question 3

How important is it to use multiple kinds of data to assess student learning?


Question 4

How often do you use multiple kinds of data to make instructional decisions?


Question 5

In my department, excellence/scholarship in teaching is rewarded at a level comparable to excellence/scholarship in research.


disagree

Explore

Teaching and learning challenges

Developing and critiquing learning objectives and assessments

Aligning objectives with assessment

Exploring tools for assessment - what is the evidence?

Implementing active learning strategies in science courses and curricula

Topics for Today

What’s up with Termites?

1. On a sheet of paper, draw two circles near each other on the center of the page.

2. Release termites onto paper.

3. Keep creatures safe. I shall collect them in their original habitat.

4. What do you observe about termite behavior?

5. Develop a question your group could explore if you had more time.

(15 minutes - select a timekeeper)

1. Develop one possible objective for this ‘inquiry’.

Team Written response.

Reporter - Recorder - Timekeeper - (10 minutes)Checker -

2. Develop an assessment appropriate for the objective.

Explore...thin air

What is going on?Teaching without learning!

Talk to your neighbor - brainstorm

1. Diagnose situation - the learning challenge

2. Where is/are the missing links?misconceptions?

In your groups:

What do you want your students to be able to DO?

Design learning objectives that address photosynthesis learning challenges.

Get to know your students and their prior knowledge.

May use pretest or diagnostic (clicker) question

At the end of this unit, students should be able to….

Generate statement of measurable performance

Learning objectives

• Once you have a set of learning goals….

• assign a Bloom-level to each goal.

Statement that indicates level of expectation of performance

What level of learning do we ask of our students?

Bloom (1956) Cognitive Domain of Educational Objectives

6 categories - KnowledgeComprehensionApplicationAnalysisSynthesisEvaluation

•

Learning Objectives... Revise

Explore more...

JigsawNew groups

Count off -- 1, 2, 3, 4, 5

All 1s work on same paper ....2s, 3s, 4s, 5s

Return to ‘home’ groups and share what you found in each of the papers.

Report out

Paper Assignments

Group 1: Climate change....

Group 2: Novel assessments...

Group 3: Practicing scientific inquiry...

Group 4: Determining confidence...

Group 5: Collaborative learning...

In your groups:Read the paper, discuss, record...

1. What are the student learning goals?

2. What is the Bloom-level of each goal?

3. Do the assessments align with the goals?

4. What are the active learning strategies?

What is assessment?

Data collection with the purpose of answering questions about…

students’ understanding

students’ attitudes

students’ skills

instructional design and implementation

curricular reform (at multiple grainsizes)

Why do assessment?

1. Improve student learning and development.

2. Provide students and facultysubstantive feedback about student understanding.

3. Challenge to use disciplinary research strategies to assess learning.

Multiple Choice … … Concept Maps … … Essay … … Interview

high Ease of Assessment low

low Potential for Assessment of Learning high

Theoretical Framework• Ausubel 1968; meaningful learning• Novak 1998; visual representations• King and Kitchner 1994; reflective judgment• National Research Council 1999; theoretical frameworks for assessment

Assessment Gradient

Identify desired goals/objectives

Determine acceptable evidence

Design learning experiences

and instruction

Wiggins and McTighe 1998

Backwards Design

Department of Plant Biology

Course Level

Mean B

loom

Level

Department of Plant Biology

% T

ota

l Q

s

Course Level*N items

Bloom Level

Explain

Assessments: Tools to Detect LearningAssessments: Tools to Detect Learning

What’s a tool used to test student attitude?What’s a tool used to test student attitude?

2.VASS (Views)

22

1. Self-Efficacy

What’s a tool used to test knowledge?What’s a tool used to test knowledge?

22

1. Exam 2. Writing 3. PBA 4. Concept I

The most common tool used is?The most common tool used is?

What is the relationship among DNA, a gene, and a chromosome?

a. A chromosome contains hundreds of genes which are composed of DNA.

b. DNA contains hundreds of genes which are composed of chromosomes.

c. A gene contains hundreds of chromosomes which are composed of DNA.





An mRNA molecule with the following sequence is being read by a ribosome: 5' CCG-ACG -UGC 3'

What was the original 'antisense' DNA sequence that encoded this? a. 5' CCG-ACG-UGC 3'b. 5' CCG-ACG-TGC 3'c. 5' GGC-TGC-ACG 3'd. 5' CCA-CGT-CGG 3'


What was the original 'antisense' DNA sequence that encoded this? a. 5' CCG-ACG-UGC 3'b. 5' CCG-ACG-TGC 3'c. 5' GGC-TGC-ACG 3'd. 5' CCA-CGT-CGG 3'

















An mRNA molecule with the following sequence is being read by a ribosome: 5' CCG-ACG -UGC 3‘

What was the original 'antisense' DNA sequence that encoded this?

a. 5' CCG-ACG-UGC 3‘b. 5' CCG-ACG-TGC 3‘c. 5' GGC-TGC-ACG 3‘d. 5' CCA-CGT-CGG 3'












a. 5' CCG-ACG-UGC 3‘b. 5' CCG-ACG-TGC 3‘c. 5' GGC-TGC-ACG 3‘d. 3' GGC-TGC-ACG 5'



a. 5' CCG-ACG-UGC 3‘b. 5' CCG-ACG-TGC 3‘c. 5' GGC-TGC-ACG 3‘d. 3' GGC-TGC-ACG 5'

Problem Identification

DEFINE a bicycle with words

DRAW a picture of a bicycle

Let’s try an experiment

OK?

Let’s try an experiment

OK?

Which parts and subsystems did you include in your drawing vs. your definition?

Now... let’s examine

the concept of a bicycle

Now... let’s examine

the concept of a bicycle

What’s a subsystem?

Propulsion

61

Joseph Novak, in Learning, Creating and Using Knowledge: Concept Maps as Facilitative Tools in Schools and Corporations 1998

“Meaningful learning results when the learner chooses to relate new information to ideas the learner already knows. Rote learning occurs when the learner memorizes new information without relating to prior knowledge.”

C-TOOLSC-TOOLS

Students could explain details of transcription & translation but not the relation... “DNA-Gene-Chromosome”

Students could explain details of transcription & translation but not the relation... “DNA-Gene-Chromosome”

Can concept mapping force students to “Think different” and confront their (mis) understanding?

Can concept mapping force students to “Think different” and confront their (mis) understanding?

Hierarchy

has

Structure

has

Concept Maps

Visual DiagramsOr Models

are represent

Knowledge or Understanding

Concepts

display

connectedwith

Linking Words

Used for

Assessment Organization

Reflection &Learning

promotes

Context

is constructedwith

NewInformation

PriorKnowledge

We’ve built some infrastructure...We’ve built some infrastructure...

With Robograder for feedback.With Robograder for feedback.

www.ctools.msu.edu

Student’s Concept Map

“In a pair of mutually complementary studies*, concept mapping has been shown to facilitate learning when constructed both before a unit of instruction, as an advance organizer, or when constructed following the unit, as a so-called ‘postorganizer.’ ”

*Willerman and MacHarg (1991), Spiegel and Barufaldi (1994)

D.E. Moody, Mapping Biology

Knowledge, 2000Concept

Maps have been tested

in the classroom.

Concept Maps have been tested

in the classroom.

1. Select a concept that is critical for your students to understand.

2. Identify 4 or 5 subconcepts that are important to understanding that concept

e.g., DNA - Gene- Chromosome - Enzyme

For a course you teach .....

3. Arrange them by rank order - top most general, bottom most specific

4. Add linking lines to make connections between two concepts

5. Add linking words that describe the relationship between two concepts

Objective (outcome)

Students will demonstrate understanding of evolution by natural selection.

•Changes in a population occur through a gradual change in individual members of a population.

•New traits in species are developed in response to need.

•All members of a population are genetically equivalent, variation and fitness are not considered.

•Traits acquired during an individual’s lifetime will be inherited by offspring.

Alternative Conceptions: Natural Selection

(AAAS 1999)

Pre-test: Essay Explain the changes that occurred in the tree and animal. Use your current understanding of evolution by natural selection.

Modified problem: populations of trees and animals

How do we develop rubrics?

Describe the goal/objective for the activity, problem, task...Select the assessment tasks aligned with goalsDevelop performance standardsDifferentiate levels of responses based on clearly described criteriaRate (assign value) the categories

Level of Achievement General Approach ComprehensionExemplary(5 pts)

• Addresses thequestion.• States a relevant,justifiable answer.• Presents arguments ina logical order.• Uses acceptable styleand grammar (noerrors).

• Demonstrates an accurate andcomplete understanding of thequestion.• Backs conclusions with dataand warrants.• Uses 2 or more ideas,examples and/or arguments thatsupport the answer.

Adequate(3 pts)

• Does not address thequestion explicitly,although does sotangentially.• States a relevant andjustifiable answer.• Presents arguments ina logical order.• Uses acceptable styleand grammar (oneerror).

• Demonstrates accurate but onlyadequate understanding ofquestion because does not backconclusions with warrants anddata.• Uses only one idea to supportthe answer.• Less thorough than above.

Needs Improvement(1 pt)

• Does not address thequestion.• States no relevantanswers• indicatesmisconceptions.• Is not clearly orlogically organized.• Fails to use acceptablestyle and grammar (twoor more errors).

• Does not demonstrate accurateunderstanding of the question.• Does not provide evidence tosupport their answer to thequestion.

No Answer (0 pts)

Scoring Rubric for Quizzes and Homework

Advantages of Scoring Rubrics

Improve the reliability of scoring written assignments and oral presentationsConvey goals and performance expectations of students in an unambiguous wayConvey “grading standards” or “point values” and relate them to performance goalsEngage students in critical evaluation of their own performance Save time but spend it well

Limitations of Scoring Rubrics

Problem of criteriaProblem of practice and regular use

Scoring Rubric website• http://www.wcer.wisc.edu/nise/cl1/flag/

Sample Rubrics for Environmental Science

• http://www.msu.edu/~ebertmay/isb202/home.html

Guidelines for Planning Research

How do instructors move from assessment to designing research on learning?

What did the assessment data suggest about student understanding?

Why didn’t students understand critical concepts?

What has been done already about students’ understanding of these concepts?

Ask Questions

Investigator’s Purpose

Improve own classroom instruction?

Really conduct research?

How and why will you select the research methods?

What kinds of data will you collect? Direct or Self-Report?

How will you analyze the data?

Design Study Collect Data

Research Designs

Data Collection

How will you analyze the data?

How could the results influence instruction?

Analyze Data

Quantitative data - statistical analysis

Qualitative data

break into manageable units and define coding categories

search for patterns, quantify

interpret and synthesize

Valid and repeatable measures

Data Analysis

Ideas and results are peer reviewed - formally and/or informally.

Science journals

Report the Study

Articles derived from journal papers

How do analogous assessment questions help us determine students’ prior understanding and progressive thinking about the carbon cycle?

Question

Concept 1: Matter disappears during decomposition of organisms in the soil.

Concept 2: Photosynthesis as Energy: Photosynthesis provides energy for uptake of nutrients through roots which builds biomass. No biomass built through photosynthesis alone.

Concept 3: Thin Air: CO2 and O2 are gases therefore, do not have mass and

therefore, can not add or take away mass from an organism.

Concept 4: Plant Altruism: CO2 is converted to O2 in plant leaves so that all

organisms can ‘breathe’.

Concept 5: All Green: Plants have chloroplasts instead of mitochondria so they can not respire.

Some Common Misconceptions about Photosynthesis & Respiration

Some Common Misconceptions about Photosynthesis & Respiration

Ebert-May et al. 2003 Bioscience

Design Experiment

Quantitative DataQualitative Data

Instructional Design

Two class meetings on carbon cycle (160 minutes)Active, inquiry-based learning

Cooperative groupsQuestions, group processing, large lecture sections, small discussion sections, multi-week laboratory investigationHomework problems including web-based modules

Different faculty for each courseOne graduate/8-10 undergraduate TAs per course

Experimental Design

Two introductory courses for majors:Bio 1 - organismal/population biology (faculty A)Bio 2 - cell and molecular biology (faculty B)

Three cohorts:Cohort 1 Bio 1 (n=141)Cohort 2 Bio1/Bio2 (n=63) Cohort 3 Other/Bio2 (n=40)

Assessment Design

Multiple iterations/versions of the carbon cycle problem Pretest, midterm, final with additional formative assessments during classAdministered during instructionSemester 1 - pretest, midterm, final exam Semester 2 - final exam

Text

Problem

Experimental setup:Weighed out 3 batches of radish seeds each weighing 1.5 g.

Experimental treatments:1. Seeds placed on moistened paper towels in LIGHT2. Seeds placed on moistened paper towels in DARK3. Seeds not moistened (left DRY) placed in light

Problem (2)

After 1 week, all plant material was dried in an oven overnight (no water left) and plant biomass was measured in grams.

Predict the biomass of the plant material in the various treatments.

Water, lightWater, darkNo water, light

Results Mass of Radish Seeds/Seedlings

1.46 g 1.63 g 1.20 g

Write an explanation about the results.

Explain the results.Write individually on carbonless

paper.

Grandma Johnson Problem

Hypothetical scenario: Grandma Johnson had very sentimental feelings toward Johnson Canyon, Utah, where she and her late husband had honeymooned long ago. Her feelings toward this spot were such that upon her death she requested to be buried under a creosote bush overlooking the canyon. Trace the path of a carbon atom from Grandma Johnson’s remains to where it could become part of a coyote. NOTE: the coyote will not dig up Grandma Johnson and consume any of her remains.

Analysis of Responses

Used same scoring rubric (coding scheme) for all three problems - calibrated by adding additional criteria when necessary, rescoring:

Examined two major concepts:

Concept 1: Decomposers respire CO2Concept 2: Plants uptake of CO2

Explanations categorized into two groups:Organisms (trophic levels)Processes (metabolic)

Code Organisms Code Processes and pathways

1 Decomposers IA Cellular Respiration

IB Release CO 2

2 IIA Pathway of Carbon

Primary producers IIA _1: through Air

IIA _2 : through Root

IIA _3 : no mention about pathway

IIB Make Glucose

II C Photosynthesis

3 Herbivore III Respiration

(glycolysis, Kreb cycle)

4 Carnivore IV Respiration

(glycolysis, Kreb cycle)

Coding Scheme

Corr

ect

Stu

dent

Resp

onse

s (%

)

Cellular Respiration by Decomposers

Bio1/Bio2 Other/Bio2

Friedmans, p<0.01

Cellular Respiration by Decomposers

Pathway of Carbon in Photosynthesis

Bio1/Bio2Corr

ect

Stu

dent

Resp

onse

s (%

)

Other/Bio2

Friedmans, p<0.05

Pathway of Carbon in Photosynthesis

Evolution and Natural

Selection

How does active learning affect students’ understanding of evolution and natural selection over time?

In guppy populations, what are the primary changes that occur gradually

over time?

a. The traits of each individual guppy within a population gradually change.

b. The proportions of guppies having different traits within a population change.

c. Successful behaviors learned by certain guppies are passed on to offspring.

d. Mutations occur to meet the needs of the guppies as the environment changes.

CINS Multiple Choice

• Day 13 in class:

Pretest: CINS multiple choice and dino essay

Artificial selection in dogs

Groups discuss and record: Why are dogs only one species?

Groups discuss and record: Breed a larger dog.

Assign concept map 4 - homework

Active Learning

Individual AssignmentMake a new concept map demonstrating your understanding of natural selection using the following concepts.

genetic variation evolution species population natural selection artificial selection selective agent

Concept Map 4

Days of Instruction

Mean

Perc

en

t C

orr

ect

MCMC

Essay

C-map

Clicker

Assessments

PreMC

PreEssay

Active Learning

•Day 14 in class

Artificial selection in plants

Group discuss and record: What traits were selected and why?

Clicker Q: Plant fitness

Natural selection

Clicker Q: Selection on beak size

Group discuss and record: What traits were naturally selected in Hawaiian finches?

Minute paper: Natural and artificial selection

• Assign homework: Guppy sexual selection

Active Learning

Days of Instruction

Mean

Perc

en

t C

orr

ect

MCMC

Essay

Assessments

PreMC

PreEssay

Clicker MC

Active Learning

http://www.pbs.org/wgbh/evolution/sex/guppy/ed_pop.html

Homework: Guppy Sexual Selection

Days of Instruction

Mean

Perc

en

t C

orr

ect

MCMC

Essay

Assessments

PreMC

PreEssay

Clicker MC

Active Learning

C-map 4

Day 15

In class: groups

Determine fitness of wild tobacco

Sexual selection

review guppy homework

Assign homework: concept map 5

Active Learning

Days of Instruction

Mean

Perc

en

t C

orr

ect

C-map 5

Assessments

C-map 4

PreMC

PreEssay

Clicker MC

Active Learning

Individual AssignmentAdd the following terms to concept map 4.

biotic factorsabiotic factorsfitnesstraitmutationreproductionmeiosisallele frequencyvariationselection pressureadaptation

Concept Map 5

2nd midterm exam

CINS multiple choice on natural selection

Final exam

CINS multiple choice on natural selection

Dinosaur post test

Assessments

Days of Instruction

Mean

Perc

en

t C

orr

ect

PreMC

Post 1MC Post 2

MC

PostEssay

PreEssay

Clicker MC

Active Learning

Assessments 2005C-map 5

C-map 4

dayday

Pre MC - Pre EssayPre MC - Post 1 MCPre MC - Post 2 MCPre Essay - Post 1 MCPre Essay - Post Essay

0.350.540.250.330.53

Cmap 4 - Cmap 5Cmap 4 - Post 2 MCPost 1 MC - Post 2 MCPost 1 MC - Post 2 MC

0.600.380.380.29

Correlations (p<0.05)N= 49-117

Days of Instruction

Mean

Perc

en

t C

orr

ect

Post 1MC

Active Learning

C-map

Assessments 2004

Post 2MC

Post 1 Essay

PreEssay

Clicker 1MC

Clicker 2MC

Correlations (p<0.05)

N=84-103

Post 2Essay

Pre Essay - Post 1 EssayClicker 2 MC - Post 1 MCPost 1 MC - Post 2 MCPost 1 MC - Post 2 Essay

0.250.250.460.42

Post 1 Essay - Post 2 MCPost 1 Essay - Post 2 EssayPost 2 MC - Post 2 Essay

0.240.290.60

SystemModel

ResearchTeaching

Is it scientific?

Documents

Pathways to Scientific Teaching