SGTS Theory 3 - Learning Outcomes, Assessment

Summer Graduate Teaching Scholars

May 9, 2013

Theory 3:

Learning Outcomes and Assessment

Scholarly approach to teaching:

SGTS Theory Stream - collegeclassroom.ucsd.edu

2

Carl Wieman

Science Education Initiative

What should

students

learn?

What are

students

learning?

What instructional

approaches

help students

learn?

learning

outcomes

(goals, objectives)

assessment

(Image: NASA)

SGTS Theory Stream - collegeclassroom.ucsd.edu

3

Introductory “Astro 101”

This course covers Chapters

1. Mercury

2. Venus

…

8. Neptune

9. other objects

10. Formation of the Solar

System

deduce from patterns in the

properties of the planets,

moons, asteroids and other

bodies that the Solar

System had single

formation event.

provide notable examples

of how comets influenced

history, art and science

4

SGTS Theory Stream - collegeclassroom.ucsd.edu

Traditional Course Syllabus Course with Learning Outcomes

Learning outcomes

SGTS Theory Stream - collegeclassroom.ucsd.edu

5

completes the sentence, “By this end of this

lesson/unit/course, you will be able to…”

begins with an action verb (more below…)

clarifies what it means to “understand” the concept

deduce from patterns in the

properties of the planets,

moons, asteroids and other

bodies that the Solar

System had single

formation event.

What is the Value of Course-Specific

Learning Goals

SGTS Theory Stream - collegeclassroom.ucsd.edu

6

What was the most interesting thing you learned from

Simon & Taylor? [1]

this

Outcomes

SGTS Theory Stream - collegeclassroom.ucsd.edu

7 Simon & Taylor (2009)

Learning outcomes are valuable to the

instructor, too.

SGTS Theory Stream - collegeclassroom.ucsd.edu

8

crystallizes what prof actually cares about

helped prof

choose clicker questions for class

write the final exam

…choose clicker questions for class

SGTS Theory Stream - collegeclassroom.ucsd.edu

9

ClassAction http://astro.unl.edu/classaction/

…write the final exam

SGTS Theory Stream - collegeclassroom.ucsd.edu

10

(10 marks) List 3 patterns of the Solar System as a

whole. Then, outline in some detail the current model for

the formation of the Solar System. In particular, make

sure you explain how the observed patterns and

regularities are related to this theory of formation.

Writing LOs – The Easy Way

SGTS Theory Stream - collegeclassroom.ucsd.edu

11

Astro 101 Learning Outcomes

learn about Saturn

Assess your LOs:

“learn”? Learn what?

how does a student demonstrate to you s/he has

“learned about Saturn?”

how does a student check that s/he has mastered the

Saturn part of the course?

Writing LOs – More Effective Way

SGTS Theory Stream - collegeclassroom.ucsd.edu

12

Astro 101 Learning Outcomes

describe in detail the size and structure of Saturn’s

rings

step through the gravitational feedback cycle that

keeps Saturn’s rings so thin

Assess your LOs:

1. Is this really what I want them to know?

2. Are they capable of that?

3. How do I assess their ability to demonstrate this LO?

Deciding on the level of a LO

SGTS Theory Stream - collegeclassroom.ucsd.edu

13

Writing learning outcomes is hard because you have to

recognize

declare

(admit)

what you want your students to be capable of doing.

A good start is picking the verb describing the action

the students will perform to demonstrate their mastery

of the concept:

Bloom’s Taxonomy [2,3]

SGTS Theory Stream - collegeclassroom.ucsd.edu

14

Remember

Understand

Apply

Analyze

Evaluate

Create

think critically about and defend a position

transform or combine ideas to create something new

break down concepts into parts

apply comprehension to unfamiliar situations

demonstrate understanding of ideas and concepts

remember and recall factual information

Bloom’s Taxonomy [2,3]

SGTS Theory Stream - collegeclassroom.ucsd.edu

15

Remember

Understand

Apply

Analyze

Evaluate

Create

higher order thinking

lower order thinking

Bloom’s Taxonomy [2,3]

SGTS Theory Stream - collegeclassroom.ucsd.edu

16

Remember

Understand

Apply

Analyze

Evaluate

Create develop, create, propose, formulate, design, invent

judge, appraise, recommend, justify, defend, criticize, evaluate

compare, contrast, categorize, distinguish, identify, infer

apply, demonstrate, use, compute, solve, predict, construct, modify

describe, explain, summarize, interpret, illustrate

define, list, state, label, name, describe

Driver’s Ed 101: How to Drive in CA

SGTS Theory Stream - collegeclassroom.ucsd.edu

17

The whiteboards on the tables are numbered. Your

group will concentrate on the question on the California

DMV Test [4] matching your board’s number.

Task: Write a learning outcome

that your group’s question assesses.

(Refer to Wieman handout

for Bloom’s levels and corresponding verbs)

Revised Bloom’s Taxonomy [2] 18

SGTS Theory Stream - collegeclassroom.ucsd.edu

Vocabulary check: assessment

is that which gives a final

judgment of evaluation of

proficiency, such as grades or

scores.

(How Learning Works, p. 139)

explicitly communicates to

students about some specific

aspects of their performance

relative to specific target

criteria, and … provides

information that helps students

progress toward meeting those

criteria…[It] informs students’

subsequent learning.

(How Learning Works, p. 139)

19

SGTS Theory Stream - collegeclassroom.ucsd.edu

summative assessment formative assessment

Feedback and Practice that Enhance

Learning (How Learning Works [5]) 20

Writing – public policy course

Presentations on research – medical anthropology

SGTS Theory Stream - collegeclassroom.ucsd.edu

Instructors’ expertise

and bias not clear to

students (or themselves)

Feedback and Practice that Enhance

Learning (How Learning Works [5]) 21

Solution: Goal-directed practice coupled with targeted

feedback are critical to learning.

SGTS Theory Stream - collegeclassroom.ucsd.edu

Images:

Excellent Shot by Varsity Life on flickr CC

Music by Piulet on flickr CC

Feedback and Practice that Enhance

Learning (How Learning Works [5]) 22

Solution: Goal-directed practice coupled with targeted

feedback are critical to learning.

[G]oals can direct the nature of focused practice, provide

the basis for evaluating observed performance, and shape

the targeted feedback that guides students’ future efforts.

(p. 127)

[T]argeted feedback gives students prioritized information

about how their performance does or does not meet the

criteria so they can understand how to improve their future

performance.

(p. 141) SGTS Theory Stream - collegeclassroom.ucsd.edu

Scenarios

SGTS Theory Stream - collegeclassroom.ucsd.edu

23

feedback at

appropriate level

feedback not at

appropriate level

productive practice unproductive practice

practice is goal-directed practice not goal-directed

timely feedback untimely feedback

Find the person with the same

colored sheet as you. Fill out

the sheet together.

Feedback at Appropriate Level Feedback not at Appropriate Level sp

ort

/hobby _

_______________

ed

uca

tion

__________________

Productive Practice Unproductive Practice sp

ort

/hobby _

_______________

ed

uca

tion

__________________

Practice Goal-directed Practice not Goal-directed sp

ort

/hob

by _

_______________

ed

uca

tion

__________________

Timely Feedback Untimely Feedback sp

ort

/hobby _

_______________

ed

uca

tion

__________________

Instructional Scaffolding 28

Needs to be given BEFORE and BUILT INTO assignment

Outlines what it takes to improve

Supports Zone of Proximal Development [8]

SGTS Theory Stream - collegeclassroom.ucsd.edu

Appropriate Level of Challenge 29

Glued to Games [6]: Psychological

exploration (and comparison with

educational practices) of characteristics

of games that drive people to spend time

and succeed.

James Paul Gee [7] “What video games

have to teach us about learning and

literacy”

SGTS Theory Stream - collegeclassroom.ucsd.edu

angrybirds.com

SGTS Theory Stream - collegeclassroom.ucsd.edu

30

Clicker question 31

Does this rubric foster a

A) fixed/performance mindset

B) growth/mastery mindset

C) neither

D) both

SGTS Theory Stream - collegeclassroom.ucsd.edu

SGTS Theory Stream - collegeclassroom.ucsd.edu

Robert Talbert

tinyurl.com/RobertTalbertRubric

Instructional Scaffolding 33

Rubrics

support growth mindsets

path to improvement

goal-directed

[G]oals can direct the nature of focused practice, provide the basis for evaluating observed performance, and shape

the targeted feedback that guides students’ future efforts.

targeted feedback

[T]argeted feedback gives students prioritized information about how their performance does or does not meet the criteria so they can understand how to improve their future performance.

SGTS Theory Stream - collegeclassroom.ucsd.edu

Assessment Strategies…

SGTS Theory Stream - collegeclassroom.ucsd.edu

34

addressing the need for goal-directed practice

addressing the need for targeted feedback

Work on the hand-out,

thinking about what you’ve

experienced or what you

aspire to do in your field.

Executive Summary:

SGTS Theory Stream - collegeclassroom.ucsd.edu

35

Plan your course

by synchronizing and

aligning your learning

outcomes, activities and

assessments.

What should

students

learn?

What are

students

learning?

What instructional

approaches

help students

learn?

References

SGTS Theory Stream - collegeclassroom.ucsd.edu

36

1. Simon, B., & Taylor, J. (2009). What is the Value of Course-Specific Learning

Goals? Journal of College Science Teaching, 39, 2, 52-57. PDF available at

www.cwsei.ubc.ca/SEI_research/files/LifeSci/Simon_Taylor_ValueOfCourseSpecif

icLG.pdf

2. Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The

Cognitive Domain. New York: David McKay Co Inc.

Adapted from Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for

Learning. Teaching, and assessing: A revision of bloom's taxonomy of educational

objectives.

Revised Bloom’s Taxonomy www.celt.iastate.edu/teaching/RevisedBlooms1.html

3. Excerpt from Wieman, C. (2007). Slides from the Wieman Learning Goals

Workshop. www.cwsei.ubc.ca/resources/learn_goals.htm

4. California DMV Sample Class C Written Test 5

www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

References, continued

SGTS Theory Stream - collegeclassroom.ucsd.edu

37

5. Ambrose, S.A., Bridges, M.W., DiPietro, M., Lovett, M.C., & Norman, M.K. (2010).

How Learning Works. San Fransisco: Jossey-Bass.

6. Glued to Games: How Video Games Draw Us In and Hold Us Spellbound

http://www.gluedtogames.com/

7. Gee, J.P. (2005). Learning by Design: good video games as learning machines. E-

Learning 2, 1, 5-16.

8. Wertsch, J.V. (1984). The zone of proximal development: Some conceptual issues.

New Directions for Child and Adolescent Development, 1984, 23, 7–18.

Bloom’s Taxonomy of the Cognitive Domain

(Levels of Learning) [Wieman, 2007] 38

6. Synthesis: transform and combine ideas to create something new

develop, create, propose, formulate, design, invent

5. Evaluation: think critically about and defend a position

judge, appraise, recommend, justify, defend, criticize, evaluate

4. Analysis: break down concepts into parts

compare, contrast, categorize, distinguish, identify, infer

3. Application: apply comprehension to unfamiliar situations

apply, demonstrate, use, compute, solve, predict, construct, modify

2. Comprehension: demonstrate understanding of ideas, concepts

describe, explain, summarize, interpret, illustrate

1. Factual Knowledge: remember and recall factual knowledge

define, list, state, label, name, describe

SGTS Theory Stream - collegeclassroom.ucsd.edu