SGTS Theory 3 - Learning Outcomes, Assessment

Summer Graduate Teaching Scholars

May 9, 2013

Theory 3:

Learning Outcomes and Assessment

Scholarly approach to teaching:

SGTS Theory Stream - collegeclassroom.ucsd.edu

2

Carl Wieman

Science Education Initiative

What should

students

learn?

What are

students

learning?

What instructional

approaches

help students

learn?

learning

outcomes

(goals, objectives)

assessment

(Image: NASA)


3

Introductory “Astro 101”

This course covers Chapters

1. Mercury

2. Venus

…

8. Neptune

9. other objects

10. Formation of the Solar

System

deduce from patterns in the

properties of the planets,

moons, asteroids and other

bodies that the Solar

System had single

formation event.

provide notable examples

of how comets influenced

history, art and science

4


Traditional Course Syllabus Course with Learning Outcomes

Learning outcomes


5

completes the sentence, “By this end of this

lesson/unit/course, you will be able to…”

begins with an action verb (more below…)

clarifies what it means to “understand” the concept

deduce from patterns in the

properties of the planets,

moons, asteroids and other

bodies that the Solar

System had single

formation event.

What is the Value of Course-Specific

Learning Goals


6

What was the most interesting thing you learned from

Simon & Taylor? [1]

this

Outcomes


7 Simon & Taylor (2009)

Learning outcomes are valuable to the

instructor, too.


8

crystallizes what prof actually cares about

helped prof

choose clicker questions for class

write the final exam

…choose clicker questions for class


9

ClassAction http://astro.unl.edu/classaction/

…write the final exam


10

(10 marks) List 3 patterns of the Solar System as a

whole. Then, outline in some detail the current model for

the formation of the Solar System. In particular, make

sure you explain how the observed patterns and

regularities are related to this theory of formation.

Writing LOs – The Easy Way


11

Astro 101 Learning Outcomes

learn about Saturn

Assess your LOs:

“learn”? Learn what?

how does a student demonstrate to you s/he has

“learned about Saturn?”

how does a student check that s/he has mastered the

Saturn part of the course?

Writing LOs – More Effective Way


12

Astro 101 Learning Outcomes

describe in detail the size and structure of Saturn’s

rings

step through the gravitational feedback cycle that

keeps Saturn’s rings so thin

Assess your LOs:

1. Is this really what I want them to know?

2. Are they capable of that?

3. How do I assess their ability to demonstrate this LO?

Deciding on the level of a LO


13

Writing learning outcomes is hard because you have to

recognize

declare

(admit)

what you want your students to be capable of doing.

A good start is picking the verb describing the action

the students will perform to demonstrate their mastery

of the concept:

Bloom’s Taxonomy [2,3]


14

Remember

Understand

Apply

Analyze

Evaluate

Create

think critically about and defend a position

transform or combine ideas to create something new

break down concepts into parts

apply comprehension to unfamiliar situations

demonstrate understanding of ideas and concepts

remember and recall factual information



15

Remember

Understand

Apply

Analyze

Evaluate

Create

higher order thinking

lower order thinking



16

Remember

Understand

Apply

Analyze

Evaluate

Create develop, create, propose, formulate, design, invent

judge, appraise, recommend, justify, defend, criticize, evaluate

compare, contrast, categorize, distinguish, identify, infer

apply, demonstrate, use, compute, solve, predict, construct, modify

describe, explain, summarize, interpret, illustrate

define, list, state, label, name, describe

Driver’s Ed 101: How to Drive in CA


17

The whiteboards on the tables are numbered. Your

group will concentrate on the question on the California

DMV Test [4] matching your board’s number.

Task: Write a learning outcome

that your group’s question assesses.

(Refer to Wieman handout

for Bloom’s levels and corresponding verbs)

Revised Bloom’s Taxonomy [2] 18


Vocabulary check: assessment

is that which gives a final

judgment of evaluation of

proficiency, such as grades or

scores.

(How Learning Works, p. 139)

explicitly communicates to

students about some specific

aspects of their performance

relative to specific target

criteria, and … provides

information that helps students

progress toward meeting those

criteria…[It] informs students’

subsequent learning.

(How Learning Works, p. 139)

19


summative assessment formative assessment

Feedback and Practice that Enhance

Learning (How Learning Works [5]) 20

Writing – public policy course

Presentations on research – medical anthropology


Instructors’ expertise

and bias not clear to

students (or themselves)



Solution: Goal-directed practice coupled with targeted

feedback are critical to learning.


Images:

Excellent Shot by Varsity Life on flickr CC

Music by Piulet on flickr CC



Solution: Goal-directed practice coupled with targeted

feedback are critical to learning.

[G]oals can direct the nature of focused practice, provide

the basis for evaluating observed performance, and shape

the targeted feedback that guides students’ future efforts.

(p. 127)

[T]argeted feedback gives students prioritized information

about how their performance does or does not meet the

criteria so they can understand how to improve their future

performance.

(p. 141) SGTS Theory Stream - collegeclassroom.ucsd.edu

Scenarios


23

feedback at

appropriate level

feedback not at

appropriate level

productive practice unproductive practice

practice is goal-directed practice not goal-directed

timely feedback untimely feedback

Find the person with the same

colored sheet as you. Fill out

the sheet together.

Feedback at Appropriate Level Feedback not at Appropriate Level sp

ort

/hobby _

_______________

ed

uca

tion

__________________

Productive Practice Unproductive Practice sp

ort

/hobby _

_______________

ed

uca

tion

__________________

Practice Goal-directed Practice not Goal-directed sp

ort

/hob

by _

_______________

ed

uca

tion

__________________

Timely Feedback Untimely Feedback sp

ort

/hobby _

_______________

ed

uca

tion

__________________

Instructional Scaffolding 28

Needs to be given BEFORE and BUILT INTO assignment

Outlines what it takes to improve

Supports Zone of Proximal Development [8]


Appropriate Level of Challenge 29

Glued to Games [6]: Psychological

exploration (and comparison with

educational practices) of characteristics

of games that drive people to spend time

and succeed.

James Paul Gee [7] “What video games

have to teach us about learning and

literacy”


angrybirds.com


30

Clicker question 31

Does this rubric foster a

A) fixed/performance mindset

B) growth/mastery mindset

C) neither

D) both



Robert Talbert

tinyurl.com/RobertTalbertRubric

Instructional Scaffolding 33

Rubrics

support growth mindsets

path to improvement

goal-directed

[G]oals can direct the nature of focused practice, provide the basis for evaluating observed performance, and shape

the targeted feedback that guides students’ future efforts.

targeted feedback

[T]argeted feedback gives students prioritized information about how their performance does or does not meet the criteria so they can understand how to improve their future performance.


Assessment Strategies…


34

addressing the need for goal-directed practice

addressing the need for targeted feedback

Work on the hand-out,

thinking about what you’ve

experienced or what you

aspire to do in your field.

Executive Summary:


35

Plan your course

by synchronizing and

aligning your learning

outcomes, activities and

assessments.

What should

students

learn?

What are

students

learning?

What instructional

approaches

help students

learn?

References


36

1. Simon, B., & Taylor, J. (2009). What is the Value of Course-Specific Learning

Goals? Journal of College Science Teaching, 39, 2, 52-57. PDF available at

www.cwsei.ubc.ca/SEI_research/files/LifeSci/Simon_Taylor_ValueOfCourseSpecif

icLG.pdf

2. Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The

Cognitive Domain. New York: David McKay Co Inc.

Adapted from Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for

Learning. Teaching, and assessing: A revision of bloom's taxonomy of educational

objectives.

Revised Bloom’s Taxonomy www.celt.iastate.edu/teaching/RevisedBlooms1.html

3. Excerpt from Wieman, C. (2007). Slides from the Wieman Learning Goals

Workshop. www.cwsei.ubc.ca/resources/learn_goals.htm

4. California DMV Sample Class C Written Test 5

www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

http://www.celt.iastate.edu/teaching/RevisedBlooms1.html

http://www.cwsei.ubc.ca/resources/learn_goals.htm

http://www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

References, continued


37

5. Ambrose, S.A., Bridges, M.W., DiPietro, M., Lovett, M.C., & Norman, M.K. (2010).

How Learning Works. San Fransisco: Jossey-Bass.

6. Glued to Games: How Video Games Draw Us In and Hold Us Spellbound

http://www.gluedtogames.com/

7. Gee, J.P. (2005). Learning by Design: good video games as learning machines. E-

Learning 2, 1, 5-16.

8. Wertsch, J.V. (1984). The zone of proximal development: Some conceptual issues.

New Directions for Child and Adolescent Development, 1984, 23, 7–18.



Bloom’s Taxonomy of the Cognitive Domain

(Levels of Learning) [Wieman, 2007] 38

6. Synthesis: transform and combine ideas to create something new

develop, create, propose, formulate, design, invent

5. Evaluation: think critically about and defend a position

judge, appraise, recommend, justify, defend, criticize, evaluate

4. Analysis: break down concepts into parts

compare, contrast, categorize, distinguish, identify, infer

3. Application: apply comprehension to unfamiliar situations

apply, demonstrate, use, compute, solve, predict, construct, modify

2. Comprehension: demonstrate understanding of ideas, concepts

describe, explain, summarize, interpret, illustrate

1. Factual Knowledge: remember and recall factual knowledge

define, list, state, label, name, describe
