(Establishing a) Research Agenda for Online Education in ... · Jackie Elser, Trish Loeblein Molly Giulliano Susan M. Nicholson-Dykstra Sara Severance Emily Quinty Mindy Gratny, Kate

6/2/13

Distance Education and Online Learning in Physics Workshop © 2013, Finkelstein 1

(Establishing a) Research Agenda for Online Education in Physics

A quick tour of theory A taxonomy of research questions

Noah Finkelstein

University of Colorado Boulder [email protected]

APS Workshop on Online Education & Tech in Physics Education

Physics Education Research group at CU-Boulder

Funded by: National Science Foundation William and Flora Hewlett Foundation American Association of Physics Teachers Physics Teacher Education Coalition American Institute of Physics American Physical Society National Math & Science Initiative Howard Hughes Medical Institute And generous donors like you.

Grad Students: Stephanie Barr Kara Gray May Lee Mike Ross Ben Spike Ben Van Dusen Bethany Wilcox

Teachers / Partners / Staff: Shelly Belleau Jackie Elser, Trish Loeblein Molly Giulliano Susan M. Nicholson-Dykstra Sara Severance Emily Quinty Mindy Gratny, Kate Kidder John Blanco, Sam Reid Chris Malley, Jon Olson Oliver Nix, Nina Zabolotnaya

Faculty: Melissa Dancy Michael Dubson Noah Finkelstein Valerie Otero Kathy Perkins Steven Pollock Carl Wieman (on leave)

Postdocs/ Scientists: Charles Baily Danny Caballero Stephanie Chasteen Julia Chamberlain Katie Hinko Kelly Lancaster Emily Moore Ariel Paul Noah Podolefsky Benjamin Zwickl

The image cannot be displayed. Your computer

6/2/13


Why Education?

Individual Empowerment Societal Empowerment Workforce / Economic

Development

6/2/13


Tools are key in education

•  Shifting learning paradigms From: acquisition/delivery To include: enculturation

•  Tools are defining elements of culture Tools (Technologies) essential to enculturate students

“The medium is the message.”

- Marshall McLuhan

6/2/13


A Word about Tools

How many people pull?

Well designed tools lead naturally to their intended use

6/2/13


Toy-theoretical Framework

Finkelstein (2005), adapted from Cole, M. (1996), Cultural Psychology

i.  tools mediate our understanding / cognitive processes

ii.  context shapes the meaning / use of tools

7960.0 10

= 796.0

7960.0 16

= Hard(er) 1F18.0

10 = 1F1.8

Easy!

Decimal (Base 10) Hex (Base 16)

Tools Mediate Thought

6/2/13


Role of New Tech in Education

Education is a social act Technologies can reorganize social processes

Built in to our classes?

6/2/13


Trad’l Model of Education Instruction via transmission Individual Content (e.g. circuits) transmissionist

Where does our model come from…

– Sumer, circa 3000 BCE

6/2/13


PER Theoretic Background Instruction via transmission Individual Content (E/M) transmissionist

Individual Prior knowledge

Content (E/M) Construction constructivist basic constructivist

Contextual constructivist context

Individual Prior knowledge Attitudes and Beliefs Student background Affect

Content (E/M)

Tools / Instructor . . .

Finkelstein, N. (2005) Context in the Context of Physics Education, IJSE Finkelstein, N. (2005-2010). NSF CAREER Grant: REC# 0448176

Many PER technological innovations

6/2/13


by actively engaging students & building on what students know

…

0

0.1

0.2

0.3

0.4

0.5

0.6

0.08 0.14 0.20 0.26 0.32 0.38 0.44 0.50 0.56 0.62 0.68

Fractio

n o

f Courses

R. Hake, ”…A six-thousand-student survey…” AJP 66, 64-74 (‘98).

traditional lecture We Are Doing Better

interactive engagement

learning gain

6/2/13


Establishing a Research Agenda

Scales of Research: Impacts of Tech on…

•  Individual Students •  Nature of Environment / Tools •  Faculty Roles / Practices •  Institutional Roles/ Practices •  Scaling / Sustaining (models of change) •  Societal impact •  What can’t be measured

6/2/13


Individual Students

Student learning: – Conceptual mastery (pre / post) – Trad’l Problem solving –  Skills / Practices

“I think we are rushing this a little bit. I haven’t seen a single study showing that online learning is as good as other learning.”

- Sebastian Thrun (AI course teacher)

21st Century Skills

6/2/13


Broad Objectives Accessible, Understandable, and Fun

Connections to Everyday Life

Conceptual Learning

Engage in Exploration

Sense of Ownership

Individual Students

Student engagement – Participation – Retention –  Interactions

•  with each other •  with instructional staff

– Student analytics •  what can we get •  what do they tell us

6/2/13


Who will benefit from (free) Ed Tech?

-Justin Reich

•  Highly interactive, dynamic, visual simulations (> 100)

•  Fun, bridge to real world, open, explicit visual/ conceptual models

•  Research-based

•  Freeware, Online or downloadable

•  Tested: design and use

•  Supporting materials: lessons and translations

Who is familiar with PhET?

http://phet.colorado.edu

6/2/13


Naïve thoughts: Wow.. PhET is cool

Therefore it will solve our educational problems.

Or What are problems can be solved with this tool? (solution in want of a problem)

Or Now, best to ignore existing tools/ practices

Research on Design •  Extensive feedback on each sim •  Research on many sims shows what

works

Research on Use •  Learning gains •  Broader educational objectives •  Effective implementation

Research with Simulations

6/2/13


Build and Study Simulations Physics Education Technology Project

Built on Learning Principles -  in Design and -  in Use

Study: -  Whether they work -  How they work -  Potential for use

Design of PhET

•  Engaging Interactive •  Dynamic feedback •  Constuctivist approach •  Workspace for play or ‘messing about’ •  Visual models •  Productive constraints

K.K. Perkins, et al, “PhET: Interactive Simulations for Teaching and Learning Physics” Physics Teacher (2006)

6/2/13


Circuit Construction Kit (CCK)

Use of Computer Sims: PhET (Chris Keller)

Study the comparative use and performance of students in:

•  Labs / Recitations •  Lectures

6/2/13


vs.

Laboratory (alg. based): CCK and Real Equip.

N D. Finkelstein, et al, “When learning about the real world is better done virtually: a study of substituting computer simulations for laboratory equipment,” PhysRev: ST PER 010103 (2005)

DC Circtuis Exam Questions

00.1

0.20.3

0.40.50.6

0.70.8

0.91

q1 q2 q3 cntlQuestion

Frac

tion

Corr

ect

CCK (N =99)TRAD (N=132)

p < 0.001

Lab (alg. based): build circuit & describe it

0

5

10

15

20

25

30

35

Time (

min)

CCK TRAD No Labp < 0.01

6/2/13


vs.

Simulation used in Lec 1, Demo used in Lec 2

0

0.25

0.5

0.75

1

Fra

ctio

n C

orr

ect

Lec 1 Silent

Lec 1 Discussion

Lec 2 Silent

Lec 2 Discussion

Real Demo used in Both Lectures

0

0.25

0.5

0.75

1

Frac

tion

Cor

rect

Lec 1 Silent

Lec 1 Discussion

Lec 2 Silent

Lec 2 Discussion

Lecture: Simulation Spurring Discussion Peer Instruction environment (Calculus)

vs.

CMU OLI in Stats…

605 of the 3,046 students enrolled in statistics �� 6 institutions OLI tools. (2012)

6/2/13


Structure of Study

•  In the treatment group, students took the course in a “hybrid” mode, in which interactive, online course materials were supplemented with one hour of face-to-face instruction per week

•  Students in a control group took a traditional course, with 3-4 hours of lectures per week and standard textbooks.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.08 0.14 0.20 0.26 0.32 0.38 0.44 0.50 0.56 0.62 0.68

Fractio

n o

f Courses

R. Hake, ”…A six-thousand-student survey…” AJP 66, 64-74 (‘98).

traditional lecture We Are Doing Better in “Lecture”


learning gain

6/2/13


0

0.1

0.2

0.3

0.4

0.5

0.6

0.08 0.14 0.20 0.26 0.32 0.38 0.44 0.50 0.56 0.62 0.68

Fractio

n o

f Courses

Pollock & Finkelstein, Physical Review, 4, 010101 (2008).

<g> = post-pre �� 100-pre

traditional lecture

Course Transformation: Engagement in Learning


CU - IE & Tutorials

CU - IE & trad recitations

learning gain

Bowen 1 year later (Mar 2013) Lack of Hard Evidence How effective has online learning been in improving (or at least maintaining) learning outcomes achieved by various populations of students in various settings? Unfortunately, no one really knows the answer to either that question or the important follow-up query about cost savings.

6/2/13


Nature of the Tool

Affordances Constraints Broad access (for those with access)

Asynchronous (on demand) (dubbed “anywhere / anytime”)

Massive / scalable / cheap?

Personalizable (minimally now, more later perhaps)

Geographically distributable

Online Ed

6/2/13


Affordances Constraints Broad access (for those with access) Limited bandwidth (comparatively) Asynchronous (on demand) (dubbed “anywhere / anytime”)

Easy exit / distraction / lack of intentionality: (does First Life trump Second Life?)

Massive / scalable Limited feedback capabilities (until the singularity?)

Personalizable (minimally now, more later perhaps)

Failure modes are catastrophic?

Geographically distributable Too focused / lack of emergence: misses the "inefficient" "incidental" "accidental" "serendipity" - the water cooler conversation No research base (yet) Don’t know how to use these tools (yet)

Online Ed

Affordances Constraints Social / human interactive Synchronous

High bandwidth Personalized / personalizable

Dynamic, responsive feedback Emergent Models of practice based on research Local / contextually relevant

Face-to-Face

6/2/13


Affordances Constraints Social / human interactive Limited size / access Synchronous Synchronous

High bandwidth Historically embedded ineffective

models Personalized / personalizable Limited geography

Dynamic, responsive feedback Lack of support Emergent Models of practice based on research Local / contextually relevant

Face-to-Face

Faculty / staff Engagement

What is same / different? What are demands?

6/2/13


Faculty Participation

Institutional Scale

What are institutional motives �� (stated and practiced)

How do these change institutional�� practice & identity

Could this change the nature of physics?

6/2/13


There is a coming revolution in teaching (online and elsewhere)

Is it?

A. Hype B. An economic necessity C.  Change the nature of (higher) education D. None of the above E.  All of the above

F. Not enough information

“Books will soon be obsolete in the public schools… Scholars will be instructed through the eye. It is possible to teach every branch of human knowledge with the motion picture. Our school system will be completely changed inside of ten years.”

- Thomas Edison (1913)

6/2/13


“It’s hard to make predictions, especially about the future”

- Niels Bohr …or Mark Twain

…or Yogi Berra

“Online education is absolutely not a replacement for the high-quality education that a top university can give. Why do people pay $50,000 a year to attend an institution like Caltech? The reason is that the real value of a Caltech education isn’t just the content. Content is increasingly available for free on the Web. The real value is the interactions with professors and other, equally bright students.”

- Andrew Ng (co-founder of Coursera)

6/2/13


Sustaining and Scaling

Can MOOCs be rolled-out, replicated: Is it easy to run a second time? Is it easy to refine?

Who ends up using these to what ends?

Institutional & Organizational Change

6/2/13


Cultural Scale

What are long term impacts? What are costs of effective education?��What can we not measure?

Loads of Blogging…

Language, history and pressures:��May 15, http://thenewinquiry.com

Pressures on education Call for slowing pace Understanding the practice Goals: certification? Learning?

6/2/13


Consider the Senate review of For-Profit Universities

6/2/13


Profit as Motive

For-profit going online . . .

6/2/13


What research would help you? and physics? and society?

•  Individual Students •  Nature of Environment / Tools •  Faculty Roles / Practices •  Institutional Roles/ Practices •  Scaling / Sustaining (models of change) •  Societal impact •  What can’t be measured

fin

http://per.colorado.edu or http://spot.colorado.edu/~finkelsn

Documents

(Establishing a) Research Agenda for Online Education in ... · Jackie Elser, Trish Loeblein Molly Giulliano Susan M. Nicholson-Dykstra Sara Severance Emily Quinty Mindy Gratny, Kate