8.5 x 11 Raw CPOP form

A New Classroom Practices Observation Protocol. Francis Jones <fjones@eos.ubc.ca> Science

Education Initiative

Instructional mode

D Delivery (lecture, probs., Stu. q’ns, etc. … see “Happening”) E Experiment / Simulation / Demo M Media: Video, Anim’n, photo (other than usual ppt) S Socratic (continuous question posing) Q Question to students, not Socratic (open, simple clicker, etc.) A Active students (eg clicker sequence, worksheets, etc.) P Presentation by student(s) T Test or quiz (include groups if two-stage or TBL, etc.)

Instructor is doing …

s Static or low key - Talking m moving, interacting with screen, etc. - Talking w Wandering around class - Talking R Real time writing (board, doc. projector, etc.) H Helping or guiding student work (eg circulating) L Listening or marking (during presentations etc. ) O One-on-one: focus on individual (class may be listening) N Nothing – waiting for activity, etc.

Happening or covering

E Explain new content, knowledge or procedure S Summarizing or synthesizing R Review, revisit, refer to prior content OR knowledge framework. P Practice/Apply; problem, thinking, analysis, etc. C Case study or example(real world; may be with “P”) F Follow up, feedback on work or thinking. B Brainstorming or novel thinking M Motivational (“here’s why we’re doing this”, etc.) H Humor or just friendly A Administration (assign hmwrk, return tests, etc) O Other – explain in comments.

Students Doing

I Individual (listening or doing) C Contributing (e.g. explain in turns, etc.) Q Student asks question P Pairs (or peer instruction) G Groups (note group size in comments) W Whole class (“shout out”, discussion, etc)

Blooms Taxonomy level (updated – more action oriented) R/U remembering / understanding A/A applying / analyzing E/C evaluating / creating Judge based on verbs that are in use (instructor or activity). Blooms Taxonomy level (original from 1956) K / C knowledge / comprehension A / A application / analysis S / E synthesis / evaluation

Eng = Engagement observations: Here’s how to do this … 1) Select N students to observer. 2) Enter “X/N” = count of #students engaged (or not “disengaged”) in what’s going on. Key: “Key Flag” helps identify key messages for feedback to the instructor.

Observation Codes

Potential purposes • Characterize courses: - Student experiences, instructor actions, others • Course Transformations Impact Assessments - After transformation; pre-post if possible • Compare courses … eg. all 1xx; core vs. elective; etc. - Busy / idle students; balance of motivation, basics, context; theory / practical; etc. • Professional and peer development for instructors: - Based on code patterns and observer feedback.

Precedent • EOS – “engagement observation protocol” (E. Lane) - Included in CPOP • UBC - Physics feedback forms (P. Newbury and C. Heiner) • RTOP – Reformed Teaching Observations Protocol • TDOP1 – Teaching Dimensions Observation Protocol1 - Foundation of CPOP - Explored in Math (W. Code)

Trials to date: EOAS courses:

1xx courses: 4 2xx courses: 5 3xx courses: 7 Eosc222: 12 classes, Jan-March

Procedure and data:

• Observation form stabile at version 10.

• Forms printed on LiveScribe paper.

• Audio records are keyed to coding on forms.

• Coded forms transferred to spreadsheet template, including color and aggregation. eosc326, Nov 16th, 2011

DSQEAMP TsmwRHLONESRPC FBMHAO I CQPGWRUAAEC

0 10%

2 20%

4 60%

6 75%

8 0%

10 0%

DSQEAMP T s mwRH L ONESRPC FBMHAO I CQPGWRUAAEC Eng

12 83%

14 69%

16 50%

18 50%

20 0%

DSQEAMP T s mwRH L ONESRPC FBMHAO I CQPGWRUAAEC Eng

22 65%

24 0%

26 0%

DSQEAMP T s mwRH L ONESRPC FBMHAO I CQPGWRUAAEC Eng

28 0%

30 94%

32 0%

34 0%

36 0%

38 0%

40 0%

DSQEAMP T s mwRH L ONESRPC FBMHAO I CQPGWRUAAEC Eng

42 0%

44 0%

46 0%

48 0%

50 0%

EngInst Mode Inst. do. Happen/CoverStu. doingBlooms

eosc326, Nov 18th (active Friday), 2011

DSQEAMP TsmwRHLONESRPC FBMHAO I CQPGWRUAAEC

0

2

4

6

8

10

DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC

12

14

16

18

20

DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC

22

24

26

DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC

28

30

32

34

36

38

40

DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC

42

44

46

48

50

DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC

52

54

56

58

60

DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC

Sumsact

lect

Inst Mode Inst. do. Happen/CoverStu. doingBlooms

Mo

de

Inst

ruct

or

Ha

pp

en

ing

Co

veri

ng

Stu

de

nts

Blo

om

s

Lecture “Active Friday”

Mo

de

Inst

ruct

or

Hap

pe

nin

g C

ove

rin

g

Stu

de

nts

Blo

om

s

T

ime

T

ime

Completed raw data form

Transcribed raw data: 2 classes of eosc326

Data

Characteristics and participation in SEI

Some features to notice:

• The “Not eosc” courses were active & application oriented, but did not use groups or peers much.

• “No SEI”: These involved more delivery, more telling, and more solo time in class.

• “SEI applied” tended to be more active, applying, in groups, and there is also a wider variety.

Data recorded at 2-minute intervals in 1st, 2nd and 3rd yr courses.

0%

50%

100%

eosc

114

eosc

112

eosc

221

eosc

373

eosc

326a

eosc

322

eosc

222a

eosc

222b

eosc

332

eosc

340

eosc

110

eosc

116

eosc

270

eosc

333

eosc

326l

atsc

201

envr

200a

envr

200b

envr

300

S,Q,A

D,E,M

0%

50%

100%

eosc

114

eosc

112

eosc

221

eosc

373

eosc

326a

eosc

322

eosc

222a

eosc

222b

eosc

332

eosc

340

eosc

110

eosc

116

eosc

270

eosc

333

eosc

326l

atsc

201

envr

200a

envr

200b

envr

300

P,C,B

E,S,R,M,H,A

0%

50%

100%

eosc

114

eosc

112

eosc

221

eosc

373

eosc

326a

eosc

322

eosc

222a

eosc

222b

eosc

332

eosc

340

eosc

110

eosc

116

eosc

270

eosc

333

eosc

326l

atsc

201

envr

200a

envr

200b

envr

300

P,G,W

I,C,Q

EOS; SEI applied

Not EOSC

EOS; No SEI

Instructional mode:Active vs Delivery

What happens in class:Applying vs Telling etc.

What students are doing:Pairs/groups vs Solo

Six weeks of EOSC222 classes: What analysis possibilities?

Address questions by combining codes:

Variations versus time:– More group work in Feb.

– “Active” tends to follows basics.

Sort by Bloom’s score:

Score = 1*R+2*U+3*A+4*A+5*E+6*U

- Higher Blooms … more active.

0%

50%

100%

active

deliver

0%

50%

100%

active

deliver

Lower Bloom’s Higher Bloom’s

0%

50%

100%

pairs/grps

solo

Students: solo vs. pairs or groups

Instructional mode

Some Patterns – 19 different classesSorted by Blooms score (see codes, below left)

low Bloom:– mostly 1st, 2nd yr. (BUT see exceptions)– high "telling" instr. Mode– high "explain or review" cover

high Bloom:– mostly 2nd, 3rd yr. (BUT see exceptions)– more students working together– exception is eosc222 poster session

eosc112– high blooms, but…– low engagement

eosc114– low blooms, but…– strong engagement

envr300– high blooms, and..– most diverse instructor doing– All guided group work

eosc340– moderate blooms– very low "explaining“– diverse instr. & stu. doing– engage starts low stays high throughout

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

eo

sc2

22

a

en

vr2

00

b

eo

sc1

14

eo

sc3

32

eo

sc2

70

eo

sc3

33

eo

sc1

16

eo

sc1

10

eo

sc3

22

atsc

20

1

eo

sc2

21

eo

sc3

73

eo

sc3

40

eo

sc3

26

l

eo

sc3

26

a

eo

sc2

22

b

eo

sc1

12

en

vr3

00

en

vr2

00

a

Blooms

C

E

A

A

U

R

0

2

4

6

8

10

12

14

16

18

20

eo

sc2

22

a

en

vr2

00

b

eo

sc1

14

eo

sc3

32

eo

sc2

70

eo

sc3

33

eo

sc1

16

eo

sc1

10

eo

sc3

22

atsc

20

1

eo

sc2

21

eo

sc3

73

eo

sc3

40

eo

sc3

26

l

eo

sc3

26

a

eo

sc2

22

b

eo

sc1

12

en

vr3

00

en

vr2

00

a

Happen / cover

R

S

E

0

2

4

6

8

10

12

14

eo

sc2

22

a

en

vr2

00

b

eo

sc1

14

eo

sc3

32

eo

sc2

70

eo

sc3

33

eo

sc1

16

eo

sc1

10

eo

sc3

22

atsc

20

1

eo

sc2

21

eo

sc3

73

eo

sc3

40

eo

sc3

26

l

eo

sc3

26

a

eo

sc2

22

b

eo

sc1

12

en

vr3

00

en

vr2

00

a

Stu. doing

W

G

P

0

2

4

6

8

10

12

14

16

18

20

eo

sc2

22

a

en

vr2

00

b

eo

sc1

14

eo

sc3

32

eo

sc2

70

eo

sc3

33

eo

sc1

16

eo

sc1

10

eo

sc3

22

atsc

20

1

eo

sc2

21

eo

sc3

73

eo

sc3

40

eo

sc3

26

l

eo

sc3

26

a

eo

sc2

22

b

eo

sc1

12

en

vr3

00

en

vr2

00

a

Instr. mode

M

E

D

0%

20%

40%

60%

80%

100%

0 10 20 30 40 50 60

Pe

rce

nta

ge o

f sa

mp

le w

ho

w

ere

"e

nga

ged

"

Minutes into the class

Engagement: 4 courses, 1 class each

eosc340

eosc114

eosc110

eosc112

Engagement: 4 courses, 1 class each

Comparing two science electives:

D67%S

0%

Q11%

E0%

A22%

M0%

P0%

T0%

Instr. mode, 373

D30%

S7%

Q13%

E0%

A50%

M0%

P0%

T0%

Instr. mode, 340

I46%

C6%

Q8%

P8%

G24%

W8%

Stu. doing 340

I78%

C7%

Q0%

P15%

G0% W

0%

Stu. doing 373

Instructional

mode

Students

doing

EOSC 340

Climate change

EOSC 373

Oceanography II

NOTES and caveats: - Each course was observed only once, and no information is used relating to what students do

outside of class. - Also, the observation protocol has not yet been validated by having several people conduct simultaneous observations. - There may be other shortcomings; the project

is definitely in only the “development” stages.

ABSTRACT: A new observation procedure is being developed aimed at helping characterize instructional practices, student actions, "Bloom's levels" and other classroom characteristics, on a timeline of the class. The intention is to develop a procedure that is useful for characterizing any type of class, rather than to make judgments about "quality" in any sense. Preliminary examples of trials are shown with some possibilities for analysis; including comparisons of several classes from one course, & single classes from 16 different courses.