21st Century Teaching and Learning Evaluation Project 2011

CFF•21st CTL Evaluation Results

Changes in Teaching Practices

Jeanne Vilberg & Robin ClausenPenn State University

What is CFF•21st CTL?

21st CTL EvaluationSince March 2007, the 21st CTL Evaluation Project has collected:

• 5,500 21st CTL Classroom Observations

• 55,000 Teacher Surveys

• 650,000 Student Surveys

• 18,000 PATI Surveys (Instructional Phase)

Analysis

Start  of  Program    (Fall  2007/2008)

Spring  2010

• The same 21st CTL teachers compared at different points in time.

• Data collection periods (fall/spring 2007 - 2010)

• Cohorts (teachers with different number of years in program)

InstrumentsObservations

Teacher/Student SurveysPATI (Instructional Phase)

http://cff.psu.edu

Findings

Has 21st CTL Changed...

• complexity of class content

• relevance of class content

• instructional style

Overall

Complexity of Content

Basic Skills Higher Order

Relevance of Content Artificial Real World

Instructional Style Didactic Constructivist

Change in the Complexity of Content

21st CTL teachers now focus more on higher order topics than they did at the beginning of the program.

Complexity of Content Describe the content your class is designed to convey.

(Almost All Higher Order Skills / More Higher Order Skills than Content Knowledge Combined)

10%

13%

16%

19%

22%

Start of Program Spring 2010

Cohort 1Cohort 2Cohort 3

Differences5.89%6.85%4.65%

Teacher Survey

Complexity of Content Percent of Time Spent in Really Complex Thinking / Problem Solving

(Quite a Lot / Almost All the Time)

35%

37%

39%

41%

43%

Start of Program Spring 2010

Cohort 1Cohort 2Cohort 3

Student Survey

Differences2.52%2.00%1.74%

Observations

Complexity of Content

50%

54%

58%

62%

66%

70%

Start of Program Spring 2010

First ThirdMiddle ThirdLast Third

Overall Comparison by Class Period - Basic - Higher Order (Ratings of 5, 6, and 7 Combined)

Differences9.12%8.35%4.06%

Classroom Content is More Authentic

21st CTL teachers report students use technology to solve real world problems more often than non-21st CTL teachers.

Teacher Survey

Cohort 14 Years

Cohort 14 Years

Cohort 23 Years

Cohort 23 Years

Cohort 32 Years

Cohort 32 Years

Fall 2007 Spring 10 Fall 2007 Spring 10 Fall 2008 Spring 10

Exercises & Assignments

17.48% 12.45% 19.71% 12.36% 16.67% 12.04%

Blend First / Last 42.48% 35.65% 45.98% 39.61% 47.97% 42.95%

Even Balance 30.22% 35.29% 25.59% 35.76% 28.73% 34.61%

Blend First / Last 7.40% 13.31% 7.40% 10.04% 6.10% 8.21%

Projects and Products

2.43% 3.30% 1.32% 2.23% 0.54% 1.65%

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Relevance of Content Artificial - Real World

How would you describe the work that students do?

Daily

Weekly

Monthly

0% 5% 10% 15% 20% 25% 30%

Cohort 1 - 4 Years in 21st CTLCohort 2 - 3 Years in 21st CTLCohort 3 - 2 Years in 21st CTLNon-21st CTL

How often do students use technology to solve real-world problems?

Relevance of Content Artificial - Real World

PATI Survey

(Instructional Phase)

Observation

Cohort 3 (2 Years in 21st CTL)Cohort 3 (2 Years in 21st CTL)Cohort 3 (2 Years in 21st CTL)

First Third of the Class

Start of ProgramFirst Third of the

ClassSpring 2010

First Third of the Class

Difference

Middle Third of the Class

Start of ProgramMiddle Third of the

ClassSpring 2010

Middle Third of the Class

Difference

Final Third of the Class

Start of ProgramFinal Third of the

ClassSpring 2010

Final Third of the Class

Difference

32.52%65.18%32.67%36.45%70.15%33.70%35.47%70.16%34.68%

Does the Lesson have a “Real World” Context (authenticity)? (More real world, rating of 5, 6, or 7 combined)

1 2 3 4 5 6 7 Real WorldArtificial

Relevance of Content Artificial - Real World

Instructional Style is Changing

21st CTL has changed the way many teachers teach; increasing percentages of teachers identify their style as constructivist.

Instructional Style Didactic - Constructivist

Teacher Survey

Cohort 14 Years

Cohort 14 Years

Cohort 23 Years

Cohort 23 Years

Cohort 32 Years

Cohort 32 Years

Fall 2007 Spring 10 Fall 2007 Spring 10 Fall 2008 Spring 10

Didactic 29.63% 26.34% 33.15% 26.21% 34.01% 29.68%

Even Balance 47.64% 52.93% 48.53% 51.83% 50.41% 52.39%

Constructivist 22.73% 20.73% 18.32% 21.96% 15.58% 17.92%

Less DidacticMore Even BalanceLess Constructivist

Less DidacticMore Even BalanceLess Constructivist

Less DidacticMore Even BalanceMore Constructivist

Less DidacticMore Even BalanceMore Constructivist

Less DidacticMore Even BalanceMore Constructivist

Less DidacticMore Even BalanceMore Constructivist

How would you describe your teaching style?

Instructional Style Didactic - Constructivist

Observation (Overall)

Class Period Term Mean Difference

First ThirdFall 09 3.951

First ThirdSpring 10 4.365 0.414

Middle ThirdFall 09 4.372

Middle ThirdSpring 10 4.801 0.429

Last ThirdFall 09 4.489

Last ThirdSpring 10 4.854 0.365

1 2 3 4 5 6 7 ConstructivistDidactic

Instructional Style Didactic - Constructivist

0%

25%

50%

75%

100%

Cohort 1 Cohort 2Cohort 3

How Assignments GradedTopics StudiedTopics of Papers or AssignmentsWay Topics StudiedWorking Together or Alone

Who makes decisions about... (Teacher Completely / Teacher Mostly)

Student Survey

Spring 2010

• What instructional strategies are being used?

Instructional Strategies

Instructional StrategiesTeacher Survey

Teaching Strategies (Valuable / Very valuable)

Cohort 1 Cohort 2 Cohort 3

Teacher Led Discussion - High 1 1 1

Problem-based Learning 2 4 3

Authentic Learning 3 2 4

Teacher Lecture 4 3 2

Teacher Led Discussion - Low level 5 6 6

Multi-modal Teaching 6 5 5

Project-based Learning 7 7 7

Collaborative Learning - informal 8 8 8

Peer Teaching 9 9 9

Collaborative Learning - formal 10 10 10

WebQuests 11 11 11

Learning Centers 12 12 12

91-97%

80-90%

33-38%

42-45%

68-78%

Instructional Strategies

Teacher Survey

Teaching Strategies (Valuable / Very valuable)

85%

87%

90%

92%

95%

97%

Cohort 1 Cohort 2 Cohort 3

Teacher Led Discussion - High Level Outcomes Problem Based LearningAuthentic Learning Teacher Lecture

Spring 2010

Use of Strategies

0%

10%

20%

30%

40%

50%

Cohort 1 Cohort 2 Cohort 3 Non-21st CTL

Collaborative Learning Differentiation of LearningInteractive Instruction Independent Study

PATI Survey

(Instructional Phase)

Spring 2010

Rate the Level of Your Current Use of the Strategy (High)

Use of Strategies

0%

15%

30%

45%

60%

Cohort 1 Cohort 2 Cohort 3 Non-21st CTL

Direct Instruction Mediating Student ThinkingInquiry Experiential Learning

Rate the Level of Your Current Use of the Strategy (High)

PATI Survey

(Instructional Phase)

Spring 2010

‣ What has been the impact of 21st CTL / CFF on teaching in your school?

‣ What impact do these changing teaching practices have on student outcomes?

Questions

Jeanne Vilberg, jav6@psu.eduRobin Clausen, rlc237@psu.edu

CFF • 21st CTL Evaluation http://cff.psu.edu/y4report

Thank you.

Image Attribution:

School Bell Towerhttp://www.flickr.com/photos/road_less_trvled/2061666068/by road_less_trvled Taken October 13, 2007 in Zionsville, PAhttp://creativecommons.org/licenses/by-nc/2.0/deed.en

Classroom with Desks against Wallhttp://www.flickr.com/photos/ransford/770039665/by Ben RansfordTaken on June 9, 2007http://creativecommons.org/licenses/by-nc/2.0/deed.en