Click here to load reader

Evaluating Creative Teaching Practices and Collaborative Learning Environments

  • View
    32

  • Download
    4

Embed Size (px)

DESCRIPTION

Evaluating Creative Teaching Practices and Collaborative Learning Environments. Ann Mastergeorge and Ingrid Roberson. UCLA Graduate School of Education & Information Studies Center for the Study of Evaluation National Center for Research on Evaluation, Standards, and Student Testing. - PowerPoint PPT Presentation

Text of Evaluating Creative Teaching Practices and Collaborative Learning Environments

Stricom 1.21.9 CC R E S S T / U C L A
Evaluating Creative Teaching Practices
and Collaborative Learning Environments
Center for the Study of Evaluation
National Center for Research on Evaluation, Standards, and Student Testing
Ann Mastergeorge and Ingrid Roberson
Annual AERA Convention
C R E S S T / U C L A
Overview
Methodological Issues in Measuring Engagement in School Reform
Discussion and Implications of Engagement for Classroom Practice
C R E S S T / U C L A
Introduction
to Creative Learning Communities
Supports school reform in schools across the country serving at-risk student populations
Promotes change centered on creative teaching practices and collaborative learning environments
Emphasizes the enhancement in students’ joy of learning in engaged learning environments
C R E S S T / U C L A
The Construct of Engagement
Substantive Engagement
Focuses on the emotional quality of student’s involvement in learning activities
Emphasizes student’s enjoyment of learning activities since it is self-motivating and self-reinforcing
C R E S S T / U C L A
Developing
Substantive Engagement depends on both student and teacher engagement in classroom learning activities
Includes the social dimension of learning
Requires a high degree of reciprocity
Learning Environments and Teaching Practices construct opportunities for students to experience engagement
Communicates support and care
Models enthusiasm for learning
C R E S S T / U C L A
Student Engagement
C R E S S T / U C L A
Methodological Issues in Measuring
Engagement in School Reform
Multiple Methods of Evaluation
Impact of School Reform on Engagement
Employs a theory of action
Provides focal points for the evaluation
Suggests a sequence in which various effects may be expected
C R E S S T / U C L A
Anchor in
curriculum, instruction
Provide
materials
on-going professional inquiry
Align with
project goals
project adjustments
Align with
Shift teacher
Improve student attitudes towards learning
C R E S S T / U C L A
Measuring the Implementation Process
Goals
Leadership
Professional Development
Teacher Survey
Teacher Interview
C R E S S T / U C L A
Teacher Survey
C R E S S T / U C L A
Teacher Interview
C R E S S T / U C L A
Measuring the Impact on Engagement
Features both Procedural and Substantive Engagement Theory
Focuses on Student Engagement
C R E S S T / U C L A
Teacher Survey
C R E S S T / U C L A
Teacher Interview
C R E S S T / U C L A
Student Survey
C R E S S T / U C L A
Classroom Practice Observation Protocol
Captures students’ attention to and enjoyment of learning activities
C R E S S T / U C L A
Dimensions of Classroom Practice
Challenge of Lesson Activity
Implementation of Lesson Activity
C R E S S T / U C L A
C R E S S T / U C L A
C R E S S T / U C L A
Challenge of Lesson Activity Exemplar
Teacher: Let’s think about all of the different conditions.
Student: My experiment needs more water.
Teacher: I heard someone else say too cold… so cold or warm.
Student: Ours is darker and theirs is lighter [amount of light] .
Teacher: Can you think of anything else?
Student: I think we should put less water and less worms.. with the
worms they will just dry, will shrivel and do nothing and
the water will just get everything damp.
Teacher: What do you think is the most important thing to make the
leaves decompose?
Student: Dry, the leaves crumble up. It’s like they turn into dirt.
Teacher: Maybe one thing we can do this week, half put water and the other half put no water and keep everything else the same. (CO.10.08.YR2)
C R E S S T / U C L A
C R E S S T / U C L A
C R E S S T / U C L A
Implementation of Lesson Activity Exemplar
After reviewing both the proof reading process and scoring rubric (both clearly written on chart paper), teacher runs through following example.
Dear Curator:
Finally, the machines were interesting to see.
Thank You, Ms. X
Teacher: Alright, so this letter… Based on that, where would you put
them if you were the teacher?
Student: 2, though it does have an introduction.
Student: 2, it doesn’t give them a sense and is supposed to be in
paragraphs.
Teacher: Anybody agree or disagree and why?
Student: It looks like a letter but it doesn’t sound like a letter
(CO.01.03.YR2)
C R E S S T / U C L A
1
2
3
4
students wander
desks chatting with
other students about
unrelated topics or
different metal wires.
They note their
observations in their
science journal. Most
Procedural Engagement
C R E S S T / U C L A
Procedural Engagement Exemplar
All students are on-task, attentive to the teacher. Some students raise their hands to answer teacher questions.
Teacher: Is this a sentence?
Student: Yes.
Teacher: So what does it have to have at the beginning?
Student: A capital letter.
Teacher: What does it have to have at the end?
Student: A question mark.
Teacher: What do I have to do to show these words came out of my
mouth? [Three students raise their hand]...come up to the
board X.
Student goes to board and puts quotation marks around the sentence.
(CO.02.01.YR2)
C R E S S T / U C L A
NA
Not
appli-
cable
that?" Other students,
while others still trying
deep concentration.
engaged. Many students shout,
when their light bulb works. Some
students smile, their eyes
on. Other students still trying
review their notes or ask another
student, “Hey, can you help me?”
Near the end, students can be
heard talking with each other
about which metal worked better
as a conducter.
Substantive Engagement
C R E S S T / U C L A
Substantive Engagement Exemplar
The students are all listening. They are riveted, looking at the teacher and the book intently. The teacher engages them in a discussion with 90% of the students participating. Students’ hands shoot up in the air waving back and forth frantically with wiggling fingers, going "oh, oh”.
Teacher: What woke him up?
Student: The sun.
Student: My mom.
Student: Sometimes my brother.
Student: My mom’s boyfriend hits me with a pillow.
Student: My dad gets me up.
Teacher: So our caterpillar comes out of the egg and how did he
feel? He was what?
C R E S S T / U C L A
C R E S S T / U C L A
C R E S S T / U C L A
Quality of Instructional Discussion Exemplar
At one table, there are three rather lively boys and as each of them works on drawing two parallel lines the following discussion ensues:
Student 2: “That’s parallel,” referring to the littlest one’s two parallel
lines.
Student 4: “Can I make it parallel?”
Student 2: “That is parallel. It doesn’t matter the size of the other
line, if they don’t intersect, then they’re parallel.”
Student 1: “If they don’t intersect, they’re parallel.”
The little one is still unconvinced by the two boys, looking at his board with a confused expression. [Student 2] continues to teach the other student the concept of parallel lines, emphasizing the word intersect and then upon seeing the continued state of confusion lets out a loud sigh and starts again. He explains that the length of the line segment or how close they are doesn’t matter as long as they don’t intersect.
(CO.05.03.YR2)
C R E S S T / U C L A
C R E S S T / U C L A
C R E S S T / U C L A
Quality of Instructional Feedback Exemplar
Teacher: “Did you measure first? If you’re going to do me, put my name in, and then you’ll have to do an estimate.”
Both students quickly go to their papers to write down their estimates on a small piece of paper.
Teacher: “Now this is in meters,” upon seeing one student write down 5.
The student nods knowingly but continues to write down the number 5.
Teacher: “Do you think I’m 5 meters high?”
He looks at her questioningly, not understanding. So she takes the yardstick and starts to raise it slowly.
Teacher: “One meter.” Raises it a meter. “Two meters.” Raises it again. “Three meters.”
Student: “Oh, two!” He vigorously erases the number five and writes down the number two. (CO.06.01.YR2)
C R E S S T / U C L A
Comparison of All and Matched Mean Ratings
Domain
2000
C R E S S T / U C L A
Comparison of Year 1 and Year 2 mean ratings
by domain across all case study schools
Chart3
1
1
2
2
3
3
4
4
5
5
6
6
7
7
9
9
10
10
11
11
12
12
13
13
14
14
15
15
16
16
17
17
19
19
20
20
22
22
23
23
26
26
28
28
29
29
30
30
31
31
32
32
33
33
Note: Responses were based on a Likert-type scale: 1= Strongly Disagree, 2= Disagree, 3=Agree and 4=Strongly Disagree
Year 1
Year 2
School Number
Figure : Mean rating by school for Influence on Student Engagement Scale
3.38529
3.41676
3.12659
3.58538
3.3721
3.438
2.9766
2.80475
2.8
3.11154
3.13612
3.49092
3.17869
3.39648
2.98609
3.08294
2.70971
3.08711
2.82767
3.05622
2.75088
3.00248
2.6602
2.32064
3.45714
3.57143
3.07713
3.08288
2.88889
3.1125
3.02121
3.08623
3.38219
3.35671
2.94135
2.91528
3.36747
3.50724
2.51176
3.11927
3.68333
3.68333
2.37305
2.76445
2.98491
3.37335
3.5797
3.45696
3.39652
3.40287
3.65048
3.5609
3.44448
3.43169
1
1
2
2
3
3
4
4
5
5
6
6
7
7
9
9
10
10
11
11
12
12
13
13
14
14
15
15
16
16
17
17
19
19
20
20
22
22
23
23
26
26
28
28
29
29
30
30
31
31
32
32
33
33
Note: Responses were based on a Likert-type scale: 1=Never, 2=Few times per year, 3= 1-2 times per month, 4=1-2 times per week, and 5=Almost Daily.
Year 1
Year 2
School Number
Figure : Mean rating by school for Creative Teaching Practices Scale
3.56555
3.56
3.30544
3.39478
3.23279
3.3956
3.16318
3.15666
3.33333
3.44505
3.37912
3.46853
3.23408
3.21488
3.65189
3.88067
3.47158
3.5154
3.36905
3.5065
3.50446
3.65457
3.38265
3.13333
3.88776
3.72175
2.76974
3.26429
3.00794
3.30357
3.25307
3.29258
3.24671
3.42857
3.46825
3.83928
3.42663
3.35
3.4898
3.47024
3.19481
3.15935
3.3254
3.31746
3.06122
3.5
3.67582
3.40468
3.48786
3.50452
3.56536
3.45157
3.34815
3.54334
1
1
Note: Responses were based on a Likert-type scale: 1= Strongly Disagree, 2= Disagree, 3=Agree and 4=Strongly Disagree
#REF!
#REF!
Figure 4: Mean rating by school for Parent Involvement Scale.
1
1
1
1
Note: Responses were based on a Likert-type scale: 1= Strongly Disagree, 2= Disagree, 3=Agree and 4=Strongly Disagree
Figure :Mean rating by school for Change in Attitude Toward Professional Development Activities Scale
#REF!
#REF!
School Number
Note: Responses were based on a Likert-type scale: 1=Change for Worse, 2= No Change, 3=Change for Better
#REF!
#REF!
Figure 5: Mean rating by school for School Change Scale.
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Year 1
Year 2
3.22
3.19
#11-I have a chance to discuss what we are learning.
3.31
3.24
2.86
2.88
#17-Students help to plan what we will do in our class.
2.93
2.47
#16-The students in my class help one another with assignments.
2.71
3.16
#10-I have no choices of different activities to do in class.
2.24
2.34
2.98
3.61
#8-Students don’t get many choices when it comes to doing assignments.
2.87
2.85
Student Participation in Discussion
Quality of Instructional Feedback
Student Participation in Discussion
Quality of Instructional Feedback
2.42
2.46
2.88
2.86
3.2
3.45
2.32
2.86
2.2
2.49
2.46
2.44
C R E S S T / U C L A
Comparison of Year 1 and Year 2 matched mean ratings by domain across all case study schools
Chart4
1
1
2
2
3
3
4
4
5
5
6
6
7
7
9
9
10
10
11
11
12
12
13
13
14
14
15
15
16
16
17
17
19
19
20
20
22
22
23
23
26
26
28
28
29
29
30
30
31
31
32
32
33
33
Note: Responses were based on a Likert-type scale: 1= Strongly Disagree, 2= Disagree, 3=Agree and 4=Strongly Disagree
Year 1
Year 2
School Number
Figure : Mean rating by school for Influence on Student Engagement Scale
3.38529
3.41676
3.12659
3.58538
3.3721
3.438
2.9766
2.80475
2.8
3.11154
3.13612
3.49092
3.17869
3.39648
2.98609
3.08294
2.70971
3.08711
2.82767
3.05622
2.75088
3.00248
2.6602
2.32064
3.45714
3.57143
3.07713
3.08288
2.88889
3.1125
3.02121
3.08623
3.38219
3.35671
2.94135
2.91528
3.36747
3.50724
2.51176
3.11927
3.68333
3.68333
2.37305
2.76445
2.98491
3.37335
3.5797
3.45696
3.39652
3.40287
3.65048
3.5609
3.44448
3.43169
1
1
2
2
3
3
4
4
5
5
6
6
7
7
9
9
10
10
11
11
12
12
13
13
14
14
15
15
16
16
17
17
19
19
20
20
22
22
23
23
26
26
28
28
29
29
30
30
31
31
32
32
33
33
Note: Responses were based on a Likert-type scale: 1=Never, 2=Few times per year, 3= 1-2 times per month, 4=1-2 times per week, and 5=Almost Daily.
Year 1
Year 2
School Number
Figure : Mean rating by school for Creative Teaching Practices Scale
3.56555
3.56
3.30544
3.39478
3.23279
3.3956
3.16318
3.15666
3.33333
3.44505
3.37912
3.46853
3.23408
3.21488
3.65189
3.88067
3.47158
3.5154
3.36905
3.5065
3.50446
3.65457
3.38265
3.13333
3.88776
3.72175
2.76974
3.26429
3.00794
3.30357
3.25307
3.29258
3.24671
3.42857
3.46825
3.83928
3.42663
3.35
3.4898
3.47024
3.19481
3.15935
3.3254
3.31746
3.06122
3.5
3.67582
3.40468
3.48786
3.50452
3.56536
3.45157
3.34815
3.54334
1
1
Note: Responses were based on a Likert-type scale: 1= Strongly Disagree, 2= Disagree, 3=Agree and 4=Strongly Disagree
#REF!
#REF!
Figure 4: Mean rating by school for Parent Involvement Scale.
1
1
1
1
Note: Responses were based on a Likert-type scale: 1= Strongly Disagree, 2= Disagree, 3=Agree and 4=Strongly Disagree
Figure :Mean rating by school for Change in Attitude Toward Professional Development Activities Scale
#REF!
#REF!
School Number
Note: Responses were based on a Likert-type scale: 1=Change for Worse, 2= No Change, 3=Change for Better
#REF!
#REF!
Figure 5: Mean rating by school for School Change Scale.
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Year 1
Year 2
3.22
3.19
#11-I have a chance to discuss what we are learning.
3.31
3.24
2.86
2.88
#17-Students help to plan what we will do in our class.
2.93
2.47
#16-The students in my class help one another with assignments.
2.71
3.16
#10-I have no choices of different activities to do in class.
2.24
2.34
2.98
3.61
#8-Students don’t get many choices when it comes to doing assignments.
2.87
2.85
Student Participation in Discussion
Quality of Instructional Feedback
Student Participation in Discussion
Quality of Instructional Feedback
2.42
2.46
2.88
2.86
3.2
3.45
2.32
2.86
2.2
2.49
2.46
2.44
C R E S S T / U C L A
Discussion and Implications for Classroom Practice
Documenting Shifts in Classroom Practice from Year 1 to Year 2
Issues in Accountability and Sustainability of School Reform
Re-focusing, Re-thinking, and Re-tooling Measures of Classroom Practice
C R E S S T / U C L A
Documenting Shifts
in Classroom Practice
Overall, small effect on Teacher Practice (Effect Size of .33) and Student Engagement (Effect Size of .201)
Between Schools, difference in degree of implementation of creative practices
Professional development linked to promoted teaching practices
Assessments to monitor progress
C R E S S T / U C L A
Issues in Accountability and Sustainability in School Reform
Combining and creating assessments that are accessible for progress monitoring
Improvements
Inquiry
Leadership
C R E S S T / U C L A
Re-focusing, Re-thinking, and Re-tooling
Measures of Classroom Practice
Re-focus efforts to include realistic expectations for reform implementation
Re-think the product and outcome model of classroom measures: measures should reflect process in action
Re-tool measures to reflect conceptual and procedural realities of classroom practice
C R E S S T / U C L A
Paper will be posted
Ann Mastergeorge
Phone 530-754-9629
E-mail [email protected]

Search related