Embed Size (px)
Citation preview
S.T.A.T. Year Three Evaluation
Dr. Jennifer R. Morrison
Dr. Steven M. RossAugust 8, 2017
S.T.A.T. Evaluation Model
Professional
Development Measurable Outcomes Goals
Classroom
Environment
Teacher
Practice
Digital
Content
Administrators
Classroom
Teachers
S.T.A.T.
Teachers
Student
Engagement
P21 Skills
Student
Achievement
Graduate Globally
Competitive
Students
MAP
PARCC
Year 1+
Year 2+
Years 3/4+
Year 1+
8/8/2017 2
Data Sources
• Interviews and Focus Groups (principals, S.T.A.T. teachers, classroom teachers)
• Classroom Teacher Survey (CRRE survey)
• Classroom observations in schools (OASIS-21 Instrument)
• Student behavioral data
• MAP data (Grades 1-3)
• S.T.A.T.-specific climate survey items (BCPS survey)
8/8/2017 3
S.T.A.T. Experience
2015-2016 2016-20172014-2015
Cohort 1:
LH Grades 1-3
Cohort 2:
LH Grades K, 4, 5
Ph 2 Grades 1-3
LH Grade 6
Cohort 3:
Ph 2 Grades K, 4, 5
Ph 2 Grade 6
LH Grade 7
LH Grades 9-1248/8/2017
Preview of Year Three Results
• Research on school-district technology integration initiatives shows1:– Higher student engagement
– Increases in student-centered instruction
– Improved student achievement
• Third-year results in BCPS show:– Continued changes from teacher- to student-centered learning
– Shifts to teacher coaching rather than presenting
– Focus on using data to customize instruction
– Strong impact on student engagement
1 Zheng, B., Warschauer, M., Lin, C., & Chang, C. (2016). Learning in one-to-one laptop environments: A meta-
analysis and research synthesis. Review of Educational Research. 8/8/2017 5
S.T.A.T. Evaluation Model
Professional
Development Measurable Outcomes Goals
Classroom
Environment
Teacher
Practice
Digital
Content
Administrators
Classroom
Teachers
S.T.A.T.
Teachers
Student
Engagement
P21 Skills
Student
Achievement
Graduate Globally
Competitive
Students
MAP
PARCC
Year 1+Year 1+
Year 2+
Year 3+
8/8/2017 6
Professional Development
• S.T.A.T. teachers continued to be viewed as critical to
implementation
• Principals viewed as supportive of teachers’
implementation
• Teacher collaboration and sharing
• Need clearly defined roles of the S.T.A.T. teacher
• Additional PD on using tools and programs, P21 skills
8/8/2017 7
S.T.A.T. Evaluation Model
Professional
Development Measurable Outcomes Goals
Classroom
Environment
Teacher
Practice
Digital
Content
Administrators
Classroom
Teachers
S.T.A.T.
Teachers
Student
Engagement
P21 Skills
Student
Achievement
Graduate Globally
Competitive
Students
MAP
PARCC
Year 1+Year 1+
Year 2+
Year 3+
8/8/2017 8
Observation Rating Scales
• Not observed: Not observed in class
• Rarely: Received little emphasis/time in class
• Somewhat/Occasionally: Receives modest emphasis/time
in class
• Frequently: Receives substantial emphasis/time in class
• Extensively: Highly prevalent in class
8/8/2017 9
Impact on Classroom Environment
8/8/2017 10
9.4%
+
6.5%
40.6%
63.2%
53.2%
50.0%
32.4%
40.3%
Cohort 1
Cohort 2
Cohort 3
Not observed General Subject Lesson Specific
Information and resources that reflect content being taught are visibly displayed
+ < 5.0%
37.5%
47.1%
53.2%
43.8%
38.2%
33.8%
18.8%
14.7%
13.0%
Cohort 1
Cohort 2
Cohort 3
Not Observed Somewhat Extensive
Information and communications that support independent thinking are visible
Summary: Classroom Environment
• Classrooms appear to be consistent with S.T.A.T. goals
• Little change in the physical environment
8/8/2017 11
Impact on Teacher Practices
12
9.4%
22.1%
15.6%
15.6%
13.2%
14.3%
28.1%
23.5%
24.7%
18.8%
19.1%
19.5%
28.1%
22.1%
26.0%
Cohort 1
Cohort 2
Cohort 3
Not Observed Rarely Somewhat/Occasionally Frequently Extensively
Teachers acting as coach/facilitator
28.1%
30.9%
44.2%
21.9%
25.0%
19.5%
37.5%
22.1%
22.1%
6.3%
13.2%
11.7%
6.3%
8.8%
+
Cohort 1
Cohort 2
Cohort 3
Not Observed Rarely Somewhat/Occasionally Frequently Extensively
Teacher presentation
+ < 5.0%
8/8/2017
Summary: Teacher Practice
• Teachers favoring coaching and facilitating instruction
over presentations
• Increased teacher collaboration, personalizing instruction,
interactive learning
8/8/2017 13
Impact on Digital Content
8/8/2017 14
+
+
8.0%
15.8%
21.1%
30.0%
36.7%
41.6%
62.0%
46.0%
32.5%
Cohort 1
Cohort 2
Cohort 3
Never At least once per month At least once a week At least once a day
Deliver instruction customized to students' needs
+ < 5.0%
72.0%
59.5%
37.3%
8.0%
10.2%
22.2%
8.0%
16.3%
24.8%
12.0%
14.0%
15.7%
Cohort 1
Cohort 2
Cohort 3
Never At least once per month At least once a week At least once a day
Post homework assignments
+ < 5.0%
Summary: Digital Content
• Successes with using technology:
– Research, students taking ownership of learning, increased
engagement
• Challenges with technology integration:
– Inappropriate/off-task use
8/8/2017 15
S.T.A.T. Evaluation Model
Professional
Development Measurable Outcomes Goals
Classroom
Environment
Teacher
Practice
Digital
Content
Administrators
Classroom
Teachers
S.T.A.T.
Teachers
Student
Engagement
P21 Skills
Student
Achievement
Graduate Globally
Competitive
Students
MAP
PARCC
Student
Achievement
Graduate Globally
Competitive
Students
MAP
PARCC
Year 1+Year 1+
Year 2+
Year 3+
8/8/2017 16
Impact on Student Engagement
8/8/2017 17
28.1%
42.6%
39.0%
37.5%
29.4%
23.4%
18.8%
5.9%
15.6%
15.6%
22.1%
22.1%
Cohort 1
Cohort 2
Cohort 3
Not Observed Rarely Somewhat/Occasionally Frequently Extensively
Student use of digital tools for learning
12.5%
16.2%
14.3%
9.4%
14.7%
5.2%
15.6%
23.5%
19.5%
40.6%
17.6%
26.0%
21.9%
27.9%
35.1%
Cohort 1
Cohort 2
Cohort 3
Not Observed Rarely Somewhat/Occasionally Frequently Extensively
Independent work
Summary: Student Engagement
• Evidence of student engagement in learning task
• Mostly positive regarding student engagement
• Student behavior
– Mixed views
– Attendance and office referral rates stable, statistically
significant increases in suspension rates
8/8/2017 18
Impact on P21 Skills
8/8/2017 19
90.6%
89.7%
83.1%
+
+
+
+
+
6.3%
5.9
11.7%
Cohort 1
Cohort 2
Cohort 3
Not Observed Rarely Somewhat/Occasionally Frequently Extensively
Project-based approaches to instruction
+ < 5.0%
56.3%
67.6%
66.2%
+
8.8%
5.2%
9.4%
10.3%
14.3%
12.5%
7.4%
9.1%
18.8%
5.9%
5.2%
Cohort 1
Cohort 2
Cohort 3
Not Observed Rarely Somewhat/Occasionally Frequently Extensively
Learning incorporates authentic/real world contexts
+ < 5.0%
Summary: Impact on P21 Skills
• Comparable observation results with baseline and across
Cohorts
• Some perceptions of improved P21 skills
• Needed PD
8/8/2017 20
S.T.A.T. Evaluation Model
Professional
Development Measurable Outcomes Goals
Classroom
Environment
Teacher
Practice
Digital
Content
Administrators
Classroom
Teachers
S.T.A.T.
Teachers
Student
Engagement
P21 Skills
Student
Achievement
Graduate Globally
Competitive
Students
MAP
PARCC
Year 1+
Year 2+
Years 3/4+
Year 1+
8/8/2017 21
Student Achievement: Lighthouse
• Mathematics
– Increases in students meeting growth expectations
– Upward trend for Grades 1-2 students
– Grades 1-3 exceeded national average
• Reading/ELA
– Increases in students meeting growth expectations
– Upward trend for Grades 1-2 students
– Grades 1-3 exceeded national average
228/8/2017
Student Achievement: Non-Lighthouse
• Mathematics
– Increases in students meeting growth expectations
– Upward trend for Grades 1-2 students
– Grade 2 exceeded national average
• Reading/ELA
– Increases in students meeting growth expectations
– Upward trend for Grades 1-3 students
– Grades 1-3 exceeded national average
238/8/2017
Perceptions of S.T.A.T.
• Principals, S.T.A.T. teachers, classroom teachers:
– Increases in student engagement, focus on instruction, variety
• Parents and students:
– Vast majority positive towards teachers’ use of technology
8/8/2017 24
Recommendations
• S.T.A.T. teacher roles and responsibilities
• Teacher instructional support
• Technology policies
• Improved communication
8/8/2017 25
Conclusion
• Highly positive perceptions of S.T.A.T. teachers
• S.T.A.T. valued for increases in student engagement, focus on instruction, support for learning
• S.T.A.T. viewed positively by all stakeholders
• Positive achievement trends on MAP
8/8/2017 26
