Embed Size (px)
Citation preview
Data Analysis for Educational Leaders
“Data Rich, Information Rich!”
“Comprehensive data analysis is tied to systematic and systemic continuous
improvement” -Victoria L. Bernhardt, Ph.D.-
1
The ResearchThe ability to accurately and
appropriately use the data … is critical to principals' effectiveness at ultimately improving student achievement (Price & Burton, 2004; Yeagley, 2001)
Holcomb (2004) found that proper training is key to effectively using data.
3
Article Review
Group A: “Developing an Inquiry-Minded District”
Group B: “Looking Deeper into Data”
Group C: “Expanding data analysis skills in educational leaders: implications for preparation programs.”
Group D: “First Things First: Demystifying Data Analysis”
4
Continuous School Improvement…..
Analyze&
Prioritize
Set S-M-A-R-TGoals
Select Instructional Strategies
Determine Results
Indicators
Progress Monitor, Examine Student
Work & Adjust through Instructional
Conversations
Collect and Chart Data
(Treasure Hunt)
Inquiry:•Question•Triangulate•Examine Angles
5
In this workshop, you will:
• Become familiar with analyzing and interpreting data.
• Learn how to translate data into SMART Goals.• Develop a school-wide method of progress
monitoring.
6
Statistics
Two types:1.Descriptive statistics are used to describe or summarize our observations without making inferences.
2.Inferential statistics are to make predictions or estimates using a sample population.
What type of Statistics will we be using?
7
Tria
ngul
atio
n
Usi
ng M
ultip
le
Mea
sure
s!
SummativeStrandLevel
Disaggregation
Evidence of instructional
practice
What does the data tell us about our students?
What are the curricular, programmatic, and instructional implications?
Why is triangulati
on essential?
Benchmark
8
Take a few minutes to answer a question:
What was your school or district goal for reading last year?
For reading, how many students at each grade level were: Below Basic? Basic? Proficient? Goal? Advanced?
9
How well did the students perform overall?
10
Is this information enough to make a school-wide goal? If so, what could one be? If not, what other information do you need?
11
Why is it important to see the proficiency levels?
CMT 2010 Data Grade 3
12
Administrators must look at “Big Picture Data”!
13
Grade
“N”(#of
Students
# Below Basic
# Basic#
Proficient# Goal
# Advanced
3 75 52% 0% 16% 13.3% 18.7%
4 52 40.4% 11.5% 13.5% 30.8% 3.8%
5 66 40.4% 12.1% 6.7% 27.3% 1.5%
aggregate 193 46% 7% 16% 23% 9%
Performance Level Scores School A
This school’s goal was to increase the
% of students scoring at
proficiency in math by 10%. Was this a good goal? Why or
why not?
14
Grade
“N”(#of
Students
# Below Basic
# Basic#
Proficient# Goal
# Advanced
3 75
75 x 52% = 39
Students
75 x 0% = 0
Students
75 x 16% = 12
Students
75 x 13.3% =
10 Students
75 x 18.7% =
14 Students
4 52
52 X 40.4%=
21 Students
52 X 11.5%=
6 Students
52 X 13.5%=
7 Students
52 X 30.8%=
16 Students
52 X 3.8%=
2 Students
5 66
66 X 40.4%=
28 Students
66 X 12.1%=
8 Students
66 X 16.7%=
11 Students
66 X 27.3%=
18 Students
66 X 1.5%=
1 Student
aggregate 193
46% 88
Students
7%14
Students
16%30
Students
23%44
Students
9%17
Students
Performance Level Scores School A
Total # of students
% of students in this level
How does this data impact your action plan? What are the instructional, professional development, curricular, and programmatic implications? Resources vs. Need?
How does this data impact your action plan? What are the instructional, professional development, curricular, and programmatic implications? Resources vs. Need?
15
16
Group Work
Work with your group to determine the overall performance of the following school. Then answer the following questions:
1.What could a goal be?2.How does this data impact your action
plan? 3.What are the instructional, professional
development, curricular, and programmatic implications? Resources vs. Need?
17
Grade
“N”(#of
Students
# Below Basic
# Basic#
Proficient# Goal
# Advanced
3 40 40 X 0%=
4 43
5 49
Aggregate3-5
132
Grade
“N”(#of
Students
% Below Basic
% Basic%
Proficient% Goal
% Advanced
3 40 0% 13% 13% 43% 32%
4 43 5% 2% 24% 35% 35%
5 49 4% 6% 8% 45% 37%
18
Grade
“N”(#of
Students
# Below Basic
# Basic#
Proficient# Goal
# Advanced
3 4040 X 0%=0 Students
40x13%=5 Students
40x13%=5 Students
40x43%=17
Students
40x32%=13
Students
4 4343 x 5%=2 Students
43 x 2%=1 Students
43 x 24%=
10 Students
43 x 35%=
15 Students
43 x 35%=
15 Students
5 4949 x 3%=2 Students
49 x 6%=3 Students
49 x 8%=4 Students
49 x 45%=
22 Students
49 x 37%=
18 Students
Aggregate3-5
132(4/132) x1001%
(9/132) x1007%
(19/132)x100
14%
(54/132)x100
41%
(46/132)x100
35%
Grade
“N”(#of
Students
% Below Basic
% Basic%
Proficient% Goal
% Advanced
3 40 0% 13% 13% 43% 32%
4 43 5% 2% 24% 35% 35%
5 49 4% 6% 8% 45% 37%
19
AYP (Reading 2010-2011=89%)
Goal= To increase the % of Students scoring At/Above Proficiency from 90% (14+41+35) to 95% (91%needed for AYP plus, in this case, because the scores are so
close to AYP, add an educational significant difference of 4%pts.) as measured by the 2011 CMT Math Score and monitored by the Common Assessments.
20
Where do we need to be? AYP vs. Safe Harbor
21
BLOOM’S TAXONOMY How We Teach
WEBB’S DEPTH OF KNOWLEDGE How We Assess
KNOWLEDGE “The recall of specifics and
universals, involving little more than bringing to
mind the appropriate material” Recall – Recall of a fact, information, or procedure
(e.g., What are 3 critical skill cues for the overhand throw?) COMPREHENSION
“Ability to process knowledge on a low level such that the knowledge
can be reproduced or communicated without a verbatim repetition.”
APPLICATION “The use of abstractions in
concrete situations.”
Basic Application of Skill/Concept – Use of information, conceptual knowledge, procedures, two or more steps, etc. (e.g., Explain why each skill cue is important to the overhand throw. “By stepping forward you are able to throw the ball
further.”)
ANALYSIS “The breakdown of a situation into
its component parts.”
Strategic Thinking – Requires reasoning, developing a plan or sequence of steps; has some
complexity; more than one possible answer; generally takes less than 10 minutes to do (e.g., Design 2 different plays in basketball and explain
what different skills are needed and when the plays should be carried out.)
SYNTHESIS AND EVALUATION “Putting together elements & parts to form a whole, then making value
judgments about the method.”
Extended Thinking – Requires an investigation; time to think and process multiple conditions of the problem or task; and more than 10 minutes to do
non-routine manipulations (e.g., Analyze 3 different tennis, racquetball, and badminton strokes for similarities, differences, and purposes. Then,
discuss the relationship between the mechanics of the stroke and the strategy for using the stroke
during game play.)
Below Basic
Proficient
Basic
Goal
Advanced
22
Grade
“N”(#of
Students
# Below Basic
# Basic#
Proficient# Goal
# Advanced
3 75
75 x 52% = 39
Students
75 x 0% = 0
Students
75 x 16% = 12
Students
75 x 13.3% =
10 Students
75 x 18.7% =
14 Students
4 52
52 X 40.4%=
21 Students
52 X 11.5%=
6 Students
52 X 13.5%=
7 Students
52 X 30.8%=
16 Students
52 X 3.8%=
2 Students
5 66
66 X 40.4%=
28 Students
66 X 12.1%=
8 Students
66 X 16.7%=
11 Students
66 X 27.3%=
18 Students
66 X 1.5%=
1 Student
aggregate 193
46% 88
Students
7%14
Students
16%30
Students
23%44
Students
9%17
Students
Performance Level Scores School A
23
Safe Harbor: An alternate method for measuring progress toward AYP. For any school and/or subgroup that does not meet the proficiency target:– Reduce the % not proficient by 10 %– Meet the additional academic indicators– Meet the 95% participation rate
requirement.– Example:% Proficient Safe
HarborMathematics 45% 55% X .10 = 5.5% 51%
Reading 60% 40% x .10 = 4% 64%24
Group Work:
Work with your partner to figure out safe harbor for this school.Use this information to develop a school-wide goalIs there anything else this data tells you?
25
This same process can be used for all sorts of data (subgroup, instructional, attendance….)!
Lets Try!
Group A: Attendance DataGroup B: Instructional DataGroup C: Subgroup Data
26
Number of Days Absent vs. Grade
Grade “N” 0-5 Days
6-10 Days
11-15 Days
16-20 Days
21+ Days
9 150 44% 20% 11% 15% 10%
10 150 35% 20% 15% 18% 12%
11 150 40% 43% 12% 3% 5%
12 150 40% 10% 25% 17% 8%
2009-2010
27
What could be a goal for grade 9?
28
Your assessment results should render information
about curriculum implementation and
instruction.
How would this data impact your action plan
(PD, support, data collection…)?
29
Grade
“N”(#of
Students
# Below Basic
# Basic#
Proficient# Goal
# Advanced
30
31
Vertical Scale Scores
32
Disaggregation
33
34
READING COMPBlack not of Hispanic origin
Proficiency Level # of Students PercentBelow Basic 15 14.85%
Basic 8 7.92%Proficient 21 20.79%
Goal 52 51.49%Advanced 5 4.95%
Total 101
White not of Hispanic origin
Proficiency Level # of Students PercentBelow Basic 0 0.00%
Basic 2 5.56%Proficient 3 8.33%
Goal 20 55.56%Advanced 11 30.56%
Total 36
Notice: there is
no overlap
35
Economically Disadvantaged (Free and Reduced Lunch)
Proficiency Level # of Students PercentBelow Basic 11 13.92%
Basic 4 5.06%Proficient 14 17.72%
Goal 44 55.70%Advanced 6 7.59%
Total 79
For the Economically Disadvantaged student
data below, to what subgroup should we compare this data?
36
The SIOP model was
implemented to reach these learners and
additional instructional hours Is it working?
%%ELL Subgroup
37
%% %%
%%ELL SubgroupNon-ELL 38
ELL Group “N”=25
Non-ELL Group “N”=50
New Arrival ”N”=1
%
39
ELL Group “N”=25
Non-ELL Group “N”=50
New Arrival ”N”=1
%
40
What questions do you have? What are
some cautions here?
41
What is Educationally Significant?
N (Male)=142N (Female)=139
Is this significant?
43
Look at the following graphs: What are the “take away” messages?
Is there any actions you would do as a result of this information?
44
45
46
N=200
47
Know the assessment
Summative, Formative, DiagnosticPoints needed for mastery (Benchmark)# and types of questions.Strand make-up
48
49
CMT READING COMPREHENSION TEST BLUEPRINT
GradeNumber of Test Items Per Content Strand/Objective + Point Value
A. General UnderstandingB. Developing Interpretation
C. Making Connections D. Content and Structure Total
3
12 mc, 1oe 7 mc, 1oe 0 mc, 4 oe 5 mc, 2 oe 24 mcA1 - 2mc - 2 pts B1 - 2mc - 2 pts C1 - 2oe - 4 pts D1 - 1mc, 1oe - 3 pts 8 oeA2 - 5mc - 5 pts B2 - 0 N/A C2 - 2oe - 4 pts D2 - 2mc, 1oe - 4 pts A3 - 1oe - 2 pts B3 - 5mc, 1oe - 7 pts D3 - 2mc - 2 pts A4 - 1mc - 1 pts A5 - 4mc - 4 pts
4
8 mc, 2 oe 10 mc, 0 oe 0 mc, 4 oe 6 mc, 2 oe 24 mcA1 - 1mc, 1oe - 3 pts B1 - 2mc - 2 pts C1 - 2oe - 4 pts D1 - 2mc, 1oe - 4 pts 8 oe
A2 - 4mc - 4 pts B2 - 4mc - 4 pts C2 - 2oe - 4 pts D2 - 2mc, 1oe - 4 pts A3 - 1oe - 2 pts B3 - 4mc - 4 pts D3 - 2mc - 2 pts A4 - 1mc - 1 pts A5 - 2mc - 2 pts
5+6
7 mc, 2 oe 8 mc, 1oe 0 mc, 4 oe 7 mc, 2 oe 22 mcA1 - 1mc, 1oe - 3 pts B1 - 2mc - 2 pts C1 - 2oe - 4 pts D1 - 2mc, 1oe - 4 pts 9 oe
A2 - 3mc - 3 pts B2 - 3mc, 1oe - 5 pts C2 - 2oe - 4 pts D2 - 2mc, 1oe - 4pts A3 - 1oe - 2 pts B3 - 3mc - 3 pts D3 - 3mc - 3 pts A4 - 1mc - 1 pts A5 - 2mc - 2 pts
7+8
6 mc, 2 oe 8 mc, 1oe 0 mc, 4 oe 6 mc, 3 oe 20 mcA1 - 1oe - 2 pts B1 - 2mc - 2 pts C1 - 2oe - 4 pts D1 - 2mc, 1oe - 4 pts 10 oeA2 - 3mc - 3 pts B2 - 3mc, 1oe - 5 pts C2 - 2oe - 4 pts D2 - 2mc, 1oe - 4pts A3 - 1oe - 2 pts B3 - 3mc - 3 pts D3 - 2mc, 1oe - 4 pts A4 - 1mc - 1 pts A5 - 2mc - 2 pts
KEY- oe = open ended mc = multiple choice
Point Value/ Strand and Questions Possible Point Value of
ObjectiveHighest Pt. Value = Priority 5 + points 7 pts. Grade 3
4 points 3 points 2 points
Lowest 1 point 0 points B2- N/A grade 3
Adapted from the CMT4 L.A. Handbook /CREC
50
Content Standard Number of Points
1. Algebraic Reasoning1.1 4-51.2 4-51.3 2-3
2. Numerical and Proportional Reasoning
2.1 3-4
2.2 8-9
3. Geometry and Measurement
3.1 3-53.2 2-33.3 5-6
4. Statistics and Probability
4.1 34.2 4-54.3 4-5
Number of Items and Points Across Standards
CAPTMathematics
OE Items(3 points each)
Grid Items ( 1 point each)
TotalPoints
Algebraic Reasoning 2 6 12
Numerical & ProportionalReasoning 2 6 12
Measurement and Geometry 2 6 12
Working with Data 2 6 12
8 OE Items 24 Grid Items 48 Points
Third Generation CAPT Mathematics Assessment Blueprint
KEY-Point Value/ Strand and
Questions Possible Point Value of
Objective
Highest Pt. Value = Priority 6+points
3-5pointsLowest 3 or below point
51
CMT Math (3rd)
52
CAPT-ReadingRtL
70-minute session
1 published fictional text
2,000-3,000-word text
10th-grade readability
4 response questions
4 dimensions assessed
6-point rubric
2-12 score
50% of CAPT reading score
RfI•45-minute session•3 published nonfiction texts•500 -1,000-word texts•10th-grade readability•12 multiple-choice, 6 open-ended questions•2 dimensions assessed•3-point rubric (0-2)•0-24 score•50% of CAPT reading score
53
Important!
CMTs at various grades differ in content, difficulty, and length. Therefore, NOT directly comparable.You can examine data longitudinally within grade and within CMT generation from year to year. This includes strand and subgroup data. Note: it is recommended that 3 years of historical data be used. CMT4 was first administered March 2006 and should be considered the benchmark .
Reference: Krisst, Abe & Martin, Paul. Guidelines for proper CMT Data Analysis. 2007. CSDE
54
Important!
Within a generation and grade, comparisons may be made on the basis of scale scores and achievement levels for all students. DRP unit scores can also be compared.Within a generation, grade and year, strand level performance can be compared to other strands. It is important that strand makeup is part of the analysis. This will help guide your action plan.
Reference: Krisst, Abe & Martin, Paul. Guidelines for proper CMT Data Analysis. 2007. CSDE
55
Trajectory
56
Demands/Skills
Time
Expectations for All Students
Baseline/Current Level of Performance
Goal
Tier 2 Projected Growth Line
4 weeks
8 weeks
6 weeks
16 weeks
Scores/Skills
What is the trajectory of the school below? If they continue on this path will they make AYP next year?
57
5% pts
58
4% pts
59
Strand Analysis- Strand scores are useful when analyzing data about
discrete parts of the content area. Do not compare strand and scale data across
grades. They are tested in different ways-different level. To find trends you should have 3 years of historical data.
Look at trends rather than small differences. Strands are not precisely comparable.
Compare within the SAME Generation. 2006 is the baseline for the generation 4.
Once trends are identified, determine if the trend is statewide. If not, which districts are exceeding?
Compare district to state, over time or for one year. Compare strand individual strand performance against
other strands for the same grade.
60
Reading1 . Forming a
General Understanding
2 . Developing Interpretation
3 . Making Reader/Text Connections
4 . Examining the Content and
StructureGrade 4 2006 64 62 34 50Grade 4 2007 81 66 36 60Grade 4 2008 81 66 49 51Grade 4 2009 86 70 59 61
61
62
63
SMART Goal
Basic Proficient Goal
Each post-it contains:Student Name: __________________
•Oct. Score: ________ •Jan. Score: ________•May Score: ________
SPED ELL ________
Below Basic Advanced
What does a S-M-A-R-T Goal look like for this class?
64
Teacher B
Basic Proficient GoalBelow Basic Advanced65
Teacher A
Basic Proficient Goal
Each post-it contains:Student Name: __________________
•Oct. Score: ________ •Jan. Score: ________•May Score: ________
SPED ELL ________
Below Basic Advanced66
Teacher B
Basic Proficient GoalBelow Basic Advanced67
Teacher C
Basic Proficient GoalBelow Basic Advanced68
School Improvement Planning Process
Analyze data to identify needs and goals
Set measurable goals and targets based on in depth data analysis
Clearly identify target levels of performance for all students
Develop SIP with timeline, benchmarks, and clear responsibilities
Align professional development with desired outcomes.
Monitor progress using interim assessments and data team process
69
Lessons Learned
Data driven decision making can be a powerful tool in changing student outcomes and promoting continuous improvement in achievement.
Progress has to be measurable. Community outreach is
essential.It has to start at the top.It takes time.
70
7 Characteristics of Effective Accountability Systems High Expectations for all studentsHigh-quality assessments aligned with standardsAlignment of resources, support and assistance for improvementSanctions and rewards linked to
resultsMultiple measuresData use in districts and schoolsInformed stakeholders
71
Bar Graphs: Points to Remember1. Bar Graphs are used to compare, each item is mutually exclusive and is collected at approximately the same time.2. Independent variables should be on the horizontal axis3. A descriptive tool that creates a picture of “What is”4. Individual bars represent independent events, groups, or individuals.Answers “how many” or “how much” groups or individuals earned or scaled at the time the data were collected.5. The most highly recommended format for descriptive data.6. Keep the easy to read as possible7. Reduce grid lines to the fewest needed for clarity8. Label data within the graph wherever possible
72
Bar Graphs: Points to Remember9. Eliminate boxes around the legend and use patterns sparingly10. You might want to move the legend to underneath if it is easier to read or include a portion of the legend in the graph11. Use type size that can be easily read and is consistent throughout the graph12. Make sure that the colors contrast enough to show the data clearly, and use colors that enable you to print or photocopy in Black and White12. ALWAYS include the “n” 13. Be sure to label both axes on the graph14. Use Data Labels when the value is difficult to see15. Dark background shading=white gridlines; white shading=dark thin gridlines16. Make sure that when presenting a series of graphs, the scales are consistent and accurate.
73
N=450
74
Line GraphsLine graphs are good at showing a series of numbers over time (for instance a disaggregated line graph is great for showing the achievement gap over time). Trends over time.To keep graphs easier to read, avoid graphing more then 5 lines.Be sure to include the number of people in each subgroup (subgroups less then 8 are too small to be included in a graph)Use a combination of colors and symbols to allow for black and white printing.Label axis and titles
75
When do you use a line graph?
76
When do you use a line graph?
77
•Explain purpose of the test/assessment•Communicate the standards and benchmarks for the assessments.•Indicate the type of assessment and reporting method (norm-referenced, percentile ranks, normal curve equivalents, standardized…)•Use a simple graph to display information.•Put data in context, with detail.•NEVER SPECULATE or infer the reason(s) for results.•Protect identities of individual test takers.•Always state your school’s plans for the results.
When communicating test results:
78
Continuous School Improvement…..
Analyze&
Prioritize
Set S-M-A-R-TGoals
Select Instructional
Strategies
Determine Results
Indicators
Progress Monitor, Examine Student
Work & Adjust through Instructional
Conversations
Collect and Chart Data
(Treasure Hunt)
Inquiry:•Question•Triangulate•Examine Angles
•Differentiation •Strong tier one•Ongoing progress monitoring using formative assessment•“None of us are as strong as all of us”
address needs of ALL students
EVERYONE is involved!All efforts align and contribute!
79
Resources
http://www.sde.ct.gov/sde/cwp/view.asp?a=2618&q=321744.
80
Thank you to the following colleagues for their support in the development of this module:
Heather Levitt, CSDE
Mary Hourdequin, CAS
Michael Wasta, CSDE
81
Thank you to the following colleagues for their assistance during the vetting process:
Megan Alubicki, CSDE
Shauna Brown, CSDE
Nina Butkiewicz, CAS
Jeff Greig, CSDE
Abe Krisst, CSDE
Mark Nolan, CAS
Beth McCaffery, LEARN
82
Contact Information:
Mara DumondEducational Specialist: Data, Assessment,
and Research Institute of Teaching and Learning, [email protected]
83
Feedback
Please take a few minutes to complete the Feedback Form.
Your comments are very important to us and to your district office, as it provides
specific information and thoughts to consider for future
professional development.
84
