Embed Size (px)
Citation preview
Measuring what we value: 21st Century Assessments for Independent Schools
Douglas Lyons, Executive Director, CAIS-CTAndrew Niblock, Lower School Head, Hamden Hall (CT)
Jim Collins
• Determine what you value most, then find a way to measure it.
• Success can be a powerful disincentive: it may be hard to become a great school if you are a very good school
Good to Great
Goals of this workshop
1. Identify measures of school quality that have historically been valued by educators and/or the public.
2. Suggest new ways to report achievement in those measures.
Why?• To better tell our story (the Value Proposition)• To make certain that these measures have stature
appropriate to their significance
Goals, cont.
3. Provide a quick preview of new and emerging assessment tools –instruments designed to measure skills that are increasing in demand in the new century
4. Describe the assessment practices in a select group of schools that define themselves – and are recognized by others – as “Schools of the Future”
Criterion 13:
The Standards require a school to provide evidence of a thoughtful process, respectful of its mission, for the collection and use in school decision making of data (both external and internal) about student learning.
“Not everything that can be counted, counts, and not everything that counts can be
counted”
“It would be easier to change the course of history…”
Powerpoint presentation is available atwww.caisct.org
look for ADVIS presentation on home page
Jim Collins – “whenever possible, use the language of metrics to define what you value”
What is the language of metrics?
4 “ways of knowing”
• Data: raw input, no context, facts, figures, symbols
• Information: organized, processed, analyzed data• Knowledge: information with higher context -
accurate, relevant, current• Wisdom: evaluated knowledge; merged with life
experience
• Heads letter in viewbook; wisdom• Description of school history and mission in viewbook:
knowledge• Course catalogue: information• SAT scores, college placement stats: data
What are people most interested in ?
Data!• conveys a lot of information - quickly• Is viewed as objective, “no spin”• Can be benchmarked, used for comparison
Data is the language of metrics
Risks / Misuses of Data
• Garbage in; garbage out• Data is easily manipulated, corrupted:
Harvard Business School caution If you torture data long enough, it will admit to anything
Data Management / Data Creation in the independent school community
The Challenge:• To frame the data that define us, or have defined
us in the past, in ways that do not elevate modestly valuable information
• To gather and/or to present new data that is beneficial to educators in our planning for the future and is data that measures performance in relation to the achievement of our highest goals.
The S.A.T.
• Decreasing in stature, but still powerful• Has poor validity statistics• Does not measure 21st century Skills• Historically, did not provide faculty with
instructionally useful information• Consider giving the School Day SAT with
Enhanced Scoring
Standardized Achievement Tests
These test are increasing in stature
What happened to elevate these tests above all other forms of data, in public education and to a lesser degree, in independent schools?
“Effects” of standardized tests today:from educating the whole child to educating
the whole test-taker
• Hyper-focus on scores, minor fluctuations• Unprecedented “score chasing” • M.D.I “measurement-driven instruction”• Mind numbing drill and practice
Most popular form of data presentation: percentiles
Math concepts
Math applications
Reading comprehension
Reading Skill
1 91 87 83 83
2 88 84 85 83
3 92 90 88 88
4 84 89 80 90
5 84 86 78 82
6 89 80 81 79
7 90 89 90 87
8 87 87 81 83
Second most popular form of data presentation: Grade Equivalence
Math concepts
Math applications
Reading comprehension
Reading skill
1 1.8 2.1 2.3 2.5
2 2.6 2.8 3.3 3.6
3 3.5 4.0 3.9 4.0
4 4.5 4.9 4.8 4.7
5 5.8 5.2 6.3 6.0
6 6.6 7.3 7.0 7.5
7 8.2 9.0 8.8 8.8
8 9.0 9.6 9.9 10.4
A better way to present achievement data
• Determine and defend your norming group• Determine a worthy and realistic goal
within the norming group• Publish data relative to the goal
Goal: to score within the top third of norming group on all subtests
Reading comprehension
Reading skills
Math applications
Math skill
1 √ √ √ √
2 √ √ -4 √
3 √ √ √ √
4 √ √ √ √
5 √ √ √ √
6 √ √ √ √
7 -2 √ √ √
8 √ √ √ √
The International Database• Most schools administer one or more normed tests
that compare American student achievement with American peers
• Is there a way to assess our international competitiveness?
• Would this data be valuable to us?
Benefits of the New York State “Truth in Testing” Law
• Thousands of released items available to educators• Released items available for NAEP tests• Released items available for TIMMS tests• Released items available for PISA tests• Construct your own “replica test” or form a
research partnership to develop replica tests
CAIS score reports for TIMMS replica test
ABC Country Day School
TIMMS “released item” test results
95% of students scored in the top 1/2 of I.A.
92% of students scored in the top 1/3 of I.A.
90% of students scored in the top 10% of I.A.
Course of study guides: how we describe our program
US HISTORY HONORS GREENWICH HIGH SCHOOL
This course addresses the events and experiences that comprise American history from the period of European colonial settlement through the Civil War (1st semester) And from the period of Reconstruction through the advent of the Second World War. The goal of the course is to provide for our students substantial opportunity to develop the ability to make informed and reasoned decisions as citizens concerned with the public good. (More text follows)
US HISTORY HONORSA study of the events and critical changes that took place from the first American settlement to the present day.We will focus on these events in the context of larger themes; including the shift from an agricultural to an industrial society, the recognition and cultural identification of different groups of people, the transition to a stronger national government, immense territorial expansion, technological change and globalization.
History at Lakeland Prep:
In the four year History sequence at Lakeland Prep, all students will complete the following Demonstrations of Learning:
• 24 research based position papers (4 to 7 pages) in which an analysis, synthesis and/ or evaluation of both original and modern sources is offered in answer to a provocative question in history.
• 6 research based position papers (10 to 15 pages) focused on a students original response to one of identified Essential Questions in American History.
• 12 oral presentations• 8 collaborative projects, • 3 projects completed in collaboration with students in other schools
and/ or countries• 4 interviews with elected officials• 6 Letters to the Editor written on a current topic in local and/or state
government
HSSSE - what do you believe your high school
emphasizes most?
• 21% memorizing facts and figures• 32% understanding information and ideas• 22% analyzing ideas in depth
• 68% my school challenges me academically• 35% my school challenges me intellectually
HSSSE: what instructional methods do you find
exciting or engaging?
• 60% discussion and debate• 60% group projects• 44% student presentations• 24% teacher lectures
Using an Engagement Survey
• Use the HSSSE and contribute your school data for national benchmarking;
• Or, create your own survey, then compare your results to the national HSSSE 2009 data, where applicable .
The Emergence of Longitudenal Data
• The National Clearinghouse has 93% of all US colleges collecting and providing longitudenal data
• Two independent school associations require student tracking for accreditation (freshman GPA):
• CAIS Canada [email protected] • ISASW www.isasw.org
Measuring teacher engagement, professionalism, attachment to school
Can that be quantified?• Longevity statistics• % faculty with advanced
degrees• % faculty participating
in Annual Giving
Occasional Teacher Absenteeism
• Reported in “school district report cards”• Measures only consecutive days absent, less than
5, 7 or 10 days. After the threshold, absence considered “long term illness”, removed from calculation
• O.C.A considered an indicator of faculty commitment, professionalism, attachment
Greenwich CT High School: State Report Card
Measure your O.T.A.
• Compare it to your local or state public school average
• Inform your Board of Trustees• Inform your Parents Association
Measuring teacher effectiveness: in the best public schools, teacher evaluation is an
informed, professional process
How can independent school leaders become more skilled in the clinical observation and evaluation of teaching?
How can we measure what we value in teaching practice?
The TIMSS video study – teaching
practices in 7 select countries
New and emerging assessment instruments – workshop goal 3
• We have just reviewed ideas for improving the use of existing data
• What about new tools and techniques? • Are the assessment instruments and practices of
earlier generations obsolete or incomplete?• If so, why is that true?
Did something occur on January 1, 2001 that changed everything?
First successful HTTP communication (modern WEB)1990
Netscape, easy to use browser 1995
Google, as a research project 1996
LiveJournal, Blogger, hosting sites 1999
Ericsson smartphone 2000
Wikipedia 2001
Facebook 2004
MIT Open Courseware 2004
YouTube 2005
Skype 2005
The World is Flat first edition 2005
Internet/Multimedia Smart phone (iphone) 2007
Global financial crisis 2008
Khan Academy (2600 videos and growing) 2009
Turn of the millenium events
100 most influential people of the 2nd Millenium
• Jonas Salk 97• Steven Spielberg 91• Elvis Presley 57• Gregor Mendel 42• Martin Luther King 33• Henry Ford 29• Michelangelo 19• Galileo 10• Columbus 6• Charles Darwin 4• Who is Number 1?
Johann Gutenberg
The printing press was information technology
What about modern day visionaries
in information technology?
What number is Bill Gates?
Steve Jobs?
What’s Past is Prologue vs The Future is Not What it Used to Be
• “What does an educated person need to know?”
• Education is defined by a remembered body of knowledge, the “canon”
• Critical, sarcastic view of the canon: Education as inoculation:” American history? I had that, Tetanus shot? I had that…)
20th century technology:
• Radio, television and film had great promise, but no demonstrable effect on schooling
What 20th century technology had a revolutionary effect on teaching and learning?
SCANTRON! Bubble answer sheets!
• 1948 – Scantron Corporation revolutionized the speed and efficiency of data collection and advanced the notion that student proficiency and school quality can be determined through mass-produced, multiple choice metrics
• Scantron, to this day, has had a greater impact on k-12 curricular design than any other technology in history.
in post scantron decades: “What gets measured is what gets
taught”.
• Tests “drive” instruction in ways that mimic both the content and format of the test.
• What gets measured is almost exclusively content• In the Information Age, we measured recall of
information
In 2012, in The Conceptual Age
· There are no books, conferences, op-ed pieces on “21st Century Content”.
· The canon has been buried under the information explosionHowever, · There is near universal agreement on a short list of 21st century
skills. There is near universal agreement on the need to employ technology in a thoughtful but robust manner
Nicholas Negroponte on applying technology in a robust
manner:
• “When you drop a penny into a glass of clear water, you get a glass of clear water with a penny in it; the change is additive.”
• “When you place a drop of red dye in a glass of clear water, you get a glass of pink water. The change is ecological.”
• Technology in education needs to be ecological; pink water
The i generation
• Defined mostly by their use of technology• Accustomed to learning things on their own and learning
from peers• Expect technology to be interactive and customizable• Non-linear thinkers; web thinkers, scanners, multi-taskers
Clay Shirke, futurist describing the i generation:
“A father sets up a new television in the living room. His 4 year old daughter is seen rummaging through the box. What is she looking for?”
Passive media experiences
will hold less appeal for this
generation
The 21st century skills movement, the Schools of the Future movement, focus on the
development of these skills:
Communication Collaboration Critical/Analytical Thinking Creativity Problem-solving
Content is still important; but content in these areas will need to be acquired through active exploration as well as through instruction.
American work in the 21st century
Non–Routine Tasks defined in the Journal of Economics, volume 118
• Gathering, synthesizing, and analyzing information.• Working autonomously to a high standard with minimal
supervision.• Leading other autonomous workers through influence.• Being creative and turning that creativity into action.• Thinking critically and asking the right questions.• Striving to understand others’ perspectives and to
understand the entirety of an issue. Communicating effectively, often using technology.
Current assessment tools do not measure these skills.
You cannot have 21st century schools using 20th century assessments.
Ideal Assessment:
Provides accurate demonstration of student proficiency
Yields information for faculty planning· Is valid as a learning experience in and of itself
· An assessment of, for and as learning
What is a performance task? Students assume roles in a scenario that is based in the "real world" and contains the types of problems they might need to solve in the future. The task requires critical thinking, analytical reasoning and problem solving. Communication skills are used in describing the solution.
Ohio Mastery Test, Grade 9
• Ms. Johnson installs new insulation to save money on heating costs, but then learns that her bills have not declined by much from the previous year. Her contractor points out that heating costs have risen and weather has been colder. Ms. Johnson wants to find out how much she has actually saved due to the insulation she installed. On the basis of the situation painted above, details about Ms. Johnson’s heating bills (rates, units of heat used), temperature changes, and some initial information to help them begin to research “heating degree days” on the internet, students are given two tasks:
• (1) Assess the cost-effectiveness of Ms. Johnson’s new insulation and window sealing.
• (2) Create a short pamphlet for gas company customers to guide them in making decisions about increasing the energy efficiency of their homes.
Students get answers to questions THEY ask
• For example – Family history – Is this breast cancer
possibly caused by abnormal oncogene expression? If so, certain types of hormonal therapy or receptor antagonists are more effective treatments.
– What level of stage III cancer, A, B or C?
CBAL
• Extended, constructed-response tasks that are delivered by computer and automatically scored.
• Pilot testing occurred in 2010 and 2011, spring of 2012. • Tests should be available for use in 2012. • Sample tests available online
Website information is in your folders.
features real-time, scenario-based tasks that measure an individual's ability to navigate, critically evaluate and understand the wealth of information available through digital technology
Ken Robinson
Age and education:
• Increase routines of behavior and habits of thought (left brain logical thinking )
• Decrease divergent thinking (free association of ideas. Right brain, creative thinking)
Creativity Index: the newstate mandate?
• Gov. Deval Patrick has made Massachusetts the first state in the country to call for the formation of a creativity index aimed at rating public schools statewide based on their ability to teach, encourage and foster creativity in students.
• Similar legislation is pending in California and Oklahoma
Torrence Test of Creative Thinking
Verbal Activity 4: Product Improvement Look at the stuffed toy elephant in the
drawing. It is six inches tall and weighs a half pound. In the space provided, list the cleverest, most interesting and unusual ways you can think of for changing this toy so that children will have more fun playing with it. Do not worry about how much the change would cost.Think only about what would make it more fun to play with.
Activity 2 and 3: Guessing Causes and Guessing Consequences measures “idea fluency”
What do you get if you solve this problem and visit the website?
The 4th goal of this workshop: Examples of assessment practices in a select group of
schools that define themselves – and are recognized by others as “Schools of the Future”
Schools that define themselves asSchools of the future
• Who are the pioneers?• What do these schools have in common?
Science Leadership AcademyPhiladelphia, Pennsylvania
All of these schools have 2 common characteristics
• urban public charter schools.
• experimenting with a dramatically different view of teaching and learning; A Collaborative, Conceptual Model
21st Century education: from coverage model to
conceptual model
• Recall of information (content) is still important• Skill in accessing and selecting information (internet
searches now deliver 2000 hits) vitally important• Ability to use or apply information in new and/or novel
settings most important (Its not what you know, but what you can do with what you know)
• Expanded role of the teacher: guide, coach, facilitator
Schools of the Future: characteristics
• Performance tasks• Project-based learning, individually and in groups• Capstone projects, individually and in groups• Independent study• Online learning, online tools (courseware, Skype, You
Tube, Ning, Moodle, Web 2.0 etc…)• Students given choices in assignments and in
demonstrations of mastery
Schools of the future, cont.• Extensive use of essential questions relating to content
area
why, how and what if questions• Computer-adaptive learning (program adjusts to student
skill level)• E-portfolios, published within the community or on the
web – seeking Facebook-type conversations in the academic community, on academic topics
• Flipped classroom strategies – routinely or occasionally• Partnerships, learning experiences beyond the school
campus• RUBRICS used to assess performance
A New Definition of School
“we need to invert the conventional classroom dynamic: instead of teaching information and content first, and then asking students to answer questions about it second, we should put the question/problem first, and then facilitate students with information and guidance as they seek the answer and hold them accountable for the excellence of their solutions and of their presentation of their results”.
-Ted Mccain Teaching for Tomorrow
What about independent schools?
• lead the nation in communication skills; writing, speaking, the performing arts
• Engagement has been supported by very strong student-faculty relationships
• An incremental approach to the challenges of the future; preserving strong, successful, traditional programs while expanding collaborative learning, online learning, project-based assessments, exhibitions of learning and use of digital portfolios
• Growing interest in “Essential Questions” theory of learning
Independent schools
http://stolaf.edu/depts/cis/wp/ebouvier/index.html
Lessons from our research:
Schools in the 21st century will define success in much broader terms
Great Schools in the 21st century will include some that have far fewer resources than independent schools. What they have is the freedom to take big risks in designing innovative cultures
“Measuring What We Value” Sites Referenced in Presentation
Hechinger Article containing multiple links of sample questions on new 2012 assessmentshttp://hechingered.org/content/are-new-online-standardized-tests-revolutionary-decide-for-yourself_5655/
Information on Torrance Testhttp://www.ststesting.com/ High School Survey of Student Engagementwww.indiana.edu/~ceep/hssse/ College Student Experiences Questionnairehttp://cseq.iub.edu/cseq_generalinfo.cfm National Student Clearinghousewww.studentclearinghouse.org/ The Self-Regulation Questionnairewww.psych.rochester.edu/SDT/measures/SRQ_text.php MHISC: Mental Health in Independent Schoolshttp://www.harthosp.org/InstituteOfLiving/OtherServices/MHISC/default.aspx
C-bal Cognitively Based Assessment for Learninghttp://www.ets.org/research/topics/cbal/initiative/ The CWRA: College to Work Readiness Assessmentwww.cae.org/cwra/ Science Leadership Academywww.scienceleadership.org/ High Tech Highhttp://www.hightechhigh.org/ New Tech Highhttp://newtechhigh.org/ Big Picture Learninghttp://www.bigpicture.org/ NYC i schoolhttp://www.nycischool.org/ Microsoft School of the Futurehttp://www.microsoft.com/education/en-us/leadership/partners_in_learning/Pages/School-of-the-Future.aspx
Performance Assessment Group of NYC Schools (check out the rubics!)http://performanceassessment.org/ Rubics – Association of American Colleges and Universities (rubics on critical thinking, creative thinking, problem-solving and others!)www.aacu.org/value/rubics Avenueshttp://www.avenues.org/ Haverford School (Decision Education)http://www.haverford.org/ Decision Education (critical thinking/character education program)http://www.decisioneducation.org/ Independent Curriculum Grouphttp://www.independentcurriculum.org/ Greens Farms Academyhttp://www.gfacademy.org/RelId/607374/ISvars/default/Capstone.htm
Hotchkiss/Loomis Collaborative Learning Projecthttp://tinyurl.com/3kq8v8f
Project-based Learning (450 sample projects – all subjects and grade levels – templates for organizing projects)http://pbl-online.org/ Siemens Challenge (sample award-winning projects)http://www.wecanchange.com/ American Society for Engineering Education (ASEE)http://teachers.egfi-k12.org/ Exploravision (sample award-winning student projects)http://www.exploravision.org/ Toyota Tapestry Grants for Science Teachers (sample grant-winning ($10,000) projects)http://www.nsta.org/pd/tapestry/
