Upload
lee-obrien
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
Challenges in
International Large-Scale Assessments
Higher School of Economics, Moscow, Russia, May 16, 2013
Challenges
1. Using information to improve quality2. Measuring beyond the academics3. Reaching out of school population4. What to measure5. Difficulty level of the tests
Challenges for effective use of assessments
• Political environment• Dissemination• Local capacity• Assessment culture
Challenge 2: Measuring beyond the academics
Success requires a combination of academic, personality and social skills
Personality is important to predict success
Personality Psychology and Economics. 2011Almlund, M., Duckworth, A., Heckman, J., Kautz, T.
Self-Discipline Outdoes IQ in Predicting Academic Performance of Adolescents
Duckworth, A. L. & Seligman, M. E. P. (2005). Psychological Science, 16, 939-944
Grit Scale
Men1st-9th percentile: < 2.5010th-19th percentile: 2.51 - 2.8220th-29th percentile: 2.83 - 3.0530th-39th percentile: 3.06 - 3.2440th-49th percentile: 3.25 - 3.3750th-59th percentile: 3.38 - 3.5360th-69th percentile: 3.54 - 3.7470th-79th percentile: 3.75 - 3.9180th-89th percentile: 3.92 - 4.0090th-99th percentile: 4.21-5.00
Women1st-9th percentile: < 2.5010th-19th percentile: 2.51 - 2.8720th-29th percentile: 2.88 - 3.1230th-39th percentile: 3.13 - 3.2440th-49th percentile: 3.25 - 3.4950th-59th percentile: 3.50 - 3.6260th-69th percentile: 3.63 - 3.7870th-79th percentile: 3.79 - 3.9980th-89th percentile: 4.00 - 4.2490th-99th percentile: 4.25-5.00
Background Questionnaires• TIMSS 2011
– Attitudes towards math and science: Like, value, confidence
• Results– Younger students have more positive attitudes– Correlation with achievement
• Strong and positive within countries• Weak among countries
Challenge 3:Reaching out of school population
73%
72%84%
91%
96%
n.a.
The PISA 2009 profiles by country/economy.http://stats.oecd.org/PISA2009Profiles/#
PISA 2006 Sample Coverage
Ferreira, F. & Gignoux, J. (2011). The Measurement of Educational Inequality. Achievement and Opportunity. Policy Research Paper 5873. Development Research Group. The World Bank
Cov
ered
Pop
/ T
otal
Pop
PISA 2006 Sample Coverage
Ferreira, F. & Gignoux, J. (2011). The Measurement of Educational Inequality. Achievement and Opportunity. Policy Research Paper 5873. Development Research Group. The World Bank
Cov
ered
Pop
/ T
otal
Pop
390
513
438
Science Scores
The “new” view in assessment
Society
PISA approach: to what extent students can apply their knowledge to real-life situations and be equipped for full participation in society.
21st Century Learning Framework
• Core subjects and interdisciplinary themes– Language, math, sciences, – Finances, health, civics
• Skills– Creativity and innovation– Critical thinking and problem solving– Communication and collaboration– Information, media, and ICT literacy
Challenge 5:Difficulty level of the tests
Measuring what student should know and be able to do?
Measuring what student actually know and can do?
Difficulty Level of the Tests
Focus on and Retrieve Explicitly Stated Information
Make Straightforward Inferences
Interpret and Integrate Ideas and Information
Examine and Evaluate Content, Language, and Textual Elements
Recognize letters
Recognize words
Read short sentences
PIRLS & PrePIRLS
EGRA, Provinha
Administrator reads test questions to the students.
Test questions are only included in the Test Administrator booklet
Difficulty Level of the Tests
Focus on and Retrieve Explicitly Stated Information
Make Straightforward Inferences
Interpret and Integrate Ideas and Information
Examine and Evaluate Content, Language, and Textual Elements
Recognize letters
Recognize words
Read short sentences
PIRLS & PrePIRLS
EGRA, Provinha
Opposed Forces
More low ability students expected to take the tests in more developing countries
Push for more difficult “higher order thinking skills” questions