Problem solving and critical thinking: assessing performance among 15-year olds in India and Vietnam

Problem solving and critical thinking: assessing

performance among 15 year olds in India and Vietnam

Padmini Iyer (Young Lives, University of Oxford)

Caine Rolleston (UCL Institute of Education)

UKFIET 2017

Oxford, 7 September 2017@yloxford

@padmini_iyer

@caine_rolleston #YLEducation

21st Century Skills: an introduction

Assessing problem solving and critical thinking in India

and Vietnam: the Young Lives study

Findings:

• How do students in India and Vietnam perform on

our problem solving and critical thinking tests?

• How can we explain performance in problem

solving and critical thinking?

OUTLINE

Introduction

21ST CENTURY SKILLS

Learning quality at secondary level: going beyond curriculum

knowledge and foundational skills

‘Critically needed skills to enable success in terms of

employment, health and personal wellbeing’ (Rankin et al 2015)

‘Crucial ingredient for disadvantaged youth to get and keep

jobs’ (Moore & Novy-Marx 2016)

21ST CENTURY SKILLS

Transferable Skills

(World Bank 2014)

21st Century Skills

(P21 2015)

Transversal competencies

(UNESCO 2016)

Assessing problem solving and critical

thinking in India and Vietnam

INTRODUCTION TO YOUNG LIVES

Longitudinal survey of 12000 children in 4 countries running for

15 years: Ethiopia, India (AP & Telangana), Peru & Vietnam

Pro-poor sample: 20 sites in each country selected to reflect

national diversity

School surveys introduced in 2010: sub-sample of Young Lives

children and their peers

2016-17 school surveys: Ethiopia (upper primary), India (lower

secondary) & Vietnam (upper secondary)

Learning progress in Maths and English; student performance in

transferable skills (problem solving and critical thinking)

ASSESSING PROBLEM SOLVING & CRITICAL THINKING

Problem solving

‘an individual’s capacity to use cognitive processes to

resolve real, cross-disciplinary situations where the solution

path is not immediately obvious’ (OECD 2004)

Items adapted from PISA 2003

Skills assessed included:

• Decision-making – deciding on the quickest and

cheapest bus journey between two points

• Trouble-shooting – working out why an irrigation system

isn’t working


Critical thinking

‘Skills such as inference and evaluation which are applied to

ill-structured problems, for which there are no definitive

solutions’ (Kuhn 1991; Thomas & Lok 2015)

Items adapted from CWRA+

Skills assessed included:

• Identifying points of agreement between two opposing

perspectives

• Evaluating reliability of information presented (including

potential bias)


Can we compare problem solving and critical thinking

performance in India and Vietnam?

Cross-country calibration process:

• IRT analysis of test data from the two countries

together

• Identifying differential item functioning

Results:

• Problem solving tests worked differently in the two

countries – so can’t directly compare problem solving

performance in the two countries

• Critical thinking tests worked similarly in the two

countries – so can directly compare critical thinking

performance in the two countries

Problem solving and critical thinking:

student performance

CRITICAL THINKING LEVELS

%19.6%

%30.6%

%49.8%

%11.3%

%37.5%

%51.2%

INDIAVIETNAM

Level 3: Competent critical thinkers

Level 2: Basic critical thinkers

Level 1: Emergent critical thinkers

Level 1: Basic problem solvers

Level 2: Competent problem solvers

Extra Classes (comprehension)

Energy Needs (comprehension)


Journey (comprehension)

Extra classes

Journey (partial credit)

IrrigationJourney (full credit)

Energy NeedsExtra Classes (partial credit)

Extra Classes (full credit)

Below Level 1: Weak or emergent problem solvers



PROBLEM SOLVING LEVELS: VIETNAM

%14.8%

%42.7%

%42.5%

PROBLEM SOLVING LEVELS: INDIA

%20.8%

%41.2%

%38.0%Extra Classes (comprehension)


Journey (comprehension)

Extra classes

Journey (partial credit)

Journey (full credit)

Energy Needs

Below Level 1: Weak or emergent problem solvers



Explaining problem solving and

critical thinking skills?

PERFORMANCE ON MATHS & PROBLEM SOLVING: INDIA

PERFORMANCE ON MATHS & PROBLEM SOLVING: VIETNAM

CONCLUSIONS

Student performance:

Performance on problem solving and critical thinking in India and Vietnam is broadly similar

• Half the students in each country are ‘emergent’ critical thinkers

• A slightly higher proportion of students in Vietnam are ‘competent’ critical thinkers

• Majority of students in both countries are ‘basic’ or ‘competent’ problem solvers

Explaining problem solving and critical thinking performance:

Relationship between high performance in school subjects and problem solving?

• Generally positive relationship – but there are schools in both countries where students

score highly in problem solving and not in Maths

• Further questions: school, teacher and student factors associated with high performance in

problem solving & critical thinking?

Thank you!

Any questions or comments?

[email protected]

[email protected]

@caine_rolleston

@padmini_iyer