Krutikova plenary ag slides ifs slides_final

© Institute for Fiscal Studies

Interpreting the evidence on impacts of programmes targeting adolescentsSonya Krutikova (Institute for Fiscal Studies, EDePo)


Main aim

The development of a strong unifying conceptual framework has hugely strengthened the state of knowledge and design of programmes in the field of Early Childhood (ECD)


Main aim

The development of a strong unifying conceptual framework has hugely strengthened the state of knowledge and design of programmes in the field of Early Childhood (ECD)

→What can we learn from the ECD experience in using programme evaluation to inform design of effective and scalable programmes for adolescents

Outline• Importance of a conceptual framework for better

programme evaluation and design• Key components of the ECD conceptual framework• Adapting the framework to adolescence• Knowns and unknowns• The potential of this approach to evaluation for

improving programme design: example from ECD


The importance of a conceptual framework for effective programme design

A key ingredient for success of ECD agenda = development of a unifying framework (Heckman et al):• Way of interpreting and organising existing evidence • Understanding the mechanisms through which

programmes work/don’t work – using the framework in programme evaluation

• Identifying outstanding questions and designing studies to answer them

• Developing scalable, cost-effective programme/policy designs based on these foundations


Framing the evidence on adolescents/young people • Clear linkages between ECD and adolescence:

– Biologically early childhood and adolescence are the key developmental stages in HC formation

– Primary aim of programmes in both areas = build human capital

– Both are fields of significant policy and programming activity

– Growing body of experimental evidence


Framing the evidence on adolescents/young people • Clear linkages between ECD and adolescence:

– Biologically early childhood and adolescence are the key developmental stages in HC formation

– Primary aim of programmes in both areas = build human capital

– Both are fields of significant policy and programming activity

– Growing body of experimental evidence• However:

– Less coherence in what we are learning from programme evaluation & how it links to progress in other areas e.g. psychology

– Less clarity about theory of change/assumptions underlying programme and policy designs

→ Scope for incorporating approaches in ECD to strengthen design and evaluation of programmes in field of adolescence


Key components of the ECD framework (Attanasio, 2015)

– Process of human capital formation:• Multidimensional • Dynamic – current HC depends on past levels, some fixed factors

(e.g. Parental education) and some factors that change over time including investment

• Windows of opportunity – investment in certain periods is particularly salient

• Investment is chosen by different actors (focus in ECD on parents)


Key components of the ECD framework (Attanasio, 2015)

– Process of human capital formation:• Multidimensional • Dynamic – current HC depends on past levels, some fixed factors

(e.g. Parental education) and some factors that change over time including investment

• Windows of opportunity – investment in certain periods is particularly salient

• Investment is chosen by different actors (focus in ECD on parents)

– Decision process determining investment behaviour:• Preferences incl. in relation to investment in a specific child, all of

the children in the household• How decision making process works within the household e.g. role

of mother and father’s preferences in final decision• Constraints:

– Financial resources – Information/beliefs re returns to investment


Adapting this framework to adolescence

• Process of human capital formation:– Change in the developmental domains of interest– Change in the types of investments that matter:

moving away from parental investments to own and school


Adapting this framework to adolescence

• Process of human capital formation:– Change in the developmental domains of interest– Change in the types of investments that matter:

moving away from parental investments to own and school

• Decision process determining investment behaviour:− Adolescent preferences may be different from adults

e.g. Importance of peers, importance of future vs present, importance of social norms

− Intra-household decision making – bargaining between parents and children

− Constraints:− Ability to make forward looking choices?


Knowns and Unknowns: Production Process• Knowns:

– Domains of importance: Windows of opportunity different from early childhood e.g. early childhood - sensory, motor and language skills; adolescence - development in executive functioning, socialisation processes, socio-emotional skills

– Types of investments:• Own investments become important • quality of school matters


Knowns and Unknowns: Production Process• Knowns:

– Domains of importance: Windows of opportunity different from early childhood e.g. early childhood - sensory, motor and language skills; adolescence - development in executive functioning, socialisation processes, socio-emotional skills

– Types of investments• Own investments become important • quality of school matters

• Unknowns:– Windows of opportunity– Can development in adolescence be effectively increases

through intervention – How does past investment affect productivity of

investments in adolescence – What are the key dimensions of school quality


Knowns and Unknowns: Decision making • Knowns:

– Adolescent preferences: Preferences and behaviour of peers matter a lot more in adolescence than other stages

– Ability to make forward looking choices: Adolescents are just as able to accurately assess risks as adults


Knowns and Unknowns: Decision making • Knowns:

– Adolescent preferences: Preferences and behaviour of peers matter a lot more in adolescence than other stages

– Ability to make forward looking choices: Adolescents are just as able to accurately assess risks as adults

• Unknowns:– How do peer preferences and behaviour enter into the

decision making process– What are the intra-household dynamics of making decisions

about adolescents– How is school quality produced


Back to programme evaluation and designing effective, scalable programmes: an example• Evidence from the Jamaica studies (Walker et a)


Back to programme evaluation and designing effective, scalable programmes: an example• Evidence from the Jamaica studies (Walker et a)• Design and evaluation of more scalable version of the

programme (Attanasio et al)



programme (Attanasio et al)• Find significant programme impacts (Attanasio et al,

BMJ)




BMJ)• Use additional data to estimate a model which allows to

disentangle the mechanisms – find that effects set in through change in maternal investment behaviour




BMJ)• Use additional data to estimate a model which allows to

disentangle the mechanisms – find that effects set in through change in maternal investment behaviour

• Design of the next project:– A) focus on measuring beliefs about the production function– B) Given this mechanism could the same effects be

attained in a (much more scalable) group setting?


Conclusion

An approach like this has potential to:

a) bring more coherence to evidence being generated through evaluation of adolescent programmes

b) Design future programme evaluation projects in a way that would systematically build on what we know


Government & Nonprofit

Krutikova plenary ag slides ifs slides_final