Evidence-based Evaluation for Afterschool Programs

Denise Huang

CRESST/UCLA

1/22/07

In the Beginning….

after school programs were available to students at a much smaller scale

The purpose is mainly to provide monitoring

The opportunities for students to engage in enrichment activities

In the 90s, the concepts of safe haven for at-risk youths

No Child Left Behind Act (2002):

An emphasis is placed on schools to improve academic achievement, particularly for students at risk

Schools that are not performing well can use after school programs as supplemental services to improve student performance

More funding becomes available, both from the government and private sector

For the last 3 years, the 21st Community Learning Center is working with a budget of over $1 billion

The Formation of Afterschool Alliance

A partnership between DOE and Mott Foundation

A coalition of public and private organizations

Dedicated to raise awareness of the importance of ASP and advocating on their behalf

The goal is to make quality ASP available to all children by 2010

The Need of Evidences to Support Expansions

• Do afterschool programs contribute to positive youth development?

• What aspects of the program functioning contribute to these positive outcomes?

Indicators for program effectiveness

• Student attendance in regular school

• Performance measures (achievement tests, homework completion, classroom grades, language re-designation, school retention, future aspirations, etc.)

• Non-cognitive measures (safety, attitudes towards school, relationships with adults, social competence, conflict resolution skills, self-esteem, and self-efficacy etc.)

• Parent involvement

• Professional development

• Long-term effects (drop out rate, life satisfaction, etc)

What is known about the impacts of ASP?

Tutoring and academic

Inconclusive

Personal and Social Outcomes

Findings in general positive :

concerns about safety

conflict resolution

academic self-efficacy

study habits

family involvement

future aspirations

Long-term Outcomes

Long-term effect study

Dropout study

DOJ study

Who Benefited Most :

Students who attended more regularly

Female students

LEP students

Students who scored low initially on attitudinal surveys (self-efficacy, work habits, conflict resolution skills)

Students with low performance in test scores

Summary of Effects from Current Meta-analyses

Youth who participated in quality ASP improved significantly in feelings and attitudes towards self and learning

Positive social behaviors

Academic improvements

Reduce gang participation and criminal behavior

What practices are effective in producing student outcomes?

Moving away from a deficit model

Emphasizing social capital, setting features, networking, collaboration, cooperation…

Purposeful staff development

Active learning and student participation

Ecological ModelAfterschool programs are social organizations embedded with cultural values

coming from families, school, and neighborhood climate and norms.

Community/Neighborhood

School LA’s BEST

Family

Child

Space / Community / Culture (structure/resources) (networks) (social, norms, sanctions)

Provisional Features of Positive

Developmental Settings (Larson et. al) Physical and Psychological Safety

Clear and consistent structure

Supportive relationships

Opportunities to belong

Positive social norms

Support for efficacy and mattering

Opportunities for skill building

Integration of family, school, and community efforts

Developmental Assets (Search Institute)Asset In Practice

Support School and other activities provide caring, encouraging environments for children

Empowerment Children are given useful roles at home and in the community

Boundaries and Expectations Schools provide clear rules and consequences

Constructive Use of Time Children participate in music, art and other creative activities at least three hours a week

Commitment to Learning Teachers encourage children to explore and engage in stimulating activities

Positive Values Children are encouraged to help others

Social Competencies Children begin to learn how to make choices at appropriate developmental levels

Positive Identity Children are hopeful about their personal future

High Scope Educational Research Foundation

“Points of service assessments”

• Safe Environment

• Supportive Environment

• Interaction

• Engagement

Evidence –based training approach (CASEL)

Sequenced - a sequenced set of activities to achieve skill objectives

Active - the use of active forms of learning

Focused – program component to be focused on specific skills

Explicit – the targeting of specific skills

Evidence Based Evaluation Model

Staff Practices

Critical voice

Classroom structure

Staff Philosophy

Shared control

Use of materials

Personal relevance

Interaction style

Student choices

InteractionPositive Social Norm & Supportive Relationship

&Opportunity to belong

NetworksIntegration of

school, family & community

efforts

ClimatePsychological

SafetyClear & Consistence

Structure

Active participants

Shared control

Lifelong learning

Autonomy Oriented Settings

Student negotiation