Best Practices in Out of School Time (OST) for Middle School Aged Children: Literature Review Matrix

Compiled by Laura Gogia, January 2013

Author(s),

Year

Article Title and

Journal

Participants and

Research Design

Major Findings Other notes

Bodilly,

McCombs,

Orr, Scherer,

Constant,

Gershwin

(2010)

Hours of Opportunity, Volume 1

www.wallacefoundation.org

Wallace Foundation-

funded programs in

Providence, NYC,

Boston, Washington DC,

and Chicago

Qualitative, replicated

case-study design

involving interviews,

business plans, other

documents

Program objectives being

reviewed:

1. Increase access and

participation

2. Improve program

quality

3. Develop information

systems to aid in decision

making

4. Plan for financial

stability

1. Early planning (needs

assessment, concerns and

desires of parents and

children, barriers to

participation like safety or

transportation)

2. MI system (enrollment,

attendance, demographics)

helps with future planning

and obtaining funding

3. Adequate recruitment and

advertising techniques,

especially:

a. program locators

(web-or paper-based

frequently distributed

through schools)

were directly linked

to enrollment rates

b. program site

coordinators who

worked with schools

to encourage

enrollment and

regular attendance

4. City-wide collaboration—

key components identified

include:

a. Mayoral support

in restructuring

agencies, increasing

funding, demanding

National best

practices

progress reports,

appointing a special

advisor with power

and authority to

ensure interagency

cooperation like

establishing memos

of understanding

(MOUs) to document

sharing of resources

and interpretation of

policy

Yohalem,

Devaney,

Smith

and Wilson-

Ahlstrom

(2012)

BUILDING CITYWIDE SYSTEMS FOR QUALITY:

A Guide and Case Studies

for Afterschool Leaders

http://www.wallacefoundation.org

/knowledge-center/after-school/

coordinating-after-schoolresources/

Documents/Building-Citywide-Systems-for-Quality-A-

Guide-and-Case-Studies-for-Afterschool-Leaders.pdf

Grounded theory

approach to create

theoretical framework of

QIS, then case studies and

local expert opinion of

ASPs in Atlanta, Austin,

Chicago, NYC, Palm

Beach Fl, and Hampden

Co MA to adjust

framework.

Framework is couched in

a continuous

improvement approach—

organizations regularly

measure against a

standard, then develop,

implement improvement

plans and then begin

cycle again

Recommendations for

citywide collaboration:

1. Shared definitions of

quality and standards

amongst stakeholders.

2. Clear leader organization

3. Engaged stakeholders

4. Continuous improvement

model including: standards

for high quality performance,

assessment tool, and aligned

improvement supports

(planning, coaching,

training)

5. Management Information

System (to collect data on

programs—attendance, etc.)

6. Clear guidelines and

incentives for participation

7. Adequate resources

National best

practices

Donner

(2012)

Making the Connections:

A Report on the First National Survey of

Out-of-School Time Intermediary Organizations

http://www.wallacefoundation.org/knowledge-

Online survey of 212

OST nonprofit

coordinating orgs

Defines OST intermediary:

1. Connects public and

private funders with direct

service providers

2. Provides technical

National best

practices and

indicators for

system design

center/after-school/coordinating-after-school-

resources/Documents/Making-the-Connections-Report-

First-National-Survey-of-OST.pdf

assistance and other supports

to direct service providers

OST intermediaries need 3

years to show a positive

impact in areas of building

data systems, increasing

investment in quality

standards and tools,

increasing kids’ access to

programs

60% of OST intermediaries

focus on increasing access to

affordable, high quality OST

programs for underserved

kids

LaFleur

Russell

Low

Romash

(2011)

THE BEACON COMMUNITY CENTERS

MIDDLE SCHOOL INITIATIVE:

Final Report on Implementation and Youth Experience

in the Initiative

http://www.wallacefoundation.org/knowledge-

center/after-school/evaluations/Documents/Beacon-

Community-Center-Middle-School-Initiative-Final-

Report-on-Implementation-and-Youth-Experience.pdf

80 Beacon Community

Centers in NYC

Community-based, located in

selected public schools,

serving youth in evenings,

weekends, holidays, and

summers. They do have

Youth Councils to provide

youth input into

programming

Provide structured

programming for grades 5-8

in academics, life skills,

career awareness, civic

engagement, physical health,

arts, and culture

Middle school initiative

asked for target enrollment

of 200 middle schoolers per

center (each center average

total student enrollment of

1200)

Exemplary

Program

Strategies

Performance

measures:

Program

Enrollment

and

Attendance

For 2009-2010, set program

level goals of 216 hours of

programming for middle

schoolers (avg of 72 days a

year, 3 hours a day

participation)

Saw slight increase in ELA

(English Language Arts) and

standardized math scores for

program participants

compared to those who did

not participate

Programs housed in middle

schools had the highest

middle school enrollment

Interaction with family was

important to promote

ongoing youth participation

Community

Indicator

Grossman,

Campbell, &

Raley,

(2007)

Quality time after school: What instructors can do to

increase learning

http://www.pop.givewell.org/files

/Cause4/East%20Harlem%20Tutorial%20Organization/

PPV213_publication.pdf

Philadelphia Beacon-

based; not designed to

measure impact but rather

what affects student

engagement/enjoyment

1. Surveys of youth on

the perceptions of

activities

2. Surveys of staff for

training profiles

3. Observations focused

on adult/youth and peer

relationships, teaching

methods, behavior

management,

youth decision-making

1. Project based learning

2. The most important ways

to ensure youth engagement

involves group management:

a. setting reasonable

ground rules

b. providing positive

reinforcement

c. being consistent

and fair in reinforcing

expectations

d. remaining firm,

but not harsh, when

ground rules were

broken.

Quantitative analysis did not

Strategy and

best practices

and youth input to further

describe what occurred in

each activity.

find a direct link

between peer affiliation or

cooperative peer learning

and participants’ level of

engagement or their

perceived level of learning.

But the more participants

reported that staff

encouraged them to work

together, the more youth

enjoyed the activity and the

more they wanted to return.

Kauh (2011) AfterZone: Outcomes for Youth Participating in

Providence’s Citywide After-School System

http://www.wallacefoundation.org/knowledge-

center/after-school/evaluations/Documents/AfterZone-

Outcomes-Youth-Participating-Providences-Citywide-

After-School-System.pdf

Quasi-experimental

design

768 6th

graders, 50%

enrolled in Afterzone and

50% not enrolled

Data collected from:

1. Youth surveys

administered at the

beginning and end of 6th

grade and end of 7th

grade

2. Administrative school

records

3. PASA MIS

Did NOT look at the

quality of the Afterzone

programs

Offers programming in

sports, skills, and arts

3 distinct sessions a year

(Fall, Spring, Summer)

2.5 hours a day, 4 days a

week, for 6-8 grade

4 key model features:

1. Single set of quality

standards with

training/support provided for

providers

2. Neighborhood

campuses—multiple sites in

geographically clustered

areas anchored by 1 or 2

middle schools where the

day begins and ends

3. Developmentally

appropriate programming

(i.e. encouraging

independence and exposing

to new experiences)

4. Check-in and check-out

process, so student presence

monitored at all times; safe

Exemplar

Program

Strategies

transport provided between

all sites

Benefits:

1. Strikingly higher school

attendance (but only in the

first year of participation)

2. Improved attitudes

towards community

resources, social skills,

increased connection to

school

3. In seventh grade, higher

grades in Math

There was a dosage effect to

benefits.

Performance

Measures

Kotloff and

Korom-

Djakovic

(2010)

AfterZones: Creating a Citywide

System to Support and Sustain

High-Quality After-School Programs

http://www.wallacefoundation.org/knowledge-

center/after-school/coordinating-after-school-

resources/Documents/Afterzones-Creating-Citywide-

System-to-Support-and-Sustain-High-Quality-After-

School-Programs.pdf

Mixed methods study to

examine how Afterzones:

engaged and retained

youth, promoted positive

youth development,

ensured high quality

programming

Methods

1. Site visits

(interviewing staff and

stakeholders)

2. Program observations

using quality assessment

tool RIPQA

3. Youth and Instructor

feedback surveys

4. Program documents

To track students through the

zone and collect participation

data, AfterZones uses

youthservices.net

4 dimensions of quality

measured for programs:

1. adult support

2. physical/emotional safety

of the environment

3. quality of adult and peer

interactions

4. ability for the youth to

make choices and plan

Exemplar

program—

system design

Durlak,

Weissburg,

Pachan

(2010)

A Meta-Analysis of After-School Programs

That Seek to Promote Personal and Social

Skills in Children and Adolescents

Meta-analysis of 69 ASPs

that foster personal and

social skills as main

objectives. Those

Evidence for following

SAFE technique for skills

training:

1. Sequenced

Recommended

strategies and

performance

measures

Am J Community Psychol (2010) 45:294–309

DOI 10.1007/s10464-010-9300-6

focused entirely on

academic outcomes were

excluded

2. Active

3. Focused

4. Explicit

Noted increases in self-

perception, positive social

behaviors, grades, and

reduction in problem

behaviors

Duffett,

Johnson,

Farkas,

Kung, Ott

(2004)

All Work and No Play? Listening to What Kids and

Parents Really Want from Out-of-School Time

http://www.publicagenda.org/files/pdf/

all_work_no_play.pdf

Commissioned by Wallace Foundation

2 national random sample

surveys, one with 609

middle and high school

students and another with

1003 parents of school-

age children

Cost, transportation issues,

and absence of the programs

within neighborhoods are all

barriers to low-income

family involvement in OST

programs

Parents are not necessarily

looking for OST to be

academically focused; 42%

would like it to help their

children develop

interests/hobbies

Research

supporting

need for early

planning/needs

assessment

Grossman,

Price,

Fellerath,

Jucovy,

Kotloff,

Raley,

Walker

(2002)

Multiple Choices after school:

Findings from the Extended-Service Schools Initiative

http://www.issuelab.org/resource/multiple_

choices_after_school_findings_from_the_

extended_service_schools_initiative

Multi-method of OST

programs in 10 cities:

Site coordinator filled out

annual organizational

survey and in-depth site

visits with interviews of

staff, students, parents,

other stakeholders, parent

telephone survey, and

observations of OST

activities

Research questions:

• Which youth came to

the afterschool

Programs and Why? Were

4 nationally recognized

models for OST programs:

Beacon, Bridges to Success,

Community Schools, West

Philadelphia Improvement

Corporation.

They all:

1. Operate out of schools

2. Involve collaboration

with community-based

organizations and/or

universities

3. Offer a range of activities:

academic, sports/rec,

enrichment

Exemplary

programs?

Strategies and

programs

the programs attracting

the young people who

could benefit most?

• What were the

characteristics of high-

quality programs?

• What were the benefits

of participation?

• Cost to operate and

ways to finance

4. Finances are controlled by

partnering organization, not

the school

Youth who attended

reported:

1. that they used less alcohol

2. handled anger in socially

appropriate ways

3. less likely to skip school

Parents in the phone survey:

1. felt the kids were getting

along better with peers

2. less likely to get in

trouble

3. more self-confident

4. to have a better attitude

towards school

Authors concluded that

enrichment programs were

most beneficial, offering

more opportunities for

fostering strong adult-child

relationships, peer

collaboration, decision-

making, and leadership skills

Locating programs in

schools serving low-income

communities was an

effective way of targeting

low-income students, but

additional resources were

required to target older youth

and the most high needs

students

Performance

Measures

Performance

Measures

Evidence

supporting

needs

assessment

and

meaningful

recruitment

Bodilly and

Beckett

(2005)

Making Out of School Time Matter:

Evidence for an action agenda

http://www.rand.org/content/dam/rand/pubs

/monographs/2005/RAND_MG242.pdf

Narrative Review Program

objectives/activities:

1. To provide safe

environment

2. Change attitudes towards

academic achievement,

achievement (test scores), or

level of attainment

(continuation to next grade,

high school graduation, jobs)

through:

a. Tutoring

b. Homework

assistance

c. Small group

learning

d. Homework

assistance

e. Field trips/College

trips

f. Career exploration

3. Change social/health

behaviors through:

a. drug/violence

counseling or

prevention

b. health education

4. Change social interactions

a. conflict-

resolution,anger

management

b. peer discussion

groups

c. parent support

groups

5. Cultural enrichment

(drama, crafts, music)—

although never assessed as

an outcome

Potential

performance

measures and

indicators

Programs/

Activities

Research

questions

remaining

Problems with

current

research

Impact of OST programs on

community beyond the youth

(ie crime reduction) has not

been measured convincingly

to date.

Current program evaluation

plagued with selection bias

and lack of control for

participation after enrollment

In general, OSTs program

eval shows very little impact

on test scores

Although no rigorously-

obtained data available, a

convergence of expert

opinion from multiple

sources suggest the

following lead to positive

youth outcomes:

1. clear mission

2. high expectations and

positive social norms

3. safe/healthy environment

4. supportive emotional

climate

5. small total enrollment

6. stable, trained personnel

7. appropriate content and

pedagogy relative to the

children’s needs

8. opportunities to engage

integrated family and

community partners

9. frequent assessment.

Problems with

test scores as

an indicator

National best

practices

Empirical evidence to

support:

1. Careful needs assessment

and market targeting,

providing ample program

advertising and recruitment

2. Monitoring enrollment

and attendance, follow-up on

absences, incentive programs

for consistant attendance

Lauer,

Akiba,

Wilkerson,

Apthorp,

Snow,

Martin-

Glenn (2006)

Out-of-School-Time Programs: A Meta-Analysis of the

Effects for At-Risk Students

Review of Educational Research 76 (2), 275-313

Meta-Analysis of 35 OST

quantitative studies

employing control or

comparison groups,

published in peer-review

journals, reporting effect

sizes based on gain scores

between pre- and post-

tests or post-test scores

between comparison

groups. Included K-12

programming

Indicators used: Math and

Reading standardized test

scores OR grades OR

classroom assessments

1. Small but significant

effects on reading and math

student achievement

2. Large positive effects for

programs with specific

characteristics such as

tutoring

3. After school versus

summer did not make a

difference

4. OST programming does

not have to focus entirely on

academics to make a

difference in indicators

Indicators

Support for

relationship

and strategies

Cooper,

Charlton,

Valentine,

Muhlenbruck

(2000)

Making the most of summer school: A meta-analystic

and narrative review. Monographs of the society for

Research in child development (Serial No. 260), 65(1),

1-118

Meta-Analysis and

narrative review of

summer school research

1. Positive academic effects

of summer school for both

low-income and middle-

income students

2. More positive results for

programs for smaller groups

of students involving

individualized and small

group instruction

Support for

relationship

between OST

and

achievement

Support for

strategy

3. Early elementary school

students benefitted more than

older elementary school

students

Fashola

(1998)

Review of extended-day and after-school programs and

their effectiveness (Report No. 24). Baltimore, Center

for Research on the Education of students.

**Have not been able to locate. This info is derived

from secondary sources.

Review of 34 extended-

day or after-school

programs

A. Programs studied fell

into following categories:

1. Language arts after-

school programs,

2. Study skills programs,

3. Academic programs in

other curriculum areas,

4. Tutoring programs for

reading

5. Community-based

programs.

Notes general paucity of data

on OST program evaluation.

For programs intending to

increase academic

achievement there is some

evidence of effectiveness of

those that provide:

1. Greater

structure/predictable routine

2. Link to the school-day

curriculum

3. Well-trained staff

4. Individual tutoring

Questions

remain in

research

Support for

national best

practices

Redd,

Cochran,

Hair, Moore

(2002)

Academic Achievement Programs and Youth

Development: A Synthesis

**Info based on abstract in ERIC

Review of 12

academically oriented

experimental or quasi-

experimental studies of

OST programs for

adolescents

Noted considerable

variability in outcomes.

Most programs had academic

achievement as only one

program objective. Other

objectives included health

and social wellbeing, self-

sufficiency.

Programs that focused on an

objective other than

academic achievement (but

included an academic

component) tended to do

better at achieving that

outcome than programs that

Inconsistency

in research

Program

objectives

Relationship

between

strategy and

outcome

focused entirely on

academics.

Quasi-experimental data

suggests participation for

longer and more intensive

durations had a more positive

effect on outcomes

Strategy

Miller (2003) Critical hours: Afterschool programs and educational

success, Quincy, MA: Nellie Mae Education

Foundation.

http://www.nmefoundation.org/

getmedia/08b6e87b-69ff-4865-b44e-

ad42f2596381/Critical-Hours?ext=.pdf

Comprehensive narrative

review of middle school

after school programs in

promoting academic

success and positive

adolescent development

McComb &

Scott-Little

(2003)

After-school programs: Evaluations and Outcomes.

Greensboro, NC: SERVE

Students who attend more,

benefit more

Barley et al

(2002)

Tutoring and peer tutoring

can be effective strategies for

improving achievement

during the school day

Policy

Studies

Associates

(1995)

Key is to engage student

attention

Hobbs,

(2012)

Effects of an afterschool program on elementary and

middle school math achievement in Georgia Schools: A

Dissertation

http://digitalcommons.liberty.edu/cgi/

viewcontent.cgi?article=1545&context=doctoral

ANCOVA comparison of

a sample of 180 at-risk

upper elementary and

middle school students

who did and did not

participate in OST

program

Indicator for study: Georgia

Criterion Referenced

Competency Test (GCRT)

Indicator for program (21st

Century Community

Learning Center):

Maintaining student

enrollment and attendance,

math and reading classroom

grades, improving behavior,

completing homework,

involving parents

Findings: No significant

difference in test scores

Niehaus,

Rudasill,

Adelson

(2012)

Self-Efficacy, intrinsic motivation, and academic

outcomes among latino middle school students

participating in an After-School program

Hispanic Journal of Behavioral Sciences 34, 118

47 Latino 6-8 grade

students in an afterschool

program specifically for

Latino students

1 year longitudinal study

1. Intrinsic motivation

positively correlated with

GPA

2. Self-efficacy was a

positive predictor for school

attendance and standardized

math achievement scores

3. Program attendance

contributed positively to

math achievement

4. Multilevel growth

modeling showed that self-

efficacy and intrinsic

motivation remained stable

across school year, unrelated

to degree of participation in

the program

5. Incorporated Bandura

(mastery experiences,

vicarious learning, social

Measurement/

Outcome

Relationships

Program

strategy

persuasions) and

opportunities to choose

activities, enhanced

interpersonal relationships by

maintaining communication

with students’ teachers and

parents.

Valentine,

Cooper,

Patall,

Tyson,

Robinson

(2010)

A method for evaluating research synthesis: The

quality, conclusions, and consensus of the synthesis of

the effects of after-school programs

Research Synthesis Methods, 1, 20-38

1. Many syntheses on ASPs

(After school programs)

aren’t published in traditional

academic outlets—are better

found by searching for

nonprofits that evaluate

ASPs, meeting notes, etc

2. Common indicators:

Academic achievement,

prevention, hobbies/interests,

social-emotional

development, safe

environment

Indicators

Little &

Harris

(2003)

A Review of Out-of-School Time Program Quasi-

Experimental and Experimental evaluation results.

Out-of-School Time Evaluation Snapshot

Harvard Family Research Project

Review of 27

experimental and quasi-

experimental OST

program evaluations

OSTs linked to better

attitudes towards school,

higher educational

aspirations, better school

performance and attendance,

less disciplinary action.

Also decreased drug/alcohol

use, sexual

activity/pregnancy, and

increased skills for coping

with peer pressure.

Positive links to these “youth

development outcomes”:

decreased behavior

problems, improved social

Research

supporting

programs/

potential

performance

measures

and communication skills,

increased community

involvement and broadened

world view, increased self-

confidence and self-esteem.

Little,

Priscilla

(2012)

A Field is Born: Reflections on a decade of afterschool

http://www.hfrp.org/out-of-school-time/publications-

resources/a-field-is-born-reflections-on-a-decade-of-

afterschool

Expert opinion Family Engagement should

be broadened to include:

1. Family involvement in

programming or onsite

2. Family ensuring kids get

to the program

3. Helping kids make

informed choices regarding

programming choices

4. Discussing kids’ progress

with staff

Six Strategies for Engaging

Families

1.Have adequate and

welcoming space to engage

families

2.Establish policies and

procedures to promote family

engagement

3.Communicate and build

trusting relationships

4.Be intentional about staff

hiring and training to

promote effective staff–

family interactions

5.Support families and their

basic needs

6.Connect families to each

other, to the program staff, to

schools, and to other

community institutions

Best practice

Bethea

(2012)

The Impact of Oakland Freedom School’s Summer

Youth Program on the Psychosocial Development of

African American Youth

Black Psychology, 38, 442

Program outcomes of

Leadership excellence,

Inc., Oakland Freedom

Schools (6 week summer

Language Arts

enrichment course for 5-

14 yos)

79 students (20 were

middle schoolers)

completed pre- and post-

ERAS for attitudes

towards reading, PRSCS

or Student Self Concept

Survey, WALLY for

problem solving, SSRS

for social behaviors,

Adolescent Survey of

Black Life for racial

identity, Social Action

Questionnaire

Program positively

influenced racial identity and

youth views toward AA

culture and precepts

Increase in pretest to posttest

scores on social skills

strategies and future

commitment to social action

No significant increases in

attitudes toward reading or

self-concept.

Program strategies—books,

field trips, games reinforce

each other in context of AA

history/culture through the

eyes of children. Emphasis

on AAs engaging in social

action or community service.

No testing or entrance

requirements for students

Performance

measures

Program

strategies

James-

Burdumy,

Dynarski,

Moore,

Deke,

Mansfield

Pistorino.

(2005)

When Schools Stay Open Late: The National

Evaluation of the 21st Century Community Learning

Centers Program: Final Report. U.S. Department of

Education, Institute of Education Sciences, National

Center for Education Evaluation and Regional

Assistance, 2005.

http://www.ed.gov/ies/ncee

**There is controversy regarding this study.

26 21st CCLC centers in

12 school districts

Synthesis of two previous

reports plus two years of

follow-up on elementary

school students

Randomly controlled field

trial

Data on students’

supervision after school,

Typical program activities:

homework sessions, academic

activities, enrichment activities

like art, drama, music, and

recreation activities.

Treatment subjects felt safer

after school but had no

improvement in academic

achievement (test scores in

math, reading, science, social

studies) and an increase in

Activities

Outcomes

See Mahoney & Zigler (2006). Translating science to

policy under the No Child Left Behind Act of 2001:

Lessons from the national evaluation of the 21st century

community learning centers. J of Applied

Developmental Psychology, 27(4) 282-294.

academic achievement,

behavior, developmental

outcomes, and feelings of

safety after school were

collected from parents,

teachers, students, and

school records in fall

2000 (baseline), spring

2001 (first followup), and

spring 2002 (second

followup) for the first

cohort of students, and

one year later for students

who applied to centers in

fall 2001.

The Stanford

Achievement Test in

reading was administered

at baseline and followup.

Implementation data from

program staff and

principals and two sight

visits to each site also

performed.

negative behaviors (measured

by suspensions, calls home

from teachers, teacher

disciplinary action), mixed

results in developmental

indicators

**Parental involvement was

measured by asking parents to

report how often they helped

their children with homework.

Parents with elementary school

children in the program

averaged helping their children

with homework 3 times a week,

significantly higher than those

not in the program

House, 1992 The relationship between perceived task competence,

achievement expectancies, and school withdrawal of

academically unprepared adolescents.

Child Study Journal 24(4), 280-299

4 year, prospective,

longitudinal study of 378

academically

underprepared students

Academic self-concept

and achievement

expectancy measured

Students’ educational

expectations are very strongly

related to eventual decision to

withdrawal from school

Relationships

between

potential

measurements

and indicators

National

League of

Cities, 2011

City Strategies to Engage Older Youth in Afterschool

Programs

http://www.nlc.org/Documents/Find%

20City%20Solutions/IYEF/Afterschool/

city-strategies-to-engage-older-youth-in-afterschool-

programs-oct-12.pdf

(The Wallace Foundation)

Strategy guide geared

towards helping large or

mid-size city programs

attract youths age 11-18

Based heavily on

Deschenes, Arbreton,

Little, Herrera, Grossman,

Weiss, Lee (2009).

Engaging Older Youth:

Program and City-level

Strategies to Support

Sustained Participation in

Out-of-School Time

Cambridge, MA: Harvard

Family Research Project.

Recommendations:

1. Coordinate systems to

support effective service

delivery (meaning strong needs

assessment, making sure

programs are accessible and

support marketing/recruitment,

track participation and impact)

2. Ensure quality programs

(meaning well-trained staff,

input from youth on types of

activities, setting clear

standards, providing staff

professional development)

3. Offer relevant programming

(older youth gravitate toward

academic, arts, recreation, and

science programs as well as

skill acquisition for reaching

long term goals)

4. Promote college attendance

and workplace readiness

Middle schoolers need

programs focusing on choice,

leadership opportunities,

cultural enrichment, health and

wellness, community service

National best

practices

Deschenes,

Arbreton,

Little,

Herrera,

Grossman,

Weiss, Lee

(2010)

Engaging Older Youth: Program and City-Level

Strategies to Support Sustained Participation in Out-Of-

School Time

http://www.hfrp.org/out-of-school-time/publications-

resources/engaging-older-youth-program-and-city-

level-strategies-to-support-sustained-participation-in-

out-of-school-time

Commissioned by The Wallace Foundation

Mixed methods—large

sample survey and in-

depth interviews from 6

cities (Chicago,

Cincinnati, NYC,

Providence, SF, Wash

DC)

Research Questions

1. What are the

characteristics of high-

participation

OST programs that

Successful in supporting high

retention:

1. Provide leadership

opportunities for youth (eg

volunteer/community service

activities, youth councils,

opportunities to design

activities for peers or younger

youth, “officer” roles, paid staff

positions)

2. Having staff keep informed

about what the youth are doing

outside of programs (eg

collecting report cards and

Research to

support

national best

practices

support sustained

participation as

measured by retention?

2. How do these

characteristics differ for

middle school

and high school youth?

3. What strategies are city

initiatives implementing

to support access to

programs and sustained

participation, and how do

OST programs perceive

the

usefulness of city-level

strategies for achieving

their

participation goals?

contacting parents regularly,

publically recognize

accomplishments outside of

program)

3. Being community based

(rather than school based)

4. Enrolling 100 or more youth

5. Holding regular staff

meetings

Middle school programs

reported that particularly

around eight grade, youth

stopped attending because they

wanted a program that felt

“older.” Consider programming

for eight graders that includes

more responsibility.

High retention programs report

using more strategies to engage

families than low retention

programs

Harris (2011) Afterschool Evaluation 101: How to Evaluate an

Expanded Learning Program

http://www.hfrp.org/out-of-school-time/publications-

resources/afterschool-evaluation-101-how-to-evaluate-

an-expanded-learning-program

A toolkit for OST

program directors for

designing and

implementing an

evaluation strategy