Phillips Brooks House Association (PBHA)’s Summer Urban Program

Overview: PBHA is a student-run non-profit with a dual mission of student development and community impact. PBHA youth programs build positive relationships with youth, families, and community partners to ensure that Boston and Cambridge youth ages 5-19 can build the social, emotional, and academic skills needed to define, access, and achieve their own success in and beyond our programs.

Site info: 12 public school buildings in Boston and Cambridge

Program Type: Mix of academic and enrichment

Students Served: 920 youth, 5-18 year olds

Staff: all sites combined have 130 FT seasonal college students, 10 FT year-round professionals, 80 FT seasonal high school students

Why Evaluation

• 423 Staff Applications• 910 Youth• 129,647 hours of programming…. But wait, there’s more!

5 Long-term OutcomesAcademic, Support System, Future, Community

Engagement, Self-Worth

Staff ManagementTraining, Coaching, Data Collection Processes

Objectives and Agenda

• Building an Inquiry Approach• Evidence-Based Improvement Process – 1. SAYO– 2. Writing Rubric – 3. APT Observation Data

• Individual Program Data • Debrief and next steps

Ex 1

Ex 2

(n=6)

1 =

Stro

ngly

Dis

agre

e; 3

= N

eutr

al; 5

= S

tron

gly

Agre

eEx 3

1 =

Stro

ngly

Dis

agre

e; 3

= N

eutr

al; 5

= S

tron

gly

Agre

e

(n=6)(n=40)

Ex 4

Validity

• Reliability: measurement tool design that is consistent each time it’s used

• Sampling: the questions relate well to the construct and representative group answered the questions

• Scaling: scores using scaled answers lead to more or less significant differences between two or more groups of results

What do you notice?

(evidence)

What do you

wonder? (focus

questions)

What more

data do you

need?

Identify a problem

we control

Plan highest

leverage strategy

Focus on Evidence and inquiry into data

Programmatic decisions and approach based on data

Case Study

Benchmark average for each index = 3.0 (Mostly True)

2014

2014APT Averages by Domain (out of 4)

n=42(no Middle Up)

Data Transparency/Sharing

• Google Presentations• Infoactive:– https://infoactive.co/plays/32998

• Tableau Public