AFTER SCHOOL Evaluation Report

2012-2013

33rd à 37th Percentile Rank Increase, Reading

27th à 33rd Percentile Rank Increase, Math

95% Parents reporting that they are more engaged with their child’s education because of BELL

90% Parents expressing satisfaction with BELL’s services

K-8 Grades Served

83% Average Daily Attendance

4,072 Scholars

85% Teachers expressing satisfaction with BELL’s program

AFTER SCHOOL 2012-2013 Report Card

experienceBELL.org Page 1 of 5

Percentile Rank, Start of BELL After School

BELL (Building Educated Leaders for Life) exists to transform the academic achievements, self-confidence, and life trajectories of children living in under-resourced, urban communities. In the BELL After School program, scholars in grades K-8 participate in literacy and math instruction and hands-on enrichment activities. The program also engages parents and positive adult role models from the community.

BELL partners with schools to expand learning time and reach the scholars who need it most. During the 2012-13 school year, 4,087 scholars participated in BELL After School at 35 schools in Oakland, San Jose, and San Rafael, CA; Baltimore, MD; Boston, MA; Detroit, MI; Charlotte, NC; Newark, NJ; and New York City, NY.

BELL measured program outcomes against three major goals for scholars: improved academic performance, enhanced self-concept and self-efficacy, and increased social skills. Additionally, evaluation activities sought to assess parent engagement and satisfaction with the program.

OUTCOMES

PERCENTILE RANK INCREASE: In BELL After School, scholars increased their percentile rank scores. Percentile rank scores range from a low of 1 to a high of 99, with 50 representing the middle score and denoting average performance at or near grade level. They provide a relative measure that compares BELL scholar performance to national norms. On average, when enrolling in BELL After School, scholars were underperforming: they entered at the 33rd percentile in reading and the 27th in math. In BELL After School, scholars increased their percentile rank scores to the 37th and 33rd percentiles in reading and math, significantly reducing their deviation from the norm. Reducing this deviation, in effect, shows that scholars are narrowing the achievement gap.

NORMAL CURVE EQUIVALENT SCORES: Scholars gained new literacy and math skills at a faster rate than their peers during the school year, according to a positive NCE score on standardized diagnostic tests. Normal curve equivalent (NCE) scores show a student’s relative position compared to others in the same grade and tested at the same time of year.  A gain in NCE indicates that the student has “grown” more than the norm group.  The average student demonstrates no change, for a NCE score of zero. BELL scholars are targeted for underperformance in school and need to learn at an accelerated rate in order to succeed academically (i.e., demonstrate a positive change in NCE scores).

Together, increases in percentile rank scores and positive NCE scores confirm that participation in the BELL After School programs boosts scholars’ engagement in learning and academic achievement. This conclusion is further validated by teacher and parent survey data.

23%

28%

27%

37%

31%

33%

12%

5%

6%

3%

5%

4%

0% 10% 20% 30% 40% 50%

Average Percentile Rank Increase by Subject

LITERACY

All Scholars

Grades K-4

Grades 5-8

All Scholars

Grades K-4

Grades 5-8

MATH

Percentile Rank Gain in BELL After School

AFTER SCHOOL 2012-2013 Evaluation Report

Page 2 of 5

95%

96%

80%

78%

97%

96%

93%

91%

84%

94%

95%

95%

85%

91%

0% 25% 50% 75% 100%

Parents reporting that scholars cooperate more with other children!

Parents reporting that scholars are better at group activities!

Teachers reporting that scholars improved their self-control skills!

Teachers reporting that scholars improved their peer relations skills!

Parents reporting that scholars improved their attitude toward learning!

Parents reporting that scholars improved their attitude toward school!

Parents reporting that scholars increased their self-confidence!

Teachers reporting that scholars improved their ability to express their ideas!

Teachers reporting that scholars improved their ability to ask for help!

Teachers reporting that scholars increased their self-confidence!

Parents reporting that scholars improved their math skills!

Parents reporting that scholars improved their' reading skills!

Teachers reporting that scholars improved their math skills!

Teachers reporting that scholars improved their reading skills!

Parent & Teacher Observations of Community Skills

Parent & Teacher Observations of Scholars' Academic Performance

Parent & Teacher Observations of Self-Efficacy Skills

TEACHER & PARENT SURVEY DATA

Page 3 of 5

METHODOLOGY

TEST DATA: Data included in this report come from BELL After School sites in CA, MD, NJ, NY, and NC, where BELL administered the Stanford Diagnostic Reading Test (SDRT-IV) (Harcourt Educational Measurement), a diagnostic assessment of grade-level literacy skills for grades K-8; and Stanford Diagnostic Mathematics Test (SDMT-IV) (Harcourt Educational Measurement), a diagnostic assessment of grade-level mathematics skills for grades 2-8. Tests were administered at the start and end of the BELL After School program.

At sites in MA, BELL collected data using a new measurement instrument. In Boston, BELL piloted STAR Enterprise assessments, computer adaptive assessments that provide our team of educators with stronger data, more efficiently, and faster.  BELL expects to roll out the STAR assessment more broadly in the summer of 2013 and the 2013-2014 school year.

SELF-EFFICACY DATA: BELL uses the Youth Outcome Toolkit, a validated, youth-respondent instrument, to measure changes in scholars’ academic behaviors, sense of self, positive core values, and life skills, because such indicators are are highly predictive of future school persistence and success.

TEACHER SURVEYS: BELL developed a teacher and tutor survey instrument to assess program implementation and efficacy from the perspective of instructional staff. Teachers and tutors complete the anonymous survey at the end of the program, which includes questions regarding scholars’ progress, training, curricula, BELL staff and service, and parent engagement.

PARENT SURVEYS: BELL developed a parent survey instrument to assess parent observations of scholar improvement, program quality, and parent satisfaction. Parents complete the anonymous survey on-site during the final week of the program.

93%

96%

90%

91%

95%

85%

97%

96%

90%

97%

95%

0% 25% 50% 75% 100%

Teachers reporting that they would recommend BELL to parents.!

Teachers reporting that they found working for BELL personally rewarding.!

Teachers reporting that BELL After School developed their professional skills.!

Teachers reporting that BELL's team displayed a positive attitude.!

Teachers reporting that BELL's team modeled BELL's mission.!

Teachers reporting high satisfaction with their experience in BELL After School.!

Parents reporting that they would recommend BELL to other parents!

Parents reporting that they would enroll their child in BELL again!

Parents expressing satisfaction with BELL's services!

Parents reporting that their child enjoyed BELL!

Parents reporting that they felt more engaged in their child's education!

Teacher Engagement & Satisfaction

Parent Engagement & Satisfaction

Page 4 of 5

EVALUATION ADVISORY BOARD

BELL’s evaluation activities are guided, advised, and endorsed by an external, interdisciplinary group, BELL's national Evaluation Advisory Board. The responsibility of the Evaluation Advisory Board is to oversee the application of principles and standards established in the evaluation and assessment fields.

Dr. Duncan Chaplin, Senior Researcher Mathematica Policy Research, Inc.

Dr. Leslie Goodyear, Senior Research Associate Educational Development Center, Inc.

Dr. Denise Huang, Project Director & Senior Researcher National Center for Research on Evaluation, Standards, and Student Testing (CRESST)

Ms. Priscilla Little, Research & Evaluation Consultant Evaluation Consultant

Dr. Beth Miller, Director of Research & Evaluation Nellie Mae Education Foundation

experienceBELL.org