34
Evaluation 101: After School Programs February 1, 2007 Region 3 After School Technical Assistance Center Conference

Evaluation 101: After School Programs

  • Upload
    allayna

  • View
    46

  • Download
    0

Embed Size (px)

DESCRIPTION

Evaluation 101: After School Programs. February 1, 2007 Region 3 After School Technical Assistance Center Conference. Evaluation 101. ASES Evaluation—student data and how to get it Prop 49 evaluation requirements in SB638 Suggested Timeline Roles and Responsibilities. ASES Evaluation. - PowerPoint PPT Presentation

Citation preview

Page 1: Evaluation 101: After School Programs

Evaluation 101: After School Programs

February 1, 2007

Region 3 After School Technical Assistance Center

Conference

Page 2: Evaluation 101: After School Programs

Evaluation 101

ASES Evaluation—student data and how to get it

Prop 49 evaluation requirements in SB638

Suggested Timeline

Roles and Responsibilities

Page 3: Evaluation 101: After School Programs

ASES Evaluation Annually by October 1 funded programs must

submit to CDE data on participating students Two years data required:

2006/2007 follow up/program year 2005/2006 baseline/prior year

Data submitted to CDE in Excel file Evaluation narrative

Page 4: Evaluation 101: After School Programs

Prop 49 Evaluation SB638 Programs must submit individual student data:

Program attendance – participation Regular school day attendance STAR test results

Program effectiveness measures (1 or more from list) STAR test data Positive behavioral change* Homework completion rates* Skill development* Additional measures to be determined by CDE

* Measured by ASP teachers or regular day teachers

Page 5: Evaluation 101: After School Programs

Prop 49 Evaluation SB638

State Board of Education will define performance measures by November 2007

Funded programs will comply with state evaluation requirements determined in collaboration with Advisory Committee

Page 6: Evaluation 101: After School Programs

Terms ASP: After School Program

CHKS: California Healthy Kids Survey

Participation: after school program attendance

Regular attendance: regular school day attendance

Baseline: prior year 2005/2006

Follow up: program year 2006/2007

Continuing students: those who participated the previous year in After School Program

Page 7: Evaluation 101: After School Programs

ASP Evaluation Keys to Success

Four distinctive types of student information needed

Build evaluation database throughout year Clear roles and responsibilities Good ongoing communication Document process and sources

Page 8: Evaluation 101: After School Programs

Evaluation data: Four Types Needed on Students

Participation in after school program: number of days attended program in the school year

Regular school attendance: days enrolled and days absent in the school year

California Standards Test proficiency level math and language arts

California Healthy Kids Survey questions at entry and exit from program (4 questions)

Page 9: Evaluation 101: After School Programs

Sources of DataData Source

Participation of students in After School Program

ASP Program staff gather and record throughout year

Regular school attendance baseline and follow up

District data system for current and prior years; prior year ASES evaluation for continuing participants

California Standards Test results on math and language arts proficiency

District test director or curriculum director receives CST data annually

California Healthy Kids Survey 4 questions

ASP Program staff gather from students at entry and exit

Page 10: Evaluation 101: After School Programs

Sample Excel File in Folder

Page 11: Evaluation 101: After School Programs

Who Needs to be involved? ASP director and site staff

District data managers, or evaluation department, or fiscal department with regular school attendance information on students

District testing and curriculum staff who obtain the CST results annually and disseminate them to schools

Staff with expertise in Excel (or database software)

Page 12: Evaluation 101: After School Programs

Tips

Create a roster of key people involved in ASP evaluation with their phone numbers, faxes and email addresses.

Do ASP evaluation update every 2-3 months to keep all informed on progress. Will raise ASP evaluation visibility

Immediately thank via email and/or phone call anyone who gives you data

Write it down – for next year!

Page 13: Evaluation 101: After School Programs

Participation Data

Page 14: Evaluation 101: After School Programs

Participation Data

Prior to program beginning: Decide on how to gather and record program attendance:

after school program software, Excel spread sheets or Access database

Find last year’s evaluation report in Excel format to get baseline data (ongoing programs)

Decide on roles and responsibilities for data entry Who records attendance Who enters data into computer

Page 15: Evaluation 101: After School Programs

Participation Data

Record child’s name (first and last), grade, school, Student ID Number and days attended by month

As new children enter, record them in the data system on an ongoing basis

Continuing participants, gather baseline for participation, regular school attendance and CST from prior year’s Excel file

Page 16: Evaluation 101: After School Programs

Tips

Keep track of Student ID Numbers – they are the key to obtaining regular school attendance and CST results

Get child’s name as they are registered in school – no nicknames. James Jones not JJ Jones or Jimmie Jones

Don’t record attendance in Word , you will need to be able to add figures and sum them up

Page 17: Evaluation 101: After School Programs

Regular School Attendance

Page 18: Evaluation 101: After School Programs

Regular School Attendance

Data needed: Days enrolled, days absent current school year

2006/07 and prior school year 2005/06

Where is it kept: District data system Link by Student ID Number and school

Page 19: Evaluation 101: After School Programs

Regular School Attendance

What you need to do: Identify where data is kept in district Contact person who can get you access to data

and what you will need: initial information query in fall

Contact in March to obtain baseline prior year attendance for students new to program

Contact in June to obtain current year attendance for all participants

Page 20: Evaluation 101: After School Programs

Tips

Don’t be surprised if there is no prior year attendance for new participants – student mobility means students may be new to the school in program year

Check data for new participants in March for reasonableness Days enrolled should be around 170 (150-181) Days absent should be between 1 and 20; rarely

over 20

Page 21: Evaluation 101: After School Programs

California Standards Tests

Page 22: Evaluation 101: After School Programs

California Standards Tests Results available July or August following end of school

year

Math score and Language Arts Score

Scores are 1 through 5 or Far Below Basic, Below Basic, Basic, Proficient and Advanced

Best source for student demographics: gender, ethnicity, English Language Learner Status

Page 23: Evaluation 101: After School Programs

California Standards Tests Data needed

Proficiency level in math and language arts for program year (spring 2007) and baseline year (spring 2006)

Demographics on students Where it is kept

Comes to district testing staff July or August on CD ROM from state testing contractor

For continuing ASP students, baseline data is on prior year evaluation report to CDE

For new ASP students, baseline data is prior year CST results 2005/2006

Page 24: Evaluation 101: After School Programs

California Standards Tests What you need to do:

Identify test contact person in district Contact person who can get you access to test data:

initial information query fall Contact in March to obtain baseline prior year

proficiency for students new to program Submit file with id numbers of new students by school with

data request Contact in June to remind them of upcoming

summer request Submit file with id numbers of all participating students in

summer for new test data and new demographics

Page 25: Evaluation 101: After School Programs

Tips

• Don’t be surprised if there is no prior year CST results for new participants – student mobility means students may be new to the school in program year

• Keep in touch with testing person for district, sometimes data comes in early

• Don’t wait until the summer, many people are not working in summer months!

Page 26: Evaluation 101: After School Programs

California Healthy Kids Survey

Page 27: Evaluation 101: After School Programs

California Healthy Kids Survey

Train staff to administer survey when program begins and when students leave the program or near the close of school year

Enter initial (pre) surveys in evaluation database as they come in.

Page 28: Evaluation 101: After School Programs

Survey Questions1.  In the past 30 days, how often have you wanted to go to school? (Never,

one or two times, About once a week, A few times a week, Everyday) 2.  In the past 30 days, how often have you studied for a test? (Never, one or

two times, About once a week, A few times a week, Everyday)  3.  In the past 30 days, how often have you felt unsafe at school? (Never, one

or two times, About once a week, A few times a week, Everyday)  4.  In the past 30 days, how often have your parents asked you about school?

(Never, one or two times, About once a week, A few times a week, Everyday)

 

Page 29: Evaluation 101: After School Programs

Tips

Be sure to gather pre surveys for students who enter the program during the year.

Train staff to look for red flags if a student reports they don’t feel safe, for example.

Gather the post follow up surveys (same 4 questions as on entry to program) beginning about one month before the program ends.

Enter into database.

Page 30: Evaluation 101: After School Programs

Master timeline

Month before program starts

Fall

Decide on database format Set up participation data entry

system Train staff on California Healthy Kids

Survey and attendance procedures Find last year’s evaluation report

Excel files Identify person in district with regular

school attendance data and CST test scores and hold initial meeting

Page 31: Evaluation 101: After School Programs

Master timeline, continued

October through March

Check to see that you can generate list of students by school with id numbers and total days attended for first three months or six months

Check that number of initial California Healthy Kids Surveys in database match number of participants

Build baseline for ongoing students using ASP participation, regular school attendance and CST results from previous evaluation Excel file

Page 32: Evaluation 101: After School Programs

Master timeline, continued

May of school year

Meet with district attendance data contact; provide list of id numbers of participating students; obtain prior year attendance for new students

Review completeness of database, look for holes on continuing students and fill them

Page 33: Evaluation 101: After School Programs

Master timeline, continued

June

July

August

Request current year attendance data for all participants, giving school list of students with ID numbers

Request CST data and demographics from district test contact

Finalize Excel file with Student ID, days of ASP attendance; regular school attendance; CHKS answers

Page 34: Evaluation 101: After School Programs

Master timeline, final

September Add test scores from CST to Excel files

Complete narrative Submit to CDE