Using Data EffectivelyUsing Data Effectively

ABE Directors’ MeetingOctober 9th and 10th, 2002

AGENDAAGENDA

Current Use of DataWhat Types of Data Can Be Used Improving Your Program: Current Practices A Model for Program ImprovementCase StudyNext Steps: Develop a Program

Improvement PlanEvaluation

Current Use of DataCurrent Use of Data

What two statistics (e.g., attendance hours, enrollment rates) do you use frequently or would you like to use?

What Types of Data Can Be UsedWhat Types of Data Can Be Used

Learner Data - Demographic / background characteristics

- Labor force status

- Educational gains

- Education and employment outcomes

- Class participation

- Community participation

- Family outcomes

- Learners’ perceptions of instructional services

Staff Data - Background of instructional staff

- Staffs’ employment and training experience

- Characteristics of instructional approaches

• Program OperationsProgram Operations

- Recruitment

- Learner intake

- Learner placement

- Structure and hours of instruction

- Assessment instruments and procedures

- Support services

- Learner transition

- Fiscal management

- Program management

- Interagency collaboration

- Data management

Improving Your Program: Improving Your Program: Current PracticesCurrent Practices

What internal and external priorities have prompted changes or improvements?

What types of data do you collect? How do you use these data? What actions have you taken, or what

changes have you made based on the analysis of data?

A Model for Program ImprovementA Model for Program Improvement

Analyze current data and determine questions to ask

Identify areas for improvement Design a program improvement plan Document and monitor use of the new practice Evaluate implementation and outcomes

Analyze Current Data and Analyze Current Data and Determine Questions to AskDetermine Questions to Ask

Identify trends in data that raise questions Analyze data related to an issue or question

that has prompted interest Identify and review additional information that

can be examined to confirm or clarify results of data analysis

Define preliminary issues or problems that need to be addressed

Identify Areas for ImprovementIdentify Areas for Improvement

Use benchmarks to assess extent of problem and target population

Determine area for change based on importance of problem and other factors

Create team to identify new

practice / process for target population; promote effort Access information about practices from online

systems, websites, reports, other programs/states

Design a Program Improvement PlanDesign a Program Improvement Plan

Determine new practice Specify learners, classes, or sites for pilot Develop communication networks / new policies Identify training and technical assistance Determine resources / staff Set timeline for training and pilot testing the

practice Specify data collection Review the change process

Document and Monitor Use of Document and Monitor Use of New PracticeNew Practice

Collect information about implementation of practice

Specify materials, staff, and resources Chart time for implementing Identify barriers to use Identify aspects of practice or use that appear to

be successful Identify factors affecting use Chart unexpected outcomes

Evaluate Implementation and Evaluate Implementation and OutcomesOutcomes

Analyze data to identify elements of the practice implemented and not implemented

Assess what worked, what didn’t, and why Identify missing elements in practice Describe outcomes from using practice Refine practice and its use

Case StudyCase Study

Funder: Local Education Agency (LEA)

Services Funded: ABE, ESOL, Family Literacy, Workplace

Sites: Five sites - three sites provide ABE and ESOL services, one site provides Even Start, and the other site

provides workplace literacy

Total Enrollments: 524

* approximately 50% of the students are ESOL - of those 30% are Brazilian

* approximately 10% of the ABE students entered at the beginning literacy level, 20% were at the ABE/Pre

ASE level, and 20% were at the ASE level

Assessment to Measure Educational Gain:

* For ESOL students, last year the program used a program-developed tool with a rubric developed by one

ESOL teacher. This year the program will use BEST and REEP.

* For Beginning Literacy ABE students, last year the program used a home grown tool translated into

Spanish. They will continue to use that tool this year.

* For ABE, Pre ASE, and ASE students, last year the program used TABE 5 Level M for all students. This

year the program will use TABE 7 and 8.

Assessments for Class Placement:

* For ESOL students, the program uses a program-developed instrument which includes an oral and writing

section.

* For all ABE students, the program uses the TABE Locator.

Instructional Methods:

The delivery of instruction has not been an emphasis in professional development in recent years. Each teacher uses methods that he/she feels comfortable with. There is no formal curriculum in place and the program does not use the Curriculum Frameworks. The ASE classes are totally individualized and student attendance is sporadic. The ESOL attendance is high yet the % completing a level is low. On the Federal Report Table 4, the overall percent for completing the educational levels is 20.56% for one site and Cognos shows that the overall percent for class attendance for one of the sites is 54%.

Program and Staff Development:

The program development is determined by the Program Coordinator who assigns hours equally to all staff members. Staff development is decided by each staff person with no plan in place.

Counseling:

Counseling hours are allotted to each ESOL teacher, and the Program Coordinator provides counseling to ABE/ASE students.

Staffing:

All staff are white. One teacher is bilingual in Spanish. All staff are part time and are paid $20/hour with no benefits. They are not compensated for prep time. They are allotted additional paid hours for assessment and follow-up.

Next Steps: Develop a Program Next Steps: Develop a Program Improvement PlanImprovement Plan

Analyze current data and determine questions to ask

Identify areas for improvement

Design a program improvement plan

Document and monitor use of the new practice

Evaluate implementation and outcomes