Embed Size (px)
Citation preview
Literacy and Numeracy
2
Agenda• Background• Key Variables for Calculating Literacy and Numeracy Rate
– Definition of a Youth– Out-of-School – Basic Literacy Skills Deficiency– Educational Functioning Level– Program Years– Reasons For Exit– Participation Years
• Testing Requirements for Youth
3
Background
• Target Population –• More than 40 million adults, or approximately 21
percent of the adult population of the United States, possess limited literacy capability—that is, they have not completed a high school diploma or equivalent.
4
Target Population Statistics 1
• 37 % (or about 8 % of the total adult population) have completed eight or fewer years of education
• 11.3 % - four or fewer years of schooling
• 33 % English is their second language.
1 http://www.ed.gov/about/offices/list/ovae/pi/AdultEd/census1.pdf
5
Background
• TEGL 17-05, Common Performance Measures Policy for the Employment and Training Accountability System
• TEGL 3-04 New Strategic Vision for the Delivery of Youth Services under the Workforce Investment System
• National Reporting System (NRS) for Adult Education – U.S. Department of Education
6
ETA & NRS Implementation Notes
• ETA is using the NRS EFL Structure • EFL Implementation
– ETA allows for reporting on various EFL– NRS (Adult ED) reports on lowest EFL
• Calculations– ETA will use a different set of calculations
• Data Quality– Both ETA & NRS are focused on data integrity and
reporting
7
Literacy or Numeracy Gains
Of those out-of-school youth who are basic skills deficient:
# of youth participants who increase one or more educational functioning levels
# of youth participants who have completed a year in the program* plus the # of youth participants who exit before completing a year in the youth
program
*(i.e. one year from the date of first youth program service)
8
Key Variables for Calculating Literacy and Numeracy Rate
9
Definition of a Youth
• There are two WISARD layouts– OMB approved layout– ETA software layout
• Differences– Fields 306 & 307 have been changed
Item OMB Approved ETA Software
306 Youth (Local) Date of First Youth Service
307 Youth [Statewide (15%) Activities]
Youth (Statewide 15% Activities)
10
Definition of a Youth (cont.)
• ETA software uses WIASRD as the basis of its record layout.
• We are using the WIASRD that we expect to be approved by OMB for the October 2006 WIASRD submission.
11
Calculation of Age for Youth
• Date of First Youth Service minus Date of Birth– Item 306 – Item 102 = Age
• Age >= 14 and Age <= 21• There is no younger or older youth differentiation
12
Out-of-School
• Out-of-School is defined as:– An eligible youth who is a school dropout, or who has
received a secondary school diploma or its equivalent but is basic skills deficient, unemployed, or underemployed (WIA section 101(33)).
– For reporting purposes, this term includes all youth except: (i) those who are attending any school and have not received a secondary school diploma or its recognized equivalent, or (ii) those who are attending post-secondary school and are not basic skills deficient.
13
Understanding Out-of-School Youth
• For Reporting Purposes:1. Not in school
2. In post-secondary education & basic skills deficient
• We could be looking potentially at anyone who is not in secondary school and is basic skills deficient
14
Basic Literacy Skills Deficient
• Is defined as:– The individual computes or solves problems, reads,
writes, or speaks English at or below the eighth grade level or is unable to compute or solve problems, read, write, or speak English at a level necessary to function on the job, in the individual’s family, or in society.
15
Basic Literacy Skills Deficient
• For the WISARD – During Year 1– Basic Literacy Skills Deficient, Item 130 = 1– Initial Pre-test score, EFL and test date must be recorded
• For WISARD – During Subsequent Years– Based on EFL (Year #1, #2, etc.), Items 709, 712, etc. <=
4
16
Educational Functioning Levels
• Based on the National Reporting Systems EFL Table
• Educational Functioning Levels are based on skill sets – a measure of adult educational progress
• They are equal to approximately 2 grade levels
17
Educational Functioning Levels
• Educational Gain – – At post-test, participant completes or advances one or
more educational functioning levels from the starting level measured on entry into the program (pre-test) – for Year 1.
– At post-test, participant completes or advances one or more educational functioning levels from the post-test level measured from the previous year’s latest post-test – for Year 2 and beyond.
18
EFL Calculation
• To increase an EFL: EFL (based on test) during the current participation year minus the prior EFL (based on prior test) (either the pre-test or one of the post-tests) > 0.
• If EFL is either 0 or -1 then there is a no success.• An increase of one EFL in any functional area
during a participation year is considered a success for that participation year.
19
EFL Calculation
• Success for year one is – – Educational Functioning Level Item 709 > than original
EFL Level, Item 706– Date must be within or equal to 1 year anniversary of
Date of First Youth Service
• Success for subsequent years– In the PRESENT year, the Educational Functioning Level
> the EFL for the previous year
20
NO
Literacy or Numeracy Gain – 1Literacy or Numeracy Gain – 1stst Yr. Yr.
Has increased one EFL within 1 year of First Youth Service Date?
NOYES
Excluded
YESYE
S
Numerator
Denominator
20
YES
Is Out-of-School?
YES
Is Basic Skills Deficient?
Is Youth?
NO
NO
21
First Year Notes• Initial test must be given within 60 days of Date of First
Youth Service.
• If a previous assessment was conducted (i.e. by a partner program) within six months prior to Date of First Youth Service, then that test score can be used.
• Participant is included in the measure even if they exit prior to end of the first year.
22
1st Program
YouthProgram
Post-test must be given 1
1 Exit can occur at anytime, but post-test must be given prior to exit2 Post-test must be given prior to or on the anniversary date
Date of Participation
Anniversary Date 2
Date of First Youth Service
Pre-test given within 6 monthscan be used*
Pre-test must be given within 60 days
1 year
First Year
23
First Year Notes (cont.)
• Post-test can be anytime within the first year, but must be given prior to anniversary date.
• Failure to test within first year or prior to exit will result in a negative outcome
• If a participant exits for an exclusion (i.e. global exclusion) and is not a success, the participant is excluded from the measure.
24
Basic Literacy Skills Deficient Revisited
R = Reading/Writing ProficiencyM = Math Proficiency
BSP = Basic Skills Proficiency
Not R OR Not M = Not BSP
Not R AND Not M = Not BSP
Not R = Not BSPNot M = Not BSP
R AND M = BSP
25
NO
Literacy or Numeracy Gain – 2Literacy or Numeracy Gain – 2ndnd Yr. Yr. & Each Year Thereafter& Each Year Thereafter
Has increased one EFL within last year based on
latest post-test score?
NOYES
Excluded
YESYE
S
Numerator
Denominator
25
YES
Has completed another full year based on
Anniversary Date*?
Is Basic Skills Deficient?
NO
*Anniversary date is based on Date of First Youth Service and Exit Date >= Anniversary Date
26
Second Year and Beyond Notes
• Test can be given anytime between anniversary dates
• For subsequent years in the program, if the participant exits prior to the anniversary of the participation year, then the participant is excluded from the measure whether or not the participant completed the appropriate post-test.
27
Post-test must be given 1
1 If exit occurs prior to anniversary date, participant is excluded from the measure2 Post-test must be given prior to or on the anniversary date
Anniversary Date 2Anniversary Date
Second Year and Beyond
28
Two Methods for Calculations
• Option A – States who are implementing in first year– The participant is included in the measure the moment
the individual has an outcome – a successful post-test, exit, or the anniversary of the participation year
• Option B – Used by States in all subsequent years– Includes the participant in the measure only on the
anniversary of the individuals participation year
29
1st Program
YouthProgram
* Option A** Option B
Date of Participation
AnniversaryDates
Date of First Youth Service
Year 1*
Two Methods for Calculations
Year 2**
Date of Exit
Common Participation Period
90 calendar days
of no service
30
Example
• Participant A enters the program on 10/01/2006• Pre-test given on 10/02/2006• Takes post-test and increases 1 EFL on
10/05/2006• Option A
– Success as of 10/05/2006, the date of EFL increase
• Option B– Success on 10/01/2007, the anniversary date of the Date
of the First Youth Service
31
Exclusions• Social Security Number –
– Can impact the ability of states to receive administrative records from partner programs• Technical and community college that use SSN as primary identifier
• Exit due to exclusion causes all outcomes to be lost– Participant had a success in the 1st year and then they were
excluded in the 2nd year • Options
1. Positive or negative outcome would be lost
2. No impact to real-time outcomes
• More clarification to come
32
Test Requirements
33
Why Test?
• Test or assessment has three purposes1: – It provides diagnostic information (formative
assessments)
– Evaluates student progress (summative assessment)
– Evaluates the overall performance of an entity (e.g., class, program, state).
1Developing Performance Assessments http://www.nrsweb.org/reports/PerformanceAssessments.pdf
34
Testing Requirements
• Pre-test – A test used to assess a participant’s basic literacy skills, which is administered to a participant up to six months prior to the date of participation, if such pre-test scores are available, or within 60 days following the date of first youth service.
• Post-test – A test administered to a participant at regular intervals during the program.
35
Testing Requirements
• All assessment tests must crosswalk to the NRS
• Programs must use test outline in Attachment C – NRS EFL Table
• States, grantees, or contractors are not required to use the same assessment tool throughout their jurisdictions.
36
Testing Requirements• Programs must adhere to the following in choosing an
assessment tool:
1. The same assessment tool is administered to the participant for pre-testing and post-testing;
2. The assessment tool and its scores must crosswalk directly to the educational functioning levels so that educational gains can be reported in terms of increase in one or more ABE or ESL levels; and
3. Tests must be administered in a standardized manner throughout the jurisdiction (i.e., used consistently and reliably across programs and produce observable results).
37
Understanding NRS Table• The NRS provides test benchmarks for EFL.
• Indicate how students functioning at each level would perform on the tests.
• Inclusion of tests does not imply that they are equivalent or the basis for assessment.
• Tests do not necessarily measure the same skills.
38
Testing Requirements
• Intervals– Participants must be post-tested at least once by the end
of one year following the individual’s date of first youth program service.
– If more than one assessment is administered after the initial test, the latest assessment should be used
• Individuals who remain basic skills deficient and continue to participate, must continue to receive basic skills remediation services.
39
Testing Youth with Disabilities
• When administering assessment tools, individuals with disabilities are to be provided with reasonable accommodations:– “[m]odifications or adjustments,” made on a case-by-case
basis, “that enable a qualified individual with a disability . . . to receive aid, benefits, services, or training equal to that provided to qualified individuals without disabilities.”*
*29 CFR 37.4
40
Testing Youth with Disabilities
• Reasonable Accommodations are to be provided in accordance with:– Section 188 of the WIA– Guidelines associated with the assessment test– State laws or policy
• Youth with disabilities are expected to achieve the same gains as other youth– Accommodations provide a “level playing field.”
41
Testing Youth with Disabilities
• For youth with one or more severe disabilities – The use of these testing instruments, even with
appropriate accommodations, may not provide a valid or reliable evaluation of the literacy and numeracy skills
• Service providers and grant recipients may use alternate assessment tools to measure gains– State has deemed alternate assessments valid and
reliable indicators of information – Comparable to the information provided through the
educational functioning levels
42
Categories of Accommodations
• Presentation – Braille or oral reading of questions
• Response – having test taker point to response
• Setting – taken in small setting
• Timing/Schedule – extending testing times.
State participation and accommodation policies for students with disabilities: 1999 update (Synthesis Report No. 33)
43
Documentation of Accommodation
• Diagnosis of the learning disability or disabilities or, diagnosis of ADHD
• Statement of how this disability substantially limits candidate’s activity
• Recommendation for accommodation(s) appropriate for the individual’s educational needs
44
Further InformationETA’s Performance and Results Website
http://www.doleta.gov/performance
ETA’s Office of Workforce Investment – Youth Services
http://www.doleta.gov/youth_services/
National Reporting System for Adult Education Website
http://www.nrsweb.org
45
Further InformationThe Iowa Literacy Resource Center
http://www.readiowa.org/
GED Testing Accommodationshttp://www.readiowa.org/GED_testing/sld001.htm
American Council on Education – GED Testing
Accommodations http://www.acenet.edu/AM/Template.cfm?Section=GEDTS&TEMPLATE=/CM/ContentDisplay.cfm&CONTENTID=12223
Synthesis Report No. 33http://education.umn.edu/NCEO/OnlinePubs/Synthesis33.html
46
Special Thanks
Mathematica Policy Research, Inc.for their assistance in preparing technical
documentation for this presentation.
