1
April 11, 2013
Doug Kosty, Assistant Superintendent Kimberly Harrington, ELD Teacher, Hillsboro
School DistrictELPA Content Panel Member
Michelle McCoy, Assessment Specialist, English Language Proficiency
Mary Seburn, Principal Research Scientist, EPIC Kathleen Vanderwall, Manager,
Test Design and Administration
2
Provide background and results of the ELP Standards Verification, including:
Impetus, Timeline and Stakeholder Engagement Evaluation and Results of the Workshops Feedback from Stakeholders Next Steps
3
4
Cohort of Superintendents requested examination of the consistency and coherency of the ELPA Performance Standards (2008)
Periodic review Expanded range of items assessing the
language functions
5
6
• External Evaluation conducted by The Educational Policy Improvement Center (EPIC)• Observed process• Conducted formal and informal Interviews with
participants• Conducted training and workshop evaluations
77
Documented implementation of best practices and technical adequacy• Noted evidence of procedural validity as
observed• Noted when relevant standards for Educational
and Psychological Testing were met• Documented technical evidence similar to what is
required by ESEA (NCLB) Peer Review• Documented adherence to best and emerging
practice
88
• Bookmarking Method• Held in two workshops• November: 76 Oregon stakeholders set bookmarks
over 4 days (Phase 1)• February: 20 Oregon stakeholders set bookmarks over
2 days (Phase 2)
• Panelists recommended changes to the performance standards for all grade levels
• External evaluators monitored process and documented observed evidence of validity
99
When are students proficient enough in English to participate meaningfully in the general education program?
When exited, what is the evidence that ELL students are performing as well as their non-ELL peers?
When are ELL students participating in essentially all aspects of the district’s curriculum?
10
Overall, feedback on the training was positive, for example:• 92% of participants agreed or strongly agreed with
the statement, “The training materials were helpful.”
• 86% of participants agreed or strongly agreed with the statement, “I am confident I understand my role in the standards verification process.”
• 83% of participants agreed or strongly agreed with the statement, “Overall, I feel well trained and prepared to complete the standards verification task.”
1111
12
In many cases, participants recommended lower cut scores, particularly for the Intermediate Performance Level and for grade 2
Impact of these recommended cut scores is “smoother” across grades and Performance Levels
The new cut scores will provide more meaningful information to students, educators, parents, policymakers and others regarding individual performance on the acquisition of English
13
14
Recommended Cut Scores
Current Cut Scores
Changes to Cut Scores (+/- when compared to
current cut scores)
Grade
Inter-mediate
Early Advanced
Advanced/ Proficient
Early Inter-
mediate
Inter- mediate
Early Advanced
Advanced/ Proficient
Early Inter-
mediate
Inter-mediate
Early Advanced
Advanced/ Proficient
Early Inter-
mediate
K 481 491 497 505 482 492 498 507 -1 -1 -1 -2
1 491 503 512 522 492 507 514 523 -1 -4 -2 -1
2 492 504 514 521 495 508 514 523 -3 -4 0 -2
3 500 511 521 526 501 514 521 529 -1 -3 0 -3
4 494 504 514 522 497 508 514 521 -3 -4 0 1
5 496 508 515 524 497 508 516 523 -1 0 -1 1
6 493 504 516 522 497 506 515 522 -4 -2 1 0
7 495 508 518 524 497 507 517 524 -2 1 1 0
8 497 509 520 527 499 508 518 526 -2 1 +2 1
HS 494 500 513 523 494 501 515 528 0 -1 -2 -5
15
How do the new cut scores compare with the previous cut scores?The participant-recommended cut scores range from slightly lower, the same, or slightly high than the previous cut scores.
Respondents concerning the proposed ELP Cut Scores◦ 67% agree or strongly agree with the proposed cut
score changes◦ 82% believe that the proposed cut scores will be a
positive change
Respondents concerning the updated Performance Level Descriptors◦ 82% agree or are neutral/no opinion concerning the
updated PLDs
16
Continuing gathering input through the survey through 4/19.
Recommend adoption of the updated cut scores by the Oregon Board of Education, which would go into effect for 2013-14.
Recommend that the State Board of Education adopt the new ELPA Performance Level Descriptors, which would also go into effect in 2013-14.
Place on the Consent Agenda, May 16-17, 2013.
17