Upload
curtis-thompson
View
216
Download
2
Tags:
Embed Size (px)
Citation preview
Texas Comprehensive Center @ SEDL
Austin, Texas
March 16–17, 2009
Making Consistent Decisions About Accommodations for English Language Learners
– Research Summit –
Dr. Jamal Abedi
CRESSTUniversity of California, Davis
School of Education
Language Factors in the Assessment of English
Language Learners
How Important is Assessment How Important is Assessment for ELL Students?for ELL Students?
Assessment results affect ELL students in the Assessment results affect ELL students in the following areas: following areas:
• Classification
• Instruction
• Accountability (the NCLB issues)
• Promotion
• Graduation
Are the Standardized Achievement
Tests Appropriate for ELLs?
The Standards for Educational and Psychological Testing (American Educational Research Association, American
Psychological Association, & National Council on Measurement in Education, 1999) elaborated on this issue:
For all test takers, any test that employs language is, in part, a measure of their
language skills. This is of particular concern for test takers whose first language is not the
language of the test.
Performance/Reliability-Gap Between ELL and Non-ELL Students
Performance-Gap Reliability-Gap
Reading 20% - 60% 15% - 40%
Science/Social Sciences
10% - 40% 12% - 35%
Math Problem Solving
8% - 25% 10% - 30%
Math Computation
0% - 10% 10% – 15%
Are the Standardized Achievement Tests Reliable and Valid for These
Students?
• The reliability coefficients of the test scores for ELL students are substantially lower than those for non-ELL students.
• ELL students’ test outcomes show lower criterion-related validity.
• Structural relationships between test components and across measurement domains are lower for ELL students.
Site 2 Stanford 9 Sub-scale Reliabilities (Alpha), Grade 9
Non-LEP Students
Sub-scale (Items) Hi SES Low SES English Only FEP RFEP LEP
Reading, N= 205,092 35,855 181,202 37,876 21,869 52,720
-Vocabulary (30) .828 .781 .835 .814 .759 .666
-Reading Comp (54) .912 .893 .916 .903 .877 .833
Average Reliability .870 .837 .876 .859 .818 .750
Math, N= 207,155 36,588 183,262 38,329 22,152 54,815
-Total (48) .899 .853 .898 .898 .876 .802
Language, N= 204,571 35,866 180,743 37,862 21,852 52,863
-Mechanics (24) .801 .759 .803 .802 .755 .686
-Expression (24) .818 .779 .812 .804 .757 .680
Average Reliability .810 .769 .813 .803 .756 .683
Science, N= 163,960 28,377 144,821 29,946 17,570 40,255
-Total (40) .800 .723 .805 .778 .716 .597
Social Science, N= 204,965 36,132 181,078 38,052 21,967 53,925
-Total (40) .803 .702 .805 .784 .722 .530
Principle of Equity and Fairness
•Provide assistance in the form of accommodations.
• Therefore, the Principle of Equity and Fairness demands assistance to
these students.
Samples of Accommodations Used for ELL Students That May Not Be
Relevant
• Test-taker marks answers in a test booklet
• Copying assistance provided between drafts
• Test-taker indicates answers by pointing or other similar method
• Paper is secured to work area with tape/magnet
• Physical assistance is provided
Samples of Accommodations Used
for ELL Students
• Enlarged answer sheets provided
• Breaks provided
• Tests individually administered
• Tests administered in small groups
• Tests administered in a location with minimal distractions
Presenting Language-Related Accommodations for ELLs
• English Dictionary
• English Glossary
• Bilingual Dictionary/Glossary
• Customized Dictionary
• Native Language Testing
• Linguistically Modified Test
Clear Language of Instruction and Assessment Works for ELLs, SWDs, and Everyone
• What is language modification of test items?
Language of Assessment
• Clear and concise language is a requirement for reliable and valid assessments for ELL students.
• It may also be an important consideration for students with learning disabilities.
• Students with learning disabilities may have difficulty processing complex language in the assessment.
• Reducing the unnecessary linguistic complexity of test items can help students with the most challenging academic career.
• Familiarity/frequency of non-math vocabulary:Unfamiliar or infrequent words changed
A certain reference file > Mack’s company
Linguistic Modification Concerns
Original:A certain reference file contains approximately six billion facts. About how many millions is that?
A. 6,000,000B. 600,000C. 60,000D. 6,000E. 600
Modified:Mack’s company sold six billion pencils. About how many millions is that?
A. 6,000,000 B. 600,000 C. 60,000 D. 6,000 E. 600
Linguistic Modification Concerns Cont.
• Length of nominals: long nominals shortened
Last year’s class vice president > Vice president
The pattern of puppy’s weight gain > The pattern above
• Question phrases: complex question phrases changed to simple question words
At which of the following times > When
Which is best approximation of the number > Approximately how many
• Conditional clauses: conditionals either replaced with separate sentences or order of conditional and main clause changed
If two batteries in the sample were found to be dead > He found two broken pencils in the sample
If Lee delivers x newspapers > Lee delivers x newspapers
• Voice of verb phrase: passive verb forms changed to active
The weights of 3 objects were compared > Sandra compared the weights of 3 rabbits
If a marble is taken from the bag > If you take a marble from the bag
The boy was bitten by the dog > The dog bit the boy
Linguistic Modification cont.
CRESST Studies on the Assessment and Accommodation of
ELL Students:
Impact of Language Factors On Assessment of ELLs
A Chain of Events
Fourteen studies on the assessment and three on the instruction (OTL) of ELL students
Examining Complex Linguistic Features in Content-Based Test Items
Feature Feature Description Categories Combined
1 I tem length 1, 2, 4, 45
2 Vocabulary 3, 26, 27
3 Nominal heaviness 5, 6, 29, 30, 31, 32
4 Verb voice 7, 33
5 Modal 8, 34
6 Relative clause 9, 10, 11, 35, 36, 37
7 Adverbial modification 12, 13, 14, 15, 16, 17, 38, 39, 40, 41
8 Conditional clause 18, 19
9 Complement clause 20, 44
10 Sentence structure 28, 42, 43, 46
11 Preferred argument structure 22, 23, 47, 48
12 Question form 21
13 Global difficulty 24
14 Content interest 25
Additional Complex Linguistic Features
More recent research has identified these additional features:
• Complex verbs
• Subordinate clauses (including relative clauses)
• Complex noun phrases
• Various entities as subjects
Studies on Linguistic Modifications
• Results of national studies are not conclusive.
• Most of the CRESST studies found significant gain for ELL students on the linguistically modified version.
• However, the outcome of national research on the impact of linguistic modification is mixed (Francis, et al. 2006).
• Factors that lead to such discrepancies include variation in methodology in implementing linguistic modification approach, sampling and power issues, variation in test items and the nature of linguistic complexity, etc.
Improvements in the Language of Instruction
• Unnecessary linguistic complexity may impact students’ understanding of instructional materials
• Opportunity to learn for ELLs and SLDs may be affected
• Research show ELLs and SLDs benefiting less from instructions with complex linguistic structure
Conclusions and Recommendations
Assessments and instructions for ELLs :
• Must be free of construct irrelevant factors• Must be free of unnecessary linguistic complexities• Must include a sufficient number of ELLs and SWDs in its
development process (field testing, standard setting, etc.)• Must be free of cultural biases• Must be sensitive to students’ linguistics and cultural needs
Impact of Linguistic Modification Research on
Practice
• Many test publishers have incorporated the concept of linguistic modifications in their test development process.
• Among these test publishers: Educational Testing Service (ETS), Harcourt Assessment, CTB/McGraw Hill, Measurement Incorporated, etc.
• Many states have adopted this approach into their assessments
• The U.S. Department of Education incorporated this process into the National Assessments (NAEP) and has highly recommended this approach to be incorporated into the state NCLB Title I assessments.
For more information, please contact Jamal Abedi at CRESST/UC Davis:
(530) 754-9150
or