CRESST ONR/NETC Meetings, 17-18 July 2003, v1

ONR Advanced Distributed Learning

Linguistic Modification of Test Items

Jamal Abedi

University of California, Los Angeles

National Center for Research on Evaluation, Standards, and Student Testing (CRESST)

July 18, 2003

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

What is linguistic modification of test items

It is not a simple text editing

It is not language simplification

It reduces or eliminates unnecessary linguistic complexity of assessment tools

It is based on principles of linguistic theory and research findings

It does not change, alter, or touch any content-related term

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

Examining Complex Linguistic Features in Content-Based Test Items

Feature Feature Description Categories Combined

1 Item length 1, 2, 4, 45

2 Vocabulary 3, 26, 27

3 Nominal heaviness 5, 6, 29, 30, 31, 32

4 Verb voice 7, 33

5 Modal 8, 34

6 Relative clause 9, 10, 11, 35, 36, 37

7 Adverbial modification 12, 13, 14, 15, 16, 17, 38, 39, 40, 41

8 Conditional clause 18, 19

9 Complement clause 20, 44

10 Sentence structure 28, 42, 43, 46

11 Preferred argument structure 22, 23, 47, 48

12 Question form 21

13 Global difficulty 24

14 Content interest 25

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

Familiarity/frequency of non-math vocabulary: unfamiliar or infrequent words changed

census > video gameA certain reference file > Mack’s company

Length of nominals: long nominals shortened last year’s class vice president > vice president

the pattern of puppy’s weight gain > the pattern above

Question phrases: complex question phrases changed to simple question words

At which of the following times > Whenwhich is best approximation of the number > approximately how many

Linguistic Modification Concerns

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

Conditional clauses: conditionals either replaced with separate sentences or order of conditional and main clause changed If Lee delivers x newspapers > Lee delivers x newspapersIf two batteries in the sample were found to be dead > he found three

broken pencils in the sample

Relative clauses: relative clauses either removed or re-cast A report that contains 64 sheets of paper >

He needs 64 sheets of paper for each report

Linguistic Modification (continued)

Voice of verb phrase: passive verb forms changed to active The weights of 3 objects were compared >

Sandra compared the weights of 3 rabbitsIf a marble is taken from the bag > if you take a marble from the bag

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

Most recent updated version of the CRESST Linguistic Modification

Rubric

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

Language Modification Concerns

Language features that complicate test items

Vocabulary (Lexicon) unfamiliar words (idioms, words from unfamiliar

contexts, long words) false cognates overuse of synonyms / indefinite pronouns / missing

or unclear antecedents derivatives of content words

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

Grammar (Syntax)

long phrases in questions (question word not at the beginning) long question phrases complex sentences (subordinating clauses) compound sentences (coordinating conjunctions, conjunctive adverbs) logical connectors / conditional / adverbial clauses unfamiliar tenses (conditional verbs, modals) long noun phrases relative clauses some prepositional phrases; multiple prepositional phrases comparative structures

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

Style of Discourse

• long problem statements; unnecessary expository material

• abstract (vs. concrete) presentation of problem

• passive voice• complex arrangement of parts of speech• negation, esp. negative questions, negative

terms, grammatical double negatives

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

Concerns Specific to Math Problems

• phrasing that confuses the order of math operations

• words with both technical and non-technical meanings

• math keywords misinterpreted

CRESST ONR/NETC Meetings, 17-18 July 2003, v1

Summary and Recommendations

The performance gap between English learners and other students can be narrowed by modifying the language of the test items to reduce the use of low-frequency vocabulary and language structures that are incidental to the content knowledge being assessed.

There is no reason why all our students should not have content-area assessments that

use clear language and provide sufficient time for them to show what they know.

The use of clear language, free of unnecessary complexity should and can be a part of

good instructional planning and assessment practice.

The specific language demands of academic materials and assessment tools should be

identified and provided to teachers so that they can ensure that students have the language resources to demonstrate their content-area knowledge and skills.