Chapter 6 Formal Assessment of Reading: Individualized Assessment

Chapter 6

Formal Assessment of Reading: Individualized Assessment

Introduction

• Development of Formal Reading Measures• Types of Scores• Administration and Scoring of Formal Tests• Determining Chronological Age• Individualized vs Group Tests• What Do Individualized Norm-referenced Measures of Reading Look

Like? • Test Bias• Selecting Formal, Individualized Instruments• Special Considerations for Formal, Individual Assessment of Adult and

English Language Learners

Test Development

• Table of specifications (blueprint) (Figure 5.2)• Development version• Pilot or field testing• Standardization version• Norm sample

• Representativeness of sample

Figure 6.1

Types of Scores

• Numerical Scales: Nominal, Ordinal, Interval, Ratio• Raw Score: Actual number correct, taking into account

the basal• Derived Scores• Standard Scores: Z Scores, IQ Scores, T Scores, Scale

Scores• Grade Equivalents, Age Equivalents• Percentiles

Assessment Terms

• Grade Equivalent• Age Equivalent• Grade-Based Norms• Age-Based Norms As a Rule, Use Age-Based

Norms to Determine Standard Scores

• Standard Scores-can Add and Subtract-Used for Comparisons

• Percentiles-Provides A Rank Order 1-99

• Stanines-A Rough Gauge 1-9

Text Box 6.1

Age and Grade Equivalents

Age Equivalent Grade Equivalent6-1 1.26 years, 1 month 1st grade, 2nd month

Percentile versus Percentage

Misha Jesse

Percentage 50 90Mastery

Percentile 16 80

Text Box 6.2

Text Box 6.3

Contrasting Standard Scores for Word Recognition and Reading Comprehension

Mean = 100; Standard Deviation = 15

Word Reading Recognition Comprehension

Shelby 85 84Saimah 95 70

Text Box 6.4

Comparing IQ Scores with Reading Achievement Scores (Mean = 100; Standard Deviation = 15)

Saimah ShelbyStandard Score IQ 94 103Reading Comprehension -70 -85

24 18

Administering and Scoring of Formal Tests

Test Administration

• Test Manual: Administration Instructions, Scoring and Norm Information, Reliability and Validity Data

• Protocol• Calculation of Chronological Age• General Guidelines for Test Administration• Raw Scores• Establishing Basals and Ceilings• Derived Scores: Percentiles, Grade Equivalents, Standard

Scores

Test Administration

• Determine chronological age• Know start and stop rules• Establish rapport with examinee• Administer subtests according to directions

• Do NOT coach, prompt, or give feedback EXCEPT as directed in the manual

• Obtain raw scores-take into account basal and ceiling• Obtain derived scores (standard scores, percentiles, etc.

Tips for Administering Standardized Tests

• Text Box 6.5

Word Recognition

Basal Rule: 3 in a row correct. Ceiling rule: 3 in row incorrect.

1. ___ 2. ___ 3. ___

→ 4. _1_ 5. _1_ 6. _1_ 7. _1_ 8. _1_ 9. _1_ 10. _0_ 11. _0_ 12. _0_

Note: Starting point for this student indicated by the →

Vocabulary

Basal rule: All items in a set correct. Ceiling rule: All items in a set incorrect.

1. ___ 2. ___ 3. ___

4. ___ 5. ___ → 6. _1_ 7. _1_ 8. _1_ 9. _1_ 10. _1_ 11. _0_ 12. _0_ 13. _0_ 14. _0_ 15. _0_

Note: Starting point for

this student indicated by the →

Figure 6.2 Sample Reading Subtests

Showing Basal and Ceiling Rules

Raw Score Conversion Table for Reading Comprehension Test

Age: 8 years, 9 monthsRaw Score

Standard Score

Percentile Grade Equivalent

Age Equivalent

↑

15 95 37 2.1 7-616 96 39 2.4 7-917 97 42 2.7 7-1118 98 45 2.9 8-319 99 47 3.2 8-620 100 50 3.5 8-9 ↓

Figure 6.3 Sample Excerpt from a Norm Table

for an Individually Administered Reading Comprehension Test for Students Age 8 years, 9 months.

Examiners calculate the raw score for the test, then look up derived scores (standard scores, percentiles, grade-

and age-equivalents) based on the examinee’s chronological age.

Determining Chronological Age

Calculation of Chronological Age

Subtract Date of Birth from Current Date:

2016 06 22

- 2009 07 14

Months always have 30 days and years (of course) have 12 months. Alwaysdouble check your work.

Standardized Diagnostic Testing

• When to Use Diagnostic Tests• Instructional Level• Informal Instruments• Probes• Direct Measurement• Domains

Individualized vs. Group Tests

Individual versus Group?

Individual• Special education eligibility• Determine goals and

objectives for IFSPs, IEPs• Progress monitoring• May be more reliable for

certain students

Group• Determine mastery of

curriculum standards• Determine if teachers are

effective (value added)• Controversial but happening!

• Accountability and general school planning

• May be less reliable for certain students

Test Interpretation

• Analyze test scores• Are scores above average,

average or below average?• Do the scores yield any

instructional information?• Compile with other data• Maintain confidentiality

• Is there a pattern of strengths and weaknesses?

• e. g. Areas of learning disability? basic reading, reading fluency, reading comprehension?

What do Individualized, Norm-Referenced Measures of Reading

Look Like?

Figure 6.4

Figure 6.5

Figure 6.6

Figure 6.7

Figure 6.8

Figure 6.9

Methods of Assessing Comprehension

• KTEA-III• Reading Comprehension subtest

• Picture identification, answering questions

• WIAT-III• Reading Comprehension subtest

• Picture identification, answering questions

• WJ-IV• Passage Comprehension subtest

• Picture identification, fill in the blank

• Reading Recall subtest• Silent reading followed by oral retelling

Comprehension

• Wechsler Individual Achievement Test (WIAT-III)• Reading Comprehension subtest• The examinee is required to read short sentences or passages and

respond to comprehension questions.

Reading Comprehension Similar to WIAT-III

Sample Maze ActivityNorm-referenced tests are standardized (without/on/for) a clearly defined group, termed the (test/sample/norm) group, and scaled so that each (score/instrument/variance) reflects a rank within the (evaluation/reliability/norm) group. Psychologists have developed norm-(standardized/referenced/rated) tests to assess, for example, (height/attitude/intelligence), reading, mathematics, writing, etc. Although we (aren’t/are/is) fortunate in having a choice of (poorly/haphazardly/well)-standardized and psychometrically sound tests (without/above/with) which to evaluate children, some (numbers/tests/practitioners) do not meet acceptable psychometric standards.

(Passage from Sattler, 2001)

Nelson-Denny Reading Test

• Comprehension• 7 reading passages• 38 comprehension questions• Literal and interpretive question types• 5 answer choices per question• 20 minutes

Methods of AssessingVocabulary

•WJ-IV: Reading Vocabulary• Synonyms, Antonyms, Analogies

•Nelson Denny• Multiple choice

Vocabulary

•Woodcock Johnson - IV Tests of Achievement (WJ-IV)

• Reading Vocabulary subtest• The examinee is required to orally state

synonyms and antonyms for printed words and orally complete written analogies.

Reading Vocabulary Similar to WJ-IV

Analogies

Synonyms

Antonyms

Nelson-Denny Reading Test

• Vocabulary• 80 items• 5 answer choices each• 15 minute time limit• Example:

Which word best completes the opening statement?

A chef works with: A. bricks B. music C. clothes D. food E. statues

Methods of Assessing Fluency• KTEA-III

•Decoding Fluency subtest• Nonsense word list (timed)

•Word Recognition Fluency subtest• Word list (timed)

• WIAT-III•Reading Comprehension subtest

• Reading timed passages

• WJ-IV•Sentence Reading Fluency subtest

• Reading sentences and marking whether statements are true or false (3 minute time limit)

•Word Reading Fluency• Choosing two words that go together from an array of choices

Fluency• The Test of Silent Word Reading Fluency-2

(TOSWRF-2)• Examinees are required to identify increasingly difficult words

that have no spaces between them by drawing lines between the boundaries of as many words as possible within the time limit (3 minutes)

• The Test of Silent Contextual Word Reading Fluency (TOSCRF-2)

• Similar to TOSWRF-2 but examinee reads connected text of increasingly difficult graded passages

TOSWRF-2: Example

Nelson-Denny Reading Test• Reading Rate

• Part of Comprehension Test• The examiner calls “Mark” after one minute has elapsed,

and the examinees are asked to record the number to the right of their current line of text and continue reading.

Methods of Assessing Phonics

• KTEA-III• Nonsense Word Decoding subtest

• Nonsense words

• WJ-IV• Word Attack subtest

• Nonsense words

• WIAT-III• Pseudoword Decoding subtest

• Nonsense words

Phonics• Wechsler Individual Achievement Test (WIAT-III)

• Pseudoword Decoding subtest • Requires the examinee to read aloud a list of nonsense words designed

to mimic the phonetic structure of words in the English Language.

Pseudoword DecodingSimilar to WIAT-III

• heb• mib• fum• bim• pon• vun• dreeb

Methods of Assessing Phonemic Awareness

• KTEA-III• Phonological Awareness subtest

• Rhyming, Sound Matching, Blending, Segmenting, Deletion

• WJ-IV• Spelling of Sounds subtest

• Rhyming, Deletion, Substitution, Reversal

• WIAT-III• Word Reading subtest

• Rhyming, Sound Matching, Blending

Phonemic Awareness

• Woodcock Johnson Tests of Achievement (WJ IV)• Spelling of Sounds subtest• Requires the examinee to use rhyming, deletion, substitution, and

reversal to manipulate sounds in words.

Spelling of SoundsSimilar to WJ-IV

• Rhyming:“What rhymes with go?”

• Deletion:“Say cowboy without saying boy.”

• Substitution:“Change /s/ in sun to /f/.”

• Reversal:“Listen to the sounds in the word pot - /p/ /o/ /t/. Now you say the sounds backward.”

Other Measures that Assess Phonemic Awareness

• CTOPP-2• Durrell-2• ERDA-2

• PAL-2 RW• TOWRE-2• TERA-3

Methods of Assessing Sight Word Recognition

• WIAT-II • Word Reading subtest

• Letter and word identification

• KTEA-II• Letter and Word Recognition subtest


• WJ-IV• Letter Word Identification subtest


Sight Word Recognition

• Kaufman Tests of Educational Achievement (KTEA III)• Letter and Word Recognition subtest• The examinee is asked to point to or name various letters, sounds, and

words.

Letter Word Recognition Similar to KTEA III

Test Bias

• Scores from a test should not provide a systematic advantage to a particular group or subgroup based on irregularities in test content, administration, or interpretation.

• If a test produces such an unfair advantage, it is said to be biased against the group that is disadvantaged (Payne, 2003).

Evidence of Test Bias

(a) Does the test assess the same content for all the groups using it?

(b) Are the constructs assessed by the test measured consistently for all groups using it?

(c) Does the test predict (later achievement) as well for all the groups using it?

(d) Does the test provide equally useful instructional information for all groups using it?

Avoiding Bias

• Expert judgment• Statistical analyses: Are the constructs assessed the same

across different groups (e.g., gender, race, SES level)?• Remember tests that are not inherently biased may be misused

(e.g., use of test in English for students with limited English skills).

Selecting Formal, Individualized Instruments

• Consider test characteristics• Consider psychometric properties• Consider needs of children, classrooms, schools, or systems• Determine whether a group or individualized test is needed

Special Considerations for Formal, Individual Assessment of Adult and


Tips for Formal, Individualized Assessment of Adult Learners

• Text Box 6.7 • Assess adult learners’ educational histories, background experiences, and interests

as well as specific reading skills. • Multiple assessments will likely be needed to provide complete information about

adult learners’ reading strengths and weaknesses.• Ensure that the test used with adults is standardized on that population.• Put adult learners at ease; establish rapport and a purpose before launching into

formal testing.• Using grade equivalent scores, and to a lesser extent age equivalent scores, is

problematic due to ease of misinterpretation of these scores, and their psychometric limitations.

Tips for Adults cont’d

• Tests should include item content appropriate for adults, rather than young students, even though the difficulty level of the content may be low in order to achieve a reasonable test floor.

• Consider entering your test data and using the instructional skills profile available at the Adult Reading Components Study website: https://lincs.ed.gov/readingprofiles/resources.htm

https://lincs.ed.gov/readingprofiles/resources.htm

Tips for Formal, Individualized Assessment of English Language Learners

• Text Box 6.8• Users should follow standards set for test administration, scoring and

interpretation in general, and for ELL and other diverse learners as developed by experts. Many of these standards are reproduced in Chapter 6; some of the most relevant are reproduced in this Text Box in abbreviated form.

• Be aware that an ELL student’s formal test scores will be compared to students in the U.S., who may not resemble the ELL in background, culture, knowledge, etc.

• Related to Number 2 above, an ELL’s scores on formal tests may be depressed because of cultural, linguistic, and knowledge differences.

Tips for ELLs cont’d

• Do not administer a language-loaded test for ELL examinees unless the language content in the test is the focus of assessment.

• Related to Number 4 above, if a student’s English language proficiency is the target of assessment, English language-loaded tests (i.e., English reading tests) are appropriate.

• Ensure that the test does not contain item content that may be offensive in the ELL’s home country or culture.

• Ensure that reliability and validity of the test have been established for ELLs.

• Ensure that the items are not biased against ELL learners by evaluating recommendations from expert panels.


• Ensure that items are not biased against ELL learners by examining evidence from statistical techniques designed to determine bias.

• When possible, use an examiner who can communicate in the ELL’s native language to establish rapport and aid in administering the test.

• Be aware that an ELL’s progress in learning a second language and ability to understand test demands may be limited by negative transfer from the native language.

• Reduce importance of speeded performance on scores; some cultures do not value speed.

• Ensure that the test only measures what you intend to measure (i.e., specific types of reading skills) and not some irrelevant content.


• If your ELL students score below the mean, ensure that if mean differences occur, the differences are not the result of bias.

• Use ample teaching items to ensure that ELL learners understand the task demands.

• Using grade equivalent scores, and to a lesser extent age equivalent scores, is problematic due to each of misinterpretation of these scores, and psychometric limitations.

Assessment at a Glance: Formal, Individualized Assessment

• Tables 6.1 and 6.2• Characteristics of Formal, Individualized, Norm-Referenced

Assessments of Reading• Psychometric Properties of Formal, Individualized, Norm-

Referenced Assessments of Reading

Summary

• Development of Formal Reading Measures• Types of Scores• Administration and Scoring of Formal Tests• Determining Chronological Age• Individualized vs Group Tests• What Do Individualized Norm-referenced Measures of Reading Look

Like? • Test Bias• Selecting Formal, Individualized Instruments• Special Considerations for Formal, Individual Assessment of Adult and


Documents

Chapter 6 Formal Assessment of Reading: Individualized Assessment