Making The Toefl® Test Accessible To Candidates With Disabilities (Freeland&Novak)

®

Making the TOEFL® test accessible

Patti FreelandJakub Novák

Educational Testing Service

March 27, 2010

®Confidential and Proprietary. Copyright © 2010 Educational Testing Service. All rights reserved. ETS, the ETS logo, Listening. Learning. Leading. and TOEFL are registered trademarks of Educational Testing Service (ETS) in the United States and other countries. TOEFL iBT is a trademark of ETS.

22

The TOEFL® test and Accommodations

• The TOEFL test is an English language test for non-native speakers. It assesses a candidate’s ability to use and understand English in an academic setting.

• Every year, the TOEFL test is administered to nearly 1 million candidates in total.

• Every year, 300-400 TOEFL® candidates request special accommodations. In 2009, 384 candidates requested accommodations.


3

Presentation Outline

• Legal framework• List of accommodations that can be approved• How we create tests with accommodations• Comparability of tests with accommodations to

standard tests • A closer look at accommodations for different types of

disabilities• How to navigate our website: applying for

accommodations• Practice materials, future plans, suggested reading• Contact information


4

Legal framework

• Section 504 of the Rehabilitation Act of 1973:“No otherwise qualified person with a disability in the United States … shall solely by reason of a disability, be denied the benefits of, be excluded from participation in, or be subjected to discrimination under any program or activity receiving federal financial assistance.”

• The Americans with Disabilities Act (ADA), 1990: “Any person that offers examinations … for secondary or postsecondary education, professional, or trade purposes shall offer such examinations … In a place and manner accessible to persons with disabilities or offer alternative accessible arrangements for such individuals.”


5

Implementing Section 504

• Regulations to implement Section 504 in testing contexts:

-- test results should accurately reflect the applicant's aptitude or achievement level ..., rather than reflecting the applicant's impaired sensory, manual, or speaking skills (except where those skills are the factors that the test purports to measure) …


6

Breakdown by type of disability

• Visual impairments

• Deaf and hard of hearing

• Learning disabilities

• Other physical disabilities


7

What accommodations do we offer for the TOEFL® PBT test?

Visual Impairments• Audio test • Braille test • Large print (16 pt. and other type sizes) • Test reader • Writer/recorder of answers • Large print answer sheet • Perkins Brailler®, Braille slate and stylus Hearing Loss

• Oral interpreter – for listening section• Listening section omitted • Sign language interpreter for spoken directions only • Printed copy of spoken directions Other

• Extended testing time • Additional rest breaks


8

What accommodations do we offer for the TOEFL® iBT test?

Visual Impairments• Screen magnification • Selectable background and foreground colors • Writer/recorder of answers • Test reader Hearing Loss

• Listening section omitted and transcripts provided for Speaking and Writing sections

• Speaking section omitted• Sign language interpreter (for spoken directions only) • Oral interpreter (for spoken directions only /with listening section

omitted)Other

• Kensington Trackball mouse, Quill mouse• IntelliKeys keyboard, Ergonomic keyboard. Keyboard with touchpad • Extended testing time • Additional rest breaks


9

How we create forms with accommodations

• Review test material for fairness. Consult with Office of Disability Policy. Use external reviewers where appropriate (for example, Recording for the Blind and Dyslexic).

• Discuss needed accommodations with disability experts (ODP and others at ETS).

• Discuss accommodations with test content specialists. (Would the relevant skills still be measured? Would the tasks be comparable?)

• Produce accommodated forms with the help of ETS technical and publishing staff and external vendors (Associated Services for the Blind; gh).


10

Comparability

Example: in what way is the read-aloud accommodation comparable to the standard test?

Criteria of comparability• Task comparability: Are the tasks on the accommodated test

comparable to those on the standard test? Do the tasks assess the relevant skills assessed by the standard test? Are elements thatundermine comparability removed? (Can be based on a priori judgment; guides the creation of accommodations.)

• Score comparability: Are the scores on the tests comparable? Do they have the same meaning, interpretation, predictive value, etc.?(Based on empirical, quantitative studies. A difficult question to answer.)


11

A definition of reading skills

Reading is decoding and understanding written text. Decoding requires translating the symbols of writing systems (including braille) into the spoken words they represent. Understanding is determined by the purposes for reading, the context, the nature of the text, and the readers’ strategies and knowledge.

A definition proposed by NARAP (The National Accessible Reading Assessment Projects)


12

Reading skills the TOEFL testfocuses on

• General skill: The Reading section measures the test taker’s ability to understand university-level academic texts and passages.

• Important sub-skills:– Reading to find information (e.g. scanning)– Basic comprehension (e.g. facts, vocabulary)– Reading to learn (organization, purpose, connections of

ideas)

• Upshot: understanding is the aspect of reading the TOEFL test focuses on.


13

Read-aloud accommodation in the TOEFL test

• Read-aloud accommodation legitimate.

• Academic context: understanding much more important than decoding. Measures the understanding of the register of the English language used in “university-level academic texts and passages.”

• Reflects real-world skills. Audio technologies are ascendant. In the United States, most blind people do not use Braille.

14®

Confidential and Proprietary. Copyright © 2010 Educational Testing Service. All rights reserved. ETS, the ETS logo, Listening. Learning. Leading. and TOEFL are registered trademarks of Educational Testing Service (ETS) in the United States and other countries. TOEFL iBT is a trademark of ETS.

How the read-aloud accommodation works in the TOEFL test

• Conveying the reading section in an audio format is a challenge:– long passages (memory load)– Listening is primarily a linear experience. Reading a recursive

experience.

• To replicate re-reading, sections of the passage can be repeated or replayed on demand. Extended time.

• Questions repeat passage information. Listen to an example.


15

Candidates who have low vision: Large Print, screen magnification

• TOEFL PBT. We have a suite of tests in several large print type sizes available on the shelf: 16pt, 18pt, 25pt, 35pt.

• TOEFL iBT. We are creating a large print version of the iBT test as well. We tested several fonts to create our base large-print form: 18pt Verdana gave us the best combination of visibility and convenient page layout. We will produce test books in other type sizes on demand.

• TOEFL iBT. Screen magnification and selectable foreground/background colors available for the online version.

• Extended time.

• Can also be approved for candidates with learning disabilities.


16

Candidates who are blind or have low vision: Braille

TOEFL PBT (TOEFL iBT to be rolled out)

• Contracted Braille

• No tactile graphics

• Extended time


17

Candidates who are blind or have low vision: Audio, Reader

TOEFL PBT (TOEFL iBT to be rolled out)

• Live reader or a cassette recording by an actor

• Extended time

• Writer/Recorder of Answers


18

Candidates who are deaf or hard of hearing

• TOEFL PBT: Listening section omitted.

• TOEFL iBT: Listening section omitted, but listening portions of Speaking and Writing sections replaced with transcripts (since listening is not the primary skill measured in those sections).

• TOEFL iBT: Speaking section omitted.

• Oral Interpreter.

• Extended time (if appropriate).

• Sign language interpreter: for instructions only.


19

Candidates with a learning disability

• Extended time

• Extra rest breaks

• Read-aloud accommodation

• Large print, screen magnification


20

Candidates with a physical disability

• Assistive devices — alternative types of mouse, keyboard

• Writer/recorder of answers

• Extra rest breaks

• Extended time


21

Other disabilities and combinations of disabilities

We will try to provide accommodations for any disability or combination of disabilities, as long as the accommodation does not compromise the comparability of the test tasks.


22

What we do not offer (and why)

• The use of some electronic assistive devices: if they require special software or if they present a security concern.

• Sign language interpreter for the listening portions of the test itself. This departs too much from the skill that the test is measuring, namely the candidate’s proficiency in English. A sign language interpreter can be approved for the test instructions.

23®


How to navigate our website information

• Go to www.toefl.org• The link Register for the Test will take you to Disabilities

information• Choose Internet-based Test or Paper-based Test• Among beige tabs choose Disabilities

http://www.toefl.org/

http://www.ets.org/bin/getprogram.cgi?test=toefl

http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=f968af5e44df4010VgnVCM10000022f95190RCRD&vgnextchannel=6cd6197a484f4010VgnVCM10000022f95190RCRD

http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=e9b72d3631df4010VgnVCM10000022f95190RCRD&vgnextchannel=e352197a484f4010VgnVCM10000022f95190RCRD


What you can find under the Disabilities tab

• A list of accommodations we offer• Printable versions of sample questions in regular and large

print (iBT)• Instructions for requesting accommodations and registering

for the test– Submit the accommodation request and the

registration form together. The request takes up to 6 weeks to process. If unsure how to proceed, contact disability services at [email protected]

• Bulletins and forms (also in screen reader-friendly versions)• Contact information• A link to general resources for test takers with disabilities

24


25

Test preparation materials• We have some test practice materials for candidates with

disabilities on the shelf. We can create others on demand. Do not hesitate to ask for materials that you need.

• Practice materials on the shelf (look under Disabilities and Test Preparation tabs):PBT

– Sample questions and transcripts can be used to prepare for some tests with accommodations

– Practice test in Audio format (mailed on request)iBT

– Sample questions and transcripts can be used to prepare for some tests with accommodations (they include an online version and printable versions in large and regular print)

– Test Practice Online can be used to prepare for some tests with accommodations

• Contact [email protected] if you need more information.


26

Plans for the future

• Roll out TOEFL iBT forms in Braille, Audio, Reader, and Oral Interpreter formats.

• Enlarge our library of on-the-shelf practice materials.

• Explore the use of other assistive technologies in accommodations.

27®


Suggested Reading

• Thurlow, M. L., Moen, R. E., Liu, K. K., Scullin, S., Hausmann, K. E., & Shyyan, V. (2009). Disabilities and reading: Understanding the effects of disabilities and their relationship to reading instruction and assessment. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment. (Focused on reading. Available, with other reports and studies, at http://www.readingassessment.info)

• Laitusis, C.C. & Cook L.L. (eds.) (2007). Large-Scale Assessment and Accommodations: What Works? Arlington, Virginia: Council for Exceptional Children. (A volume of many reports, focused mostly on K-12 assessment.)

• Willingham, W. W., Ragosta, M., Bennett, R. E., Braun, H., Rock, D. A., & Powers, D. E. (eds.) (1988). Testing handicapped people. Boston: Allyn & Bacon. (Focused on higher-education assessments such as SAT and GRE. Older, but still a good introduction to the issues.)

http://www.readingassessment.info/


Contact

• ETS Disability services: [email protected]: 1-866-387-8602 (toll free in U.S. and Canada); 1-609-771-7780

• Information on the website: (www.toefl.org PBT/iBT register disabilities)

• Presenters: Patti Freeland: [email protected] Novák: [email protected]

28

29®


Questions, comments?

Documents

Making The Toefl® Test Accessible To Candidates With Disabilities (Freeland&Novak)