0915206 - Language Access...LanguageAccess and the Role Certification and Licensure Exams Play Portland, Oregon September 15, 2016 Council on Licensure, Enforcement and Regulation

Language Access and the Role Certificationand Licensure Exams Play

Portland, OregonSeptember 15, 2016

Council on Licensure, Enforcementand Regulation 2016 Annual Educational Conference 1

Language Access and theRole Certification andLicensure Exams Play

Panelists

• Stacey Saunders, Washington StateDepartment of Health

• Carmen Castro-Rojas, Judicial Council ofCalifornia

• Ron Ripa, Prometric Test Development

Washington Department of Health

Home Care Aide

Certification Exam Program




Washington State Facts

• Population: 6,968,170

• Language other than Englishspoken at home: 1,310,016 or 18.8%

• Limited English: 601,376 or 9.15%

• Languages Spoken: Over 200

Washington State Office of Financial Management, US Census data

Washington State Map-County

Seattle Area Map-Census Tract




Home Care Aide Overview

• Certification required by law

• Must pass knowledge and skills exam

• Exam in 12 translated languages,individual interpreters

Translated Languages

• Arabic

• Chinese

• Khmer/Cambodian

• Korean

• Laotian

• Russian

• Samoan

• Spanish

• Somali

• Tagalog

• Ukrainian

• Vietnamese

Why Have Language Access?

• DEMAND: Consumers want a caregiverfrom their culture and language.

• PUBLIC SAFETY: Home Care Aides arecaregivers, English proficiency notrequired.

– Caregiving transcends culture and language.




Benefits of Language Access

• Consumers have caregiver from theirculture/language

• In-home care costs less

• Profession open to Limited Englishworkers

• Health equity

Unintended Barriers

• Old exam designed for proficient Englishspeakers educated in the US.

• Standardized multiple choice questionswith plausible distractors.

• Lower pass rates for translatedlanguages.

Home Care Aide New Exam• Diverse expert panel for all steps

• No multiple choice with plausibledistractors

• True/false




New Exam Cont’d• Numbers instead of words

• Supplement with pictures

• Animated scenarios

Sample QuestionsTrue or False: Gloves can be disinfectedand reused.

True or False: Proper body mechanicsinclude bending at the knees.

Results




Results (Cont’d)

Results (Cont’d)

Lessons Learned

• Know the demand

• Maintain public safety

• Verbatim translation not sufficient

• Importance of dialect




Lessons Learned-Testing• Multi-cultural/multi-lingual expert panel

all stages of test development

• Multiple choice questions not standard inall countries

• Easily understood questions

• Oral option for all at test sites

• Translate supporting documents/training

Speaker Contact Information

Stacey Saunders, Home Care Aides ProgramDirector, Washington State Department ofHealth, [email protected], 360-236-2813

California State Courts

Testing of Court Interpreters

and Bilingual Staff




California Facts

• 39,144,818 estimated population

• 44% Speak a language other than Englishat home

• 19% Speak English “less than very well” –known as Limited English Proficient (LEP)

• Over 250 languages and dialects spoken

Source: U.S. Census Bureau (2015)

Primary Languages Spoken by LEPs:

eaking English "less than very well'

Source: U.S. Census Bureau (2015)

Reasons for and benefits of languageaccess in courts

• FAIRNESS: Provide access to justice for 7million LEP Californians

• PUBLIC INTEREST: Enable the court itself, thejudges and all court participants to understandwhat everyone is saying




Court Interpreter and Bilingual StaffOverview

• 7.5 Million court cases filed per year

• Over 800,000 interpretations provided

• 1,900 credentialed interpreters

• Highly specialized interpreting skills

• Bilingual staff provide language access

Interpreter Testing Overview• Certified court interpreters:

– tested on actual interpreting skills

– 14 most used languages: 1,681 interpreters

• Registered court interpreters:

- tested on language speaking ability

- all other languages: 211 interpreters

• Both tested on court process, interpreterethics, complex English terminology

Certified Spoken Languages• Arabic

• Armenian (Eastern andWestern)

• Cantonese

• Farsi

• Japanese

• Khmer

• Korean

• Mandarin

• Portuguese

• Punjabi

• Russian

• Spanish

• Tagalog

• Vietnamese




Bilingual Interpreting Exam• In-person exam, one candidate in room

• 4 sections: sight translation English to pairedlanguage, paired language to English,consecutive interpretation, simultaneousinterpretation

• Audio recorded prompt materials, earphonesand recorder

• National Center for State Courts

Rating Interpreting Exams• Each exam oral rendition recorded

• Recording sent securely to trained raters

• At least two raters score, consensus

• Objective scoring designated word units

• Scoring guide “acceptable” and “unacceptable”for each scoring unit

• Distributed among language categories

Written Exam

• Tests English language terms, court terms,court processes, and court interpreter ethics

• Computer based testing, multiple choice

• Required for certified and registered status

• 884 candidates in prior year




Oral Proficiency Exam (OPE)• Tests speaking fluency in the language

• Taken at a test center, on the phone from aprivate testing room

• Language Testing International (LTI)

• American Council of Teachers of ForeignLanguage (ACTFL) scoring system

• Used for registered status interpreters andbilingual staff– with different score levels

Lessons and Challenges

• Appropriate allocation of financial resources

• Extensive materials accompany each examversion

• Difference in candidate volume betweenlanguages

• Constant need for maintenance and refreshingexam and materials

Testing Challenges

• Interpreting – specific set of skills

• Open testing pros and cons

• Expensive

• Exam raters complex to recruit and expensiveto train

• Some small language communities





Carmen Castro Rojas, Court Analyst

Judicial Council of California

[email protected]

(415)865-7787

Test Development and Psychometrics




Factors for Certification – when language access isimportant

• Washington - Ability to display proficiency regardless

of language in order to determine minimal competence

of subject information

• California – Ability to interpret in order to determine

professional competency for a professional credential.

• Ensuring quality, reliability, validity and fairness

What do we mean by reliability, validity andfairness?

• Reliability – Are the results of the test consistent?

• Validity – How well does the test measure what it issupposed to be testing?

• Fairness – items are not controversial or offensive, donot reinforce stereotypes and content is appropriatefor what is to be measured.

Translation is complex• English MC item:

• An individual who constantly complains aboutheadaches that will not subside could be displayingsymptoms of:

• Italian translation:

• Un individuo che si lamenta continuamente di mal ditesta che non si abbasserà potrebbe essere lavisualizzazione di sintomi di

• English:

• An individual who continually complains of headachethat will not subside the symptoms might be viewing




What are some steps that are needed in order tohave quality translations?

• Outsource and manage language services to a qualifiedtranslation service

• Quality control – Review for quality translations

• Use SMEs (subject matter experts) to review items forlinguistic and cultural accuracy

• Importance of testing

Candidate Performance

• Many studies of candidate performance have beendone.

• Factors for consideration:

• A. Need

• B. Cost

• C. Yielding results that are valid, reliable, fair and

psychometrically sound (DIF – Differential Item

Functioning)

Candidate Performance

• Tests serve different purposes – What is thepurpose of the test? Determination ofcompetence or determination of fluency skills?

• Does the cost benefit analysis of translationoutweigh the need?

• Will the sample size yield sound psychometricresults?





Ron Ripa, M.Ed., Senior Test Developer

Prometric

[email protected]

(443)455-6057

Documents

0915206 - Language Access...LanguageAccess and the Role Certification and Licensure Exams Play Portland, Oregon September 15, 2016 Council on Licensure, Enforcement and Regulation