Shannon Wang, M.A., CCC-SLP Nancy Castilleja, M.A., CCC-SLP Marie Sepulveda, M.S., CCC-SLP

Importance of Conceptual Scoring to Language Assessment in Bilingual

Children2011 ASHA Convention, San Diego, CA

November 19, 2011

Shannon Wang, M.A., CCC-SLP

Nancy Castilleja, M.A., CCC-SLP

Marie Sepulveda, M.S., CCC-SLP

Mark H. Daniel, Ph.D.

AgendaOverview: Assessing bilingual children

Conceptual score approach to language assessment

Data collection

Research results

Overview: Assessing Bilingual Children

IDEIA Statute: Reduce the inappropriate over-identification of children, especially minority and limited English-proficient children, as having a disability. Statute: Title 1.D.664.b.2.D.vii

Normal bilingual phenomena can look similar to a disorder to those unfamiliar with 2nd language acquisition

Some typical characteristics of bilingual speakers in the U.S.

• Arrest: The level of proficiency in the language does not change.• Attrition: Language loss and language forgetting• Avoidance: Specific element of a language is not used• Language non use (silent period): a language is not used for

communication purposes• Overgeneralization: a language rule is applied in an unrestricted fashion• Language transfer: phonological, morphological, syntactic, semantic and/or

pragmatic characteristic is used in another language• Fossilization: an inaccurate rule stabilizes to the point of continual usage(Region 4 Educational Service Center, 2005)

Result: Bilingual children often misdiagnosed• Low test scores in both Spanish and English

Assessing Bilingual Abilities

“The lower vocabulary of bilinguals at certain stages of development may have nothing to do with handicaps or dominance questions but probably more with a smaller variety of linguistic input in each language taken separately.” Hugo Baetens-Beardsmore, 1986

Assessing vocabulary in bilingual children: best practice is to test both languages H. Kayser, 1989; H. W. Langdon, 1989

Conceptual Scoring

“Conceptual scoring” is scoring the meaning of a response regardless of the language in which it is produced. B. Pearson, S. Fernandez, & D.K. Oller, 1993

Bilingual children benefit from conceptual scoring, especially when tested in Spanish L. Bedore, E. Peña, M. Garcia, & C. Cortez, 2005

Different ways of combining test scores across languages were tested—combining scores across two languages in a composite or selecting combinations of better task or language performance to use as a basis for decision-making…Classification can be more accurate when scores in both language are used systematically for decision-making.E. Peña and L. Bedore, 2011

Conceptual Scoring ---> Dual Language Score

“Conceptual scoring” is based on literature examining semantic language development (vocabulary and other semantic skills).

PLS-5 Spanish targets oral language (semantic and morphosyntactic skills) and early academic skills.

Does the dual language score approach provide a more valid representation of a bilingual child’s language skills?

Studies Examining a Dual Language Approach for PLS-5 Spanish

• PLS–4 Spanish bilingual pilot study

• PLS–5 Spanish • bilingual tryout study• bilingual standardization

study

Development of a dual-language scoring procedure• Bilingual expert panel

– Hortencia Kayser, Ph.D. – Henriette Langdon, Ph.D. – Elizabeth Peña, Ph.D.

• Developed PLS–4 Spanish English Record Form supplement

• Administered PLS–4 Spanish to participants

• After administration of the PLS-4 Spanish, items the child missed in Spanish were re-administered in English

PLS–4 Spanish Bilingual Pilot Study

PLS–4 Spanish Bilingual Pilot StudyParticipants n=28

Ages 3:7-6:10

Countries of origin– Mexico– Caribbean– Central & South America

Caregiver education level– 11th grade or less 37%– High school graduate or GED 22%– 1–3 years of college or technical school 22%– 4 or more years of college 19%

Fluency in Spanish

Exposure to Spanish• Primary caregiver speaks Spanish to child• Child is Spanish-English bilingual• Child may be enrolled in bilingual classes

Language comprehension• Understands Spanish and a little English OR• Understands both Spanish and English OR• Understands some concepts only in Spanish and some

only in English

Language expression• Speaks Spanish, a little English OR• Speaks both Spanish and English


Results• 93% received additional points in AC

Score difference range: 0 to 6 points (mean = 2.9)

• 75% received additional points in ECScore difference range: 0 to 13 (mean = 3 points)

• 32% of sample earned scores that moved from language-disordered range of performance to typically developing range


PLS–5 Spanish Bilingual Tryout Study

Participants n=200

Ages 2:0 through 7:11

Diagnosis TD: n = 166NonTD: n = 34


Criteria for Language Disorder

Inclusionary Criteria• Diagnosed with a moderate to severe language disorder (< 77

on standardized test) in either receptive language, expressive language or both

OR• Diagnosis based on non-standardized tests results; plus

statement provided by clinician indicating a moderate to severe language disorder

• Must be enrolled in language therapy


Criteria for Language Disorder (cont.)

Exclusionary Criteria• history of hearing impairment, middle ear infections/ otitis

media/PE tubes, or hearing aids• phonological disorder• verbal apraxia or dyspraxia, or exhibits deletions of final

sounds or syllables • Exceptions

– aspirated final /s/, common in a Puerto Rican dialect– Consistently substitutes final /s/ with another phoneme

PLS-5 Spanish Bilingual Tryout Study: Sample Demographics

TD Non-TD

N 166 34

Age: Mean 4:11 5:5 SD 1:7 1:5

NTD group: Expressive language 19% Receptive language 4% Both 77%


Method• PLS-5 Spanish Tryout edition

• Items were administered in Spanish first• Any items missed in Spanish were re-administered in English

• Items were scored based on:• Spanish performance• Spanish-English performance (dual language scoring)

• Data analysis compared Spanish-only scores and Spanish-English scores

PLS–5 Spanish Bilingual Tryout Study Findings

Gain from dual language scoring • Beneficial for children ages 4:0-7:11

• Strongly related to rated proficiency in English(Children with “little English” show little gain)

• No relationship to caregiver education level

• No relationship to whether or not the child is typically developing

• For children 2:0-3:11• Children still in the early language acquisition process • There was not the same pattern of gains with dual language

scoring as with older children

PLS–5 Spanish Bilingual Tryout Study Findings (continued)

• Children with typical language development showed equal gains in academic and non-academic language

• Children with a language disorder showed greater gains in non-academic language

PLS-5 Spanish Standardization:Dual Language Study

Bilingual Children’s Levels of Fluency in Spanish and English

• Primarily Spanish speaker with some English abilities

• Bilingual Spanish-English speaker

Language Comprehension in Spanish

1. Child understands Spanish, but no English [monolingual]

2. Child understands Spanish and a little English [bilingual]

3. Child understands both Spanish and English [bilingual]

4. Child understands some concepts in Spanish and some in English (e.g., home concepts in Spanish; school concepts in English) [bilingual]

5. Child understands English and some Spanish [not included in sample]

6. Child does not understand Spanish; only understands English [not included in sample]

Expressive Language in Spanish

1. Child converses in Spanish, speaks no English [monolingual]

2. Child converses fluently Spanish and speaks Spanish most of the time. He or she speaks a little English [bilingual]

3. Child converses fluently in both Spanish and English [bilingual]

4. Child converses fluently in English and speaks English most of the time. He or she speaks a little Spanish. [not included in sample]

5. Child converses fluently in English; speaks no Spanish [not included in sample]

Exposure to SpanishAlmost always: [monolingual] • Interacts in a Spanish speaking environment only• Leisure activities in Spanish• Speaks Spanish with family and friends

Often: [bilingual]• interacts in both Spanish and English environments • may prefer to speak Spanish with friends and family OR• may switch languages without a preference for either

Occasionally: [bilingual]• Interacts with friends or family members who speak Spanish only• Speaks Spanish but prefers English with family and friends

Seldom or Almost Never [not included in the study]• Interacts with friends or family members who speak Spanish only, but do not live in

child’s home (seen infrequently)• Communicates a few messages in Spanish

Dual Language STDZ Study: Length of Time Residing in the U.S.

* 17% did not report length of time in the U.S.

• 0-11 Months• 1 Year• 2 Years• 3 Years• 4 Years• 5 or more Years• Born in the U.S.

83% of the children living in the U.S. wereborn in the U.S. or have lived in the U.S. for more than 5 years

PLS–5 Spanish Dual Language Standardization Study: Overview

Field Research• PLS-5 Spanish Standardization edition

– Items were administered in Spanish first– Any items missed in Spanish were re-administered in English

Scoring• Items were scored based on:

– Spanish performance– Spanish-English performance (dual language scoring)

Data Analysis• Data analysis compared Spanish scores to Spanish-English

scores

Dual Language Record Form(Draft)

PLS–5 Spanish Dual Language STDZ Study

Matched Bilingual and Monolingual Samples


Matched Monolingual and Bilingual Samples:Age and Gender

Bilingual Monolingual Bilingual Monolingual Bilingual Monolingual

N

Age: Mean SD

Gender: Female 44% 44% 46% 51% 46% 46% Male 56% 56% 54% 49% 54% 54%

4.80.8

76

7.00.6

Ages 0-2 Ages 3-5 Ages 6-7

81

1.50.8

151


Matched Monolingual and Bilingual Samples:Caregiver Education


Caregiver education:

< high school grad 22% 22% 30% 33% 26% 18%

high school grad 25% 26% 27% 31% 34% 34%

some college 24% 26% 21% 20% 17% 29%

college graduate 30% 26% 23% 17% 22% 18%



Matched Monolingual and Bilingual Samples:Region


Region:

Northeast 4% 3% 5% 0% 4% 0%

Midwest 0% 0% 6% 1% 3% 0%

South 51% 42% 69% 48% 66% 13%

West 44% 26% 17% 7% 22% 1%

Puerto Rico 1% 30% 3% 44% 5% 86%



Matched Monolingual and Bilingual Samples:Country of Origin


Country of origin:

Mexico 61% 53% 66% 41% 75% 15%

Puerto Rico 11% 31% 5% 43% 9% 85%

South America 16% 6% 17% 4% 8% 0%

Central America 10% 4% 10% 7% 4% 0%

Cuba 3% 5% 1% 4% 4% 0%

Dominican Rep. 0% 1% 1% 1% 0% 0%



Standard Score by Administration Method:Matched Monolingual and Bilingual Samples

Ages 0-2

Scale Score Type M SD M SD ∆ t pSpanish 102.0 15.7 102.5 12.3 0.5 0.20Dual-Language 102.0 15.7 103.6 12.2 1.6 0.73

Spanish 102.9 13.4 103.9 11.1 1.0 0.55Dual-Language 102.9 13.4 104.3 11.1 1.4 0.74


Difference

80

81

80

Monolingual Bilingual

AuditoryComp

ExpComm

TotalLang

N per group



Ages 0-2

Auditory Comprehension

95

100

105

110

Spanish Dual-Language

MonolingualBilingual

Expressive Communication

95

100

105

110



Total Language

95

100

105

110





Ages 3-5

Scale Score Type M SD M SD ∆ t pSpanish 97.8 11.7 100.4 14.6 2.6 1.77Dual-Language 97.8 11.7 106.4 14.8 8.6 5.66 <.001

Spanish 99.9 11.8 100.5 16.3 0.6 0.37Dual-Language 99.9 11.8 104.3 16.3 4.4 2.60 .01

Spanish 98.8 11.6 100.6 16.0 1.8 1.03Dual-Language 98.8 11.6 105.9 16.1 7.1 4.14 <.001

145

144

Monolingual

AuditoryComp

ExpComm

TotalLang

BilingualN per group

150

Difference



Ages 3-5


95

100

105

110




95

100

105

110



Total Language

95

100

105

110





Ages 6-7

Scale Score Type M SD M SD ∆ t pSpanish 98.6 11.1 97.7 16.9 -0.9 -0.49Dual-Language 98.6 11.1 106.6 13.4 8.0 4.73 <.001



75

75

74

DifferenceN per group

TotalLang

Monolingual Bilingual

AuditoryComp

ExpComm



Ages 6-7


95

100

105

110




95

100

105

110



Total Language

95

100

105

110




Norm Sample(includes a representative number of clinical cases)


Standard Score by Administration Methodand Fluency Group (Norm Sample)

Ages 0-2

Scale Score Type N M SD N M SD N M SDSpanish 98.7 16.3 101.5 12.5 102.4 12.3Dual-Language 98.7 16.3 102.2 12.5 107.6 11.3

Difference 0.0 0.7 5.2





AuditoryComp

ExpComm

TotalLanguage

26

Bilingual

14

15

14

Primarily Spanish

26

26

Monolingual

286

286

286



Ages 0-2


95

100

105

110


BilingualPrimarily SpanishMonolingual


95

100

105

110



Total Language

95

100

105

110





Ages 3-5








54

49

49

AuditoryComp

89

88

ExpComm

TotalLanguage

305

305

88305



Ages 3-5


95

100

105

110




95

100

105

110



Total Language

95

100

105

110





Ages 6-7







84

80

80

Bilingual

69 41

69 43

AuditoryComp

69 43

Primarily SpanishMonolingual

ExpComm

TotalLanguage



Ages 6-7


90

95

100

105

110




90

95

100

105

110



Total Language

90

95

100

105

110




Clinical Samples


Matched Clinical and Nonclinical Samples:Age and Gender

Expressive Language Disorder

Receptive Language Disorder

Exp & Recept Language Disorder

N 69 53 48

Age: 1 3 3 32 11 11 113 12 6 64 14 10 85 11 10 86 10 8 77 8 5 5Mean: 4.7 4.6 4.5

Gender: Female 28% 23% 21%Male 72% 77% 79%


Matched Clinical and Nonclinical Samples:Caregiver Education



Expressive & Receptive Language Disorder

Caregiver education:

< high school grad 55% 64% 67%

high school grad 20% 25% 23%

some college 13% 2% 2%

college graduate 12% 9% 8%


Matched Clinical and Nonclinical Samples:Region




Region:

Northeast 17% 23% 23%

Midwest 0% 0% 0%

South 35% 34% 35%

West 30% 32% 31%

Puerto Rico 17% 11% 10%


Matched Clinical and Nonclinical Samples:Country of Origin




Country of origin:

Mexico 59% 62% 60%

Puerto Rico 20% 13% 13%

South America 4% 6% 6%

Central America 10% 13% 15%

Cuba 4% 4% 4%

Dominican Rep. 1% 2% 2%


Standard Score by Administration Method:Matched Clinical and Nonclinical Samples


Note: All clinical vs. nonclinical differences are statistically significant (p < .001).

Scale Score Type M SD M SD ∆Spanish 78.8 15.5 96.1 14.9 17.3Dual-Language 79.7 15.2 98.7 13.4 19.0

Spanish 76.5 12.1 97.8 14.2 21.3Dual-Language 77.1 12.0 99.0 14.4 21.9


ExpComm

64

TotalLanguage

64

N per group

Clinical Nonclinical

AuditoryComp

69





70

75

80

85

90

95

100

105


NonclinicalClinical


70

75

80

85

90

95

100

105


NonclinicalClinical

Total Language

70

75

80

85

90

95

100

105


NonclinicalClinical








Expressive Communic

49

TotalLanguage

49

N per group


Auditory Comprehe

53





70

75

80

85

90

95

100

105


NonclinicalClinical


70

75

80

85

90

95

100

105


NonclinicalClinical

Total Language

70

75

80

85

90

95

100

105


NonclinicalClinical








Expressive Communic

45

TotalLanguage

45

N per group


Auditory Comprehe

48





70

75

80

85

90

95

100

105


NonclinicalClinical


70

75

80

85

90

95

100

105


NonclinicalClinical

Total Language

70

75

80

85

90

95

100

105


NonclinicalClinical


1. Little effect of dual-language scoring below age 3.


1. Little effect of dual-language scoring below age 3.2. Nevertheless, dual-language scoring significantly

raised the average standard scores of bilingual children. Children who primarily speak Spanish but know some English had a smaller increase than children who are more fully bilingual.

PLS–5 Spanish Standardization:Dual Language STDZ Study

1. Little effect of dual-language scoring below age 3.2. Nevertheless, dual-language scoring significantly raised the

average standard scores of bilingual children. Children who primarily speak Spanish but know some English had a smaller increase than children who are more fully bilingual.

3. Dual-language scoring did not affect the scores of children with language disorders.

Testing in Spanish and English: Dominance and Proficiency

“The concept of a ‘dominant’ language is losing favor as there is more evidence that proficiency in two languages occur on a continuum, with individuals being able to understand or express some concepts better in one language and others in another language.”(Peña, Bedore, & Zlatic-Giunta, 2002)

“…notions such as proficiency and dominance are moving targets altered with differences in tasks, topics, and demands”(Goldstein, 2004)

Testing in Spanish and English: Dominance and Proficiency

“Language proficiency measurement is not as concerned as to which language is stronger or dominant, but rather its goal is to provide a description of the language development of the child in listening, speaking, reading, and writing.(Kayser, 2001)

ReferencesBaetens-Beardsmore, H. (1986). Bilingualism: Basic Principles (2nd Ed.). San Diego, CA: College-Hill Press.

Bedore, L., Peña, E., Garcia, M., & Cortez, C. (2005). Conceptual vs. monolingual scoring: when does it make a difference?. Language, Speech, and Hearing Services in Schools, 36, 188-200.

Kayser, H.R. (1989). Speech and language assessment of Spanish-English Speaking Children. Language, Speech, and Hearing Services in the Schools, 20 (3), 226-244.

Kayser, H. (2001) “Assessing Language Proficiency and LanguageDominance.” From the Hart. October 2001. Bilingual Therapies, Inc.http://www.bilingualtherapies.com/kayser-newsletter/2001/assessinglanguage-proficiency-and-language-dominance/

Individuals with Disabilities Education Improvement Act of 2004 (IDEIA), Public Law 108-446 (2004.) 118 Stat. 2647 (2004)

References (continued)

Langdon, H.W. (1989). Language Disorder or Difference? Assessing the Language Skills of Hispanic Students. Exceptional Children, 56 (2).

Pearson, B., Fernandez, S. & Oller, D. K. (1993). Lexical development in bilingual infants and toddlers: comparison to monolingual norms. Language Learning, 43 (1), 93-120.

Peña, E.D. & Bedore, L.M. (2011). “It takes two: improving assessment accuracy in bilingual children. ASHA Leader, 16 (13), 20-22.

Peña, E., Bedore, L., Zlatic-Giunta, R. (2002) Category-GenerationPerformance of Bilingual Children: The Influence of Condition Category andLanguage. Journal of Speech Language and Hearing Research, 45, 938-947.

Region 4 Educational Service Center (2005). Houston, TX (Author).

Contact Information

Shannon WangSenior Research [email protected]

Nancy CastillejaSenior Product [email protected]

Marie SepulvedaResearch [email protected]

Mark DanielSenior Scientist for Research [email protected]

mailto:[email protected]




Documents

Shannon Wang, M.A., CCC-SLP Nancy Castilleja, M.A., CCC-SLP Marie Sepulveda, M.S., CCC-SLP