Apr 19, 2023 1

Screening for Disability

Washington Group on Disability Statistics

Apr 19, 2023 2

Screening for Disability on Censuses

• The main purpose of a screening instrument is to be as inclusive as possible in order to identify all eligible people for further study.

Apr 19, 2023 3

Screening Questions

• To be inclusive, questions should cover all three of the main dimensions described in the ICF: body structure & function, activity (basic actions), and participation (complex activities).

• Once the disabled population is identified, the follow-up survey could focus obtaining more information on a wide range of aspects of disability related topics.

Apr 19, 2023 4

Screening Questions

• Screening reduces total respondent burden and survey costs.

• However, the length of the ideal screener could be prohibitive, and

• The more inclusive the screened population – the larger the sample to be followed and the higher the costs

• Disability Supplement to the US NHIS screener took 25 minutes to administer and identified over 25% of the population

Apr 19, 2023 5

Canadian Experience

• The Census is used to develop a frame for a disability survey

• The Participation and Activity Limitation Survey (PALS) determines disability status

Apr 19, 2023 6

Census Filter Question - 20061. Does this person have any difficulty hearing,

seeing, communicating, walking, climbing stairs, bending, learning or doing any similar activities?

2. Does a physical condition or mental condition or health problem reduce the amount or the kind of activity this person can doAt home? At work or at school? In other activities, for example, transportation?

Response categories:Yes, sometimes; Yes, often; No

Apr 19, 2023 7

2006 PALS

• 2006 Census• 2006 PALS collection (46,000 persons)

• Post collection follow-up• Face to face interviews with 50 false positives• Face to face interviews with 100 false negatives• Face to face interviews with 50 mild disabilities

results

False + false -

Apr 19, 2023 8

Apr 19, 2023 9

False Negatives

• A person with a disability that does not get picked up by the filter question –• Proxy reporting• Timing of the Census – cyclical or

episodic disability• Mild to medium pain limitations• Psychological limitations• Perception of activity limitation

• Related to age, cultural identity, language

Apr 19, 2023 10

False Negative - Results• False negatives: persons with disabilities who say No

to the original screening questions but who WOULD have reported a limitation if they were administered the PALS survey. 

• Following the 2006 PALS a small scale exercise for adults was conducted. 

• Persons in Toronto and Ottawa who said No to the Census filters were administered the PALS survey. 

• Some reported limitations and this cast some doubt on the effectiveness of the filter questions. 

• But on analysis it was found that almost all respondents were coming in on the pain module - middle aged persons with sore knees or shoulders. 

• For the adult population, the filter questions were not seen as missing large numbers of persons with disabilities.

Apr 19, 2023 11

False Positives

• A person without a disability who answers ‘Yes’ to a filter question –• Proxy responding• Short term disability or injury• Cyclical or episodic disability• High false positive rates for children

Apr 19, 2023 12

False Positive - Results• For adults, about 25% of PALS respondents are false

positives, i.e. they said Yes on the census but report no limitation on the PALS survey. 

• The most common reasons are:1. short term injury 2. proxy effect as the census is completed by a

single household member and 3. mistake, often language based. 

• For children, the false positive rate is about 50% - here, the proxy effect is less clear as both the Census and PALS are proxy for children.  

Apr 19, 2023 13

Irish Experience• 2004 - Government decision to conduct

post-censal National Disability Survey (NDS)• Purpose

• Establish prevalence, severity and impact of disability in Ireland

• Identify improvements needed in policy and service provision

• Targeted survey - sample based on responses to 2006 Census questions on disability

Apr 19, 2023 14

Census questions

9 Disability types in NDSSeeingHearingSpeechMobility and dexterityRemembering and concentratingIntellectual and learningEmotional, psychological and mental healthPainBreathing

15

NDS thresholds for disability

Level of difficulty in every day activities

Disability typeNo

difficultyJust a little

A moderate

level

A lot of difficulty

Cannot do at all

A. Seeing

B. Hearing

C. Speech E. Remembering &

concentrating

F. Intellectual & learning

G. Emotional, psychological, & mental health

H. Pain I. Breathing

16

Apr 19, 2023 17

Census and NDS – comparison• NDS private household sample matched

to Census data file• Analysis based on unweighted sample data

only• Comparison of distribution of ‘Yes’ and ‘No’

responses in each - ‘false positive’ / ‘false negative’ responses to Census question

• Types of disability reported in Census and NDS

• Consistency of reporting of disabilities between Census and NDS

Apr 19, 2023 18

Comparison of Yes and No responses

NDS Response

Census response No Yes Total NDS

No 1,321

8.4%

295

1.9%

1,616

10.3%

Yes 1,910

12.4%

12,062

77.4%

13,972

89.7%

Total Census 3,231

20.8%

12,357

79.3%

15,588

100.0%

Apr 19, 2023 19

Census ‘false positives’

• 14% of persons reporting a disability in Census did not report a disability in NDS

• 25% 65 and over• Report lower levels of limitation• 53% - Census category “Other

including chronic illness”

Apr 19, 2023 20

Census ‘false negatives’

• 18% of people who did not report disability in the Census did report disability in the NDS

• 51% aged 65 or over• Highest proportion of responses:

• Pain disability category - 51% • Mobility & dexterity disability category -

46%• Report lower levels of limitation

Apr 19, 2023 21

Prevalence rates

• Census: 9.3%• NDS: 18.5%

• 11.5% of false negatives are classified as disabled on the NDS; this group has large weights

Apr 19, 2023 22

Conclusions - Ireland• Matching of NDS records to Census at person level

increases statistical value of NDS output• A high proportion of the Census false positive responses

are due to responses in the residual ‘Other’ category• >3/4 of sample reported a disability in both surveys• The Census questions & methodology resulted in a

much higher level of single disability reporting than NDS• Learning related difficulties prevalent among children in

the sample• Mobility and Pain most prevalent disabilities for older

people• People of working age had a more mixed range of

difficulties