Presentation on Field testsMargie Schneider

Dan Mont6th Meeting of the Washington

Group

Kampala, Uganda

10 – 13th October 2006

Why the field tests?

• How does the short set of questions function in different countries?

• Are the questions cross culturally comparable?

• Do the questions capture what the intention of Q is to measure?

Specific objectives

• Determination of whether the single question per domain is representative of that domain

• Determination of whether the questions produce comparable data across countries

• Determination of how the Washington Group questions work as a set in comparisons with other questions used by the country

Analysis

• Endorsement rates for single question vs detailed questions for three different definitions of ‘disabled’

• Endorsement rates for composite score on short set vs composite score on detailed set of Qs

• Comparison of WG Qs to specific country Qs used in previous Census/survey

Analysis - process

• Recoding of data into D1, D2 and D3 and ED1, ED2, ED3

• Frequencies for each question and composite for detailed questions

• Crosstabulations for short question vs individual detailed and composite of detailed Questions

• Comparison of composite scores for WG short set and detailed set

Countries that did the field test

• Vietnam: – population based sample in one province; sample

size of 3540 individuals; – WG short set + detailed questions from cognitive test

+ country specific questions on causes of difficulty

Gambia: – Purposive sampling of 302 – Disabled and non-disabled respondents– WG short set + detailed set from field test (no

additional domains) + own country questions

Countries that did the field test

• South Africa: – Purposive sample of ‘disabled’, ‘non-disabled’

and ‘unsure’ status – 185 adults and 38 parents of children (disabled and non-disabled)

– WG short set + detailed set (incl additional domains) + Census 2001 question and ‘Are you/is your child disabled?’

• Brazil, Paraguay and ? to do field test in November 2006

Cognitive test data analysis

• Used responses to the following: – WG short set questions– Questions that are included in the field test detailed

set of questions

• Sample of 1639 cases from 13 countries• Not population based sample • Applied same analysis as for the field test• Self care – limited because of detailed questions

were not included in the Cognitive test

General comments on results

• Clear progression from low to high endorsement numbers when using different cutoff points (D1 – D3; ED1 – ED3)

• Short set/core single questions seem to be capturing more than the detailed individual questions in most domains

• Use of additional domains increases overall endorsement

• WG short set: more endorsement than Qs with ‘disability’ or ‘disabled’

Cognitive test composite scores

0

200

400

600

800

1000

1200

1 2 3

No. o

f res

pond

ents Core set

Detailed set

Gambia Comparison of composite scores

0

50

100150

200

250

Composite of 6 coreQs

Composite of detailedqs

Using composite ofQs for each domain

No. o

f res

pond

ents

D1/ED1 D2/ED2 D3/ED3

Cognitive data - vision

0

100

200

300

400

500

600

W1 V

ision

Near vi

sion

Far vi

sion

2 Qs

No

. o

f d

isab

led

res

po

nd

ents

D1/ED1

D2/ED2

D3/ED3

Cognitive data - Mobility

0100200300400500600700800

No

. o

f d

isab

led

rsp

on

den

ts

D1/ED1

D2/ED2

D3/ED3

Cognitive data - Cognition

0

100

200

300

400

500

600

W4

Cognit

ion

Conce

ntrat

e 10

minu

tes

New ta

sks

Find so

lution

s3Q

sNo

. o

f d

isa

ble

d r

es

po

nd

en

ts

D1/ED1

D2/ED2

D3/ED3

Gambia short vs detailed

0

20

40

60

80

100

120

140

no

. of

resp

on

den

ts

D1 D2 D3 ED1 ED2 ED3

South Africa testing process

• Stats SAs needs: Testing a disability schedule for Census 2011

• Process includes: – Focus group study (2006)– Household survey (2006/07)– Further qualitative work (?) (2008)– Pilot of Census (2009)– Census (2011)

Focus group distribution

• 5 focus groups with parents of children (3 with disabled children and 2 with non-disabled children)

• 21 focus groups with adults: – 9 groups with ‘disabled’– 6 groups with ‘non-disabled’– 6 groups with ‘unsure’

• Mixed in terms of urban/rural and language groups• All completed questionnaire before the group

discussion– Background info including ‘are you disabled?’ and Census

2001 Qs– Proposed Qs for Census (WG short set)– Detailed Qs

• Whole questionnaires translated into all languages

• Tape recorded

• Transcribed into original language

• Translated into English

• Themes in discussion: – Understanding of disability– Consistent responses on proposed questions

across different groups– Issues not covered– Relevance of issues– Sensitive information on disability

South Africa : N=185D vs ED (2, 3 or 4 Qs)

01020304050607080

No.

of r

espo

nden

ts

D1 (High) D3 (Low) ED1 (high) ED3 (Low)

South Africa: Additional domains (N=185)

020406080

100120140160180

*Participation incommunityactivities

**Learning **Socialinteractions

**Emotionalfunctioning

No. o

f res

pond

ents

D1 (High) D3 (Low) ED1 (high) ED3 (Low)

South Africa: Composite Scores (N=185)

0

50

100

150

200

D: Overall withoutparticipation

D: Overall withparticipation

ED: Overallwithout additional

domains

ED: Overall withadditionaldomains

No. o

f res

pond

ents

D1 (High) D3 (Low) ED1 (high) ED3 (Low)

Core vs detailed sets

• No major differences if use core domains for D1 vs ED1(except self care); more differences when comparing D3 vs ED3

• Significant increase if use additional domains of learning, social interactions and emotional functioning. – ED1 without additional domains: 128/185– ED1 with additional domains: 166/185

South Africa: Comparison of 3 Qs

020406080

100120140

WG Core 6 ‘High’(D1)

WG core 6 ‘Low’(D3)

Census 2001 ‘Are you/is yourchild disabled?

No. o

f res

pond

ents

Adults (N=185) Children (N=38)

Group WG D1 (6 questions) Census 2001 ‘Are you disabled?’ ‘Disabled’ Disabled Non

disabled Disabled Non

disabled Disabled Non

disabled English urban psychiatrically ill

7 1 7 1 6 2

English urban visually impaired

9 0 2 7 9 0

English urban physically impaired

6 0 3 3 6 0

Tsonga rural visually impaired

9 1 5 5 10 0

Tswana rural physically disabled

8 2 4 6 7 3

‘Unsure’ English urban HIV/AIDS

10 0 2 8 0 9

English urban chronic illness

7 0 6 1 0 7

Zulu urban older people

7 2 5 4 2 7

‘Non-disabled’ English urban youth 2 7 1 8 0 9 SeSotho urban adults 3 7 0 10 0 9 Afrikaans rural adults 4 5 1 9 1 9

Reactions to questions• Generally understood easily• Recall period: caused some difficulties (‘can’t

remember everyday for last month’) or ignored (just answered as ‘usual/normal’)

• Distance reference: needs to be clear• Sensitivity of questions: sometimes hurtful but not

sensitive (emotional functioning); important to ‘tell the world’

• Sensitive topics: sexuality, toileting, direct questions about impairment, reading/writing for Deaf people, HIV status

Reactions to questions (contd)

• Relevance of questions: – Generally relevant especially for ‘disabled’

and ‘unsure’ groups– Reflected their own lives – ‘unsure’ group but

not ‘non-disabled’ group– Identification with ‘disabled’: ‘unsure’ and

‘non-disabled’ groups did not identify– Identification is used politically to get access

to services (e.g. Deaf group)

What is disability• Descriptions provided by participants

suggests that disability is: – Permanent– Visible and physical– Not ‘curable’ or ‘solvable’– ‘cannot do anything by themselves’ vs ‘can’t

do’ only in impaired domain of functioning– ‘Them’ and ‘us’ across all groups– Not positive connotation

What is disability? (contd)

• ‘Difficulty’ is less severe than disability and can be solved;

• Disability includes difficulty but difficulty does not include disability

• Disabled group were most aware of role of environmental factors on their functioning

• Health problem is a temporary feature and can be cured and not directly linked to disability

• Seeing, hearing, concentration, remembering, participation etc. are not health

• Aches and pains are health

Recommendations for changes to proposed questions

• Introductory phrase: change use of ‘health problem’

• Change to introductory phrase of communication question

• Clear distance reference (‘across the road’)• Training on phrase ‘even if wearing glasses or

hearing aid’• Retain question on participation• Need further research on E questions• Avoid mention of ‘disability’• Add question on learning for young children (if

space permits)

Additional areas for survey questions

• More on mental functioning• Issues of transport• Accessibility of built environment• Availability of services and meeting of needs• Employment issues• Attitudes towards disabled people• Personal assistance• Needs such as home based care• Stress• Issues of sleep and energy (psychiatric illness)• Medication• Balance (Deaf)

Revised Qs for Survey GENERAL HEALTH AND FUNCTIONING

The following questions ask about any difficulties that the person has because of a health problem or condition.

a) Does the person have difficulty in doing any of the following?

• Seeing (even with glasses if he/she wear(s) them)• Hearing (even with a hearing aid if he/she wears one)• Walking a kilometre or climbing a flight of steps• Remembering• Concentrating• With self-care, such as washing all over or dressing• In communicating in his/her usual language, including sign

language (i.e. understanding others or being understood by others) • Joining in community activities (for example, festivities, religious or

other activities) in the same way as anyone else can

• Response options: 1 = no; 2 = some; 3 = a lot; 4 = unable to do; 5 = don’t know

b) Does the person use any of the following? • 1 = Yes• 2 = No• Eye glasses• Hearing aid• Walking stick or frame• A wheelchair• Chronic medication