Copy of Developing Instrument Data Collection (1) 2 Mei

8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei

1/36

Developing Instrument Data

Collection (1)

Kumboyono


2/36

Choosing an instrument

Appropriateness

Validity

Responsiveness

Precision

Interpretability

Acceptability

Reliability


3/36

Choosing an instrument

Appropriateness

Is the content of the instrument appropriate to thequestions which the study is intended to address?

Reliability

Does the instrument produce results that are reproducibleand internally consistent?

Validity

Does the instrument measure what it claims to measure?

Responsiveness

Does the instrument detect changes over time that matterto patients?


4/36

Choosing an instrument (2)

Precision

How precise are the scores of the instrument?

Interpretability

How interpretable are the scores of the instrument? Acceptability

Is the instrument acceptable to patients?

Feasibility

Is the instrument easy to administer and process?


5/36

Overview of Lecture

What are questionnaires? Why use them?

Ideal properties of a questionnaire

Planning a questionnaire (general principles)

(Choosing/adapting a data collection method, Questions and response categories, Use

of existing data collection instruments)

Questionnaire completion issues

Piloting

Conclusions


6/36

What are questionnaires?Why use them?

Why use them? Measures of outcome (i.e. in a trial)

Audit/Quality assurance (patients and clinicians)

Assessment of health care needs/ preferences

Assessment of individual patient care/satisfaction with that care

List goes on!

A Quantitativesurvey method


7/36

Planning a questionnaire

What to ask?

Who to ask? When to ask?

How to ask?


8/36

Questionnaire design steps

1. Literature Review and initial planning

2. Choose the most appropriate method of data collection (e.g.self-completed questionnaires vs. interviews)

3. Design or adapt data collection instruments (Layout, Presentation,Types of instruments)

4. Conduct pilot work and refine methods and instruments

5. Design and select sample

6. Conduct data collection (often termed field work)

7. Process data8. Analyse data

9. Report findings


9/36

Initial Planning & Literature Review

So where to start?

List topics of interest - If research question is already known!

(why ask?)

Review existing literature on your topic, collate existing

questions and scales that can be used (What worked? What

didnt? Dont re-invent the wheel!)

First opportunity to think about - timing, frequency, content,

wording, layout & administration mode


10/36

Choosing the data collection

method (administration methods) Self-completion

Self completion postal survey (retrospectively)

Self completion concurrent survey

Computer-assisted (internet based?) Paper based

Interviewer administration

Face-to-face (paper/computer assisted) Telephone interviews


11/36

Advantages/ Disadvantages

Postal Administration ?


12/36

Advantages/ Disadvantages

Postal

Advs:

Cheap

Easy to administer (large volume of participants can be accessed

easily) Privacy for sensitive/ stigmatised questions to be explored (more or

less likely to divulge without someone present?)?

Participants can verify responses over time (checking)

Dis: Missing data?

Low response rates?

Misunderstanding of questions (meaning and flow)?


13/36

Design/adapt data collection instruments(Questionnaire layout)

Introduction

Anonymity & confidentiality

Deadlines

Feedback (potentially)

Instructions

Return envelope (reply paid)

Personalisation

Covering

letter?


14/36

Design/adapt data collection instruments(Presentation issues)

Length

Shorter more likely to be completed

Dont need to know, dont ask!

Visual Impact

Paper colour & quality

Font size

Layout


15/36

General principles of questionnaire design

(1)

Ensure all questions and sub questions are clearly labelled

(1,2a,2b,3,4a, etc)

Start with easier (not too sensitive initially!)

Use a logical order but if a long questionnaire start with most

important first

Put demographic questions at the end

Ask about behaviour before attitudes

Ask about past behaviour before current behaviour

Ask general questions first, then more specific


16/36


(2)

Use funnelling-(start broad then narrowdown)

Use filtering

Use familiar words (avoid technical jargon)

Avoid vague terms/abbreviations

Avoid ambiguity

Keep questions short (if possible)


17/36


(3)

Avoid double negatives

Avoid manipulating responses (by conveying

an assumption)

Avoid leading questions

Use hypothetical questions carefully

Avoid bias (mention more later on this)


18/36

Types of questions

Behavioural

Factual

Knowledge

Attitude

Primary consideration: Response formats

2 basic kinds - Open Response

- Closed Response


19/36

Open and closed question responses

Open are just that, left open

- e.g. How do you feel about your health today?

Closed questions have pre-coded answers

- Do you feel well today? Yes No

- Today my health is excellent:(Tick)

Agree Neither agree or disagree Disagree


20/36

Open format

Open questions useful when.

replies unknown

too many/complex to pre-code

good for search for explanations to answers

Open questions are NOT useful when

need to be coded somehow possibility of

bias introduced difficult to analyse and generalise


21/36

Closed format

Closed format are useful because.

Easy and quick to fill in

Easy to code, record and analyse

Closed Format are not useful because..

need cover all possible responses(comprehensive)

usually mutually exclusive no room ambiguity


22/36

Examples of response categories

Yes/no


23/36


Scales (1-10)

Height/ weight etc


24/36


Frequency, duration, distress


25/36

Format of response categories

Circling or tick boxes

Skip & branch


26/36

Design/adapt data collection

instruments (Types of instrument)

Can use data collection instruments (with already defined

response categories) and just adapt them for your work.

Existing data collection instruments tend to be specific for a

particular health related topic.

For e.g.

- disease specific

- site or region (of the body!) specific

- dimension specific- generic


27/36

Design/adapt data collection

instruments (Types of instrument)

Disease specific

The Gastrointestinal Symptom Rating Scale (GSRS)

The Arthritis Impact Measurement Scales (AIMS)

Hospital Anxiety & Depression Scale (HADS)

(overleaf)


28/36

HADS

(Hospital Anxiety &Depression Scale)


29/36

HADS

(Hospital Anxiety &Depression Scale)


30/36

Visual

analogue scale(EQ5d)


31/36

The ICIQ -Internal

Consultation onIncontinence

Modular

Questionnaire


32/36

Questionnaire completion issues

Under-reporting

Over-reporting (exaggeration)

Social desirability bias

Observer bias (in interview situation only ?)

Memory errors (time frame consideration? Too soon,

cloud judgement? Too long forgotten?) Ill-considered responses (possibly inaccurate on

reflection?)

Skewed/misleadingresponses


33/36

Piloting and Evaluation

A pilot project - Serving as a tentative model for future experiment or development(serving or leading as guide)

(Ref: Teijlingen Van E et al 2001)

Piloting1. Development

& testing ofresearch

instruments

2. Assement ofthe feasibility of

a full scalesurvey

3. Identification oflogistical problems

with the questionnaire


34/36

Pilot and evaluate

Prepare first draft and pre-test

Distribute questionnaire to small sample

Evaluate responses & redraft

Points to check: (adapted from Bowling 2002)

The questions measure what is intended

The wording is understood by all and mean a similar thing to all

Instructions are easily understood/followed

Do the responses suggest all relevant issues have been covered? Any questions systematically/frequently missed etc

Basically test for bugs


35/36

Conclusions

When it comes to questionnaire design

No universal recommendations, just issues

to be aware of, consider and balance out Take into account aims of study, population

and resources available

Principal objective is to collect data that isRELIABLE, VALID & UNBIASED


36/36

References

Oppenheim AN. Questionnaire design, interviewing and attitudemeasurement. 2nd ed. London: Pinter; 1992.

McColl et al (2001). Design and use of questionnaires: a review of bestpractice applicable to surveys of health service staff and patients.

HTA: 5 (31). Bowling A. Research methods in health: Investigating health and

health services, 2nd edition. Open University Press, 2002.

Teijlingen Van E, Rennie AM, Hundley V, Graham W, The importance ofconducting and reporting pilot studies: the example of the ScottishBirths Survey. Journal of Advanced Nursing 2001; 343; 289-295

www.natcen.ac.uk

www.statistics.gov.uk
http://www.natcen.ac.uk/http://www.statistics.gov.uk/http://www.statistics.gov.uk/http://www.natcen.ac.uk/

Documents

Copy of Developing Instrument Data Collection (1) 2 Mei