Upload
erick-berry
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
Dr Sally Boa and Dr Joan Murphy
Professor Pam Enderby
Funded by NHS Education Scotland
Conducted by Talking Mats Limited
© Talking Mats Ltd 2014
Background
A Right to Speak (Scottish Government 2012)1. AAC services to demonstrate the effectiveness of AAC interventions.
2. National statistics on AAC to be gathered by relevant agencies to support future gathering of cost effectiveness data on AAC to ensure that AAC funding is sustained in the longer term.
© Talking Mats Ltd 2014
Two stage project
1. Scoping of current methods of measuring outcomes
2. Testing the validity and inter-rater reliability of TOM AAC (Enderby et al 2006)
© Talking Mats Ltd 2014
Main points arising from scoping project
• Agreed on what we mean by ‘Outcome Measure’
• Agreed on what is the best measure to use in AAC
• Identified the need for a measure to be workable across agencies
© Talking Mats Ltd 2014
Definition of an outcome measure
Outcome Measures:Should measure changeCan be repeated over timeCan be used across clients, settings and disciplinesCan be used to give feedback to services, professionals, carers and clients (about what works as well as what doesn’t)Should be robust/standardised/well respectedShould inform dischargeShould be short and straightforward to useShould allow for comparison with different types of AAC/No AAC
© Talking Mats Ltd 2014
Main contender for outcome measure in AAC
• TOMs AAC• (still in development – needed to be tested for
reliability and validity)
© Talking Mats Ltd 2014
Stage 2
• Testing TOM AAC for validity and inter-rater reliability
© Talking Mats Ltd 2014
Therapy Outcome Measure (TOM)• An Outcome Measure used by many rehabilitation
professionals to measure the impact of their interventions
• Based on the World Health Organisation’s International Classification of Functioning, Disability and Health (WHO ICF, 2001)
• Administered following assessment/intervention of an individual by a professional
• Individuals are rated using an 11 point ordinal scale with 6 defined points. People are rated in relation to four descriptors: Impairment, Activity, Participation and Well-being.
© Talking Mats Ltd 2014
Normal limits0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
MildProfound Severe Severe/ Moderate Moderate
Therapy Outcome Measure
Impairment/ Body
Disability/ Activity Participation Well-being
Carer Well-being Date
AdmissionIntermediateFinal
Reason for discharge
© Talking Mats Ltd 2014
TOM AAC
• Adapted scale for use with an AAC population (children and adults)
• Domains of activity and participation can be rated with and without AAC
© Talking Mats Ltd 2014
Recruitment of AAC professionals(n = 29)
Profession Numbers
SLT 16
Teacher 7
Rehab engineer/technologist 2
Social work 1
OT 1
Physio 1
Nurse 1
© Talking Mats Ltd 2014
Validity testing
• Training provided• Each professional was asked to rate 10 AAC
users using TOM AAC• Feedback provided through:– Online survey– Focus groups
© Talking Mats Ltd 2014
Findings from the survey
Coherence and relevance of the descriptors•Strong agreement that the descriptors in TOM AAC were appropriate and relevant for use with an AAC population•Respondents reported a good understanding of the difference between impairment, activity, participation and well-being
Usability of the measure in practice•Most agreed that the measure was quick and easy to use •Most agreed that the TOM AAC would help them to report on outcomes on their work with individuals who use AAC
© Talking Mats Ltd 2014
Findings from Focus Groups
• TOM AAC was straight forward to use– “I think it’s really useful. I think it’s a really quick,
useful tool.” (Participant from focus group 3)
• Comments about wording of the impairment sections - a distinction between ‘expression’ and ‘comprehension’ rather than ‘speech’ and ‘language’
© Talking Mats Ltd 2014
Strong face validity – changes made to descriptors of TOM AAC
• Wording changed to ensure consistency across the descriptors
• Within the impairment section, distinction was made between ‘expression’ and ‘comprehension’ rather than ‘speech’ and ‘language’.
• More detailed explanation was also provided for each descriptor to increase the clarity of meaning for each section
© Talking Mats Ltd 2014
Inter-rater reliability testing
• AAC professionals (n = 17) provided with ten case histories to rate using TOM AAC
• Scores collated and analysed using Intraclass Correlation Coefficient (ICC)
© Talking Mats Ltd 2014
Findings
• Agreement was generally acceptable–highest for physical impairment (0.887),
comprehension (0.831) and wellbeing (0.816)
– the lowest was for expression (0.486)
© Talking Mats Ltd 2014
Inter-rater reliabilityTOM domain ICC
Physical Impairment 0.887
Cognitive Impairment 0.726
Sensory Impairment 0.605
Expression 0.486
Comprehension 0.831
Activity 0.622
Participation 0.703
Wellbeing 0.816
© Talking M
ats Ltd
Limitations
• We may not have provided sufficient information in case histories five and six, as ratings for these case histories gave the largest range between individuals.
• The domain of ‘expression’ appeared to produce the most variation between raters – need for clearer definition of descriptors
• Further investigation needed around ‘expression’
© Talking Mats Ltd 2014
With adaptations
© Talking Mats Ltd 2014
Greig
© Talking Mats Ltd 2014
Example
© Talking Mats Ltd 2014
Summary
• TOM AAC has been tested for face validity and inter-rater reliability
• TOM AAC has the potential to provide a quick, simple and reliable for all AAC professionals
• TOM next edition will include AAC
© Talking Mats Ltd 2014
ReferencesCommunication Matters (2012) http://www.communicationmatters.org.uk/sites/default/files/downloads/standards/aac_outcome_measurement_sept_2012.pdf
Framework for Measuring Impact http://www.measuringimpact.org/
Scottish Government (2012) ‘A Right to Speak’http://www.scotland.gov.uk/Publications/2012/06/8416/0
Lacey, A. & Luff, D. (2001). Trent focus for research and development in primary healthcare: An introduction to qualitative analysis. London: Trent Focus.
Khangura et al 2012 Evidence summaries: the evolution of a rapid review approachhttp://www.systematicreviewsjournal.com/content/1/1/10
TOMS: Enderby P., John A., & Petheram B. (2006) Therapy Outcome Measures forRehabilitation Professionals: Speech and Language Therapy, Physiotherapy,Occupational Therapy, Rehabilitation Nursing & Hearing Therapists. Second Edition, JohnWiley & Sons Ltd.
CODES: http://keycomm.weebly.com/codes-framework.html
Shrout, PE, Fleiss, JL. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin 1979; 86(2): 420-428
© Talking Mats Ltd 2014