Using Research to Inform Professional Decision Making in a Context of Risk Brian J Taylor Department of Social Work

Using Research to Inform Using Research to Inform Professional Decision Professional Decision Making in a Context of RiskMaking in a Context of Risk

Brian J Taylor

Department of Social Work

SSRG 09 April 2008 [email protected] 2

Social Work takes place in a context of uncertainty or ‘risk’ Assessing & managing ‘risk’ is a

central feature of practice, involving decision making in situations of uncertainty*

*Skills for Care and Development (2005) “National Occupational Standards for Social Work” London: SCD (paraphrased)


Assessing and Managing Risk What do we do when we “assess risk”? We assess ‘Risk’, BUT what do we do next? What do we mean by “manage risk”? What are ‘reasonable steps to minimise risk’? Is it all about Health & Safety at Work

legislation? What are the issues for organisations,

professions, policy makers, managers, trainers and practitioners?


Organisational assessment of risk

Department of Health (2003) Controls Assurance Standards January 2003 London: Department of Health

Department of Health, Social Services & Public Safety (2003) Risk Management: Core Standard Belfast: DHSSPS

LIKELIHOOD

CONSEQUENCE

1

Insignificant

2Minor

3Moderate

4Major

5Catastro

phic

5Almost certain

Low5

Medium10

Medium15

High20

High25

4Likely

Low4

Medium8

Medium12

High16

High20

3Possible

Very Low3

Low6

Medium9

Medium12

Medium15

2Unlikely

Very low2

Low4

Low6

Medium8

Medium10

1Rare

Very low1

Very low2

Very Low3

Low4

Low5


Organisational management of risk Publicly accountable members of the Board

must have in place systems so that they know what the major risks are in their health and social care organisation Financial Legal Professional care and treatment (statutory duty

of quality)Department of Health, Social Services And Public Safety (2001) Best

Practice – Best Care: A Framework for Setting Standards, Delivering Services and Improving Monitoring and Regulation in the HPSS: A Consultation Paper Belfast: DHSSPS


Professional management of risk Accountability of professions for managing

risk as part of corporate risk management “… focus on safety, accountability and

improvement of services”* “… culture of self-evaluation and

continuous improvement.”*Department of Health, Social Services and Public Safety (2002)

Departmental Guidance HSS(PPM) 10/2002 Governance in the HPSS – Clinical and Social Care Governance: Guidelines for Implementation Belfast: DHSSPS

*Simmons L (2007) Social Care Governance Workbook Belfast: DHSSPS and London: SCIE for Clinical and Social Care Governance Support Team


By now we are all familiar with undertaking a ‘risk assessment’ on activities – even Dilbert is gradually getting used to the idea!

copyright©1997 United Feature Syndicate, Inc. Reproduced by permission


How meaningful is this risk assessment?

What value does it have?

How does risk assessment fit with professional values, client values, & the professional task?

copyright©1997 United Feature Syndicate, Inc. Reproduced by permission


Do we all see “risks” the same way ?!

copyright©1997

United F

eature S

yndicate, Inc. R

eproduced by perm

ission


Three examples of using research in relation to risk and decision making Qualitative research

in work with Older People Experimental research

in Mental Health ‘Theoretical research’

in Family and Child Care


EXAMPLE 1: Professional Perspectives on Risk and Decision Making To make organisational &

professional approaches to assessing & managing risk effective, we need to understand how practitioners ‘do the risk business’

What tools and concepts do professionals use to ‘do risk’?


Perceptions of health & social care staff on decision-making on long-term care of older people Grounded theory approach Sample: 4 of 11 HSS Trusts in NI that

deliver community health & social services one in each Board area (policy effects) at least one providing only community

health & social services at least one ‘integrated Trust’ (with acute

services)(also a proxy for rural and urban)


Data gathering method & sample Focus groups with social workers,

community nurses, occupational therapists, hospital discharge support teams, home care managers

Interviews with consultant geriatricians, general medical practitioners, older people (with their carers, where appropriate)

99 staff in total; 9 older people


Identified Risk Factors for Admission to Institutional Care (fits the evidence!) mental impairment

including dementia falls & fractures ADL limitations managing medication incontinence health-related needs sleep problems nutrition


What evidence do you use in assessment & decision making? We only predict from our experience and

knowledge (Care Manager) We know the number of people who end up

in institutional care after a fracture … most of that [predictive data on the probability of harm] would be helpful (Geriatrician)

BUT generally little explicit use of research or synthesised data in relation to individuals


Use of theory (based on evidence?) (1) I don’t think we have put <risk> on the

training agenda explicitly (Geriatrician) There is no tick chart or formula that I

work to. It’s just what is staring me in the face when I go out there (GP)

I’m used with the Brearley model to make my mind focus on issues … You know when people are shouting at you “they are at risk out there” (SW)


Building A Model of Admission Decisions Some findings led to a new conceptualisation of decision making about admission of an older person to a residential or nursing home

We illustrate using one factor >


Rated very seriously

And then there is so much robbery now and beating old people up. If she <elderly aunt> falls she would always get better again but if somebody came in and beat her up or robbed her, she would never get over it. It would kill her. (Family carer interviewed with her aunt, an older woman living at home against professional advice)


Client fears – even refusing services Everybody knows who you are. I’ve got my pension and I suppose they know how much you are worth every week and are liable to break in. Sure it’s happening all the time. (Older Man)

She needed a handrail at the front door but <refused it> because there were so many break-ins recently she thought that it might make her look more vulnerable if she had a rail. (Occupational Therapist)


But not regarded as a legitimate or effective reason for admission Researcher: Is this a factor in admission to

institutional care, this vulnerability to intruders or burglary that you mentioned?

Social Worker: It would not be of course officially in an assessment

This and other elements led to a new model:

Taylor BJ & Donnelly M (2006) ‘Professional perspectives on decision making about the long-term care of older people’ British Journal of Social Work 36 (5): 807-26.


Elements of Admission Process

Person living alone loses confidence

Person living with family carer who is no longer able or

willing to meetneeds

Professionals assessing risk considering law, policy, theory, etc

Availabilityof servicesincl, funding & people

APPLYING THIS EVIDENCE IN:Development of a Single Assessment Tool for the Health and Social Care Needs of Older People – Jan 2006 to Jun 2008 (DHSSPS)http://www.dhsspsni.gov.uk/ec-single-assessment-tool

PROJECT TEAM – University of UlsterProfessor Brendan McCormack, Institute of Nursing ResearchDr Brian Taylor, Social WorkMrs Bridget Murray, Rehabilitation SciencesMrs Joanne McConville, Project OfficerDr Paul Slater, Research Fellow

http://www.science.ulster.ac.uk/sat/


Elements of the Tool – developed with 350+ stakeholders Contact Screening Core Assessment >1 GP and Medical Practitioner Report Specialist Referral Specialist Summary & Recommendations Carers’ Assessment Complex Assessment >2


Core Assessment – informed by a range of health & social care research evidence1. Physical Health2. Mental Health & Emotional Well-being3. Awareness & Decision Making4. Medication Management5. Communication & Sensory Functioning6. Walking & Movement7. Personal Care & Daily Tasks8. Living Arrangements & Accommodation9. Relationships10. Work, Finance & Leisure


Complex Assessment Focus on risks to independent living in

relation to each domain in Core Component of the Tool Recent events, crises and their impact on the older

person, including their confidence to live at home Disruption or distress due to identified needs

(concerns, risks) ability to cope with needs rehabilitation and learning potential health promotion possibilities additional care or support required

How do identified difficulties impact on ability to make choices and decisions about daily living?


Example 2: Applying experimental evidence (knowledge) to manage uncertainty

Basic ethical principle ‘primum non nocere’ = first, do no harm ““Counselling and psychotherapy* is probably pretty Counselling and psychotherapy* is probably pretty

useless and therefore mostly harmless” useless and therefore mostly harmless”

*‘psychotherapy’ = “A generic term for the treatment of mental illness or emotional disturbances primarily by verbal or nonverbal communication” [Medline database] ie includes art therapy, drama therapy as well as most varieties of ‘counselling’

BUT let’s look at an example of emerging evidence:


5

10

15

20

25

30

35

40

Baseline 4 months 3 years

Imp

ac

t o

f E

ve

nt

Sc

ale

Debrief (Hi) Control (HI)

Critical Incident Stress Debriefing Study

Richard Mayou et al, 2000


Risks in inappropriate counselling? MAY make people worse than without treatment In depression – behavioural activation is more

effective than just talking about problems After the Omagh bomb in Aug 1998 the local Health

and Social Care Trust activated its emergency plan & set up a Community Trauma and Recovery Team (CTRT)

Of people coming because of Post Traumatic Stress Disorder (PTSD) 50% had previous unsuccessful psychological therapy or counselling which generally added to their sense of hopelessness

WHY MIGHT THIS BE? (theory)


Omagh bombing study - factors most highly associated with Chronic PTSD Injured .38 Beliefs: ‘Unsafe World’ .38 In Market Street .40 Thought would die .42 Thought /Emotion Suppression .43 Rumination .54 Negative View of Symptoms/Self .72

Duffy M, Gillespie K & Clark DM (2007) ‘Posttraumatic Stress Disorder in the Context of Terrorism and Other Civil Conflict in Northern Ireland: A Randomized Controlled Treatment Trial’ British Medical Journal 334(7604):1121-2.


What will help? How do we know? The Omagh CTRTeam (included Social Work,

Psychiatric Nursing, Psychology, Psychiatry, Occupational Therapy) had to face the question:

What helping approach shall we use? How will we know if it is effective? The team leader Michael Duffy (SW) and colleagues

developed a randomised controlled trial of the effectiveness of Cognitive Behavioural Therapy (the main treatment used) for the treatment of Post Traumatic Stress Disorder in the context of civil conflict – a previously untested application of CBT


CBT for common problems recovery

rateMajor depressive disorder 50%Panic disorder 75%Posttraumatic stress disorder 80%Social phobia 75%Generalised anxiety disorder 50%Obsessive compulsive disorder 45%Specific phobias 75%

Recovery rate is the approximate proportion of individuals who do not have the problem at the end of intervention


The end of psychosocial treatments? ABSOLUTELY NOT!! BUT We need to note & appraise the evidence which is much stronger for STRUCTURED counselling models E.g. “…problem-solving therapy, interpersonal

psychotherapy, and pharmacotherapy would be considered efficacious interventions for major depression, with cognitive-behavioral and cognitive therapy considered possibly efficacious.”

Wolf NJ & Honko DR (2008) ‘Psychosocial and pharmacological interventions for depressed adults in primary care: A critical review’ Clinical Psychology Review, 28(1):131-61


Report by Lord Richard Layard

Mental illness is the UK’s greatest social problem - £3 million to be set aside for training in evidence based psychotherapies 2008-2010 (in England)

Layard R (2004) ‘Mental Health: Britain’s Biggest Social Problem?’ Paper presented to Government Strategy Unit, London


Social Work & Psychosocial Interventions If we in Social Work lose our role in

relation to psychosocial interventions it will be because of our entrenched

attitudes and systems not because such treatments do not work

or could not be shown to work or because of a lack of public funding

cf Layard R (2006) ‘The case for psychological treatment centres’ British Medical Journal 332 (7548) 1030-2


Using Evidence:

in commissioning new services

in commissioning training

in decisions about

individuals


Applying evidence to decisions about care of individuals Going back 5 slides to the one on recovery

using CBT CBT was generally shown to be effective for

a range of conditions, BUT Between 20% and 55% of people were NOT

helped sufficiently to no longer ‘have that problem’

What approach for THIS individual? Can we refine our decision making?


Example 3: Slated by (ab)use of research?!

“Now tell me Miss Social Worker, in relation to my client Mrs Whitehouse and her beloved child Amy, have you read this piece of research by Sniff and Jones about successful treatment of addictions …?”


Research Evidence for Court Decisions What value is one research study? How will we judge the quality of a study? What value is a theory?

E.g. attachment and bonding; stigma; loss; rehabilitation; nurturing environment; discipline

Can we put more weight on a synthesis of studies than a single study?

What criteria do we set for a good synthesis? How does this study apply to Mrs Whitehouse? How will we train Social Workers for this work?


Research-based reasoning in court?

“Effective clinical reasoning requires skill in developing arguments, establishing the relevance of information to an argument, and evaluating the plausibility of assertions or claims…” Osmo R & Landau R (2001) ‘The need for explicit argumentation in ethical decision–making in social work’ Social Work Education 20(4) 483-492 (page 489)

“Expressions of opinion must be supported by detailed evidence and articulated reasoning” (Munby J in Re M (Care Proceedings: Judicial Review) (2003) 2FLR 171 p.183)

“Mrs R’s Article 8 rights required that her child should not be taken from her unless every feasible alternative was thoroughly explored and rejected for good reason.” (AR v Homefirst Community Health and Social Services Trust (2005) p.18 para 101)


One model of explicit argumentation1. CLAIM Make a claim or conclusion

2. WARRANT Explain relationship between claim & data

3. DATA Provide grounds, data & evidence

4. JUSTIFY relationship between evidence and claim

5. QUALIFY the claim (degree of confidence)

6. Highlight LIMITATIONS & reservations to the claim (including conditions where it might be weakened)

7. JUSTIFY (2) with additional evidence in relation to qualifiers and limitations

Stephen Edelston Toulmin (1958) ‘The Uses of Argument’ Cambridge: Cambridge University Press (adapted)


A Child Care example Ms G has three children who are in care –

problems of alcohol abuse & neglect Child A (12) & Child B (14) are with different

foster families, Child C (15) is in residential care

Ms G has been erratic regarding contact and there have been problems of aggression

Decision required about contact – considered detrimental to the children


Evidence in child protection (1)?1. CLAIM - Rights under Article 8 HRA should

be over-ridden in the interests of the children2. WARRANT – outline issues & relationship of

these To whose benefit is continued contact? Emotional well-being of children: identity, distress,

future prospects, stability Impact of Ms G’s addiction on children

3. DATA on visits, views of children etc, & research evidence related to these issues to inform professional judgment

4. JUSTIFY connection: evidence and concerns


Evidence in child protection (2)?5. QUALIFY – research relating to severity &

likelihood of harm and benefit for continuing contact and for ceasing it

6. LIMITATIONS – different situation of each child, possible change in problematic maternal behaviour

7. JUSTIFY (2) – research on resilience, capacity to change once addicted etc


What is the research process to test & bring this model to practice, if it proves suitable at each stage?1. Consider application of theoretical model

2. Apply to historical or hypothetical cases?

3. Seek professional views?

4. Pilot studies?

5. Controlled trial of effectiveness, usability & acceptability?!


Key Issue: Prediction of Harm Is it ‘safe’ to return this child home?

Has this parent recovered sufficiently from mental illness or addiction?

Is it ‘safe’ to discharge this person from psychiatric hospital in terms of possible violence or suicide?

Is it ‘safe’ for elderly Mrs Brown to return home from hospital after her stroke?

Is it ‘safe’ for ‘us’ to support this step towards independent living of young Mr O’Kane with a disability?


Predicting (Screening for) ‘problems’ Suicide rate for Northern Ireland 2006

227 male; 64 female Treat as approximately 300 in a population

of about 1.5 million i.e. 2 in 10,000

Could we use a screening tool to identify these people so as to target services?


Predicting rare events If we had a test that was 90% accurate and we are trying to predict a rare event such

as abuse, violence or suicide that occurs say 2 in 10, 000 people per year (as in this example)

amongst e.g. a population of 100,000 people EXERCISE: IDENTIFY ISSUES IN USING

THE TOOL BY CALCULATING HOW MANY ARE CORRECTLY AND INCORRECTLY IDENTIFIED


Effectiveness of a tool to predict harmTOOL: yes no

REALITY

yes <1> <2>

TP FN

No <4> <3>

FP TN


Predicting rare events

The real incidence of abuse or suicide or other risk would be 20 cases per year in the population of 100,000 [2 in 10,000], and so

there would be 99,980 cases that year where this harm did not occur.

[total = 100,000]


False Negatives

With a 90% accurate tool, we would detect

18 of the 20 cases of harm correctly (item 1, TP),

but would miss the other 2 “false negatives” (2, FN) because the test wrongly said there was no risk


False Positives

OF the 99,980 cases that were not a risk, the test would correctly show 89,982

[90%] as being no risk situations (3, TN) but would incorrectly pick up 9,998 (10%)

of the no-risk situations as being risky “false positives” (4, FP) because the test incorrectly labels them as risky when they are not.


Issues in using a tool to predict harmTOOL: yes no total

REALITY

yes 1818 22 20

No 9,9989,998 89,98289,982 99,980

Total 10,016 89,984100,000


Using a 90% accurate tool to predict NI suicides (pop 1.5M)TOOL: yes no total

REALITY

yes 270 (TP) 30 (FN) 300

No 149,970 1,349,730 1,499,700

(FP) (TN)

Total 150,240 1,349,760 1,500,000


Problem of False Positives

Trying to predict rare events such as abuse, violence or suicide in a much larger population presents particular problems

Many non-risky situations will be identified as being “risky” with consequences for individuals and for workloads of professions, teams & organisations

How can we know for certain whether or not this individual is the one where the harm will occur?

IMPOSSIBLE!


The Best Prediction Tools The very best prediction tools in social work

and criminal justice are achieving rather lower accuracy than the example above (perhaps 70% - 80%)

e.g. predicting the likelihood of a convicted offender being re-convicted by committing a similar type of serious crime

False positives & false negatives are an even greater problem with a less accurate tool


“A Continuum of Risk” “Research therefore cautions us that [in

mental health] as in other fields such as medicine and child protection there is no such thing as a ‘risk free’ assessment….

… There are no criteria which enable us to place individuals into sharply-defined, once-and-for all categories of ‘dangerous’ or ‘not dangerous’. Rather there is a continuum of statistical risk with uncomfortably limited predictive capacity.”

Perry, J and Sheldon, B (1995) Richard Phillips Inquiry Report, London: City of Westminster, and Kensington & Chelsea & Westminster District Health Authority (p.18)


Prospects for Actuarial Risk Assessment Tools continually improving for predicting probability

of harm esp. criminal justice & mental health Actuarial prediction often discriminates better

between risk levels than clinical judgment Might inform priority for services Recognise limitations regarding predicting harm Take a broader view of ‘assessment’ Accept more diffuse approaches to targeting

preventive measures E.g. postcode vs identifying individual ‘children in need’

Study how we make professional decisions


Conclusions (1) To make organisational & professional

approaches to assessing & managing risk effective, we need to understand how practitioners ‘do the risk business’

Assessment tools to support decision making can be valuably informed by research and can support professional clinical judgment but we cannot predict rare events (harm) accurately

Be confident in our SW skills in psycho-social interventions – but we need to focus on treatments based on robust evidence


Conclusions (2) Evidence can be applied more readily to

commissioning services and training than to decisions about care of individuals

We need a ‘network’: to energise research on risk, evidence and

decision-making in Social Work to build theoretical models of decision making To develop models for bringing research from

theoretical concepts through empirical studies and into practice

Using Research to Inform Using Research to Inform Professional Decision Professional Decision Making in a Context of RiskMaking in a Context of Risk

THE END

[email protected]

Documents

Using Research to Inform Professional Decision Making in a Context of Risk Brian J Taylor Department of Social Work