Randomised Trials in UK Social Work: A Discussion of Possible Approaches

Donald ForresterProfessor of Social Work Research

Director of the Tilda Goldberg Centre

Proposed RCT

• Early stages of development (2012-14)

• Want to openly discuss issues and challenges– Not got all answers and would rather have errors pointed

out now...

• Focus: whether Motivational Interviewing improves engagement of parents who misuse drugs or alcohol

• But first a bit of history, theory and background

Social Work’s Evidence Base: Emperor’s New Clothes?

RCTs are one of the greatest intellectual

achievements of the 20th century

What is an “RCT”?

What are advantages?

• The best way of ruling out bias• Bias defined as explanations other than the

intervention that might explain results

• Key types of bias:– Self selection bias– People are active and resolve problems– Social work is about maintenance not cure...– Self-report not good indicator of impact – people usually

appreciate services

History of RCTs

• First comparative trial – in the Bible. Daniel compares vegetarian diet with meat and drink.

• RCTs in education and social work 1930s

• Medicine from the late 1940s

• Now dominant approach in medicine

Selective history of RCTs

Cambridge and Sommerville Study• USA 1930s• Intensive, long-term befriending from social

worker for boys considered at risk of offending• Much appreciated by young men• Increased offendingFrom Oakley, 2002

Social work and RCTs

• Dominant social work approach in 1960s, psycho-dynamic casework

• Reid and Shyne tested and found:– Long-term work LESS effective than short term– Developed task-centred approaches based on

approach

Social work and RCTs

• Homebuilders Intensive Family Preservation Service

• Aimed at preventing children entering care• Success rates from 70 – 100%• Large-scale RCT – no impact

Field social work RCTs in the UK 1970-2010

Problems with RCTs

Bias

ContextGeneralisability

ContextPawson and Tilley – Realist critique – context crucial for

understanding nature of intervention and its impact

Orford (2008) from within the RCT field – reflecting on the “Dodo Bird” effect

• Whatever is happening across interventions more important than differences between interventions

• Context of service, client or research

• Requires emphasis on understanding context, process as well as outcomes

Generalisability• RCTs on drugs establish general knowledge• This isn’t true for psychosocial interventions:– Implementation fidelity and innovation effects– What the control group receives eg “nothing works

in Sweden…”– Or Dodo effects – everything works equally…

• This is a fair criticism of naïve positivism• But ALL research makes claims to generalisability – and

all should be considered critically

Outcomes and social work: the politics of research

Client’s View

Expert View

Society’s View

Reimagining RCTs• Starting point to develop a critical form of

RCTs that addresses these concerns – a characteristically and perhaps distinctively social work approach to RCTs

• Many of problems with RCTs are of RCTs as done not as should or could be

• So how should we do RCTs...

Early social work RCTs

• Reid and Shyne (1969) – brief vs open-ended social work with middle class couples with relationship problems– Randomization purely on length of intervention– Taped interviews and found this led to more focussed

approach– Extensive outline of context of intervention– Both women and (particularly) men preferred this– Social work, service user and standardised measures of

outcome all agreed ST work produced better outcomes at 12 months

Goldberg (1970): Helping the Aged

• Randomised older people to “normal service” (with unqualified workers) or service from qualified social workers

• Detailed description of needs, services and outcomes and combination of qualitative and quantitative methods

• High levels of need, improvements in both groups

• Major problem: the “intervention” differed in multiple ways

Evaluating Helping the Aged

• The descriptive work was fantastic• The research is transparent and self-critical

• Contributed to the programme of work that culminated in “Community Care”

• Should have been the start for RCTs in social work...

Field social work RCTs in the UK 1970-2010

Not the research questions

• Can we show that any of the approaches taught to social workers makes a positive difference – compared to “normal practice”?

• Can we do this in a “normal” local authority setting? (chosen child and family work)

• Can we make an RCT “work” in social work?

Key Considerations: Outcomes

• What outcome/s to measure?

• Legal basis (Children Act 1989)– Protection from harm– Meeting needs

• Every Child Matters outcomes (2004)– Health– Safety (eg from abuse)– Education– Making positive contribution– Achieving economic wellbeing

Key Considerations: Outcomes

• Complex factors influencing outcome/s– Social situation (eg poverty, racism etc)– Personal problems of parents (eg addiction)– Difficulties in child (eg challenging behaviour)– Patterns of family functioning and parenting

• Complexity of social work “intervention”– Direct work– Interagency linkage– Risk assessment and decisions– Not voluntary clients seeking help

Key Considerations: Outcomes

Single biggest problem likely to be family participation rate in research

Commonly as low as 5% - we need 60-70%Or a good proxy...

Key Considerations: Intervention

• Chose Motivational Interviewing as:– Strong evidence base in substance misuse field– Very flexible style of working– General style of communication aimed at engaging

resistant clients– Consistent with social work values– Initial piloting study undertaken – which

highlighted challenges

Simple Overview of Intervention Process

Improved Communicatio

n Skills

Increased Engagement

Positive Impact on Parent and

Family

Child Protected and

Thriving

Piloting approach to outcomesImprove

Engagement of Parents

•Tapes– parent response (MISC and bespoke) •Parent views of SW (simple ratings)•Number and length of visits (worker and parent)•Engagement with other services (worker and parent)

Improve Parent and Family Wellbeing

•Reduced family stress (PSI)•Reduced alcohol and drug use (MAP)•Reduced parental stress (GHQ)

Protect child and improve welfare

•Whether abused/neglected (problematic) – using parent and social worker measures and one SI•Child’s emotional and behavioural welfare (SDQ)•NOT expecting to show an impact...?

Current Pilot

• Combining small-scale RCT of MI with quasi-experimental study of “Hackney Model”

• 40 workers in 10 teams/units in 2 contrasting LAs receive intensive MI input

• 40 workers do not

• Piloting MI input, types of data collection and success of different approaches to families

Elements of Data Collection Piloted• Direct tapes of practice

– Measured for MI-ness – Social work skill ratings being developed with group of experts by experience through

iterative process

• Brief questionnaires to all families allocated– With follow-up from administrators

• Research interviews with families using various standardised instruments– Key problem likely to be low response rate

• Social worker questionnaires on families – Key problem validity – To be tested by comparing with research intvws

• Exploring using LA data measures

Concluding Thoughts

An RCT is the most robust test of effectiveness, because it minimises bias

Social work needs to engage with the complexity of RCTs in order to evaluate whether what we do works

A critical social work perspective is also likely to create better RCTs

Concluding Thoughts

Developing RCTs fit for use in social work settings is not straightforward or easy

Doing so will not provide all of the answers

However, hopefully it will provide some evidence about what skilful social work is and the difference it can make...

“If we knew what it was we were doing, it would not be called research, would it?”

Einstein

ReferencesForrester, D. (2010) Playing with fire or rediscovering fire? The perils and Potential for Evidence

Based Practice in Child and Family Social Work”, in Ayre, P. and Preston-Shoot, M. (Eds) Children’s Services at the Crossroads. A Critical Evaluation of Contemporary Policy for Practice, Russell House Publishing; Dorset

Goldberg, E.M., with Mortimer, A. and Williams, B.T. (1970) Helping The Aged—A Field Experiment In Social Work National Institute for Social Work Training, No. 19, George Allen & Unwin: London. 1970.

Holosko, M. J. (2010) “What Types of Designs are We Using in Social Work Research and Evaluation?”Research on Social Work Practice; Nov2010, Vol. 20 Issue 6, p665-673

Oakley, A. (2000) Experiments in Knowing: Gender and Method in the Social Sciences, Polity PressOrford, J. (2008) “Asking the right questions in the right way: the need for a shift in research on

psychological treatments for addiction”, Addiction, Volume 103, Issue 6, pages 875–885, June 2008

Reid, W.J. and Shyne, A.W. (1969) Brief and Extended Casework, Columbia University PressTorgerson, D.J. and Torgerson, C.J. (2008) Designing Randomised Trials in Health, Education and

the Social Sciences, An Introduction, Palgrave Macmillan; Basingstoke