Non-strategic ignorance: considering the potential for a ...repository.essex.ac.uk/22691/1/Strategic ignorance Health pre-print Jul2018.pdfFelicitas Rost2 [email protected] Sukhjit

1

Health (2018). In press

Non-strategic ignorance: considering the potential for a paradigm shift in evidence

based mental health

Susan McPherson1 (Corresponding Author)

[email protected]

Felicitas Rost2

[email protected]

Sukhjit Sidhu1

[email protected]

Maxine Dennis2

[email protected]

1 School of Health and Social Care, University of Essex, Colchester CO4 3SQ, UK

2 Tavistock & Portman NHS Foundation Trust, 120 Belsize Lane, London NW3 5BA, UK

Abstract

Randomised Controlled Trials (RCTs) form a central building block within the prevailing

Evidence Based Mental Health (EBMH) paradigm. Both methodology and paradigm have

been widely problematized since their emergence in the mid-late twentieth century. We draw

on the concept of ‘strategic ignorance’ to understand why the paradigm still prevails. We

present focus group data gathered from 37 participants (service users, public, carers, GPs,

commissioners) concerning the way they made sense of an RCT of psychotherapy for

treatment resistant depression. Thematic analysis of the findings revealed an overall critique

of RCT methods which we refer to as ‘non-strategic ignorance’. Specifically, participants

problematized the construct of depression, unseating the premise of the RCT; they were

sceptical about the purpose and highlighted its failure to show how therapy works or who

might benefit; the RCT was seen as inadequate for informing decisions about how to select a

therapy. Participants assumed the treatment would be cost effective given the client group

mailto:[email protected]:[email protected]:[email protected]:[email protected]

2


and nature of the therapy, irrespective of any RCT findings. Each area of lay (‘non-strategic’)

critique has an analogous form within the methodological expert domain. We argue that

‘expert’ critiques have generally failed to have paradigmatic impact because they represent

strategic ignorance. Yet parallel non-strategic critiques have common sense appeal,

highlighting the potential power of lay voices. The discussion considers whether the EBMH

paradigm is faced with epistemological problems of such complexity that the conditions exist

for a new paradigm in which service user views are central and RCTs peripheral.

Keywords: RCTs; depression; psychoanalytic psychotherapy; patient and public

involvement; evidence based mental health

3


Introduction

This article presents a paradigmatic critique of Evidence Based Mental Health (EBMH),

specifically its veneration of Randomised Controlled Trials (RCTs). Yet, rather than review

the many expert methodological critiques abundant in existing literature, the thoughts of non-

researchers (service users, carers, members of the public, GPs and commissioners) were

sought, specifically in response to a recent RCT of psychoanalytic psychotherapy for

treatment resistant depression. First, we consider the historical background to EBMH using

the concept of ‘strategic ignorance’ as a lens through which to understand its persistence in

face of widespread epistemological critique. We then present empirical data to highlight the

potential strength of a paradigm critique derived from an alternative position of ‘non-strategic

ignorance’, which has common sense appeal. The analysis of this data provides a forerunner

to a discussion, using Kuhn’s concept of scientific revolutions, on whether the paradigm

could shift towards one in which service users are central, rather than an epistemological

problem to be solved.

A Brief History of EBMH

The concepts incorporated within Evidence Based Medicine (EBM) are generally attributed

to Bradford-Hill, a British statistician involved in the first RCT in 1948; Cochrane, a British

doctor who wrote a pamphlet on clinical effectiveness in 1972; and Sackett, a Canadian

doctor who coined the term EBM in the 1970s (Devereaux & Yusuf, 2003). Healthcare

reforms in the UK enhancing the paradigmatic status of EBM spanned Conservative and

Labour governments: ‘Working for Patients’ (1989), ‘Promoting Clinical Effectiveness’

(1996), ‘A First Class Service’ (1998), ‘The NHS Plan’ (2000) and guidelines produced by

the National Institute of Clinical Effectiveness (NICE) from 1998 onwards. Similar reforms

occurred in the USA and Canada over the same period.

4


EBMH followed suit from the 1950s, hand-in-hand with psychotropic drug research. EBMH

rose to paradigm status around the late 1990s when SSRIs dominated the anti-depressant

market. According to Rees, past-president of the UK Royal College of Psychiatrists, the

pharmaceutical industry learned RCT methods from psychiatry and adopted them to get new

drugs passed by the Committee for Safety of Drugs (Rees & Healey, 1997).

The political agenda central to EBMH was evident in early accounts of its development by its

proponents. Paris (2000) drew a direct contrast between EBMH and psychoanalysis,

describing 1950s and 1960s psychiatry as dominated by psychoanalysis, opinion and

authority which he pejoratively termed ‘archaic’. Paris praised the shift in types of articles

published in the Canadian Journal of Psychiatry between 1950 and 2000, which moved from

‘clinical inference’ to EBMH, depicting EBMH as an advancement of science and a rational,

logical step in psychiatry. This celebratory tone among early proponents fits with Wieringa

et al’s (2017) account of EBM as a modernist movement, ‘able to provide “facts” of biology

and epidemiology, as far as possible devoid of human intuition or experience’. Paris set out

psychopharmacology and genetics as developments of reason and logic, bringing psychiatry

to be equal with other medical specialties. Similarly, Joyce (2002) invoked Popper to

construct EBMH as a powerful philosophical tool for the advancement of psychiatry.

Nevertheless, early critics were acutely aware of the permissive influence of the

pharmaceutical industry on the development of evidence and knowledge and on clinical and

academic freedom and enterprise. Harari (2001) noted that EBMH was based on

empiricism, the notion that knowledge can be built by making observations of the world and

organising observations into theories. Harari traced the history of the philosophy of science

through Husserl’s phenomenology, Galileo’s observations of gravity and Einstein’s

5


epistemological concerns to illustrate that observations in any field of enquiry cannot be

neutral nor direct the observer to the truth because of the many theories the observer may

already assume to be true before making the observation. An empiricist EBMH was

therefore a threat to ‘epistemic pluralism’ in psychiatry.

Specifically critiquing RCTs, Healy (1999) argued that they formed part of a ‘sanctioned

narrative’ employed by psychiatry to evaluate and promote pharmacotherapy, forcing

psychotherapies to adopt the same evaluative methods. He challenged the autocracy of this

narrative arguing that clinical trials in mental health used ‘suspect’ observer rating scales and

convenience samples for marketing needs. Dixon and Goldmann (2004) noted loss of

‘fidelity’ as intervention programmes began to drift away from the original trial model and

varied outcomes began to emerge, highlighting the question of whether it is possible to

implement treatments in real clinical practice that RCTs have found to be effective.

These and other critiques have been rehearsed many times in many fora since. The paradigm

has been debunked from a range of disciplinary perspectives including anthropology

(Kleinman, 1995) and philosophy (Devish & Murray, 2009). Yet RCTs remain central to

EBMH and the same serious concerns continue to be raised today. Shedler (2017), for

example, describes new American Psychiatric Association trauma guidelines as ‘harmful’

and ‘unethical’ because of their reliance on RCTs. He notes:

Copernicus, Galileo, Darwin, Einstein, Niels Bohr, Marie Curie, Stephen Hawking.

What do they have in common? None of them ever conducted an RCT. Most scientific

knowledge does not come from RCTs.

6


Strategic Ignorance in EBMH

We now consider the concept ‘strategic ignorance’ to understand the persistence of EBMH in

the face of enduring critiques. Informing the development of a sociology of ignorance,

McGoey (2010) analysed the debates which emerged when Kirsch (2008) published a meta-

analysis of antidepressant trials following freedom of information requests to pharmaceutical

companies. This could have been a watershed moment for EBMH, yet McGoey proposed that

the arguments problematizing RCTs only served to reify EBMH because of a widespread

investment in ‘strategic ignorance’.

Partly because of the regulatory and clinical dependence on RCTs, problems within

individual RCTs are rarely seen as indictments of the methodology as a whole:

instead, they are used as evidence of the need to refine and perfect the methodology

further (McGoey, 2010).

McGoey hones in on Kirsch’s conclusion that the summed effect of antidepressants has a

Cohen’s d of 0.32, below NICE’s threshold of 0.5, itself an arbitrary marker. She points out

that in using an argument derived from EBMH principles in order to unseat pharmaceutical

claims, Kirsch further reifies the paradigm and its principles.

It is the very methodological weaknesses of RCTs that imbues them with the authority

they hold: for to deny the reliability of a particular study, one must reach for more

data, more studies, larger RCTs, in order to justify the validity of one’s objections. Of

course, individuals are free to suggest that RCTs themselves are incapable of

arbitrating in the debate before them. But what data do they possess, what

representation, what visuals, what inscriptions… does such a dissenter have at hand

to convince others of the value of her or his interpretation over others’? The problem

is not that individuals are incapable of or restrained from challenging RCTs, but that,

7


unless they have the resources to defend the scientific rigour of their objections,

preferably through RCT evidence, their interlocutors are equally free to remain deaf.

(McGoey, 2010)

Strategic ignorance is the behaviour of key players in which they are aware of

epistemological flaws and yet, in order to make their case and appear authoritative, they must

also appear ignorant of those flaws. McGoey (2010) concludes that the paradox within

EBMH is that “never before have the inadequacies of RCTs been so apparent to so many.

Yet, equally, never before have those in positions of authority – from regulators, to NICE

policy-makers, to doctors – relied so extensively on RCT evidence”.

‘New’ EBM

It has been suggested by recent commentators that the EBM paradigm is changing. Wieringa

et al (2017) drew on Latour to suggest that EBM is evolving beyond its ‘modernist’ roots to

embrace Latour’s ‘middle kingdom’. They suggest that, rather than dichotomising nature and

culture, EBM now acknowledges the ‘hybridisation’ of nature and culture. They claim this

results in more pluralism and less hierarchy in research methodology; that evidence is treated

as subjective, requiring interpretation dependent on culture and context; that individual

clinical decisions take individual context into account rather than reifying aggregate

evidence; that healthcare can be inference based, recognising the limits of scientific evidence.

Wieringa et al (2017) find the proof of this ‘evolution’ in the ALLtrials campaign which

promotes full disclosure of trial results, preventing selective publication; apps facilitating

individualised clinical decision making, taking into account patient views and affordability;

and increasing acceptance of other forms of evidence such as pragmatic RCTs.

8


However, Latour’s thesis on nature and culture went beyond the concept of ‘hybridisation’

referred to by Wieringa et al (2017). Latour proposed a stance of ‘symmetrical anthropology’

or ‘relative relativism’, which recognises the symmetry of nature and culture in constructing

individuals. He proposed that ‘nature-cultures’ can be studied, differences recognised,

measured and explained; that the relationship between nature and culture is essential to

understanding the relationships between nature-cultures and the former relationship acts as a

measuring instrument for the latter.

By considering EBM as a form of nature-culture that we can study; and taking the claims of

Wieringa et al (2017) that a ‘new EBM’ is emerging; it might be argued that a persistent

feature of EBM as a nature-culture is its tendency to cultivate strategic ignorance. We suggest

that ‘new EBM’ is a fallacy and that EBMH remains replete with epistemological problems

which cannot be solved through ‘evolution’.

Lay Perspectives of an RCT

Experts will likely continue to critique RCTs from a position of strategic ignorance while

proponents may continue to claim the methodology is evolving to address these problems.

We consider RCTs from a different perspective and explore how a non-psychotherapy

research audience understands the design, purpose and findings of an RCT of psychoanalytic

psychotherapy for treatment resistant depression (Fonagy et al, 2015). Psychotherapy

researchers have rehearsed the problematisation of RCTs as diligently as others (e.g.

Richardson, 2000). Thus, the very undertaking of an RCT (which the current authors were

involved in as researchers) might be seen as a form of strategic ignorance, fully embracing a

methodology known to be flawed.

9


The RCT was designed by Richardson in consultation with clinical and research colleagues to

meet the criteria NICE would apply in developing guidelines for depression. This was

achieved in spite of the many contradictions in terms of the nature of the treatment and the

intended outcomes of psychoanalytic psychotherapy which are broadly more relational than

symptom based. It being necessary to target and define a diagnostic group for study purposes,

the category ‘treatment resistant depression’ was selected with comorbidity and complexity

screened in (see Taylor et al, 2012) because of the importance of demonstrating cost as well

as clinical effectiveness of a treatment widely considered to be expensive (Richardson &

Hobson, 2000).

In the present study, we set out to explore the ways in which service users, members of the

public, carers, GPs and other non-psychotherapy researchers would understand and make

sense of this RCT. The aim was to explore how RCTs are understood from without the

paradigm, from where it may be possible to consider the method and findings without relying

on EBMH methodological expertise or investing in ‘strategic ignorance’.

Methods

Two research engagement events were held in London, January 2017. The events were

advertised through social media, directed at a range of service user and carer groups, GPs and

mental health commissioners in and around London. A weblink was provided on the flyer for

people to register for the event. The registration process included ticking consent statements

to indicate understanding that the event would involve taking part in focus groups which

would be recorded and that data would be anonymised, stored securely and analysed for

research purposes. In total 37 people took part (3 GPs, 17 service users, 6 members of the

public, 4 carers, 1 commissioner, 6 non-researcher mental health professionals). Ethics

10


approval was obtained from the University of Essex Faculty of Science and Health Ethics

Committee (Ref 16004).

The events began with a one-hour presentation in four parts. First, the RCT design and

findings were presented using slides to convey the material in lay terms, explaining terms

such as ‘control group’ and ‘randomised controlled trial’. An accompanying leaflet

containing the same information was provided to all participants on the day

(https://ris.essex.ac.uk/repository.html?pub=111822). Next, two service users spoke about

their experiences of receiving psychodynamic psychotherapy for depression. Thirdly, a GP

who had referred patients to the RCT spoke about the impact of therapy on the patients she

referred. In the final part, a therapist who had delivered therapy within the RCT spoke about

her work with one patient and her perception of how the therapy impacted on the patient.

After the presentations, participants were divided into focus groups. There were six groups

across the two events ranging from 3-8 participants. Four groups were made up of service

users, members of the public and carers; two groups were made up of GPs, commissioners

and other health professionals.

At the start of each focus group, participants were reminded of the consent and

confidentiality statements they had checked when registering and advised that they were free

to leave if they no longer wanted to take part. No participants withdrew. Each group had two

facilitators who encouraged the group to discuss four topics: initial reactions to the

presentations; thoughts on the methodology; whether the presentations would influence them

in any way; what further research would be useful. Focus groups were audio-recorded,

transcribed and analysed using thematic analysis (Braun & Clarke, 2006).

11


Findings

Five overall themes emerged, each with subthemes indicated in italics (see Table 1).

Quotations from participants are identified in terms of whether the participant was in one of

the ‘Professional’ focus groups (GPs, commissioners, other professionals); or in one of the

patient, public or carer (‘PPC’) focus groups. Overall, the findings represented a broad

critique of the RCT from a non-strategic perspective, partially blind to its venerated status in

the expert domain. Since participants were mostly non-experts in psychological treatment

trial methodology, this overarching finding is referred to as ‘non-strategic ignorance’. The

analysis has also highlighted where these critiques have a parallel form in the expert

methodological domain, hence noting where ‘strategic ignorance’ and ‘non-strategic

ignorance’ are partly analogous.

Table 1: Themes and subthemes

Problematising depression Trauma underlies mental illness

Trauma is unlocked by psychotherapy

Depression is an inevitable response to trauma

Who cares about RCTs? RCTs cannot capture the complexity of effect in

psychotherapy

Individual ratings on scales are unreliable

Completing questionnaires is harmful

Who will benefit?

Why doesn’t therapy work for some people?

Wariness of universal claims

Bowing to the audience on study design

Service user experiences influence funding decisions

Commissioning processes are based on headline messages

RCT will not increase access to therapy

12


Understanding how therapy

works

Short term therapies unlikely to lead to real recovery

Relationship with the therapist key to the process

Psychoanalytic therapy provides longer term learning and

skills

Unmasking the process

Therapist and GP need to work together

Ending therapy needs transitional support

Selecting a therapy is

complex

Therapy should be tailored not manualised

Therapy duration should depend on individual need

You have to find what’s best for you

Therapy should not be based on profiling or a formal diagnosis

Hearing other's experiences of therapy is important for one’s

own choice

Choice is unsupported by medical professionals

Overcoming stigma comes before choice

Inevitability of cost

effectiveness

Intuitively holistic sense of the broader value of the therapy

Patients are not burdens

Problematising depression

The RCT in question was constructed as a trial for ‘treatment resistant depression’. This label

was problematised by participants and seemed intrinsic to participants’ determining the value

of the RCT. In this theme, the construct of ‘treatment resistant depression’ was challenged by

both PPC and professional groups, indicating an inherent problem with framing an RCT

around a single label and defining it through inclusion and exclusion criteria. The disease

category label selected was also felt to infer a biological disease inhabiting a patient who is

somehow unbending or resistant to the proper medical cure, which is fundamentally different

conceptually to participants’ idea of depression as a natural response to past or current

trauma. The theme illustrates how participants’ common sense non-medicalized concept of

13


depression undermined the core tenets of the RCT. In this sense, participants, through

speaking to their experience, have alighted on a parallel expert critique of the medicalization

of mental illness (e.g. Rapley, Moncrieff & Dillon, 2011), which overlaps with expert

critiques of EBMH at a foundational level of defining the ‘diseases’ around which ‘evidence’

is framed.

There was a widespread assumption underlying participant comments that trauma underlies

mental illness and that the client group that participated would have had high levels of trauma

which could be unlocked by psychotherapy:

…quite a lot of what I think emerges from this… long term psychotherapy is actually

quite a lot of people realising that they’ve hung on to something very traumatic.

(Professional)

Connected to this was the understanding of depression as an inevitable response to current

trauma:

…if you’ve got severe arthritis and you can’t move and you never go and you never

see anyone, how the hell you are supposed to not be depressed I do not know, ‘cause

your life is hell and my mother was like that and your life is just misery, you’re just

lying there and you never see anyone, plonked in front of the television, you can’t

move and you’ve had all of these operations I mean, how can you expect someone

like that to not be depressed. (PPC)

14


Along with advocating a trauma model, there was a general rejection of the medical model of

depression:

…the chemical imbalance theory [ ] of depression is a totally unproven theory…that

psychiatrists have been trying to say it’s proven for donkey’s years. (PPC)

There was an assumption, mainly among professionals, that the client group described had

personality difficulties which would impact on the usefulness of therapy:

…quite a few of these people are going to have quite complex personalities as well

so… I just think a sense of carefully selected people I think a bit longer would be my

suggestion. (Professional)

These assumptions about the nature of the population raise questions about the validity of a

single diagnosis approach employed in RCTs and NICE guidelines, highlighting the potential

for manipulation and/or distortion where there is comorbidity in RCT populations. The

possibility of labelling the RCT as a trial for personality disorder treatment was raised and

dismissed by participants because of the stigmatising effect of that label; but there was a

sense expressed by PPC participants that the term ‘treatment resistant depression’ was

stigmatising even without invoking the personality label, perhaps because the implication was

there anyway:

People that don’t benefit from an inverted commas treatment, whether drug or therapy

because there’s something not working about the treatment, it’s not in the person that

15


they’re resistant, it’s just that they’re not being helped as much or in the way they

need or as well as they need it, so I don’t like the term treatment resistant. (PPC)

Who cares about RCTs?

In this theme, participants appeared ‘ignorant’ of the status of RCTs in EBMH, rejecting any

veneration of RCTs not for any technical paucity in the conduct or design of this particular

trial, but from a confidently common sense notion of what type of information is meaningful.

This is congruent with expert critiques drawing on issues around the complexity of process in

psychotherapy which renders RCT methodologies unsuitable because they over-generalise

interventions.

While commenting on the apparent quality of the RCT and positive findings (significant

treatment effects at 2-year follow-up), participants seemed sceptical about its relevance. In a

broad sense, some of this was about the way in which the RCT over-simplified issues. For

example, there was an idea that the methods used can’t capture the complexity of the effect of

psychotherapy:

I suppose that’s the problem I have with statistics because I don’t think you can really

capture… you feel better but I mean… you might be having one panic attack a week

instead of once a month or something. (PPC)

This was coupled with more specific criticisms of the use of questionnaires for measurement

both generally and specifically because individual ratings on measures are unreliable:

16


I've done the BDI stuff… they give it to you before crisis after crisis to see… how

much you've gotten better, before therapy, after therapy… it also depends on how you

feel on that day at that particular hour, so for example you've just broken up with your

boyfriend, of course you're gonna say yes life is miserable… I just think that when

you're really, really depressed the last thing you want to do is fill in questionnaires, so

I don't think they're wholly accurate. (PPC)

There were some particularly strong views, taking this further, that completing questionnaires

is harmful for service users:

…that questionnaire is just the most depressing thing… I started reading that,

someone gave me that and I felt like shit… I mean, do you feel miserable, blah, blah

do you want to kill yourself and… I get to about half way through it and I cannot bear

to complete it. (PPC)

In this and other comments, there is a sense that the technical inaccuracy of measures goes

hand-in-hand with the moral problem of asking service users to complete forms at times of

distress; and that these flaws cannot be uncoupled.

Related to concerns with measurement were more general concerns that the controlled design

is not adequate to account for processes:

…how we know or how we cannot review or establish … improvement in treatment is

due to the psychotherapy itself, to the psychotherapy variables, to their interactions,

with their relationship with the psychotherapist all the techniques… I’m not sure that,

17


even if the RCT is considered the gold standard for effectiveness of treatment, we can

easily establish that and pin point exactly the variable the elements that make that

different psychoanalytic treatment versus treatment… (PPC)

Whilst psychotherapy researchers might similarly argue that process research is necessary to

accompany outcome research to help understand which techniques work and why, there is a

slightly different nuance here in that knowing whether this specific treatment is helpful

cannot be determined by the RCT because the RCT can’t explain how it helps. This is

discussed further below in relation to the main theme Understanding how therapy works.

Other reasons for being sceptical about the findings were that it could not reveal who might

benefit or why therapy didn’t work for some people. These concerns suggest that the RCT

method was perceived as too blunt to provide useful information, relegating the fundamental

tenet of RCTs to find out if on a group aggregate level an intervention has broadly better

outcomes than a control intervention.

…it would be really good if we knew more about who was … more likely to benefit,

because that’s obviously the commissioners’ question, but that’s a service user’s

question as well, they don’t really want to go through something and feeling you

haven’t benefited. (PPC)

This wariness of universal claims derived from RCTs also emerged from the Professional

focus groups, albeit expressed using more technical language:

18


…if you look at the efficacy results of effectiveness then… the effect sizes are not

brilliant really and they’re not good for antidepressants… they’re a bit better for CBT,

but we haven’t really got there… I don’t think psychoanalytic psychotherapy is the

holy grail either… one size does not fit all. (Professional)

Conscious of these flaws, there were a number of ways in which participants suggested that

conducting the RCT was simply bowing to the audience on study design:

…it’s as if you’ve been forced to use them because you are competing for resources

with other kinds of mental health care so you have to… adopt an internationally

recognised method. (PPC)

One participant felt particularly strongly about this, suggesting that the host Trust had

adopted a ‘pseudoscientific’ approach for political purposes. This is very similar to the

concept of ‘strategic ignorance’, but for the participants concerned this seemed to border on

undermining the value of the psychoanalytic approach entirely, indicating a different

internalised paradigm for conceptualising ‘science’.

This alternative view of scientific value was also tangible in the professional focus groups in

which emerged a consistent view that service user experiences are influential in funding

decisions:

…numbers are not enough, I think there needs to be that narrative that goes with it,

that personal experience I think is very, very important actually. (Professional)

19


In contrast to EBMH doctrine, professionals also indicated that the local commissioning

processes are based on headline messages in what seems to be the antithesis of a systematic

rigorous process:

I have been involved in helping [the CCG] when they do, you know, proposals, calls

for bids, and you often have people who are not specialist in the area at all, possibly

not even mental health trained…just trawling through …the literature just to see what

they can come up with so, the clearer you can be with your messages in different

places the more likely you are to catch the eye of the commissioners. (Professional)

In parallel was an assumption across all groups that the RCT was almost irrelevant in terms

of commissioning because it would inevitably not increase availability of this type of

therapy. This implies a general understanding that this type of therapy cannot use RCT

methods to support a case for its increasing availability.

Understanding how therapy works

This theme also corresponds to expert concerns that the complexity of psychotherapy

processes are ignored or even warped by RCT methods. However, this theme reflects sense

making which is partially ignorant of psychotherapy research jargon about process versus

outcome. It represents instead an intuitive sense of what matters in a therapeutic relationship

that cannot be captured in an RCT.

Understanding the way in which therapy might work or not work seemed to be of more

relevance to participants than aggregated group outcomes. There was a fairly consistent view

that for the client group in the trial, short-term therapies were unlikely to lead to real

recovery, largely because of the connection to deep seated trauma discussed above. This was

20


coupled for some with a sense that short-term therapies were often all that was on offer and

could not be refused:

I will engage because it makes it looks like I'm engaging with my ill health, which is a

positive, so I'm engaging but not actually fully engaging because it's six weeks which

is never gonna get to the real of this thing that I've been burdened with for as long as I

can remember… so I'll tick that box because it shows that I've done it and I'm really

trying. (PPC)

Part of the reason short-term therapies were seen as unlikely to work was to do with the

relationship with the therapist being key to the process. Trust was seen as intrinsic and

something which required gelling with the right therapist and time:

it goes hand in hand with trust… I very, very openly, when I first went into any sort of

therapy, have said very sceptical and I don't trust you and until I can trust you I can't

talk to you about the things that I don't even know are there, that I do know are there,

but I don't know what they are because they're so suppressed… and until I trust you I

can't do that. (PPC)

Psychoanalytic psychotherapy was described as a ‘deeper form of treatment’ and was thought

to provide longer term learning and skills than other treatments, enabling you to ‘know

yourself’, ‘empowering you to move forward’, make ‘longer lasting changes’ which can carry

on ‘for the rest of your life’.

21


Some of these ideas invoked a sense of mystery around the process which was openly

acknowledged by some, and thus the presentation by the therapist was described as very

illuminating, an unmasking of the process:

I’ve never heard a therapist talk in that way… I thought it was a kind of closed shop

really and that they had these methods and these things that they were never going to

disclose to the great unwashed, so to have her be so upfront and honest it was

extraordinary and I thought, yes, very brave, especially when she was saying that she

felt ashamed that I didn’t do a bit better and I was very surprised to hear that, so that

in itself kind of overturned my idea of therapy and therapists. (PPC)

What seemed to be illuminated was the key role of the relationship in the therapy,

emphasising the points above. This was also related to the view that improvements in

relationships are the most important outcomes of therapy; that outcomes should be more

subjective and that functioning outcomes are critical. These ideas were largely antithetical to

the focus on symptom outcomes as presented in the study findings. In commenting on one of

the service user presentations, a participant noted:

…the importance for her was in the improvement in relationships and for a lot of

people that’s what they want, … that’s the end goal to get to that point where they can

have that confidence and have that kind of that, that better existence because they are

able to relate to people across the board… she was saying to be able to have

relationship with another person, but to have the relationship with the therapist, sort of

the family and the whole works. (PPC)

22


Because of the importance of the relationship and that at the end of therapy, the relationship

with the therapist must come to an end, it was felt that the therapist and GP need to work

together and that ending therapy needs transitional support, emphasising the perceived

relational context of therapy within the wider support system.

Selecting a therapy is complex

NICE guidelines compare treatments and formulate a hierarchy, with ‘first-line’

recommendations for treatments being those with the greatest weight of RCT evidence. The

within-paradigm critique of this is ‘the Dodo Bird Verdict’ that all psychotherapies may be

equally effective (e.g. Wampold, 1997) and that individual therapist effects are more

significant that therapy modality effects. Discussions in the focus groups about how to select

a therapy seemed to be based neither on the equivalence nor the hierarchical position, rather

the arguments illustrated in this theme about choosing a therapy come from without the

EBMH paradigm and speak to participants’ experiential notions of how to choose a therapy.

There was a view that several factors will influence the outcome of therapy including

intelligence of the therapist, trust in the therapist, client variables such as social support,

gender, sexuality, lifestyle, motivation, commitment, expectations. Relatedly, it was felt that

therapy should be tailored not manualised: “a good therapist, there is no set rules”; which is

antithetical to testing manualised interventions as required by RCT quality standards. It was

also felt that therapy duration should depend on individual need, further problematising the

concept of RCTs which require identical length treatments for comparability. This idea of

individualised interventions came across in both the PPC and professional groups:

23


It may be that… very intensive therapy for perhaps slightly shorter times, with more

of a guarantee that people would go for a longer period of time into groups as they

transition out of it, it might be… an economically sensible model. (Professional)

Some people will come with come with lots of… deep rooted issues and some won't,

so it’s got to be accordingly. (PPC)

The multiplicity of ‘best’ treatments was also conveyed in the idea that you have to find

what’s best for you:

For some people that might be enough, or they might need it with something else,

pills or CBT or whatever it is, but… I think for everyone I think it’s finding what

works for you. (PPC)

There was a sense that the right therapy for an individual should not be based on profiling or

a formal diagnosis which was depicted as a frustrating aspect of accessing NHS treatments.

They keep pushing me back into a template… I said please can you listen to what I'm

saying, I don't fit into that profile, can you try to offer me kind of alternative

therapy… so at the moment they're not really listening to me at the moment. (PPC)

This concept of individual needs and individual problems related to an implicit notion of

patient choice. There was a premise that the client should have a role in deciding which

therapy they need and that hearing other's experiences of therapy is important for one’s own

choice. For this reason the service user talks were felt to be valuable, with the proviso that

24


people would have liked to have heard more about what didn’t work and why things might

work and not work for different people. However, active patient choice was felt to be

unsupported by medical professionals:

In terms of offering the various options to service users… sometimes the information

is not being given to the service user in order for them to make an informed decision,

as soon as they say they want to go the holistic way, straight away the alarm bells start

triggering with the medical profession for whatever reason, and then you get ..shunted

into a different sort of arena where they label you as a kind of problem maker. (PPC)

There was a view that choosing a therapy was secondary to overcoming stigma which can

prevent people even discussing their problems and subsequently their treatment options with

professionals:

Psychotherapy or talking therapies of any form, are not something people will be

familiar with… or, if they are familiar with they will also be quite reluctant to take it

on… stigma anyway… attached to mental health… so I think maybe health

professionals need to be aware of that… as well and try to encourage people. (PPC)

This emphasises that RCTs in mental health have limited relevance to the larger population of

potential beneficiaries who are battling stigma or suffering in silence rather than worrying

about which intervention might help them.

25


Inevitability of cost effectiveness

Formal cost effectiveness analyses often only compare outcomes with participants’ health

costs up to one year preceding the intervention; a technical within-paradigm critique might

emphasise this limitation with chronically ill populations. However, participants were in a

sense ‘ignorant’ of this and as illustrated in this theme, were speaking about an intuitive sense

of value in two senses: one that costs shouldn’t matter and two, that because of the extent and

duration of the difficulties which placed burdens throughout the health and care system as

well as families and friends for many years, anything that helped would inevitably save costs

when considering everything holistically.

Although no findings relating to cost effectiveness of the intervention were presented, there

were views expressed which implied that cost effectiveness seemed inevitable. This view

appeared to be based on factors that are unlikely to feature in a formal cost effectiveness

analysis and hinged on an intuitively holistic sense of the broader value of the therapy. This

was sometimes expressed as de facto common sense in that “costs shouldn’t come into it…

cause if something works yeah it will then help in the long term”. This seemed to need no

further explanation, although others did elaborate on this, for example:

They are really depressed for a very long time, so that’s an awful lot of kind of GP

and man hours and so on, which is what at the end of the day the system and the

commissioner and so on are looking for, but also… benefiting those people and you

can’t put a cost on that. (PPC)

I think it is worth funding, because the amount of energy and time that is spent by me,

very interestingly the most intractable patient also complains of memory problems

26


which… has ended up with referral so very cleverly shops around and asks the GPs to

get new referrals and now he’s under the wing of some… professor … I think it’s all a

part of the same story and the number of scans… I can’t tell you, what I think

actually, it is a, I mean, my instinct is it’s a worthwhile investment. (Professional)

There was a sense that those responsible for making financial decisions about treatment were

not driven by the right sort of considerations:

I don't know how the government or people who are funding it, will interpret that,

because they want quick numbers, they want to literally immediate statistics, so they

just think well two years down the line… we've got other people who need… bypass

surgery, now we want quick results, so I don't know how they see that as actually is

that money worth spending. (PPC)

There was acknowledgement that this type of therapy would be more expensive than shorter

therapies; yet there was a clear sense that thinking about patients as burdensome was not just

a short-term narrow way of thinking about costs but also out of keeping with efforts to

destigmatise mental illness:

The GP has the audacity to say to the patient that you are an expensive patient he

cannot keep on coming and seeing me, right so, this unfortunate this stigma is really,

really extreme and in some sort of parity with mental health and physical health, it

doesn’t really exist… I think there is still a long way to go. (PPC)

27


Discussion

McGoey (2010) described technical critiques of EBMH using within-paradigm arguments as

a form of strategic ignorance which has served to bolster the paradigm. The very undertaking

of an RCT of psychoanalytic psychotherapy also falls into the trap of strategic ignorance, as

arguably do all psychotherapy RCTs. The lay critiques of RCTs in the empirical data

presented each have analogous forms in the expert domain, from where experts have

attempted to debunk RCTs using a plethora of arguments. The data suggest that people who

are not expert psychotherapy researchers place much less value on RCTs in this field than

NICE. The undermining of RCTs in the data, while congruent with expert critiques, also has

some nuance attributable in part to its origin in experience rather than methodological

expertise. The way in which participants made sense of this RCT, critiquing both its premise

and usefulness from outside of the paradigm, points towards the potential for a different

paradigm.

The RCT findings seem to have been made sense of with an intuitive sense of the value of the

treatment irrespective of the findings of the RCT. This intuition applied to the clinical and

cost value and derived from a sense of the importance of a relationship based treatment and

its likely impact on people who have deep seated traumas contributing to their long-standing

misery. This way of making sense of the client group and the treatment rendered almost

irrelevant the details of the carefully prescribed RCT methodology and have been referred to

here as ‘non-strategic ignorance’, since it does not attempt to employ the rules of EBMH in a

strategic way to critique specific examples of within-paradigm evidence. We now consider,

using Kuhn’s thesis on scientific revolutions (1962), whether the conditions might be present

for a shift towards a paradigm in which service user views are central, rather than a

paradigmatic problem to be solved.

28


Kuhn argued that ‘normal science’ operates within a fairly rigid framework which aims to

protect the paradigm from disconfirmation. Psychoanalytic psychotherapy is the antithesis of

EBMH, partly because its adherents held most political sway among the psychiatric

profession in North America immediately prior to the paradigm shift from ‘clinical inference’

to EBMH from the 1950s onwards. An RCT of psychoanalytic psychotherapy is problematic

for EBMH because it brings a practice associated with clinical inference into the category of

‘evidence based’. Such a contra-indication has contributed to the NICE (2017) draft update to

the depression guideline (the final version still postponed at the time of writing) designing its

review methodology in a way that means the RCT concerned is deemed not to provide good

evidence of efficacy, contrary to the actual findings. RCTs clearly remain “normal science”

for now.

Yet the notion of ‘scientific revolutions’ contains the premise that paradigms will come and

go. Kuhn (1962) defined a paradigm as a mode of thinking that is "sufficiently unprecedented

to attract an enduring group of adherents away from competing modes of scientific activity,"

and "sufficiently open-ended to leave all sorts of problems for the redefined group of

practitioners to resolve." A new paradigm attracts enough adherents away from the prevailing

paradigm to create a shift in the political power attributed to respective groups of adherents.

We might think of EBMH as having been a new paradigm in its time that, from the 1950s

onwards, drew away adherents of ‘clinical inference’ and psychoanalysis towards modernist

principles. But EBMH has not resolved the problems inherent within it. Might a shift occur

again?

29


One of the key problems unresolved within EBMH is Patient and Public Involvement (PPI).

PPI refers to the principle, now dominant in UK health policy, that patients and the public

should have a direct role in all levels of healthcare planning, delivery and research, offering

the potential for ‘increased democratic accountability’ (Madden & Speed, 2017). The idea

that service users might have valuable insights to contribute to this level of planning and

delivery is also linked to increasing recognition that service user experience (in the form of

qualitative research) has value. In a similar vein is the idea of shared decision making in

clinical practice in which clinicians and patients share information and come to joint

decisions about treatment (Charles, Gafni & Whelan, 1997).

Early in the emergence of EBMH as a new paradigm, Faulkner and Thomas (2002) set out a

vision for EBMH which included more patient-centred forms of evidence. They set out ethics

and morality as the primary arguments for including user-led qualitative research on patient

experience as essential to the repertoire of EBMH. Within EBMH and specifically NICE,

commissioning and health policy making generally, PPI has been widely endorsed and

encouraged. Yet, as noted by Madden and Speed (2017), PPI has largely become an empty

signifier, “a ghastly composite of a zombie policy that continually pops up, offering (but

never providing) a solution to purported deficits in democratic engagement.” PPI has

therefore been a particular thorn in the side of EBMH because it is messy and subjective and

threatens the purified dichotomy of nature and culture in Latour’s terms. As Madden and

Speed (2017) note, PPI tends to be no more than “an opportunity for professionals to

consume affective individual testimony without the need to engage with wider publics or

more contextualised forms of research”.

30


Kuhn notes that there must be a crisis in the current paradigm to create the conditions for a

paradigm shift. ‘Complexity’ is a necessary aspect of a crisis in that the ongoing adjustment

of the theory creates an ever-increasing complex theory that cannot sustain itself. To

conclude, we return to ‘new EBM’ to see where this complexity may lie. The ‘new EBM’

described by Wieringa et al (2017) suggests an ever-increasing complexity with their

acknowledgement that there is no resolution to the nature-culture messiness:

[EBM] can never fully embrace this middle kingdom without effectively rejecting the

very concept of evidence on which its central claims are based. Instead of resolving

this paradox, the [EBM] movement must coexist with it and ‘muddle through’.

This acknowledges that ‘new EBM’ is now unfathomably messy and illogical and yet there is

a wish for it to prevail. "Though they may begin to lose faith and then to consider

alternatives, they do not renounce the paradigm that has led them into crisis." (Kuhn, 1962).

The data presented in the current paper provides inklings that an alternative paradigm may

already be (or always has been) available outside of the EBMH expert community, outside of

positions of power, policy and commissioning. This alternative could potentially coagulate

around Madden and Speed’s call for a more democratic and less technocratic approach to PPI

in which populations are ‘critically involved’ and engaged in decision making. This would

mean not a ‘new EBM’ but something epistemologically different, representing a different

form of nature-culture with service users in a central, not peripheral role, and which could of

course be studied comparatively to the current nature-culture formation that is EBMH.

The authors declare that there is no conflict of interest.

References

31


Braun V and Clarke V (2006) Using thematic analysis in psychology. Qualitative Research in

Psychology 3: 77-101.

Charles C, Gafni A, Whelan T (1997) Shared decision-making in the medical encounter:

what does it mean? (or it takes at least two to tango). Social Science and Medicine

44(5): 681–92.

Cochrane A (1972) Effectiveness and efficiency. Random reflections on health services.

London: Nuffield Hospitals Trust.

Department of Health (1989) Working for Patients. NHS Executive.

Department of Health (1996) Promoting Clinical Effectiveness. NHS Executive.

Department of Health (1998) A First Class Service, Quality in the New NHS. NHS Executive.

Department of Health (2000) The NHS Plan: A Plan for Investment. A Plan for Reform. NHS

Executive.

Devereaux PJ and Yusuf S (2003) The evolution of the randomized controlled trial and its

role in evidence-based decision making. Journal of Internal Medicine 254(2): 105-

113.

Dixon L and Goldman H (2004) Forty years of progress in community mental health: the role

of evidence-based practices. Administration and Policy in Mental Health 31(5): 381-

392.

Faulkner A and Thomas P (2002) User-led research and evidence-based medicine. British

Journal of Psychiatry 180(1-3): 1-3.

Taylor D, Carlyle J, McPherson S, Rost F, Thomas R, Fonagy P (2012) The Tavistock Adult

Depression Study (TADS): A randomised controlled trial of psychoanalytic

psychotherapy for treatment-resistant/treatment-refractory forms of depression. BMC

Psychiatry 12:60.

32


Fonagy P, Rost F, Carlyle J, McPherson S, Thomas R, Fearon P, Goldberg D, Taylor D

(2015) Pragmatic randomized controlled trial of long-term psychoanalytic

psychotherapy for treatment-resistant depression: the Tavistock Adult Depression

Study (TADS). World Psychiatry 14(3): 312-321.

Harari E (2001) Whose evidence? Lessons from the philosophy of science and the

epistemology of medicine. Australia and New Zealand Journal of Psychiatry 35(6):

724-730.

Healy D (1999) The three faces of the antidepressants: a critical commentary on the clinical-

economic context of diagnosis. Journal of Nervous and Mental Disorder 187(3):

174-180.

Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ and Johnson BT (2008)

Initial severity and antidepressant benefits: A meta-analysis of data submitted to the

Food and Drug Administration. PLoS Medicine 5:e45.

Kuhn T (1962) The Structure of Scientific Revolutions. Chicago: University of Chicago Press.

Latour B (1993) We Have Never Been Modern. Harvard University Press.

Luborsky L, Barton S, Luborsky L (1975) Comparative studies of psychotherapies: Is it true

that "everyone has won and all must have prizes"? Archives of General Psychiatry

32(8): 995-1008.

Madden M and Speed E (2017) Beware Zombies and Unicorns: toward critical patient and

public involvement in health research in a neoliberal context. Frontiers in Sociology.

doi.org/10.3389/fsoc.2017.00007

McGoey L (2010) Profitable failure: antidepressant drugs and the triumph of flawed

experiments. History of the Human Sciences 23(1): 58-78.

NICE (2017) Depression in adults: treatment and management. Draft. Accessed from:

www.nice.org.uk/guidance/indevelopment/gid-cgwave0725

33


Paris J (2000) Canadian psychiatry across 5 decades: from clinical inference to evidence-

based practice. Canadian Journal of Psychiatry 45(1): 34-39.

Rapley, M., Moncrieff, J., Dillon, J. (Eds) (2011). De-medicalising Misery. Palgrave

Rees WL and Healy D (1997) The place of clinical trials in the development of

psychopharmacology. History of Psychiatry 8(29 pt 1): 1-20.

Richardson PH and Hobson RP (2000) In defence of NHS Psychotherapy. Psychoanalytic

Psychotherapy 14: 63-74.

Richardson PH (2000) Evidence-based practice and the psychodynamic psychotherapies. In

Mace C, Moorey S and Roberts B (eds) Evidence in Psychological Therapies: A

Critical Guide for Practitioners. London: Routledge, pp.157-173.

Shedler J (2017). Selling bad therapy to trauma victims.

https://www.psychologytoday.com/blog/psychologically-minded/201711/selling-bad-

therapy-trauma-victims

Wampold BE, Mondin GW, Moody M, Stich F, Benson K and Ahn H (1997) A meta-

analysis of outcome studies comparing bona fide psychotherapies: Empirically, "All

must have prizes." Psychological Bulletin 122: 203-215.

Wieringa S, Engebretsen E, Heggen K, Greenhalgh T (2017) Has evidence-based medicine

ever been modern? A Latour-inspired understanding of a changing EBM. Journal of

Evaluation in Clinical Practice, 23, 964-970.

Documents

Non-strategic ignorance: considering the potential for a ...repository.essex.ac.uk/22691/1/Strategic ignorance Health pre-print Jul2018.pdfFelicitas Rost2 [email protected] Sukhjit