Evaluating Expert Witness Psychological Reports in Family Court Professor Jane L. Ireland, University of Central Lancashire, Mersey Care NHS Trust and

Evaluating Expert Evaluating Expert Witness Witness

Psychological Psychological Reports in Family Reports in Family

CourtCourtProfessor Jane L. Ireland, University of Central Lancashire, Mersey Care NHS Trust and CCATS

Dr Carol A. Ireland; Dr Fiona Wilks-Riley;CCATS and University of Central Lancashire

John Pinschof, Private Practice.

1Copyright Professor Jane L. Ireland, 2012

Presentation presented by Dr. Carol A. IrelandPresentation presented by Dr. Carol A. Ireland

ContentsContents• General introduction to the area• Introducing the research• Sample• Research highlights• Specific issues• Questions?

Copyright Professor Jane L. Ireland, 2012

2

Unfortunately, Unfortunately, problems are not a problems are not a

new concept to new concept to psychological psychological reports…….reports…….


3

Previous problems noted with Previous problems noted with psychological reports (e.g.’s)psychological reports (e.g.’s)

• Presenting psychological evidence as scientific fact, when it is simply speculation (Hagan, 1997);

• Absence of psychological theory (Ireland, 2012);

• Failure to provide evidence that is outside the knowledge of a juror (Omerod and Roberts, 2006)

• Over-use of psychometrics, not applicable and over-use of jargon (Weiner, 1999)


4

Previous problems noted with Previous problems noted with psychological reports (e.g.’s)psychological reports (e.g.’s)

• Psychological risk assessments based on older approaches (Hart et al, 2007);

• Allegations reported as fact (2 Cr App R7);

• Emotive terms that could prejudice a decision (Goodman-Delahunty, 1997)


5

The ResearchThe Research• Preliminary piece of research

• Assessed the quality of expert psychological assessments presented in Family Court

• Currently, the quality of psychological reports are left to non-psychologists (e.g. Judge), and who may not be in the best position to determine this.o More problematic when the reports are not

accessible to other experts in the UK – such as in private or family proceedings – restricts opportunity for peer review


6

The ResearchThe Research• Aimed to draw on admissibility criteria for

expert evidence that has been developed elsewhere (Daubert criteria):o The theory or technique must be testable;o It has been subject to peer review;o Has a known or potential error rate (that is,

to get it wrong);o The theory and technique is generally

accepted by a relevant scientific community.


7

The ResearchThe Research• Current criteria used in the UK is the

Turner ruleo This notes if evidence is outside typical

knowledge of a juror, and is therefore admissible on the grounds of being helpful as a result (helpfulness)

o Yet, it does not allow for the direct testing of the evidence content


8

OutlineOutline• First research of it’s kind to look at those reports

specifically in family proceedings

• 126 expert psychological reports reviewed covering child and adult assessments;

• Three Courts

• Four raters using a structured proforma

• Focus: on quality with regards to UK Civil Procedure Rules and Daubert


The sample….The sample….• 79% claimed to be a full member of a British

Psychological Society (BPS) Division;• 81% Clinical Psychologists• 14% Educational Psychologists• 11% Forensic Psychologists

• 90% of experts not maintaining a clinical practice outside of Court Work – sole role appeared to be that of completing expert reports.

• 21% were not qualified psychologists (that is, not a member of a Division or even a member of the BPS in some circumstances).


Research HighlightsResearch Highlights• 65% of reports were rated as ‘poor’ or ‘very poor’;• 35% of reports ‘good’ or ‘excellent’;

• Problems in reports failing to report the data from where inferences were drawn; more in the direction for those unqualified;

• Problems in reports failing to evaluate the quality of their evidence; more in the direction for those unqualified;

• Absence of psychological theory across reports.


Some specific issuesSome specific issues

• 29% of problematic reports presented insufficient facts and went immediately to opinion;

• 22% failed to present the data (e.g. psychometrics) on which they had presented an opinion;


Some specific issues Some specific issues cont:cont:

• 77% of reports included psychometrics;o 19% rated as not relevanto 40% ‘somewhat’ relevanto 41% ‘greatly/completely’ relevant

• Reports used between 1 and 11 tests. Unqualified psychologists significantly more likely to use tests than qualified.

• Across the reports 90 DIFFERENT sets of tests were employed…..


Example self-generated ‘tests’•Life snake test•Buttons test•Time Machine

•Only cognitive tests (e.g. WAIS), those for Clinical Psychopathy (PCL-R/SV) & some clinical tests (MMPI) would meet Daubert.

•Lack of information to allow Courts to judge value (e.g. error rates)

•Some extremely out of date tests being used (e.g. WAIS-R, not WAIS-III or WAIS-IV), or just being used incorrectly.


• Risk assessments being completed without structured clinical guides: oOf those requesting a violence risk

assessment, 1% used a structured test

• Using assistants to complete tests and interviews and then writing a report on the basis of this.


Process – use of language

•Use of emotive terms - only 6% used this ‘greatly’. 21% as ‘somewhat’. Evidence of “interesting” language:

o“He seemed grumpy”;o“Uncouth child”;o“Huffed and puffed”;o“Self-centred young woman”;o“She is inadequate”.


‘‘Experience’ of expertsExperience’ of experts• Based on provided CVs:

o 30% did not have experience of mental health assessments, yet were completing such assessments;

o 22% of experts deemed not to have the competence to complete the instructed assessment.

o Mainly unqualified psychologists for both e.g.• No forensic experience noted for what is a

forensic assessment; • States clinical psychologist but is

educational; • No child experience apart from a 6 month

placement.17


Straying out of Straying out of remit/area…..remit/area…..

• 50% stayed in remit; 50% did not, e.g.

• “even smoking cigarettes will have a serious impact on his health given the present condition of Mr X’s lungs”;

• “given the symptoms, I think this child is suffering from diarrhoea”

• “The dog X was extremely well behaved and extremely patient”.


Reports that were ‘poor’ or ‘very Reports that were ‘poor’ or ‘very poor’....themes for poor practice:poor’....themes for poor practice:

o Use of graduate or assistant psychologists;o Overuse of psychometrics;o Absence of support for opinion e.g.

• “Concludes he has advanced cognitive ability but this has not been assessed”;

o Making uninformed psychological statements:• “Does not understand what mental illness and

personality disorder is”;• “States he does not have a personality

disorder and not assessed it, and then says he does have a personality disorder”.


o Failing to assess instruction questions;o Reports excessive amounts of expert work

e.g. • “200 reports a year; over 1000 report in the

last few years”o Knowingly using tests that are not

relevant;• “I have carried out this test with X although it

does not apply as it is only normed for individuals 18 and above”

o Misleading about their qualifications;• Alluding to be members of professional (BPS)

divisions, but does not state what type (full, student)


o Limited opinion/report length;o Irrelevant comments:• “She was dressed appropriately for her age

and weather”;• “She was dressed in a low cut pink dress”• “He likes spaghetti bolognaise and puts his

pots away”;o Acting without ethics

• E.g. “completed an assessment on the mother without actually seeing her”;

• “Completed an assessment in a room in a general visit area with no door, assessment was on sexual disorder”.


21

o Poor use of English;o Misunderstanding their role and

boundaries;o Narrative comments.

• “When I am talking about foster carers and prospective adopters, I often use the story of Pinocchio as an example of such a struggle…..at present in school, Ms X is like the child’s Jiminy Cricket and that feedback that Pinocchio obtains from extending his nose is the system of rewards and sanctions that are so effectively maintained within the special provision that has been made for X”.


Criterion for a ‘good’ Criterion for a ‘good’ reportreport

• Adherence to CPR;• Limited use of psychometrics;• Concise language with limited use of jargon;• Avoidance of poor methods;• Use of structured clinical guides for risk;• Remaining within remit;• Expert involved in all aspects of the report and its

preparation;• Inclusion of data supporting opinion;• Answering the instruction questions set.


Recommendations Recommendations (e.g.’s)(e.g.’s)

• Experts should be registered with their regulatory body (in the UK: HPC) and have full membership of an APPLIED division with the BPS; Need to educate on this;

• Competence and continuing practice is key. Need to re-think about experts who are not in practice (and how you define ‘in practice’);

• Not using graduate psychologists or assistants to conduct assessments/interviews for reports;

• Periodically reviewing quality of reports; Working with professional bodies on this, perhaps including peer review?;

• Encourage provision of provisional and alternative opinions;


• Periodically reviewing quality of reports; Working with professional bodies on this, perhaps including peer review?;

• Being very cautious over psychometric data – Daubert criteria would assist here;

• Using well developed psychological theory and EXPECTING this in reports;

• Not paying for reports that do not meet quality expectations (e.g. that do not answer instruction questions), or reducing the fee to reflect this;

• Training the judiciary (and other professional groups) on what to expect for good quality reports;


• Not paying for reports that do not meet quality expectations (e.g. that do not answer instruction questions), or reducing the fee to reflect this;

• Training the judiciary (and other professional groups) on what to expect for good quality reports;

• Providing experts with feedback on their report quality;

• Enforcing the CPR as much as possible (e.g. Not paying unless compliant or reducing payment);


• Any questions?

• [email protected]

• [email protected]


27

Documents

Evaluating Expert Witness Psychological Reports in Family Court Professor Jane L. Ireland, University of Central Lancashire, Mersey Care NHS Trust and