Terry Kukor, Ph.D., ABPP Rick DeMier, Ph.D., ABPP...May 04, 2019 · Dixit?” 4) “How can I decide if something qualifies as a mental illness if you won’t name that mental illness?”

Terry Kukor, Ph.D., ABPP Rick DeMier, Ph.D., ABPP

May 9, 2019

Columbus, OH 1

Controversies in Forensic Mental Health Assessment

May 9, 2019Association of Ohio Forensic Evaluation Centers

Ohio Department of Mental Health and Addiction ServicesColumbus, OH

Terry Kukor, Ph.D., ABPP, and Rick DeMier, Ph.D., ABPP

Why This Workshop?• Is there a fundamentally “right way” to complete forensic mental health evaluations?

• Certainly, there are some things about which we would all agree, but even the most esteemed experts disagree about certain mattersthe most esteemed experts disagree about certain matters

• Friedrich Nietzsche: “You have your way. I have my way. As for the right way, the correct way, and the only way, it does not exist.”

• This workshop is not about the answers; it is about how we approach the questions

2

The Making of this Workshop

3


May 9, 2019

Columbus, OH 2

Argumentation

4

What Argumentation Is

1) Argumentation = the study of effective reasoning

2) Arguing = the process of reason‐giving, where the reasons given are justifications or support for claims

3) The alternative to reason‐giving is to accept or reject claims on whim or command

a) Ipse Dixit (“He himself said it,” or, “Because I said so”), an unsupported statement that rests solely on the authority of the individual who makes it

5

What Argumentation Is Not

6


May 9, 2019

Columbus, OH 3

Forensic Controversies

1) Diagnoses: should we provide them or not?

2) Ultimate Issue: should we ever directly answer the question?

3) Data Inclusion Issues: how do we best balance data relevancy3) Data Inclusion Issues: how do we best balance data relevancy with thoroughness?

4) Bias: should it be acknowledged?

7

Forensic Controversies

5) Psychological Testing: yes, no, or sometimes (and under what circumstances)?

6) Role Issues with Attorneys: consultant versus expert

7) Recording forensic evaluations or using third‐party observers(TPO): yes, no, or sometimes?

8) Should social media data be used in forensic evaluations?

8

Critical Thinking Can Help

•Helps us evaluate the arguments (and the evidence on which they are based) on both sides of a forensic controversy

•Helps us identify explicit and tacit assumptions in argumentsHelps us identify explicit and tacit assumptions in arguments

•Helps us formulate the kind of probing questions (of ourselves and others) to uncover the soundness of the arguments on both sides of a forensic controversy

•See Handouts #1 and #2 for critical thinking techniques9


May 9, 2019

Columbus, OH 4

The Forensic Controversies

10

#1: Should Diagnoses be Offered in Forensic Reports?

11

Diagnosis in Forensic Reports

• The controversy: Some argue that formal diagnoses are an essential part of a forensic report, while others argue that diagnoses are unimportant and can be misleading

• The question: If not required, should diagnoses be offered in forensic reports?

12


May 9, 2019

Columbus, OH 5

Why Should I Care? Diagnoses are Required Where I Practice

• Even forensic examiners who are required to include a diagnosis in forensic reports would benefit from understanding the arguments against doing so, as they clarify risk and unintended g g g , y yconsequences that can be associated with diagnoses

• Understanding counterarguments can help clarify considerations in accurately representing diagnoses

13

Cautionary Statement for Forensic Use of DSM‐5

• “In most situations, the clinical diagnosis of a DSM‐5 mental disorder… does not imply that an individual with such a condition meets legal criteria for the presence of a mental disorder or a specified legal standard (e.g., for competence, p f g ( g , f p ,criminal responsibility, or disability).”

• “For the latter, additional information is usually required beyond that contained in the DSM‐5 diagnosis, which might include information about the individual's functional impairments and how these impairments affect the particular abilities in question.”

14

Cautionary Statement for Forensic Use of DSM‐5

• “Nonclinical decision makers should also be cautioned that a diagnosis does not carry any necessary implications regarding…the individual’s degree of control over the behaviors that may be associated with the disorder.”

• “When DSM‐5 categories, criteria, and textual descriptions are employed for forensic purposes, there is a risk that diagnostic information will be misused or misunderstood. These dangers arise because of the imperfect fit between the questions of ultimate concern to the law and information contained in a clinical diagnosis.” (same as DSM‐IV‐TR)

15


May 9, 2019

Columbus, OH 6

The Question of Diagnosis

PRO

1) AAPL Practice Guideline

2) Borum and Grisso (1996):

CON

1) Morse (1978): formal diagnosis does not answer the legal question

87% of psychologists and 93% of psychiatrists indicated a formal diagnosis was “essential” or “recommended”

2) DSM‐5 caution: “In determining whether an individual meets a specified legal standard…additional information is usually required beyond that contained in the DSM‐5 diagnosis.”

16

The Question of Diagnosis(Melton et al. 2007)

ADVANTAGES

1) A useful construct to gather and evaluate data

2) Diagnostic criteria can be useful

DISADVANTAGES

1) Diagnosis itself is useless to the Court

2) The presence of a diagnostic in evaluating the validity of claimed impairments

3) Some jurisdictions have specific diagnoses that are excluded from consideration

entity does not necessarily mean that the criminal behavior was caused by that entity

3) Can lead to confusing and largely irrelevant battle about whether diagnostic criteria are met

17

Unmasking Forensic DiagnosisGreenberg, Schuman, and Meyer (2004)

This landmark article poses critical questions to help frame our thinking on this issue:

1) “When do diagnoses assist and when do they confuse in ) g y fthe forensic setting?” (emphasis added)

2) Are they trustworthy (i.e., reliable) enough to be used in court?a) DSM‐5 field studies and reliability ‐more

18


May 9, 2019

Columbus, OH 7

Unmasking Forensic Diagnosis(Greenberg et al., 2004)

1) A diagnosis can confuse rather than enlighten

a) The Court may misunderstand the validity of the diagnostic categories themselves, which have largely remained a collection of consensus‐based disorders

i Withi th di ti t i f A ti i l d Ob ii. Within the diagnostic categories of Antisocial and Obsessive‐compulsive personality disorders, it is possible for two cases to share no features at all

b) A diagnosis implies a false sense of certainty, accuracy, and precision

c) Forensic use of diagnostic categories medicalizes behavior and endows it with an aura of objectivity

19

Unmasking Forensic Diagnosis(Greenberg et al. , 2004)

2) Experts should offer diagnoses judiciously and realize that a functional analysis provides the Court with more helpful information without the confusion that diagnoses create

) h ld l dd l ll l b h3) Experts should always address legally relevant behaviors, capacities, and functioning

a) If the expert can show that offering a diagnosis adds information that is reliable, predictive of legally relevant functioning, and not unfairly prejudicial, then it should also be offered

b) If these criteria cannot be met, then diagnoses should not be included in the expert’s testimony

20

Unmasking Forensic Diagnosis(Greenberg et al. , 2004)

4) Whether someone is properly labelled as having a particular diagnosis is a clinical issue; whether that classification should serve as a legal determinant for competence, compensation or commitment is a legal issue”or commitment is a legal issuea) “It is for the law to decide if a particular diagnosis should

be an essential element in one legal context and not in another.” (emphasis added)

21


May 9, 2019

Columbus, OH 8

DSM‐5 and Reliability(Regier et al., 2012)

• Kappa = a measure of interrater reliability that accounts for chance agreement; ranges from ‐1.0 to +1.0

• Very good (kappa .60‐.79): 5 diagnoses ‐ PTSD, major neurocognitive disorder, autism spectrum disorder, ADHD, (revised) complex somatic symptoms disorder

• Good (kappa .40‐.59): 9 diagnoses ‐ schizophrenia, schizoaffective disorder, bipolar I disorder, binge eating disorder, alcohol use disorder, mild neurocognitive disorder, borderline personality disorder, avoidant/restrictive food intake disorder, ODD

22

DSM‐5 and Reliability(Regier et al., 2012)

• Questionable (kappa .20‐.39): 6 diagnoses ‐major depressive disorder (adults and children), generalized anxiety disorder, mild TBI, antisocial personality disorder, disruptive mood dysregulation disorderdysregulation disorder

• Unacceptable (kappa <.20): 3 diagnoses ‐mixed anxiety‐depressive disorder (adults and children), nonsuicidal self‐injury

23

DSM‐5 and Reliability (Frances, 2012)

• Only the five of the 23 DSM‐5 diagnoses that achieved kappa levels of agreement between .60 to .79 would have been considered "good" in the past

• The nine DSM‐5 disorders in the kappa range of .40 to .59 previouslyThe nine DSM 5 disorders in the kappa range of .40 to .59 previously would have been considered poor, but DSM‐5 describes them as "good"

• DSM‐5 calls “acceptable” the six disorders that achieved low reliabilities with kappas of .20 to .39, and finds unacceptable the three diagnoses that were below <.20 (which is barely better than chance)

24


May 9, 2019

Columbus, OH 9

DSM‐5 and Reliability

• Point: in the DSM‐5 field trials, results that found kappas at values which for thirty‐five years would have been judged “poor” or “unacceptable” suddenly became “good” (Cooper, 2014)

• Counterpoint: it is unrealistic to expect that the quality of psychiatric diagnoses can be much greater than that of diagnoses in other areas of medicine, where interrrater kappa values range between 0.6 and 0.8 (Kraemer et al., 2012)

• Regier et al. (2013): reliability of specific diagnoses25

Uncomfortable Questions

1) “So you are saying this defendant has a mental illness. Does this mental illness have a name?”

2) “How can I cross examine the witness on the issue of threshold diti if h ’t id tif h t th t diti i ?”condition if she won’t identify what that condition is?”

3) “If you won’t identify the specific diagnosis, isn’t that just Ipse Dixit?”

4) “How can I decide if something qualifies as a mental illness if you won’t name that mental illness?”

26

Diagnosis: Questions to Consider

1) Who asked you?

2) Is it required in the jurisdiction in which you practice?

3) Is the Court expecting it?

4) Are you prepared to answer questions about reliability and4) Are you prepared to answer questions about reliability and validity?

5) Will it help the judge understand the nature of defendant’s mental condition?

6) Will it help the judge understand the severity of the defendant’s mental condition?

27


May 9, 2019

Columbus, OH 10

Diagnosis: Questions to Consider

7) Will it lead to a distracting and possibly irrelevant battle over whether diagnostic criteria are met?

8) Would a careful description of symptoms be more useful than a diagnosis?g

9) If you must report a diagnosis, would be it be more useful to report a diagnostic class (e.g., a Schizophrenia spectrum and other psychotic disorder) that highlights specific symptoms?

28

#2: Ultimate Opinions

29

Ultimate Opinions

• The controversy: Professionals disagree on whether one should offer an ultimate opinion (e.g., the examinee is incompetent, disabled, committable) in a forensic report – some always do, some never do and some do in certain circumstancessome never do, and some do in certain circumstances

• The question: What factors should be considered when making a decision about including an ultimate opinion?

30


May 9, 2019

Columbus, OH 11

Definitions

• Ultimate Opinion: explicitly stating whether the defendant meets the statutory criteria e.g., “It is my opinion that the defendant is incompetent to

stand trial at this time.”

• Penultimate opinion: using statutory language to address the court’s question e.g., “It is my opinion that as a result of a mental illness, the

defendant presently lacks the capacity to understand the nature and objectives of the legal proceedings and to assist in her defense.”

31

Ultimate Opinions• Specialty Guidelines for Forensic Psychologists (Adopted by APA Council of Representatives, August 3, 2011) ‐ silent on this issue

• AAPL Practice Guideline: Insanity Evaluations (2014) – Since some jurisdictions do not allow psychiatric testimony regarding the ultimate issue “full and detailed reasoning based on thethe ultimate issue, full and detailed reasoning based on the standards of the jurisdiction’s insanity test should be discussed.”

• AAPL Ethics Guidelines for the Practice of Forensic Psychiatry (2005) ‐ neither precludes nor endorses

• Federal Rules of Evidence – Allows it for all issues except sanity32

To Offer or Not to Offer an Ultimate Opinion: An Unresolved Debate

OFFER• Rogers & Ewing (1989)1) Finders of fact are not

usurped by an ultimate opinion

DON’T OFFER• Tillbrook et al. (2003)

1) Our job is to articulate the nature and extent of clinical impairment

2) Up to the fact finder to2) Semantic brinksmanship is not meaningful

3) Not doing so leads to:

a) Testimony that is too broad

b) Difficult cross‐examination

2) Up to the fact finder to determine if #1 meets the legal criteria

3) There are no established criteria for determining how much impairment is needed for such things as “substantial capacity”

33


May 9, 2019

Columbus, OH 12

Slobogin – The “Ultimate Issue” Issue

Pro1) Many of the relevant legal

terms (e.g., appreciate) do have relevant clinical

Con1) Such testimony is not the

province of mental health professionals

content

2) Forensic opinions are often in close proximity to the legal test (e.g., impaired impulse control and capacity to conform)

2) Would have undue influence on the fact finder

34

Slobogin – The “Ultimate Issue” Issue

• Can help organize and clarify the expert’s evidence for the fact finder

• If we refrain from ultimate opinions, the court may feel we are t b i t i htf d t i t hid thinot being straightforward or trying to hide something

• Ultimate opinions should be allowed if the protections afforded by cross‐examination, rebuttal evidence, and cautionary jury instructions are usedThe more “ultimate” testimony becomes, the more vigorous should be the testing of our inferences

35

Heilbrun on Ultimate Opinions

• Principle #30: Carefully consider whether to answer the ultimate legal question

• Guideline: Focus on conclusions about forensic capacities butGuideline: Focus on conclusions about forensic capacities but not the larger legal question(s). If ultimate legal question is answered, it should be in the context of a thorough evaluation clearly describing data and reasoning, and with the clear recognition that this question is in the domain of the legal decision‐maker

36


May 9, 2019

Columbus, OH 13

Is it OK not to Have an Opinion?

• Have you obtained an adequate base of reliable data to answer the legal question?

• If not:If not: you do not have to have an opinion you better have an explanationA case example

• “Opinions must not exceed the data provided.”Packer (2009)

37

Ultimate Opinions ‐ Recommendations

1) Be aware of the statutory provisions of the jurisdiction in which you are doing the evaluation

2) Be familiar with the differing recommendations of best practice source documentspractice source documents

3) Understand the arguments for and against

4) Maintain an awareness of what the judge expects

5) If the judge directly asks, the law allows, and the data support such an opinion, safe to proceed with providing onea) And if not…

38

Final Thought: A Red Herring?

• Why was this “controversy” raised to begin with?

• Morse (1978): Mental health professionals can contribute to legal decision‐making by providing scientific information that islegal decision making by providing scientific information that isnot otherwise available to the law. This means explanation

• Elwork (1984): In practice, clinicians have been ill‐prepared to explain or defend their positions. They rely too heavily on Ispe Dixit reasoning

39


May 9, 2019

Columbus, OH 14

A Red Herring

• Judge David Bazelon (1974):

• “One might hope that psychiatrists would open up their reservoirs of knowledge in the courtroom. Unfortunately ... they try to limit their testimony to conclusory statements couched in psychiatric terminology.”

• “Thereafter they take shelter in a defensive resistance to questions about the facts that are or ought to be in their possession. They thus refuse to submit their opinions to the scrutiny that the adversary process demands.”

40

A Red Herring

• When we adequately explain our opinions, the fact finder is free to draw conclusions independent of those proposed by the psychologist

• Skeem and Golding (1998) went so far as to assert that providing an opinion is “the least important part” of the forensic examiner’s task

• Only failure to explain one’s reasoning will “usurp the judicial decision‐making role”

41

#3: Data Inclusion Issues: How does one Balance Relevance and Thoroughness?

42


May 9, 2019

Columbus, OH 15

Data Inclusion

• The controversy: Some say that a good forensic report should be inclusive, “tell the story” of the examinee, and guided by comprehensiveness. Others assert that a good forensic report should have a sharp focus on the issue at hand, be more limited in scope, and guided by relevance

• The question: How deep do we go into personal (and often sensitive) history if the data have limited correspondence to the functional legal capacities being examined?

43

Arguments for Comprehensive

• Support for a traditional structure as best practice:1) Melton et al. (2007): a) “Attention to detail and accuracy of information is

required.” (p. 583)b) “ l h l d l d ”b) “Some encourage lengthy, overly detailed reports”…

“that ‘cover all the bases’ and leave the examiner an ‘out’ from a tight situation in testimony.” (p. 585)

c) A useful report should be “comprehensive…relevance to the fact finder, not tactical considerations… should guide the clinician in condensing data.” (p. 586)

44


2) Specialty Guidelines (Comprehensive and Accurate Presentation of Opinions in Reports and Testimony, 11.04): “Forensic practitioners strive to offer a complete statement of all relevant opinions … the basis and preasoning underlying the opinions, the salient data or other information that was considered in forming the opinions, and an indication of any additional evidence that may be used in support of the opinions to be offered.”

45


May 9, 2019

Columbus, OH 16


3) AAPL Ethics Guidelines: enhance honesty and objectivity by basing “forensic opinions, forensic reports and forensic testimony on all available data”

46


4) Things we have heard:a) “I was trained to be thorough.”

b) “If I write shorter reports it looks like I was cutting corners and decreases the confidence of the reader in my opinion ”and decreases the confidence of the reader in my opinion.”

c) “If I testify I want ALL the facts in my report for easy reference.”

d) “It’s easier to just throw it all in there.”

47

The Traditional Forensic Report• HistoryIn clinical settings, referrals for psychiatric and psychological evaluations

Wide variety of referral questions

A broad net encompassing history, mental status, clinical interview,A broad net encompassing history, mental status, clinical interview, and psychological test results

• Forensic evaluation reports adopted this structure from clinical assessments

• Reid Meloy, Ph.D. – applying this model to forensic assessments is like putting a square peg in a round hole

48


May 9, 2019

Columbus, OH 17

The Traditional Forensic Report

1) Reason for Evaluation

2) Confidentiality Limitations

3) Evaluation Procedure

4) Sources of Information

5) History (social, educational, occupational, military, medical, mental health, substance use, legal)

6) Collateral

7) Mental Status

8) Forensic Issue (e.g., CST, Sanity, Custody, Mitigation)

9) Opinion49

The Traditional Forensic Report

• At risk for being guided by the principle of being comprehensive: throw it all in there

• Provides the opposing side an opportunity to carefully evaluate the data for evidence that could be interpreted differently

• Often focuses on “telling the story” of the individual being examined against the backdrop of the referral question

• Question: is all the data really there or are we filtering?

50

Levels of Filtering

• Question: Why would we filter data?

• Answer: Because we can’t help it!

1) We don’t write down everything an examinee says

2) We don’t record every behavioral observation we make

3) We don’t simply transcribe our notes and put everything from them into our report

4) We don’t include in our report every thought, musing, etc. we have about an examinee

51


May 9, 2019

Columbus, OH 18

What Does “Relevant” Mean in a Forensic Context?

• Per Merriam‐Webster.com:

1) Having significant and demonstrable bearing on the matter at hand at hand

2) Affording evidence tending to prove or disprove the matter at issue or under discussion (e.g., relevant testimony)

52

Relevancy

• Definition: pertinent connection with the matter in hand

• “Relevance” to us is like “probative” to the Court

• Traditional report str ct re allo s for and ma enco rage a• Traditional report structure allows for – and may encourage – a good deal of irrelevant, non‐probative data, some of which may be sensitive in nature

• The best practices literature in forensic psychology strongly encourages data relevancy, and any report structure should maximize it

53

Admissibility of Evidence

LEGAL

1) Material: has to do with one of the case elements

2) Probative: tends to prove a

FORENSIC

1) Material: has to do with the capacity in question

2) Relevant: data must make an2) Probative: tends to prove a proposition, no matter how small the step

3) Competent: not excluded by a rule (e.g., hearsay)

2) Relevant: data must make an inference more or less likely, no matter how small the step

3) Competent: no excluded by professional standards

54


May 9, 2019

Columbus, OH 19

Relevance in the Specialty Guidelines

Specialty Guideline 9.01: examine the issue “from all reasonable perspectives and seek information that will differentially test plausible rival hypotheses.”

Specialty Guideline 11.04: practitioners should limit discussion of historical data “that does not bear directly upon the legal purpose…” and avoid information “that is irrelevant and does not provide a substantial basis of support for their opinions except when required by law.”

55

Grisso on Relevance

Grisso (2014):

• Reports should be written with “enough detail to establish important facts and track essential inferences, yet in a style of a digest rather than a meticulous thesis ”of a digest rather than a meticulous thesis.

• “The more detail you include, the greater the risk that the message will get lost in the words.” (p. 100)

56

Principles of Forensic Mental Health Assessment ‐ HeilbrunRelevancy noted in 5 separate principles

a) Principle #8: Identify relevant forensic issuesb) Principle #18: Use relevance & reliability (validity) for seeking

information and selecting data sourcesinformation and selecting data sourcesc) Principle #19: Obtain relevant historical informationd) Principle #20: Assess clinical characteristics in relevant,

reliable, and valid wayse) Principle #21: Assess legally relevant behavior

• Some data points matter more than others57


May 9, 2019

Columbus, OH 20

Relevancy in Court

• Our legal system controls what gets introduced as evidence in a court of law in order to:

1) Avoid wasting of time and resources2) Providing undue advantage or disadvantage to one side or ) o d g u due ad a tage o d sad a tage to o e s de o

another3) Avoid the sense of impropriety or imbalance in our justice

system

• Relevancy is the primary sifting tool used to determine whether or not a piece of information should be brought into the courtroom during a case

58

Relevancy

• Relevancy is not a characteristic of types of evidence

You would never be able to properly say, “This type of evidence is always relevant.”

• Relevancy is contextualA characteristic of any given piece of evidence that depends upon the relationship between that piece of evidence and a matter to be proved in that particular case

59

Balancing Relevancy & Sufficiency

1) FRE 702: examiner’s opinion must be based on “sufficientfacts or data”

2) FRE 703: “If experts in the particular field reasonably rely on those kinds of facts or data in forming an opinion on thethose kinds of facts or data in forming an opinion on the subject, they need not be admissible for the opinion to be admitted. But if the facts or data that would otherwise be inadmissible, the proponent of the opinion may disclose them to the jury only if their probative value in helping the jury evaluate the opinion substantially outweighs their prejudicial effect.”

60


May 9, 2019

Columbus, OH 21

FRE 403: Probative vs. Prejudicial

• The Court has to weigh each piece of evidence to determine if that evidence is more useful to ascertaining the truth in this case than it is prejudicial to the side opposing it

• Probative = evidence that tends to prove something

• Prejudicial = an undue tendency to suggest decision on an improper basis, commonly, though not necessarily, an emotional one

61


• AAPL Practice Guideline for Forensic Assessment (2015): the word “relevant” is used 46 times “The evaluator can first ask for a full, uninterrupted

account of the events in question, followed by a q ysecondary review with questions probing for detail, consistencies, contradictions, and relevant facts.”

62


• ABA Criminal Justice Mental Health Standards: “…evidence relevant to mens reamay often come in the form of expert testimony. The only limitation on such testimony should be relevance and the normal requirements for expert opinion ”relevance and the normal requirements for expert opinion.

63


May 9, 2019

Columbus, OH 22

Federal Rules of Evidence

• Rule 401. Test for Relevant Evidence

• Evidence is relevant if:a) it has any tendency to make a fact more or less probable

h ld b h h d dthan it would be without the evidence; andb) the fact is of consequence in determining the action

• The standard of probability: “more probable than it would be without the evidence”

64

Data Inclusion: Recommendations

1) The whole point of some forensic evaluations (e.g., death penalty mitigation) is to “tell the story” – in this context, be inclusive and comprehensive

2) If the point of the evaluation is more focused on a specific question2) If the point of the evaluation is more focused on a specific question, then focus on data that help answer the question

a) If a data point does not move you towards or away from a conclusion or inference, why is it there?

b) Example: “The defendant is a 62 year‐old, left‐handed Caucasian male of average height and weight.”Which of these data points are relevant in a violence risk assessment?

65


3) Consider how the core guiding value of relevance can help determine what should and should not be included in a reporta) What is relevant in one report may not be in the next

4) If focusing on Relevance, take care to avoid Confirmation Bias, a tendency to restrict reported data points to those that support your opinion

66


May 9, 2019

Columbus, OH 23


5) Dealing with Confirmation Bias:a) Disconfirmatory ≠ irrelevant

b) A reader who does not like your opinion does not have the option of scouring the report for data that do not supportoption of scouring the report for data that do not support your opinion or could be interpreted differently

c) Report and explain data that do not support your opinion

d) A structural remedy

67

#4: Bias

68

Bias

• The controversy: Almost everyone agrees that bias should be avoided, but people differ regarding the extent to which they should explicitly acknowledge bias or discuss how it might limit one’s opinions

• The question: How can one deal with bias in evaluations, discuss bias in reports, or acknowledge bias in testimony without potentially undermining one’s own opinion?

69


May 9, 2019

Columbus, OH 24

Definition of Bias

• Black's Law Dictionary: "A predisposition to decide a cause or an issue in a certain way.”

• West & Kenny (2011): “Any systematic error (i.e., not random error) that determines judgment other than the truth.”

• Free Online Dictionary: “A preference or an inclination, especially one that inhibits impartial judgment.”

70

Bias in Forensic Evaluation(Neal & Grisso, 2014)

• There is a negative connotation associated with an inappropriate personal or emotional involvement on the part of the evaluator

1) Explicit: bias that stems from purposefully putting a “spin” on the evaluation – the “hired gun”

2) Implicit: bias that is outside the examiner’s awareness

71

Biased about Bias?

• “Objectivity” is held out as a core foundational value in forensic (SGFP, 2013) and psychological assessments (APA Ethics Code, 2002)

• Neal and Brodsky (2014): forensic‐clinical psychologists are ti ll i li d t b li t l th t h ld b toccupationally socialized to believe not only that we should, but

that we can and do practice objectively

• Faust and Ahern (2012): forensic mental health clinicians underestimate the prevalence and severity of the influence of bias in our work

72


May 9, 2019

Columbus, OH 25

Awareness of Bias in Forensic MHA (Neal and Brodsky, 2015)

• Falls on a continuum from denial to the belief that it is inevitable

• Bias can be associated with systematic errors in decision‐makingmaking

• Bias in forensic decision‐making has been established empirically (Murrie et al., 2013) and identified as a concern by judges (Krafka et al., 2002)

• We tend to be more concerned about bias in others than in ourselves (the “bias blind spot”)

73


• Bias is associated with:1) Evoked cognitive and emotional reactionsa) Disliking the defendant

b) Anger about the offense

c) Being too sympathetic or empathic

d) Defendant reminds us of someone about whom we have strong emotional reaction

74


• Bias is associated with (Cont’d):2) Pre‐existing personal, moral, or political valuesa) Policy ideologies (e.g., “The DA is charging younger and

younger adolescents as adults and I am against that.”)you ge ado esce ts as adu ts a d a aga st t at )b) Death penalty attitudesc) Limited cultural competenciesd) Reluctance to adopt shifting social normse) Time and professional experience (e.g., “Some clinical

subgroups never really had a chance.”)

75


May 9, 2019

Columbus, OH 26


• Bias is associated with (Cont’d):3) Influence of interested othersa) Non‐obvious alliance with referral sourceb) Influence of colleaguesb) Influence of colleagues

4) Economic effecta) Financial factors (e.g., Will my opinion influence future

referrals?)

76

What to do about Bias ‐ Options

1) Nothinga) What me worry?b) I don’t have biases but everyone else does

2) Get educated and be aware2) Get educated and be awarea) Forensic examiner, know thyselfb) Introspect

3) Apply critical thinking and de‐biasing techniquesa) Actively apply debiasing strategiesb) Be prepared to discuss this on the witness stand

77

Option #1: Do NothingRISKS

1) Status quo

2) How can we grasp, consider, and evaluate

BENEFITS

1) If I can ignore this “so called” threat I will have more time for doing evaluationconsider, and evaluate

alternative points of view (not necessarily accepting them as equally valid) without considering bias?

2) Considering bias could lead to a naive or unexamined relativism in which all perspectives are valued equally

78


May 9, 2019

Columbus, OH 27

Option #2: Get Educated and be AwareRISKS

1) Research demonstrates that simple passive awareness of the source of cognitive bias is insufficient by itself to

BENEFITS

1) A good foundation

2) Build up a comfortable and false sense of confidence (“Forewarned isy

prevent a person from being trapped by biases (Ariely2008; Cialdini 2001)

( Forewarned is forearmed.”)

3) No real material impact on minimizing bias due to passivity

79

Option #3: Apply Critical Thinking and De‐biasing Techniques

RISKS

1) Change is hard

2) There is no cure: remedies must be applied over and

BENEFITS

1) Having a strong answer to this question: “Dr., what steps did you take to must be applied over and

over again

3) What works on one case may not work on another

p yminimize the influence of bias on this case?”

80

Bias Correction Strategies(Neal and Brodsky, 2016)

1) Embracing professional pridea) Develop pride in one’s professional identityb) Preventing negative perceptions of one’s workc) Striving to meet peer expectations

2) Formal and informal educationa) Training about objectivity (workshops, reading)b) Observing others who manage their personal biases

successfully (supervision, peers, learning from mistakes)

81


May 9, 2019

Columbus, OH 28


3) Relying on dataa) Investigating all relevant data before forming an opinion

(resisting early conclusions, collecting thorough data from multiple sources)

b) Taking time to think about the evaluation information rather than immediately writing the report

c) Critically examining conclusions (considering alternative hypotheses, seeking disconfirming data)

82


3) Relying on data (cont’d)d) Being an active consumer of scientific information

(being grounded in science, being aware of common sources of bias and using correction strategies)

e) Basing conclusions and opinions on sound data (grounding conclusions in data)

f) Examining patterns of personal decision‐making (agreement with referral party preferences)

83


4) Restricting the scope of opinionsa) Restricting conclusions and opinions to scientific

foundations (recognizing the limits of science, restricting opinions appropriately)p pp p y)

b) Clarifying the referral question and limiting the scope of inquiry and report to the referral (understanding the referral question and limiting inquiry to that issue)

84


May 9, 2019

Columbus, OH 29


5) Using procedural and structural supportsa) Using structured evaluation methods (using objective

measures that reduce subjective judgment)

b) Taking careful notes during an evaluation and relying on them

c) Consulting with colleagues about issues of potential bias (seeking feedback from colleagues, fostering a culture of openness to discussing bias awareness)

85


6) Resisting adversarial allegiancea) Being aware of and resisting allegiance effects (limiting

contact with retaining party, developing financial referral‐based independence, seeking referral source variety)

b) Avoiding advocacy for one side or the other (being aware of the subtle influence of colleagues)

86


7) Introspecting to recognize biasa) Fostering a continuing commitment to objectivity

(nurturing humility)

b) Continuous introspection about potential biasesb) Continuous introspection about potential biases (introspecting about potential biases, paying attention to strong reactions, examining countertransference feelings)

87


May 9, 2019

Columbus, OH 30


8) “Controlling” biasa) Intentionally controlling existing bias

b) Attending to wording choice in reports to edit out value‐laden languageladen language

c) Accepting referrals only for cases in which bias is unlikely (anticipating bias to avoid compromise of objectivity, compromised objectivity may require withdrawal)

88


9) Disengaging emotionally from cases

a) Limiting empathy and rapport in forensic cases

b) F t i ti l d t h tb) Fostering emotional detachment

c) Diffusing responsibility as a strategy to distance self

89

Neal and Brodsky (2016) ‐ Results

1) Highest rated strategies in terms of usefulness: a) Critically examining conclusions (e.g., considering

alternative hypotheses)b) Investigating all relevant data before forming an opinionb) est gat g a e e a t data be o e o g a op o

2) Lowest rated strategies in terms of usefulness: a) Accepting referrals only for cases in which bias is unlikelyb) Limiting empathy and rapport in forensic cases

90


May 9, 2019

Columbus, OH 31

Do we Kneel at the Altar of Introspection?

• Yes

• Pronin et al. (2007): introspection identified by forensic evaluators as the primary strategy to combat bias

• Neal & Brodsky (2016): bias awareness falls on a continuum, with some board‐certified (ABPP) specialists denying the potential for bias, others seeing it as inevitable

100% of respondents reported attempting to minimize bias via introspection

• Zapf et al. (2017): 87% of respondents reported they could minimize bias by consciously trying to set aside biases and expectations

91

Should we Kneel at the Altar of Introspection?• No

• Nisbet & Wilson (1977): introspection is not an effective bias reduction strategy due to the inherent difficulty in accessing biases that take place outside our awareness

• Pronin & Kugler (2007): the Introspection Illusion leads us to• Pronin & Kugler (2007): the Introspection Illusion leads us to overemphasize the effectiveness of conscious introspection to combat implicit bias that happen outside awareness

• Bias Blind Spot: first identified by Pronin, Linn, & Ross (2002) as a tendency to see bias in others rather than in oneself

• Zapf et al. (2017): 79% see bias as a problem in others, 52% see bias as a problem in their own evaluations

92

Does Education about the Ineffectiveness of Introspection Help?

• No

• Zappala et al. (2018): Anything You Can Do I can Do Better: Bias Awareness in Forensic EvaluatorsProviding an article about the fallibility of introspection to mitigate bias did not result in lower ratings about the value of introspection

• The worst thing you can do: passively assume that forewarned is forearmed

93


May 9, 2019

Columbus, OH 32

So What Are We supposed to Do about Bias?

94

Managing Bias in Forensic Evaluation Shuman & Zervopoulos (2010)

1) Use your expertise, background knowledge, and examination data to generate plausible alternative explanations that explain the data in light of the legal question being asked

2) Actively challenge each plausible alternative explanationa) generate reasons why your conclusions may be wrong and

why another possibility may be correct

95

Reducing Bias in Forensic PracticeNeal and Grisso (2014)

1) Changing practice about errors and bias requires not just knowledge but motivation and practice

2) Anchor with base rates and critically evaluate the strength of case‐specific information

3) Consider the opposite

4) Structured approaches increase reliability (an imperfect but useful substitute for improving validity)

5) Identify 4 to 6 variables that are key to the question

6) Forensic due process: two sides each working from common data base

96


May 9, 2019

Columbus, OH 33

Bias – Recommendations

1) In forensic evaluations, take reasonable steps to minimize bias

2) In forensic reports strive for objectivity but without2) In forensic reports, strive for objectivity but without explicitly drawing attention to how you handled potential bias

3) In testimony, be prepared to respond to a question about how you accounted for potential bias

97

Read the Other Expert’s Opinion First?YES

1) Allows for consideration of consistency across examiners

2) We should consider all relevant data

3) We are not so fragile that doing so

NO

1) Specialty Guidelines 1.02 and 2.07 indicate we need to avoid bias

2) Doing so exposes us to 3) We are not so fragile that doing so would corrupt us

4) Everything we think is shaped by something else

5) Thoroughness and sound logic outweigh independence

6) Essential detail we might otherwise miss

) g pframing and anchoring bias

3) None of us are immune from bias and we overestimate our ability to deal with it

98

Dealing with Bias in Report Writing

• Demonstrate objectivity by presenting countervailing data, then analyzing and explaining why you do not think such data are persuasive

• “In this case, there are data weighing on both sides of the t s case, t e e a e data e g g o bot s des of t eissue.”

• “Weighing on the side of X are…”

• “Weighing on the side of Z are…”

• “In my opinion, the data indicating X are more persuasive, for the following reasons…”

99


May 9, 2019

Columbus, OH 34

Dealing with Bias in Testimony

• “I’m glad you asked. Bias is a potential threat to objectivity in any issue involving interpretation of data.”

• “In this case, I took the following steps, including…”) b d d f l l1) Obtained data from multiple sources2) Used structured data acquisition3) Considered alternative explanations for the data

4) Tested alternative explanations to see which best fit the data5) Investigated all relevant data before forming an opinion6) Consulted with colleagues

100

#5: Psychological Testing

101

Psychological Testing

• The controversy: Some forensic psychologists routinely use psychological testing and/or forensic assessment instruments, while others do so only in specific circumstances

• The question: What factors should be considered when making decisions about including psychological testing and/or forensic assessment instruments as part of forensic assessment?

102


May 9, 2019

Columbus, OH 35

Heilbrun on Psychological Testing

• Principle #26: Use testing when indicated in assessing response style

• Guideline: Administer test(s) sensitive to response style, Guideline: Administer test(s) sensitive to response style, particularly when there is concern about the accuracy of self‐report

103

The Value of Psychological Testing in CST Evaluations (Stafford and Sellbom, 2012)

• Competency assessment instruments do not measure capacity, or whether there is an underlying condition that contributes to a defendant’s poor functioning

P h l i l t ti b f l i bj ti l i h th• Psychological testing can be useful in objectively assessing whether displayed deficits are:1) genuine or malingered2) the product of intellectual disability3) associated with cognitive deficits due to TBI or brain

dysfunction4) associated with mood or psychotic disorders

104

Data Collection: Psychological Testing

• May be warranted to provide further information about specific, relevant characteristic or condition

• Chosen according to particular issue to be examined & characteristics of the defendant to be testedcharacteristics of the defendant to be tested

• Clear link to relevant legal issue of CSTGeneral intelligence, memory, contact with reality, motivation, reasoning or problem solving, emotional control

• As much as possible match the characteristics of the population from which the defendant belongs

105


May 9, 2019

Columbus, OH 36

Data Collection: Testing

• Advantages of FAIs (Grisso, 2003)1) Conceptuala) Structureb) Communicationb) Communicationc) Empirical foundation

2) Procedurala) Reduction of error and bias b) Meaningful comparison across time c) Collection of data on normative samples

106


• Borum & Grisso (1995):

1) 66% of psychiatrists and psychologists rated testing as “essential” or “recommended”

2) Results do not support a standard such that testing should be used in every case

3) Psychologists should be able to explain why they did not do testing in competency cases

4) Most commonly used tests:

a) MMPI (94%)

b) WAIS (78%)

c) Rorschach (32%)107


• Goldstein et al. (2003): most useful when used judiciously and when inferences from testing are conservatively made & verified by corroborative, real life data

C b d t• Can be used to: 1) Help establish mental disorder2) Assess malingering3) Establish enduring psychological traits, characteristics and

behavior patterns that may be relevant to personality disorder

108


May 9, 2019

Columbus, OH 37

Nomothetic v. Idiographic

• Nomothetic: instruments based on generalized normative interpretationTests/FAIs scoredScores interpreted against norms

• Idiographic: instruments based on specific and contextual characteristics of the individual being examined Tests/FAIs may or may not have formal scores (e.g., structured professional judgment)No norms

109

Psychological Testing ‐ Recommendations

• Kukor’s position: Test when necessary1) What might testing tell me that I do not already know?a) Ambiguous casesb) Establishing a threshold conditionb) Establishing a threshold condition

2) Might I use testing to produce idiographic rather than nomothetic data?

3) Stewardship of resources4) Don’t ever just copy/paste from computer‐generated

interpretations

110

DeMier’s Schema for Discussing Psychological Testing

• Identify the test and its purpose

• Explain why it was chosen in this case

• Identify and explain any deviations from standardizedIdentify and explain any deviations from standardized administration

• Describe the results in plain language

• Link those results to your hypotheses about the individual

• Discuss hypotheses raised by discrepant test data

111


May 9, 2019

Columbus, OH 38

#6: Role Issues with Attorneys

112

Role Issues

• The controversy: Some psychologists maintain a strict boundary between the roles of “expert” (evaluate and testify) and “consultant” (identify issues in the report of another expert, assist the attorney regarding strategic decisions) while others do not, seeking to “win.”

• The question: Is it ever appropriate to provide consultant‐level litigation support services (e.g. assistance in the development of questions for cross‐examination) and expert testimonial services in the same case?

113

Specialty Guidelines for Forensic Psychology Guideline 1.01: Integrity

• “Forensic practitioners strive for accuracy, honesty, and truthfulness in the science, teaching, and practice of forensic psychology and they strive to resist partisan pressures to

id i i h i h d b i l diprovide services in any ways that might tend to be misleading or inaccurate.” (emphasis added)

• But, are the Specialty Guidelines setting us up to fail?

114


May 9, 2019

Columbus, OH 39

Adversarial Allegiance

• Subjectivity and bias exist in forensic science procedures, including those that would not seem prone to bias, such as DNA and fingerprint analysis

C b t f i di l t b k• Concerns about fairness among medical experts go back over a century Hand, L. (1901). Historical and practical considerations regarding expert testimony. Harvard Law Review, 15, 40–58

115


• Adversarial allegiance defined: The presumed tendency for experts to reach conclusions that support the party who retained them (Murrie et al., 2009)

• Trials of sexually violent predators provide an ideal context for studying this phenomenon, as experts often use the same data and instruments

• Field studies of SVP trials strongly suggest adversarial allegiance in those proceedings (Murrie et al., 2008; Murrie et al., 2009)

116


• What else could explain the field trial findings suggestive of adversarial allegiance?1) Selection effects: Experts chosen by attorneys2) Attorneys may have chosen particular experts because the

attitudes and biases of those experts were already knownattitudes and biases of those experts were already known to be conducive to the case

3) Attorneys may have consulted many experts, but arranged testimony only from those whose opinions supported the case

• Need for a true experiment117


May 9, 2019

Columbus, OH 40

Murrie et al. (2013): Are Forensic Experts Biased by the Side that Retained them?

• Over 100 experienced forensic clinicians were recruited to receive two days of CE training in the PCL‐R and Static‐9R

• They were then asked to review four cases, for a fee, and were led b l h h h l k h fto believe that this might lead to more work in the future

• They were told, randomly, whether they were consulting to the prosecution or the defense

• Manipulation checks for clinician attitudes and efficacy of experimental deception

118


• Murrie et al. (2013) found “a clear pattern of adversarial allegiance.”“Some experts who score ostensibly objective assessment instruments assign scores that are biased toward the side that retained them.”“The pull of adversarial proceedings tends to influence opinions by paid forensic experts.”

• What to do about these findings? (Hint: The answer is not to ignore them.)

119

Adversarial Allegiance: Recommendations

• Recognition of the bias is the first step, but empirical studies indicate passive awareness of potential for bias is not enough

• Employ the cognitive tactics that we have been describing all day

• If asked on the witness stand about this bias (or these articles), what would you say?

120


May 9, 2019

Columbus, OH 41

Switching Roles

• How does one avoid sliding between the roles of testifying expert and consultant?

• Is the line between the two clear or fuzzy?

• Who can describe an experience when they felt pressure by an attorney to slide into another role?

121

Consultant Expert

• This can happen when you have been hired to consult on a case, and the attorney decides you would be an excellent expert

• Importance of role clarification at outset • Are there circumstances under which this is permissible?What would they be?What would they be?What are the risks?

• Are there circumstances under which this would never be permissible?

Perhaps the answer lies in whether you have developed (and identified) a partisan biasIf so, stay off the witness stand

122

Expert Consultant

• In our experience, this is more common

• Why does it happen?1) Economics – You are already there, so why hire a

cons ltant?consultant?2) Simplicity – You are already there and know the case, so

why hire a consultant?3) It probably never occurs to most attorneys that this would

pose an ethical conflict for psychologists

• Importance of role clarification at outset 123


May 9, 2019

Columbus, OH 42

Expert Consultant

• Is it legitimate for an attorney to ask how your opinion differs from that of an opposing expert?

• Is it legitimate for an attorney to ask you to help design a cross‐i ti f th t i t?examination for that opposing expert?

• Are these two questions the same or different?

• Should I testify as a consultant?

• Should I write a report as a consultant?

124

Expert Consultant

• Our stance:

1) It is perfectly legitimate for an attorney to ask how your opinion differs from that of an opposing expert

2) F i d t d d th t d

125

2) Fairness and transparency demand that you respond honesty

3) Focus your responses on explanations about your own opinion, not explaining why the other expert is wrong

4) Explicitly helping with cross‐examination crosses the line

#7: Audio or Video Recording Forensic Evaluations and Using Third‐Party Observers (TPO)

126


May 9, 2019

Columbus, OH 43

Recording Evaluations and Third Party Observers

• The controversy: some argue that recording evaluations or having 3rd party observers fundamentally affects the validity of evaluation

• The question: when and under what circumstances should recording and/or 3rd party observers be used, and if they are, what steps should be taken to protect the integrity of the evaluation?

127

Does Recording = Observing?

128

“Psychologists appear to be particularly perplexed about the appropriateness of allowing third parties to be present during psychological evaluations conducted in the context of legal

proceedings ”proceedings.

~ Otto & Krauss, 2009

129


May 9, 2019

Columbus, OH 44

Who are these Observers, Anyway?

• They might include:1) Other mental health professionals2) Attorneys3) Interpreters4) Students4) Students5) Security personnel

• What are their roles?

• Will their presence affect the validity of the evaluation?

• Do attorneys have a right to be present? (In some places, and some types of litigation, the answer is yes. See Otto & Krauss, 2009)

130

An Inherent Contradiction?

• Hedge and Brodsky (2013) note that despite the concerns about third party observers, psychologists and neuropsychologists routinely allow the observations – and even participation – by students, interns, and fellows

• If the presence of other observers risks an adverse consequence, how are these individuals any different?

• Need to balance ethical practice with valuable graduate student learning experiences

131

Graduate Students as TPOs (Hedge & Brodsky, 2013)

• NAN considers graduate student observation of neuropsychological evaluation to be an exception to their objections to TPOs

• Graduate students may be different than other TPOs, particularly if they are actively participating in the evaluation

• APA expects graduate students to be trained to observe in a manner that minimizes threats to the evaluation process (e.g., test security issues)

• No known formal empirical studies on impact of graduate student observers on forensic evaluations

132


May 9, 2019

Columbus, OH 45


• The big question: do the benefits outweigh the risks?

• The big answer (per Hedge & Brodsky): Yes, because the small negative impact on standardized measures is outweighed by the benefits on training

133


• Recommendations:

1) Inform retaining parties and include an explanation during notification procedures about graduate student participation

2) Acknowledge graduate student presence in reports2) Acknowledge graduate student presence in reports

3) Allow for a period of adaptation during the interview before testing

4) Establish ground rules (e.g., maintaining confidentiality)

5) Consider the case and consider the student

134

Does it Make a Difference with TPOs if Testing is Involved?

• Yes

• Psychologists have ethical obligations regarding test security

• The presence of a TPO may impact standardized test administration, BUT did normative settings include:1) High‐stakes, high‐money litigation2) Testing in a tiny, freezing room in a jail3) Having been coached to some unknown degree by someone4) Having researched tests on the internet to do well/do poorly

135


May 9, 2019

Columbus, OH 46

But…

• How can it be asserted that one cannot make sense of test data that were gathered while a TPO was simply present observing when we routinely interpret test data obtained:1) With the assistance of an interpreter (whose presence is

arguably more obtrusive than the presence of a TPO)or

2) Via a psychometrist whom the psychologist sometimes observed

136

Concerns about Third‐Party ObserversOtto & Krauss (2009)

• Negative effects on the examinee’s responses and participation

• Interruption of the flow of information from the examinee to the examiner

• Threats to the validity of conclusions that can be drawn from the evaluation

• Threats to the security (and future utility) of psychological assessment techniques and tests

137

What Do the Neuropsychologists Say?

• See Cramer and Brodsky (2007) for a historical overview of this controversy

• The position of the National Academy of Neuropsychology (NAN) i th t thi d t b h ti i t this that third party observers have a negative impact on the evaluation and violate standardized testing procedures

• American Academy of Clinical Neuropsychology (AACN) distinguished between “involved” and “uninvolved” observers

138


May 9, 2019

Columbus, OH 47

What Data Do We Have on This Issue?

• Binder and Johnson‐Greene (1995); McCaffrey, Fisher, Gold, & Lynch (1996); Lynch (2005); Howe & McCaffrey (2010);

• The studies often referenced by neuropsychologists objecting to recording or observers eco d g o obse e s

• These studies do show evidence of some negative effect on performance

• However, much of the research was done on college students, so there may be problems with generalizability to those undergoing a forensic evaluation

139

Do We Have Ecologically Valid Data?

• No

• Ecological validity = a measure of how test performance predicts behaviors in real‐world settings

• Ecologically valid research on this issue would need to demonstrate that when you have a criminal defendant who may be mentally ill or cognitively impaired and facing serious time in prison or a death penalty, or that when you have a plaintiff who has millions of dollars at stake for a brain injury claim – that there is some ADDITIONAL effect of being observed/recorded AND it is definitely negative

140

Validation Samples of the Tests We Have

• Few tests employed by psychologists were normed on individuals completing them in litigation contexts

• Some psychologists believe that they can account for this significant difference as they interpret the test data

• Yet many of these same psychologists apparently believe that there is just no way to account for the effects of recording or TPOs when interpreting test data (or maybe they believe that, although they can account for such effects, they should not have to)

141


May 9, 2019

Columbus, OH 48

Eastvold et al. (2012)• Meta analysis of all known articles dealing with the effects of being observed or recorded during cognitive task performance

• 210 studies, 62 of which they deemed to meet criteria for comparisons

• Most of the studies of the effect of having an observer or knowledge that one is being recorded during neuropsychological examination show noone is being recorded during neuropsychological examination show no effect or have the effect of improving performance

• Some studies show a negative effect of being observed or recorded, decreasing performance

• The overall effect across all studies included for review is near zero

• No systematic phenomena from being observed or recorded

142

Eastvold et al. (2012)

143

Other Potential Factors Influencing Test Results1) A criminal defendant who may be mentally ill and/or cognitively

impaired and facing a significant time in prison if found competent and/or sane

2) A plaintiff in a civil action who has millions at stake for a brain injury claim

• In such situations, is there some ADDITIONAL and negative effect of being video recorded?

• We know of no research examining such factors, but suspect few would dispute that such factors may have a material influence on test results

• Weigh this risk against advantages of transparency & accountability144


May 9, 2019

Columbus, OH 49

A Summary View of Current Evidence on TPO

• What evidence exists is so far from ecologically valid that it is just not very persuasive

• What's more, we are not going to get ecological validity in an empirical studyp y

• There is some evidence from recorded evaluations that deviations from standardized administration may be more frequent than we think

• We are not above scrutiny, and recording would hold us accountable for the quality of our work

145

A Moderate Approach

“Because of the possible advice‐giving, antagonism, and tension, the preferred posture is that opposing attorneys should not be present in person. However, we see no reason they should not be permitted to view a videotaped examination, under the conditions that the videotaping equipment is unobtrusive andconditions that the videotaping equipment is unobtrusive and out of the direct vision line of the evaluee.”

~ Cramer and Brodsky (2007)

146

APA Committee on Psychological Tests and Assessment (2007)• Statement on Third Party Observers in Psychological Testing and Assessment: A Framework for Decision

• In some cases, the presence of a third party may help develop and sustain rapport in order to facilitate test result validitysustain rapport in order to facilitate test result validity

• Options to consider when a TPO is requested:

1) Do it while taking steps to minimize the impact (e.g., seating arrangements, observing but not participating)

2) Introduce the least intrusive option (e.g., one way mirror)

3) Utilize tests less affected by observation

147


May 9, 2019

Columbus, OH 50

APA Committee on Psychological Tests and Assessment (2007), Cont’d

4) Recommend that the request be withdrawn

5) Decline to do the evaluation

• Make a decision about whether the inclusion of a TPO:

1) May affect validity of an evaluation or threaten test security and copyright

2) May facilitate validity and fairness of the evaluation or be required by law

148

Which Evaluations Should be Recorded?

• Some states require recording of all interviews in child abuse or neglect investigations

• Scholars have recommended recording various types of forensic evaluations:1) Sanity (Goldstein, Morse, & Shapiro, 2003)2) Competence to execute a will (Drogin & Barrett, 2003)3) Competence to be executed (Zapf, Boccaccini & Brodsky,

2003)

149

Recording: Practical Considerations

1) How should your Notification of Purpose be adapted?

2) What should the standard be in terms of review (i.e., do you always review every second of a recorded interview? We hope not )hope not…)

3) How will you accommodate storage of digital files?

4) How will such digital files be released with appropriate authorization to do so?

5) Should psychological testing be recorded?150


May 9, 2019

Columbus, OH 51

Recording and Third Party Observers: Recommendations

1) If counsel wants to attend, make a phone call to ask why, then explore alternatives to meet the expressed concerns

2) If you agree to counsel being present, set forth clear ground rules so that you are doing it on your terms

3) Let transparency be your guiding principle in making decisions about recording

4) Think through and prepare responses in advance to the “Practical Considerations”

151

#8: Should Social Media be used for Collateral In Forensic Evaluations?

152

Social Media as Collateral

• The controversy: some argue that any publically available information that is relevant should be considered, while others argue that actively seeking out data on social media is an undue invasion of the examinee’s privacyan undue invasion of the examinee s privacy

• The question: if they are used at all, under what circumstances and with what caveats should data points from social media be used as a source of collateral information?

153


May 9, 2019

Columbus, OH 52

Social Media – What Me Worry?

• Contemporary use of search engines (e.g., Google), social networking sites (e.g., Facebook, LinkedIn), and commentary vehicles (e.g., Twitter, Snapchat) is so commonplace that it is almost universal

• Use of data from these sources is routine in employment hiring and university admissions decisions

• Individuals might disclose probative data on social media which they might otherwise be unwilling to disclose in a face‐to‐face interview

154


• If a defendant is making such information publically available, how is it different than any other collateral source of information we might ordinarily use?

• Is data from Facebook (requiring a friend status) different than• Is data from Facebook (requiring a friend status) different than Twitter (open to anyone)?

• How is using social media any different than newspaper reports or clips from the television and radio? 1) Example: what about media interviews with witnesses and

possibly even the defendant?

155


2) Is there any difference between accidently seeing/hearing such information versus actively investigating social media for a digital fingerprint?

3) Is there any difference between actively investigating social3) Is there any difference between actively investigating social media for relevant data versus being provided with such data by an attorney, disability insurer, examinee?a) If yes, then where’s the harm in letting the referral source

do the “investigation” and you do the “interpretation” so long as you “stay in your lane?”

156


May 9, 2019

Columbus, OH 53

Social Media: A Clinical Perspective(Kaslow et al., 2011)

• Ethical Principles and Legal Requirements ‐ Clinical*

1) Privacy & Confidentiality:

a) A clinician who deliberately seeks out such information (as opposed to obtaining it via happenstance) may be breeching an

fexpectation of privacy

2) Informed Consent

a) Clinicians have an affirmative duty to inform clients about the rules that apply to their relationship

* “Ethical codes cannot do our questioning, thinking, feeling, and responding for us.” (Pope & Vasquez, 1998)

157

Social Media: A Clinical Perspective (Kaslow et al., 2011)

3) Best Interests of the Clienta) Analogous situation in marriage/family therapy where the

clinician is privy to knowledge that the client does not know one has: the revelation of what is “secret” (or at least (undisclosed) can become a complicated problem for all

b) Acting upon information that a client has not provided may be inconsistent with respect for the individual’s privacy and autonomy

158

Social Media: A Clinical Perspective (Kaslow et al., 2011)

4) Recommendations

a) The lack of specific standards on use of social media does not mean we are free to violate the EPPCC

b) Refrain from action that has no clear benefit and which may place li l (j t b d tyou on a slippery slope (just because you can does not mean you

should)

c) If you use internet searches, make that known and understood up front (part of informed consent)

d) If information is public and/or obtained inadvertently, nothing unethical about using it

e) Be skeptical, unless you think everything on the internet is true

159


May 9, 2019

Columbus, OH 54

Social Media: A Clinical Perspective (Clinton et al., 2010)

Six Questions to Ask before Searching Social Media1) Why do I want to conduct this search?2) Would my search advance or compromise treatment?3) Should I obtain informed consent from the patient?) p

4) Should I share the search results with the patient?5) Should I document the search findings in the health‐care record?6) How do I monitor my motivations and the ongoing risk– benefit

profile of searching?• Does my boss know?

160

Social Media: A Forensic Perspective(Pirelli et al., 2016)

• Dearth of literature on professional use of internet as part of forensic mental health assessment

• Forensic psychiatry is ahead of forensic psychology in considering the complexities in this area

• When conducting forensic evaluations, psychologists are obligated to comply with the EPPCC, and although the APA considers all of its published guidelines to be aspirational, psychologists should act in ways consistent with them

161


• Recommendations

1) Forensic practitioners who use internet data should conceptualize them as a type of collateral informationa) Decisions about the use of such information should bea) Decisions about the use of such information should be

made in the same way as decisions about use and access of other collateral data sources

b) Even examinees’ own words from blogs or other online postings should not be considered self‐report data in the context of a forensic assessment

162


May 9, 2019

Columbus, OH 55


c) Like any data obtained as part of a forensic examination, internet data should be weighed with respect to its level of utility (i.e., as having more or less weight), rather than as valid or invalid

d) Evaluators must consider the source of the data, its characteristics (e.g., the date and context in which it was written), and, ultimately, its convergence or divergence with other available data

163


2) Although searching for/using internet‐based data is not prohibited by the EPPCC or SGFP, forensic practitioners should consider conducting internet searches on a case‐by‐case basis, weighing the potential utility versus the potentially prejudicial effects of such data

a) No current standards or guidelines can be interpreted as prohibiting examiners from seeking such data

b) No current standards or guidelines can any be interpreted as requiring such data to be collected

164


c) Recupero (2008, 2010): such data can provide a more complete picture of the examinee and inform judgments about response styles

• consider the credibility or reliability of the source for y yinternet‐based collateral information

d) Metzner & Ash (2010): be cautious about forensic examiners’ abilities to accurately understand and interpret such information

165


May 9, 2019

Columbus, OH 56


e) AAPL Practice Guideline: permissible to use internet‐based information

• Recupero (2010): May help to confirm, corroborate, refute or elaborate on th hi t i t’ l i i f ththe psychiatrist’s general impression of the person

Digital evidence can be especially useful in assessments of impairment, credibility, and dangerousness or risk, particularly when the evaluee is uncooperative or unreliable in the face‐to‐face psychiatric examination

166


• Metzner and Ash (2010): possible complications: i. Meaning of the message may be difficult to understand

(e.g., deciphering a text message)ii. People present themselves on the internet differently eop e p ese t t e se es o t e te et d e e t y

than they do in face‐to‐face contexts, so interpretation is not always clear

iii. Possible generational differences about use of the internet – there may not be consensus of what constitutes “normal” internet use and communication

167

168


May 9, 2019

Columbus, OH 57


3) With rare exceptions, forensic practitioners who gather and/or rely on internet‐based data (or anticipate such) should discuss this practice during the retention and notification processes

a) Explain to examinees the manner in which these data have been or will be collected, integrated, and disseminated

b) Make same disclosures to those who retain us

169


4) With rare exceptions, forensic practitioners should provide examinees with data points gathered via the internet and allow them to address thema) Allows examinees to challenge the content of what was ) g

yielded, or explain its context, which is critical to considering the weight the examiner should give it

b) If don’t = vulnerable to cross examination, especially if data reportedly an examinee’s own words (e.g., “Dr., you didn’t even ask what she meant by that, did you?”)

170


5) Forensic practitioners should be explicit about their use of and reliance upon any data gathered via the internet in their reports and testimony

a) EPPCC and SGFP both stress the importance of identifyinga) EPPCC and SGFP both stress the importance of identifying sources of information used and data considered (and whether the examinee was provided the opportunity to respond to such data)

171


May 9, 2019

Columbus, OH 58

Sauce for the Gander

• Kolmes & Taube (2016): clients seeking on‐line information on their therapists1) ~70% of clients reported “finding” personal information about

their therapists; of this group, 86% sought it out intentionally2) Nearly all used a search engine, >50% social networking sites,2) Nearly all used a search engine, 50% social networking sites,

~3% paid for it3) Most frequent reason = curiosity, helped them make decision

about initial meeting or to avoid asking a direct question4) ~75% did not tell their therapist about this5) Assume clients (and non‐incarcerated examinees) can and will

find personal information

172

Sauce for the Gander

• What if a forensic examinee is investigating me on social media?

• Implications for forensic evaluation1) D ’t b i d1) Don’t be surprised2) Respond with benign curiosity if reported3) Consider several possible motivations, and get input from a

trusted peer if your reaction may establish bias4) Carefully manage your online presence to minimize

preventable privacy intrusions

173

Social Media: Recommendations

1) If you plan on seeking out such data, first check any relevant sources of authority in your jurisdiction: e.g., “predatory friending” (Patrick, 2016) as unethical

2) If you plan to use such data, note it in your notification statement and allow the examinee to respond to themthe examinee to respond to them

3) Be careful about your assumptions regarding accuracy of the information or what it meansa) A cautionary Facebook tale…

4) Your motive for seeking out such information should be the same as any other collateral: neutral and objective pursuit of data that may be relevant to the issue at hand

174


May 9, 2019

Columbus, OH 59

Questions?

175

Documents

Terry Kukor, Ph.D., ABPP Rick DeMier, Ph.D., ABPP...May 04, 2019 · Dixit?” 4) “How can I decide if something qualifies as a mental illness if you won’t name that mental illness?”