The impact of discredited evidence

David LagnadoNigel HarveyEvidence project, UCL

Discredited evidence

How do people revise their beliefs once an item of evidence is discredited? For example, in a murder trial, when the

testimony of a key witness is shown to be fabricated, how does this affect juror’s beliefs about the testimony of other witnesses, or even other forensic evidence?

OJ Simpson trial

Normative Models

Bayesian network models Normative model for combining probabilistic

evidence E.g., forensic DNA evidence; paternity cases

(Dawid) Formal modelling of ‘manipulated evidence’

(Baio) Starting to be applied to crime cases

But a lot depends on network construction No ‘normative’ method for this?

Crime case (Leucari, 2005)

Explaining away (Pearl, 1988)

SS = Suspect commits crimeP(S|C) > P(S)

Finding out C raises probability of S

CC = Suspect confesses

FF = Police force confession

P(S|C&F) < P(S|C)

Finding out F too lowers the probability of S

Despite its simplicity and ubiquity, this pattern of inference is hard to capture on most psychological models of inference (e.g., associative models, connectionist, mental models, mental logic)

Psychological models

Belief-adjustment model Hogarth & Einhorn (1992)

Story model Pennington & Hastie (1981, 1988, 1992,

1993)

Belief adjustment model

For Evaluation tasks Evidence encoded as +ve or –ve relative to

hypothesis Adding model

Sk = Sk-1 + wk s (xk)

Sk = degree of belief in hypothesis given k items of evidence

Sk-1 = prior opinion

S (xk) = subjective evaluation of kth item (-1 ≤ s (xk) ≤ +1)

wk = adjustment weight (0 ≤ wk ≤1)

Online vs. Global Processing

Two processing modes Online (step-by-step)

Belief adjusted incrementally with each item of evidence

Global (end-of-sequence) Belief adjusted by aggregate impact of all

items

Sk = S0 + wk [s (x1,…, xk)]

When is each process used?

Online Global

Online All tasksComplex items

and/or long series

Global ImpossibleSimple items

and short series

PROCESS

RESPONSE MODE

Processing load - Aggregation can be costly in terms of mental resources whereas step-by-step integration makes minimal demands

Evidence for Belief Adjustment Order effects in online processing

Evaluation mode (adding not weighted average) None with consistent evidence (e.g, ++ or --) Recency with mixed evidence (-+ > +-) over-weight

last item Supported in Exps 1-5

Model is quite flexible – designed to account for rich patterns of primacy and recency evidence

But does not address relations between evidence items (assumes independence?)

Story model

Evidence evaluated through story construction Stories involve network of causal relations

between events Causal narratives not arguments

People represent events in the world, not inference process

Stories constructed prior to judgment or decision Stories determine verdicts, and are not post hoc

justifications

Evidence for story model

Verbal protocols 85% of events causally linked

Verdicts covaried with story models Recognition memory tests

More likely to falsely remember items consistent with story model

Story vs witness order More likely to convict when prosecution evidence

in causal order, defence in witness order, and vice-versa

Current experiments

Investigate effect of discredited evidence Look at relations between items of evidence Do these modulate how people revise their

beliefs? Once an item of evidence is discredited, do people

simply return to their prior level of belief? Or does this change permeate their belief network?

What factors affect this? Relations between evidence Order of presentation of evidence

HYPOTHESIS: Suspect S did it

Scenario: House burglary, local suspect S apprehended

EVIDENCE 1

Neighbour 1 says that S often loiters in area

EVIDENCE 2

Neighbour 2 says S was outside house on night of crime

Neighbour 2 is lying because he dislikes S

?P(S)

Under-discounting

Over-discounting

Generalisation

When do people generalize from the discrediting of one item to other items?

Dependent on relatedness of generating mechanisms?

SAME E.g. two statements from same neighbour

SIMILAR E.g. two statements from two different

neighbours DIFFERENT

E.g., one statement and one blood test

HYPOTHESIS: Suspect S did it

DIFFERENT Scenario: House burglary, local suspect S apprehended

EVIDENCE 1

Footprints outside house match suspect’s

EVIDENCE 2

Neighbour says S was outside house on night of crime

Neighbour is lying because he dislikes S

?P(S)

Under-discounting

Over-discounting

Experiment 1

Each subject completes 12 problems (4 scenario types x 3 levels of relatedness)

‘Relatedness’: SAME, SIMILAR, DIFFERENT

Four probability judgments (of guilt)1. Background information2. Evidence 1 (E1)3. Evidence 2 (E2)4. Discredit evidence 2 (D2)

Compare 2 and 4 (E2 vs. D2) If D2 > E2 then under-discounting If D2 < E2 then over-discounting

Example of SAME condition

BackgroundYou are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene

Evidence 1The police have a statement from the current wife of the suspect, confessing that the suspect had previously revealed a desire for the victim to be dead

Evidence 2The same police station has a confession from the suspect, admitting that he killed the victim

Discredit 2The confession from the suspect was made under extremely pressured circumstances at the police station

Example of DIFF condition

BackgroundYou are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene

Evidence 1Laboratory tests revealed that blood found at the crime scene matched the blood type of the suspect.

Evidence 2The police station has a confession from the suspect, admitting that he killed the victim

Discredit 2The confession from the suspect was made under extremely pressured circumstances at the police station

Exp 1: Results

Significant ‘over’-discounting (D2 < E1) in all conditions SAME: t(23)=3.74, p<0.05; SIM: t(23)=2.71, p<0.05; DIFF: t(23)=3.13,

p<0.05

Amount of over-discounting greater in SAME vs. SIM, t(23)=1.91, p=0.07; no differences with SAME vs. DIFF, or SIM vs. DIFF

Main effect of DIFF due to physical test as E1

0

10

20

30

40

50

60

70

80

90

100

B E1 E2 D2

Evidence

Pro

ba

bili

ty o

f gu

ilt

DIFFSIMSAME

Individual analysis

Over-discount (D2 < E1)

None (D2 = E1)

Under-discount (D2 > E1)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

SAME SIM DIFF

Relatedness

% p

art

icip

an

ts

Over

None

Under

Conclusions

Difficult to interpret on either BA or Story model

BA model Does not predict effect of relatedness Predicts recency effect with mixed evidence ++- (overweight last item) Asymmetric rebound effect?

Story model Predicts story construction only with global

judgment Does not predict over-discounting

Experiment 2

Better test of two modelsOrder of evidence

LATE - discrediting info presented after both itemsB E1 E2 D

EARLY – discrediting info presented after first itemB E2 D E1

Relatedness SAME, DIFFERENT

HYPOTHESIS: Suspect S did it

DIFFERENT & EARLY

EVIDENCE 2

Neighbour says S was outside house on night of crime

Neighbour is lying because he dislikes S

EVIDENCE 1

Footprints outside house match suspect’s

?P(S)

Model predictions

BA predicts recency Final judgment for early > late Because +-+ > ++- (overweight last item)

Story model predicts recency with online but not global SAME ≠ DIFF for global condition (take account

of relatedness)

Example of SAME and EARLY

BackgroundYou are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene

Evidence 2 The police station has a confession from the suspect, admitting that he killed

the victim Discredit 2

The confession from the suspect was made under extremely pressured circumstances at the police station

Evidence 1The police have a statement from the current wife of the suspect, confessing

that the suspect had previously revealed a desire for the victim to be dead

Example of DIFF and EARLY condition

BackgroundYou are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene

Evidence 2The police station has a confession from the suspect, admitting that he killed the victim

Discredit 2The confession from the suspect was made under extremely pressured circumstances at the police station

Evidence 1Laboratory tests revealed that blood found at the crime scene matched the blood type of the suspect.

Online judgments:Early vs. Late

EARLY

0102030405060708090

100

B1 E1 U1 E2

EVIDENCEPr

obab

ility o

f guil

t SAME

DIFF

Early condition –more sensitivity to relatedness

NB no diff between B1 & U1 in EARLY rules out asymmetric rebound effect

LATE

0102030405060708090

100

B1 E2 E1 U1

EVIDENCE

Prob

abilit

y of g

uilt SAME

DIFF

Late condition – less sensitivity to relatedness (but note that over-discounting (E1 > U1) only sig for SAME not DIFF)

Problematic for both models

BA cannot explain early condition because does not consider relations between evidence

Story model needs to be applied/adapted to online processing, and somehow explain difference between early and late

Any other models? Needed: online model that takes relations

between items into account, but can also explain early/late difference

Speculations

Even with online processing people construct network fragments

As evidence is accumulated it is compactly stored /integrated

Natural to integrate items according to valence (+ve or –ve wrt hypothesis)

E.g., group +ve evidence together

Late condition

Positive evidence A and B integrated

GUILT

A

+

B

+

GUILT

A&B

+

C discredits both A and B (irrespective of relatedness)

C

Early condition DIFF

B unaffected by C’s discredit of A

GUILT

A

+

B

+

C

GUILT

A

+

C

Early condition SAME

GUILT

A

+

A*

A* discredited by C too (because similar to A)

C

GUILT

A

+

C

Ongoing research

Look at both witness and alibi statements E.g., How does discrediting of an alibi affect

evaluation of a positive witness? Are there asymmetries in dealing with positive

vs. negative evidence? More generally, look at positive and

negative evidence (including forensic tests) Are there differential affects of discrediting? Can evidence integration idea explain these?

Manipulate deception vs error