Adventures in Forensic Statistics - James Curran

Adventures in Forensic Statistics

Professor James M. Curran

Dept. of Statistics, University of Auckland

10th October 2013

JM Curran (Statistics, Auckland) Forensic Statistics 2013-10-10 1 / 48

Forensic evidence

ã Forensic evidence has been used in the courtroom for a very long time(take Sherlock Holmes for example)

ã However it was not really until the late parts of the 20th century thatthe public really became aware of its power and usefulness

ã This was mostly because of the advent of DNA evidence

ã In the last few years forensic science has become glamorized due tothe “CSI effect”







Major trials in NZ and around the worldThink of some famous cases in New Zealand and around the world. Didthey contain forensic evidence? What kind?

Lindy Chamberlain(convicted and later acquitted of murdering daughter Azaria at Ayres Rock)



David Bain(convicted and later acquitted of murdering his father, mother and sisters)



Orenthal James (O.J.) Simpson(accused of murdering Nicole Brown Smith and Ron Goldman. Convicted of “wrongful death”)



Orenthal James (O.J.) SimpsonConvicted of armed robbery)



William Jefferson (Bill) Clinton(accused of having “sexual relations” with Monica Lewinsky)



Scott Watson(convicted of murdering Olivia Hope and Ben Smart)



Joseph Thompson(South Auckland serial rapist)



Malcolm Rewa(South Auckland serial rapist)



Mark Lundy(convicted of murdering his wife and daughter)


Forensic evidence and the court

ã People who are not forensic examiners (such as judges, lawyers andjuries) often have trouble deciding whether a certain piece of evidenceis important or relevant

ã To address this problem, the court appoints experts to give theirexperienced opinion on the evidence

ã However, generally an expert is not appointed independently.

ã That is, the prosecution and defence hire experts whom they believewill strengthen their respective cases


What does the court want to know?

The weight of the evidence“How much more likely (or less likely) does this evidence make it that theaccused is guilty?”

ã Statistics offers a framework in which evidence can be consistentlyevaluated

ã That means that two experts who analyse the evidence in the sameway will come up with the same statistic or conclusion

ã It is for this reason alone that more and more judges and lawyers aredemanding the use of statistics in conjunction with forensic evidence


Where does glass evidence come from?


Characterizing/quantifying glass evidence

Colour, shape, density, refractive index, elemental composition


Measuring refractive index (RI)


Measuring RI

Silicone oil is heated until the optical density matches that of the glass.This (average) match temperature is converted into RI with a calibrationline


Distribution of refractive index measurements (in NZ)


Some useful questions

ã Are pieces of glass from the same source more likely to be similar toeach other than to glass from other sources?

ã Is glass homogeneous within a source?


Surface effects


Crown glass


An example

ã On 3 March, 1991, a float glass window was smashed in a pharmacyin Hamilton, New Zealand

ã The offenders took drugs and prescription medicines worth thousandsof dollars

The suspects

ã Police apprehended two suspects, Michael Johnston and JohnMacKenzie, 90 minutes later

ã Their clothing was taken but the drugs were not found


The evidence

ã Recovered from Johnston’s clothing

- small flakes of paint - indistinguishable from crime scene

- 11 fragments of glass

ã MacKenzie’s clothing

- 3 fragments of glass

ã 3 fragments were original float surfaces

ã 9 control fragments taken from scene window

ã Evidence quantified using RI


The evidence


Match/non-match framework

ã Very typical at the time of this case to use some criterion todetermine whether the recovered measurements match the controlmeasurements

- Eyeballing

- Range overlap tests (range or 2/3 σ)

- t-test

ã Johnston: t = 3.06, 19 df, P = 0.006

ã Mackenzie: t = 3.38, 10 df, P = 0.011


Three principles of interpretation

Evett and Weir (1998) proposed three basic principles of evidenceinterpretation

1 To evaluate the uncertainty of any given proposition it is necessary toconsider at least one alternative proposition

2 Scientific interpretation is based on questions of the kind “What isthe probability of the evidence given the proposition?”

3 Scientific interpretation is conditioned not only by the competingpropositions, but also by the framework of circumstances within whichthey are to be evaluated


A likelihood ratio (LR) approach to evidence interpretation

Many of my colleagues and I are proponents of what is called the“Bayesian,” or “LR,” or “logical” approach to evidence interpretation

This way of thinking encapsulates all of the ideas on the previous slide

We believe all forensic scientists should present evidence in the form of alikelihood ratio

Odds form of Bayes’ TheoremPr(Hp|Evidence)Pr(Hd |Evidence)︸︷︷︸Posterior Odds

=Pr(Evidence|Hp)

Pr(Evidence|Hd)︸︷︷︸Likelihood Ratio

× Pr(Hp)

Pr(Hd)︸︷︷︸Prior Odds


A likelihood ratio for our case

ã In the hierarchy of propositions the levels are offense, activity andsource (Cook et al., 1997)

ã I will propose two competing hypotheses at the activity level

ã These are:- Contact: The suspect was in contact with the crime scene

- Contact: The suspect was not in contact with the crime scene

ã To compute the LR I must assess the probability of the Evidenceunder each of these hypotheses


The LR under consideration

ã The denominator of the LR is

Pr(Evidence|Contact) = P1SLf

ã This formula represents the probability of the evidence if the suspectwas not at the crime scene

ã If the suspect was not at the crime scene then the possible reason forpresence of glass on his person might be

- he had one group of glass from another source on his clothes- and it just happened to match the crime scene source by chance


The LR under consideration

ã The numerator of the LR is

Pr(Evidence|Contact) = TLP0 + T0P1SLf

ã This formula represents the probability of the evidence if the suspectwas at the crime scene

ã If the suspect was at the crime scene then the possible reason forpresence of glass on his person might

- no glass was transferred from the scene- and he had one group of glass from another source on his clothes- and it just happened to match the crime scene source by chance

OR

- a large group of glass was transferred from the scene window- and he had no glass on his clothing from other sources


Interpretation of the LR

ã Survey estimates give a likelihood ratio of 25 for Johnston and 10 forMacKenzie

ã “The evidence is 25(10) times more likely if the suspect was at thecrime scene than if he wasn’t”

ã This method of interpretation gives a far more intuitive and usableresult

ã The disappointing truth is that most people (including the judge, thelawyers and the jury) find this statement incomprehensible

ã These are examples of how this statement is incorrectly interpreted- “It is 25 times more likely that Johnston committed the crime”- “Johnston is 25 times more likely than anyone else to have committedthe crime”


Multivariate glass evidence

ã I have been discussing glass evidence measured on a single variable(RI)

ã I will now talk briefly about glass evidence measured on manyvariables

ã We have had the capability to analyse substances at an elementallevel since the 1940s (NMR)

ã These techniques have improved dramatically since then. Specificallythey have become

- very sensitive - elements can be measured in the low parts per billionrange

- non-destructive - laser ablation (LA) techniques mean that specimensare no longer destroyed

- cheap(er) - a modern ICP-MS setup will cost around $US100,000 -down from $US500,000


A LA-ICP-MS laboratory


A LA-ICP-MS laboratory


LA-ICP-MS at work

A series of LA “shots” on a human hairA human hair is between 20 and 200 microns or 20 − 200 × 10−6 m wide


Statistical interpretation of elemental evidence

Sequential comparison of recovered items to intervals defined by thecontrol source

Fe Mn Ba Sr Zr CrControl Min. 1978 53 166 143 70 1494Control Max. 2322 62 200 169 90 1771

Recovered 2320 62 192 166 99 1766

An example of a range test with elemental concentration data

“Improvements” to this are using standard deviation, not range, and doingtwo-sample t-tests on an element-by-element basis


Dependency

ã Probably the fundamental issue in evaluation of multivariate evidenceis dependency

ã This is common to both trace evidence and DNA evidence

ã Dependency takes many forms and it affects the results in a variety ofways

ã Correlation is one measure of dependency – but it is poorlyunderstood


What does DNA evidence look like?RFLPs, VNTRs and autoradiographs

Sir Alec Jeffreys ... had a “eureka moment” in his lab ... at 9:05 am on Monday10 September 1984,... - [Wikipedia]


What does DNA evidence look like?PCR + Amplitype® Polymarker

Kary Mullis (Nobel prize in Chemistry 1993)“Science has been just one of the keen interests in Dr. Mullis’s life, competing withpsychedelic drugs and women” - Nicholas Wade, NY Times“...his only slides were photographs of his art which depicted naked women with coloredlights projected on their bodies.”

Kit available around 1992


What does DNA evidence look like?PCR + STRs: DNA profiles – what I see

Thanks to IntergenX for the profiles



Thanks to IntergenX for the profiles



Thanks to IntergenX for the profilesJM Curran (Statistics, Auckland) Forensic Statistics 2013-10-10 32 / 48

The statistics of DNA evidence – a simple case

ã A house is broken into and burgled

ã Entry was gained by breaking a window and opening a door

ã The burglar cut himself, leaving a blood stain

ã Hours later the police apprehended a suspect (Curran)

ã Suspect had a cut on his hand

ã Suspect denies knowledge and involvement in the crime

ã DNA sample taken from suspect matches the crime scene


Did this blood come from that person?

ã The offender and the suspect have the same genetic type at thislocus, i.e. the same genotype

ã Does that mean the suspect is guilty?

ã Forensic evidence can never answer this question directly. It can onlymake it more or less likely that it is true

ã Forensic evidence can (usually) only address source level questions,not activity level questions


Why?

Some questions you might ask yourself are:

ã “How many other people have the same genotype?”

ã “I inherited my DNA from my parents. Wouldn’t their types andthose of my brothers and sisters be more similar than an unrelatedperson’s?”

ã “What other reasons might there be for the presence of DNA?”


DNA profiles

ã The questioned item has one or two alleles present at each locus

ã If there is only a single contributor to the stain then there will be atmost two alleles per locus

ã The profiles are said to match or that the suspect cannot be excludedas a contributor of this item

ã How do we assess this evidence statistically?

ã One question people may ask is “How rare is this evidence?” or “Howmany people in the population have this profile?”

ã This is something we can answer statistically with the populationfrequency of the profile


Population frequencies

ã A traditional and commonly used approach is to calculate thefrequency of the profile in the population

ã This is the right answer to the wrong questionã Whilst the answer to this question may be interesting, it ignores one

basic fact: we already know one person has this profileã All probabilities are conditional

We are interested in the answer to the following question:We know person X has this profile.

What is the probability someone else has it?


Propositions

What might the propositions be in our example case?

Hp: the suspect (Curran) is the person who left this blood

Hd : someone unrelated to the suspect left this blood

Note these propositions are just that. Also they are not mutuallyexhaustive

The probability of the evidence under Hp in this case is one. This reflectsthe idea that if the suspect did leave the blood then we expect to see hisDNA profile (with certainty)

How about the probability of the evidence under Hd? We need a model


The defence proposition

Under the defence hypothesis Hd we then must ask “What is theprobability that someone else, other than the defendant, has thisgenotype?”

To answer this question people often make some simplifying assumptions.These are:

a. Independence of alleles within a locus (HWE)b. Independence of loci (LE)c. Independence of individuals in a population

These are popular assumptions because most people remember that ifevents are independent then we can multiply

However, most people also forget the “if events are independent”


Model assumptions – Hardy Weinberg Equilibrium

For HWE to hold, a population must satisfy five conditions

1. Completely random mating – including selfing2. No mutation3. No migration4. No selection5. Infinite population size

HWE cannot possibly be true in any human population


We cannot all be unrelated


The likelihood ratio (LR) in this case

The LR (across all loci) using my profile, and the New Zealand Caucasianallele frequencies is 3.8× 1013

Courtroom statementI would interpret this as: “The evidence is 3.8× 1013 times more likely ifthe suspect was the donor rather than if someone unrelated to the suspectwas the donor”

Again, this is frequently misunderstood, misused, and misquoted


This statement is backwards


This statement is backwards


How do we (the jury) use this number?

ã Assume we believed that the “odds on guilt” before we heard theDNA evidence were 1000 to 1 against ? i.e. we think it is a thousandtime more likely that the suspect is innocent that guilty

ã We can update these odds by multiplying by the LR

ã E.g. the new odds are 3× 1013 : 1× 1 : 1000 = 3× 1010 : 1

ã That is, it is 30 billion times more likely that the suspect (Curran) isthe donor rather than someone unrelated to Curran (given theevidence)


How do we (the jury) use this number?

ã That is, after we heard the DNA evidence, we changed our beliefsfrom “leaning (strongly) towards innocence” to “very likely guiltythan innocent”

ã In practice, people rarely explicitly carry out this multiplication

ã However, the thought process is the same: if the expert gives a bignumber they move towards guilt; if it is a small number (less than 1)they move towards innocence


Complicating the issues

ã What if the alternative explanation is not that “the suspect isinnocent and someone unrelated to him did it” but is “it wasn’t me itwas my brother”?

ã How about if the crime was a rape and the DNA evidence had amixture of two people’s DNA (victim and rapist) or three people(victim, victim’s boyfriend and rapist).

ã How do we come up with the values for the allele frequencies?Sampling? So is there a margin of error?


Relatedness and sampling uncertainty

Balding and Nichols gave us a way of modelling departures from HWE

pA|A = pA + (1− pA)θ

This quantity is greater than pA for any value of θ > 0. John Buckleton,Chris Triggs and I have done considerable research on:

ã generalisations of calculations incorporating these effectsã the behaviour of the resulting estimatorsã and the determination of appropriate values of θ for different

populations.The last is extremely important because in general increasing θ decreasesthe LR

We have also done considerable research into methodology for reflectingsampling uncertainty


Thanks

I could not have done anything without the support and collaboration of:

John Buckleton


Thanks


Chris Triggs


Thanks


Chris Triggs


Thanks


Bruce Weir


Thanks


Jo Ann Bright


Thanks


Family andfriends


Technology

Adventures in Forensic Statistics - James Curran