Embed Size (px)
Citation preview
A Clarion Call to Improve the Underlying Science, Laboratory Efficiency and Cost Associated with Tes?ng of Complex Mixtures and Interpreta?on
Charlo'e J. Word, Ph.D., Michael D. Coble, Ph.D., Robin W. Co'on, Ph.D.
(John M. Butler, Ph.D. and Catherine Grgicak, Ph.D.)
May 9, 2013
PresentaFon Outline
• CharloDe Word (consultant) – IntroducFon to Current Issues in DNA TesFng – ValidaFon Studies
• Michael Coble (NIST) – Why Mixtures are Difficult – Lessons Learned from Training Workshops
• Robin CoDon (Boston University) – EducaFon and Training – ReporFng and Court
Most parFcipants are here* through the sponsorship of NIJ
• NIJ Forensic Science Training Development and Delivery Program
• NIJ Grant # 2008-‐DN-‐BX-‐K158, awarded to Biomedical Forensic Science Program at Boston University School of Medicine
• SupporFng registraFon for ~450 parFcipants from state and local laboratories
*Slide presented at Promega ISHI Mixture InterpretaFon Workshops
Your Presenters are:
John Butler NIST
Mike Coble NIST
Robin Cotton Boston University
Catherine Grgicak Boston University
Charlotte Word Consultant
617-‐638-‐1952 rwco'[email protected]
301-‐975-‐4049 [email protected]
617-‐638-‐ 1968 [email protected]
301-‐975-‐4330 [email protected]
301-‐527-‐1350 [email protected]
Alaska
Hawaii
Mixture Workshop A'endees 50 states and 25 other countries
Green = parFcipants
ISHI 2010 (N=200) ISHI 2011 (N=160) ISHI 2012 (N=145)
Federal Labs FBI ATF AFDIL USACIL
* *
*
* 4 regional workshops (N=200) Puerto Rico
NIST Webinar April 12, 2013
>1000 conFnuing educaFon cerFficates
What is the discipline you have the most experience in?
1 2 3 4 5 6 7 8
0% 0% 0% 0%0%0%0%0%
1. Controlled substances 2. Firearms/toolmarks 3. Fingerprints/pa'ern
evidence 4. Biology/DNA 5. Crime scene 6. Arson/explosives 7. Toxicology 8. Trace evidence
Changes in DNA TesFng in Recent Years
• Case and Sample Acceptance Policies – Then: High profile cases, homicides, sexual assaults
• Lots of DNA, single source, two-‐person mixtures – Now: Burglaries, Car jackings, Possession
• Handled items with “touch” DNA, small amount of DNA (Low Template DNA), complex mixtures, clothing (“wearer” DNA)
• Bulk of samples accepted in many labs
Now accepFng samples that would never have been accepted in the early STR
tesFng days
• Increased SensiFvity of PCR test kits – Use of enhancement techniques
• Many more STR test kits available • OpFons for types of tests
– Autosomal STR – Y (male) STR – mini-‐STR (degraded DNA) – May use all 3 tests on a sample if sufficient DNA
Changes in DNA TesFng in Recent Years (cont.)
• ExisFng SOPs may not be adequate – Low Template (LT) DNA – Complex Mixtures – RelaFves in mixtures – Enhancement techniques
• SWGDAM InterpretaFon Guidelines issued in 2010 (for single source and 2 person mixtures) – Need defined analyFcal and stochasFc thresholds – Need interpretaFon methods that fit with available staFsFcal methods (limited available)
Changes in DNA TesFng in Recent Years (cont.)
Likely need to modify SOPs and do addiFonal validaFon studies
ValidaFon OpFons
• New ExtracFon Kits and Columns – Manual – Automated
• Automated/RoboFc instrumentaFon, soqware, documentaFon
ValidaFon OpFons (cont.)
• New QuanFficaFon Kits – Human and Y – Human, Y and degradaFon
• New AmplificaFon Kits – Higher sensiFvity
• IdenFfiler® Plus, PowerPlex® 16 HS – More loci
• PowerPlex® Fusion (Promega) • GlobalFiler™ (Life Technologies)
More Fme needed for analysis, interpretaFon and technical review
ValidaFon OpFons (cont.)
• New AmplificaFon Kits – Y STRs
• Yfiler®, Yfiler® Plus • PowerPlex® Y23 • Rapidly mutaFng Y loci?
– MiniSTRs • MiniFiler™
– Phenotypes • IrisPlex (hair and eye color)
– In/Del? (DIPlex, Qiagen) – Rapid DNA?
ValidaFons Needed
• New Capillary Electrophoresis GeneFc Analyzer – ABI 3500 – Different data collecFon soqware – OpFmal peak heights MUCH higher than with previous CEs (e.g., 6000-‐14,000)
– Need to define analyFcal thresholds and stochasFc thresholds
• May be different for different colors – Requires different GeneMapper ID-‐X soqware
User BulleFn Applied Biosystems 3500/3500xL GeneFc Analyzers h'p://tools.invitrogen.com/content/sfs/manuals/cms_095698.pdf
ValidaFons – More Data Needed
• SensiFvity Studies – Be'er understanding of Low Template (LT) DNA and StochasFc Effects
• Single diluFon series NOT adequate – Aid in establishing one or more analyFcal thresholds and stochasFc thresholds
• Low amount of DNA vs. high amounts of DNA
• Mixture Studies – Complex mixtures, if accepFng and interpreFng samples with >2 contributors
ValidaFons – More Data Needed
• Enhancement Techniques for LT DNA – Decreased amplificaFon volume – Increased amplificaFon cycles – Increased injecFon Fme or voltage – Increased product in sample prep for CE – Post-‐amplificaFon clean-‐up
• Must do validaFon studies for ALL condiFons with all kits
ValidaFons (and TRAINING) Needed
• ProbabilisFc Modeling Soqware for LT DNA and complex DNA mixtures – Modeling of drop-‐out – Modeling of drop-‐in – Other stochasFc effects – LIKELIHOOD RATIOS – HELP!!
ValidaFons OpFons
• Case work vs. Databasing – Direct amplificaFon kits (no extracFon or quanFficaFon)
– Small amounts of DNA vs. higher amounts – Mixtures vs. single source
• InterpretaFon for CODIS entry vs. case work interpretaFon – How different are they?
ValidaFons need to include:
EvaluaFon of all aspects of tesFng procedures 1) Technology performance (kits, instruments) 2) Assessment of data with known contributor(s)
LimitaFons of each aspect of the test system 3) Development of SOPs that reflect validaFon done,
including interpreta.on guidelines
TesFng of samples from known individuals that reflect casework acceptance policies 1) Low Template DNA 2) Complex Mixtures
New ValidaFon Studies
• Technical leader will need lots of help and .me to conduct and evaluate appropriate studies
• MulFple samples will need to be tested • May need addiFonal training or assistance to evaluate data (e.g., staFsFcs)
• InterpretaFon SOPs will be much longer and more complicated and detailed
Who in your organizaFon is responsible for biology/DNA sample acceptance policies?
1 2 3 4 5 6 7
0% 0% 0% 0%0%0%0%
1. Biology/DNA supervisor 2. DNA Tech Leader 3. Lab Director 4. Evidence Tech 5. DetecFve/invesFgator 6. Prosecutor 7. Other
ConsideraFons for Cost ReducFon
• Review samples received and test results – Successes vs. inconclusives for samples with various kits
• Review case acceptance policies – Limit sample number/case – PrioriFze samples/case – Limit samples with low likelihood of results
• What tests are really needed? – What does your lab need to validate vs. outsource? (e.g., Y STRs, MiniFiler)
• Is the cost of validaFon, training, proficiency tesFng, & QC of non-‐expired kits worth it?
ConsideraFons • Implement a plan for evaluaFon of reported cases when interpretaFon SOPs change – Minor or significant change in SOP leading to minor or significant change in interpretaFon?
– Change in conclusions (e.g., inclusion to inconclusive or exclusion – most likely)?
• Possible Brady issue (exculpatory evidence) – Possible opFons:
• Sampling of 10-‐20% of cases form plan • Re-‐review when discovery requested and/or when requested to tesFfy
• When addiFonal tesFng being done in a case
But what about…?
• All the other cases • Post convicFon cases • Cases that pled out based on the DNA report conclusions
When we receive an evidence sample, we know:
1 2 3 4 5 6 7
0% 0% 0% 0%0%0%0%
1. If it is from a single source or a mixture
2. The raFo of donors to a mixture
3. The number of donors to a mixture
4. How it got there 5. All of the above 6. None of the above 7. Some of the above
Why are Mixtures Difficult to Interpret?
1. We don’t know a priori the relaFve contribuFon of each DNA component
Photo from the Amanda Knox crime scene
100% complainant? 50/50 mixture? 10% perpetrator?
2. STR kits are like “Goldilocks”
Allele Drop In
1ng
8pg
Data from Debbie Hobson (FBI) – LCN Workshop AAFS 2003 Input DNA
Allele Drop Out
50 µL PCR
5 µL PCR
Heterozygote Allele Imbalance
PHR = 87%
PHR = 50%
3. DNA ExtracFon/Recovery is about 20%
0.5 ng 100 -‐ 150 pg ~ 70-‐80% sample loss
ExtracFon process
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50K 100K 200K
% R
ecov
ery
(n
g re
cove
red/
ng in
put)*
100
Cells Added
EZ1
Salt Out
4. It can be difficult to determine the number of contributors in the mixture
or
+ or or or
+ or or or + or or
Single Sou
rce vs. M
ixture?
592 responses
Profile 9
The problem with this approach…
Actual Contributors 28,28 30,32.2 27,32.2
29,32.2
28 + 29 = 2,894 RFU 32.2 = 2,247 RFU
“Mythical Major” (potenFal for false inclusion!)
The 2010 SWGDAM Guidelines do not address 3 person/ complex/low level mixtures
5. We can’t say “NO”
Types of Offenses (2012)
Slide courtesy of Sarah Chenoweth
Lessons Learned
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Yes No No Change Working on it
n/a
Has your lab implemented changes to your SOPs based on the new guidelines?
1. Yes 2. No 3. Reviewed SOPs but
no changes needed 4. Working on it 5. Not applicable (I don’t
work in a forensic lab)
Data from 297 responses ISHI Mixture Workshop (Oct 2011) 4 regional laboratories (2011)
88% have undergone recent changes or were in the midst of changing SOPs for mixture
interpreta?on
CPI Stats
• CPI staFsFcs have a very narrow range of uFlity – 2 person mixtures with sufficient quanFty of DNA. It is currently being misapplied to a wide range of mixtures (relaFves, low level, 3 and 4 person mixtures, etc…). “Maslow's hammer”
• We MUST have a paradigm shiq in the U.S. and make the move to LRs if we wish to interpret complex mixtures where drop-‐out is possible.
Data from 636 responses NIST Mixture Webcast (Apr 2013)
ConsideraFons
• Case acceptance policies and “stop tesFng” procedures can prevent the unnecessary and costly tesFng/Fme spent interpreFng “messy” mixtures.
• This is not to say that ALL complex mixtures should be avoided – soqware programs that use probabilisFc approaches can be useful for interpretaFon – PopStats is not the soluFon!
ConsideraFons
• This will require training so the analyst can fully understand what the soqware is doing.
• Simply applying CPI stats to every mixture may produce evidence to include an innocent suspect, or exclude a true perpetrator.
Thoughts on Education and Training
Specific Requirements for DNA
• Technical leader – MS – 3 years experience – Specific courses; Biochem, GeneFcs, Mol Bio, Stats/PopulaFon GeneFcs
• Analyst – BS – Specific courses; same as above
The DNA secFon in my lab contains____# or staff.
1 2 3 4 5 6
0% 0% 0%0%0%0%
1. ≤ 3 2. 4-‐10 3. 11-‐20 4. 21-‐40 5. 41-‐60 6. > 60
From the workshops, we are concerned that some analysts may not:
1. Understand the scienFfic basis of the steps in the DNA process and/or the instrumentaFon used.
2. Be familiar with the data and data analysis that provides the scienFfic basis for the DNA interpretaFon procedures in their SOP.
3. Understand their role as an expert witness and the requirement to interpret the data from a posiFon of neutrality
Scien?
fic basis fo
r the
interpreta?o
n of th
e DN
A da
ta: How were the RFU levels set for your
laboratory ‘s stochasFc threshold?
Most A
nalysts feel preDy good
abo
ut going to
cou
rt. For me tesFfying in court ….
Analysts do en
coun
ter p
ressure whe
n serving as an
expe
rt witn
ess:
49%
Have there been any occasions when you have felt pressure from an a'orney to make stronger statements than you would otherwise make?
Other court related quesFons we have been asked:
What do I do when: • the a'orney won’t meet with me before trial? • my technical leader insists that I sign a report that I do not agree with?
• I need to tesFfy regarding a report that I no longer agree with?
• the prosecutor misrepresents data that I have just tesFfied is inconclusive?
What journals (hard copy or on line) are available to your staff?
1 2 3 4
0% 0%0%0%
1. Mainly JFS 2. JFS and FSI GeneFcs 3. JFS, FSI GeneFcs,
chemistry and toxicology journals
4. Other online journals in addiFon to the above
Importance of Access to Journals
• Develop a culture in your laboratory read the literature and share informaFon
– Join AAFS and/or ISFG, IAI – Develop a relaFonship with a local university in order to get access to the latest journal arFcles
– Note that-‐non forensic journals are needed in numerous disciplines such as chemistry, toxicology and trace evidence
Number of Ar?cles Published on DNA and DNA Mixtures
Journal Name “DNA” “DNA mixtures”
“DNA mixtures”
in 2012 Forensic Sci. Int. / FSI Gene1cs
1484 68 15
J. Forensic Sci. 1196 45 2 Int. J. Legal Med. 659 39 5 Croa1an Med. J. 155 12 4 Science & Jus1ce 73 5 0
PubMed.gov search conducted September 14, 2012 using “DNA” or “DNA mixtures” and journal name with and without “and 2012”
169 26
Which of the topics below would be your first choice for addiFonal training?
1. Relevant literature 2. How to validate
thresholds 3. How to develop
relevant SOPs 4. Interpreta?on of
low level mixtures 5. Sta?s?cs ISHI 2010
ISHI 2011 ISHI 2012
3 of 4 Regional Workshops N = 526
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Literature Valida?on SOPs Low level DNA Sta?s?cs
ConFnuing EducaFon: • On going training for pracFFoners is needed but only 8 hours/year is required by exisFng standards.
• One day workshops once per year are not enough
• DNA is not the only discipline with significant training needs
What is needed = Math – ∑ %μ ≈ ϵ x θ/β – Undergraduate staFsFcs and probability
• Measurement uncertainty • StaFsFcal basis of sampling procedures • StaFsFcal approaches for DNA mixtures
– Personal message to forensic scienFsts from Albert Einstein:
“Do not worry about your difficulFes in mathemaFcs. I can assure you mine are sFll greater.”
Is there help anywhere? People want to help because Forensics
is cool…….. • Don’t underesFmate your local university’s interest in working on projects you may have
• You will not always need discipline specific courses, exisFng courses may be perfect
• Maybe the course could come to the lab – Schools are more flexible than they used to be. – Adjuncts are affordable: about $5000.00 or 1.45 (STR kit), 3% of a Mass spec, or 357 roles of evidence tape.
Looking for new employees? Students…
• Arrive for graduate school with: – BS degrees in chemistry , biology or forensic science with chemistry or biology focus
– Incoming grades above 3.0 – Graduate record exam scores > 60th percenFle
• Depart with: – MS degree – More knowledge in one area such as chemistry or biology – Experience with a research project – Oqen have very good court training
Typical issues coming into and out of a MS program in forensic science
• Math and staFsFcs skills are variable • WriFng skills are variable • Work ethic is variable
• and….InsFtuFons need help with internship sites which assist students in having clarity in knowing what they want in a job.
What an employer might expect: • Programs can provide accurate references and assessment of students – Good but not brilliant – Brilliant but difficult – Great manager but not TL – Brilliant but bad work ethic
• Grades ma'er-‐ they do reflect how well the person did • Upon compleFon
– Many have taken FSAT which breaks out scores into disciplines – Thesis projects by themselves are generally too small to publish but some are very good
• Students oqen have very large loans • Not afraid of computer applicaFons, computers are friendly
RecommendaFons
• Analysts need more support than they currently receive to get addiFonal training
• Training requirements should be increased – Low cost soluFons can be effecFve with sufficient development and feedback
• Procedures need to be in place to resolve differences of opinion; analyst/analyst and analyst/ TL
RecommendaFons
• Court tesFmony review done by laboratory staff whenever possible-‐not a'orneys
• Analysts must understand the data behind procedure development
• Clarify to all staff that no one is required to sign a report they do not agree with.
END
Thank you to ASCLD and all the contributors of the information found in these slides !
