Validation of New Molecular Assays

Kimberlee Musser, PhD

Wadsworth Center-NYSDOH

August 12, 2008

Clinical Laboratory Evaluation Program

All clinical laboratories in New York State required to be permitted through the Clinical Laboratory Evaluation Program (CLEP).

Any Laboratory (including Wadsworth Center) wishing to perform a molecular assay must submit an application for approval to use on patient specimens

http://www.wadsworth.org/labcert/TestApproval/index.htm

ValidationExamples

CLEP Submission Guidelines

Nucleic Acid Amplification Tests

* SYBR Green, melt point analysis alone is not sufficient

CLEP Approval Process for Use of Nucleic Acid Amplification Assays

SOPMDetailed protocolMolecular workflow detailsQuality Assurance

Evidence of proficiency test program including # of samples; passing rateValidation dataReferencesSpecimen Result Report (positive, negative, and inconclusive samples with all disclaimers)References including package inserts

Where to Start?

CLIA regulations (42 CFR Section 493.1253) indicate that for laboratory-developed assays, commonly referred to as “home-brew assays,” the following performance characteristics be determined: analytical sensitivity, analytical specificity (including the effect of interfering substances), precision, accuracy, measuring range of test results, and reference intervals (normal values).

http://www.wadsworth.org/ labcert/TestApproval/index.htm

Assay DevelopmentIn-silico design- target gene, database search, multiplexing assessment

Primer and Probe optimization

Mastermix and MgCl+ optimization

Sensitivity (Determination of LOD with and without matrix)

Specificity

Inter/Intra assay reproducibility

Wadsworth Center Process for Molecular Assay Validation

Example

Mycobacterium tuberculosis complex-real-time PCR test

Experience with real-time PCRInstrumentationEase of use< 2 hoursPCR flow, closed systemInhibition test<$5.00 a test

Assay Development

Choose Target-IS6110Insertion element with a multi copy target (1 to 25 copies)Previously evaluated, Increased sensitivity, Sequence data

Evaluated 2 different primer/probe setsSavelkoul et al. *Desjardin et al. (primers were modified from publication)

Constructed inhibition controlTOPO cloned segment of fruit fly DNA containing primer binding sites specific to MTB assayDetected by additional probe while using same primers designed for IS6110 target

Optimize assayMastermix, MgCl2, probe, and primer titrations

*Savelkoul et al., J of Microb Methods 66 (2006) 177-180

Assay Validation

SensitivityAnalytical - M. tuberculosis (1 copy of IS6110) in bufferMatrix- M. tuberculosis (1 copy of IS6110) in sputum

Specificity37 members of MTB complex 24 M. tuberculosis strains (IS6110 copies from partial to 21)

49 NTM and other respiratory pathogens

Reproducibility

Sensitivity Results-analytical

Sensitivity determined to be 0.05 colony forming unit (cfu) per reaction

Sensitivity Results-sputum

Sensitivity determined to be 0.5 colony forming unit (cfu) per reaction in sputum specimens

Specimen Preparation Evaluation

Heat kill straight with no other treatmentEpicentre Masterpure DNA extraction KitEasyMag extractionProcess using IDI (Infectio Diagnostics, Inc., Quebec, Canada) lysis extraction

Currently: BD GeneOhm Lysis Kit Simple protocol: Vortex 5 minutes in lysis tube containing glass beads, heat @ 95 °C for 2 minutes

Assessment of best approach to achieve high sensitivity, no PCR inhibitors, ease of use, time

Validation of Home Brew Assays

BLINDED Study

Compare results to gold standard or FDA approved assayCompare results to results from another CLEP approved assaySpike clinical samples and compare to expected results

30 positives samples-10 must be close or no more than 10x LODMust be performed in matrix (blood, sputum). If no clinical samples available, seeded specimens can be utilized

Assay Validation

Side by side testing with GEN-PROBE® AMPLIFIED™ Mycobacterium Tuberculosis Direct (MTD) test (FDA approved)

Tested 291 samples for validation

87 0

9 190

3� 2�

Real-time PCR

MTD test NEG

NEG

POS

POS

INCONCLUSIVE

tDetermined to be culture positivelDetermined to be culture negative

*1 specimen PCR inhibited

Use of Appropriate Controls

Extraction controls- Lysis/ExtractionBacteria or virus preferred, if unavailable can use DNASeeded at a low level

Amplification controls (positive, NTC)

Inhibition testing controlControl that many labs do not test for or test for improperlyIdeally used as internal control in every sampleFor well established method can submit data from 100 samples per specimen type at a low level

CLEP approval of Broad-range Sequence based assays

New guidelines recently established

Detailed SOPM (controls, Taq, acceptable minimum sequence length, base calling)

Reporting algorithm (100%, <95%)

Databases utilized (public, private)

Blinded validation -30 samples (different species)

In silico proficiency test- 10 DNA sequences

Target specific guidelines in progress

CLEP Approval- Sequence-based Methods

Wadsworth Center has CLEP approved rpoB assay to determine mutations associated with rifampin resistance in MTB from culture

Development and validation of pyrosequencing assay for rapid sequence analysis of rpoB

Validation ongoing- all MTB complex PCR positive-specimensComparison to CLEP approved assay and gold standardPotential to have PCR and rpoB sequence results on day of specimen receipt<$1.00, 4 hours to resultAll specimens remain tested by cultureand traditional susceptibility panels

PPiATP Time

Light

Acknowledgements

Tanya Halse, Liz Nazarian, Danielle Wroblewski

Keith Derbyshire, Vincent Escuyer, Phyllis Cunningham, Jeff Driscoll

Christina Egan, Ron Limberger

Kirsten St. George, Amy Dean, Monica Parker, Jan Keithly, Vishnu Chaturvedi, Daryl Lamson

Mike Ryan, Vicky Derbyshire

Dick Jenny, Mike Neal- CLEP

DesignDesign

F1

TCTTC

Seq1

Rev

Mutations located in the 81 bp ofM. tuberculosis rpoB associatedwith rifampin resistance

Seq2

40%17%

3%1%

6%6%

GluGAA

3%

16% 7%

Mutations at WC 10/05 thru 12/07(n=70)

1%

1%

74%