Monitoring the Performance of Nucleic Acid Tests using Data Generated from EDCNet and DigitalPT

Wayne Dimech, Darren Jardine, Thu-Anh Pham and the staff of the NRL

Presentation OutlinePresentation Outline

NRL’s ever-widening role in QA

Why Internet-based technology (IT)

IT implications and examples for:External quality assessment schemes (EQAS)

Quality control (QC) programmes

Specificity monitoring

Summary - effective quality monitoring

NRL’s Ever-widening Role in QANRL’s Ever-widening Role in QA

Monitors the performance of in vitro diagnostic (IVD) reagents for BBV in Australia including nucleic acid testing (NAT)

Provides quality assurance internationally (200+ laboratories)

Extending collaborations with like organisations

Developing IT to support its client laboratories

Why Internet-based TechnologyWhy Internet-based Technology

Standardise data collection and storage

Manage increasing data complexity

Improve maintenance of data

Offer better statistical analyses

Give instant feedback to laboratories to ensure their ability to monitor performance

Individualise reporting

Offer easy administration of QA functions

Offer international accessibility

Facilitate international collaborations

NAT EQAS Design NAT EQAS Design

EQAS for NAT are designed to examineLimits of detection

Inter- and intra-run variability

Contamination/carry over

Inhibition

Instrument/method variability

Develop QC samples

Geno-subtype detection

IT Implications on NAT EQASIT Implications on NAT EQAS

NAT EQAS concepts in DigitalPTAssay

Test Process

Kit

Instrument

Analyte

Unit

DigitalPT ConceptsDigitalPT Concepts

Laboratory

Assay

Analyte

Instrument

Extraction

Technique

Kit

Analyte

Instrument

Amplification

Technique

Kit

Instrument

Detection

Technique

Kit

Units

Kit: Roche MagNA Pure LC TNA Isolation KitInstrument: MagNA PureKit: Roche COBAS Amplicor HIV-1 Monitor v 1.5

Instrument: COBAS Amplicor

Kit: Roche COBAS Amplicor HIV-1 Monitor v 1.5Instrument: COBAS Amplicor

IT Implications on Quality Control IT Implications on Quality Control

Collects, stores and analyses QC data

Allows for inter-laboratory comparisons

Requires peer groups for assessment of accuracy

Can monitor imprecision and bias

Can identify sources of variation

Quality Control InformationQuality Control Information

Data collected can be used to Monitor reagent batches

Troubleshoot laboratory problems

Provide oversight of regional laboratories

Review QC sample stability

Collect information for assay standardisation

Calculate uncertainty of measurement

Chlamydia NAT QC ProgrammeChlamydia NAT QC Programme

Roche COBAS Amplicor Chlamydia

QC sample produced by NRL

18 month period

15 laboratories, 28 instruments

3,570 observations;

CV (%) of all results = 9.14%

CV (%) ranged from 4.31 - 16.12%

Chlamydia NAT QC SummaryChlamydia NAT QC SummaryQC sample, CTNG:QL:NAT:01, was tested in the Roche COBAS Amplicor

Chlamydia in laboratories designated by number. The instrument, number of observations, mean and variations around the mean over the period 01/01/1999 to 21/05/2007 are shown.

Lab Machine n Mean SD x-2SD x+2SD Min Max %CV

A  COBAS 3 44 3.66 0.41 2.84 4.48 2.60 4.00 11.20

A  COBAS 4 30 3.86 0.17 3.52 4.20 3.36 3.96 4.40

A  COBAS 5 28 3.63 0.35 2.93 4.33 2.56 4.00 9.64

B  COBAS 1 266 3.67 0.22 3.23 4.11 2.22 4.00 5.99

B  COBAS 2 287 3.71 0.37 2.97 4.45 0.02 6.67 9.97

B  COBAS 3 233 3.82 0.24 3.34 4.30 2.72 4.00 6.28

B  COBAS 4 285 3.82 0.21 3.40 4.24 2.76 4.00 5.50

B  COBAS 5 28 3.71 0.16 3.39 4.03 3.38 4.00 4.31

B  COBAS 6 28 3.67 0.22 3.23 4.11 3.27 4.00 5.99

C  COBAS 1 18 3.52 0.37 2.78 4.26 2.98 4.00 10.51

C  COBAS 2 116 3.72 0.37 2.98 4.46 1.83 4.00 9.95

C  COBAS 3 101 3.75 0.45 2.85 4.65 1.36 4.00 12.00

C  COBAS 4 74 3.88 0.24 3.40 4.36 2.97 4.00 6.19

C  COBAS 5 32 3.86 0.23 3.40 4.32 2.81 4.00 5.96

18

287

Laboratory C:CV(%)

COBAS 5 5.96COBAS 4 6.19 COBAS 2 9.95COBAS 1 10.51COBAS 3 12.00

Mean:3.52 to 3.86

Blood screening NAT QC SummaryBlood screening NAT QC Summary

Three Chiron TMA assays monitored

Period of 15 months

21 laboratories, 31 instruments

Two batches of QC samples for each analyte (HBV, HCV and HIV)

More than 16,000 data entries

Blood screening NAT QC SummaryBlood screening NAT QC Summary

 QC Sample PeliSpy Pro HCV

Batch number 600403 625610

Assay Duplex Ultrio TIGRIS Duplex Ultrio TIGRIS

Number 2995 925 285 2958 173 66

Mean 8.63 6.92 8.06 7.95 6.73 7.27

SD 1.98 0.65 1.09 1.87 1.11 1.08

CV (%) 23.0 9.5 13.6 23.6 16.5 14.8

Blood screening NAT QC SummaryBlood screening NAT QC Summary

 QC Sample PeliSpy Pro HIV

Batch number 600402 629201

Assay Duplex Ultrio TIGRIS Duplex Ultrio TIGRIS

Number 3044 1222 290 2743 127 65

Mean 20.98 12.19 13.55 21.09 14.16 15.06

SD 3.88 1.31 1.97 2.61 1.22 1.21

CV (%) 18.7 10.7 14.6 12.4 8.6 8.1

Blood screening NAT QC SummaryBlood screening NAT QC Summary

 QC Sample PeliSpy Pro HBV

Batch number 600404 629102

Assay Ultrio TIGRIS Ultrio TIGRIS

Number 952 292 172 65

Mean 14.23 15.67 14.27 15.01

SD 1.17 1.04 1.03 1.41

CV (%) 8.2 6.6 7.2 9.4

Chiron TMA - DuplexChiron TMA - Duplex

Results of two batches of AcroMetrix PeliSpy Pro samples tested in the Chiron Duplex TMA Assays

DPSHCV600403 DPSHCV625610 DPSHIV600402 DPSHIV629201

Batch Number

0.00

10.00

20.00

30.00

40.00

HCV HIV

A B A B

QC sample batches

Chiron TMA - UltrioChiron TMA - Ultrio

Results of two batches of AcroMetrix PeliSpy Pro samples tested in the Chiron ULTRIO TMA Assays

UPSHCV600403UPSHCV625610

UPSHIV600402UPSHIV629201

UPSHBV600404UPSHBV629104

Batch Number

0.00

5.00

10.00

15.00

20.00

HCV HIV HBV

A B A B A B

QC sample batches

Chiron TMA - TIGRIS Chiron TMA - TIGRIS

Results of two batches of AcroMetrix PeliSpy Pro samples tested in the Chiron TIGRIS TMA Assays

TPSHCV600403TPSHCV625610

TPSHIV600402TPSHIV629201

TPSHBV600404TPSHBV629102

Batch Number

0.00

5.00

10.00

15.00

20.00

HCV HIV HBV

A B A B A B

QC sample batches

IT Implications on Specificity MonitoringIT Implications on Specificity Monitoring

Collects and analyses the initial (IRR) and repeat reactor rates (RRR) and compares them with confirmed positive rate

Monitors assay and laboratory performance

Increase in IRR may indicate contamination

Results may be analyzed in conjunction with QC results

ConclusionConclusion

Internet technology allows:More complex data collection and analysis

Standardisation of data collection

International participation

Greater scalability

Facilitation of collaboration

Immediate feed back to oversight bodies and manufactures on assay performance