(c) 2002 Jeanne Moldenhauer1 Validation of Rapid Microbiology Systems Based upon PDA Technical Report No. 33

(c) 2002 Jeanne Moldenhauer 1

Validation of Rapid Microbiology Systems

Based upon

PDA Technical Report No. 33


Task Force Members

• Brian Bauer, Ph.D., Merck & Co.• Mark Claerbout, Lilly Research Laboratories • Warren Casey, Ph.D., GlaxoWellcome R&D• Anthony M. Cundell, Ph.D., Wyeth-Ayerst Pharmaceuticals, Pearl River, New York (Chair)• Martin Easter, Ph.D., Celsis Ltd., • Edward Fitzgerald, Ph.D. Consultant, (USP Microbiology Subcommittee)• Carol Gravens, BioMerieux, Inc.• David Hussong, Ph.D., CDER, FDA• Michael Korcynzski, Ph.D., PDA Training Institute (USP Microbiology Subcommittee)• Robin Lerchen, American Pharmaceutical Partners• Frederic J. Marsik, Ph.D. CDER, FDA• Amy Meszaros, StatProbe Inc• Jeanne Moldenhauer, Ph.D., Jordan Pharmaceuticals, Inc.• Manju Sethi, Qualicon• Scott Sutton, Ph.D., Alcon Laboratories (USP Microbiology Subcommitee)• Martin Tricarico, Chemunex (USA)• Amanda Turton, Millipore Corp• Christine Vojt, Johnson & Johnson Diagnostics Inc.• Kirsty Wills, Celsis Ltd.• Jon Wuannlund, Becton Dickinson Microbiology Systems


Disclaimer

• Presentation represents the views published in the PDA Technical Report

• May not in all ways be consistent with my personal views regarding the validation methods


Document Contents

• Part I : Selection of New Microbiological Methods– 1.0 Introduction– 2.0 Technology Overview

• Generic Description of Types of Methods• Technology Review• Growth-based technologies• Viability-based technologies• Artifact based technologies• Nucleic Acid-based technologies

– 3.0 Regulatory Review


Document Contents

• Part II: How to Validate and Implement New Microbiological Methods– 4.0 The Validation Process

• Equipment Model• Validation Criteria• Special Considerations for Microbiology

– 5.0 Glossary– 6.0 References


Implementation Requires Validation

• May require regulatory prior approval• Many are not equivalent to traditional

methods• May be little or no guidance available for

validation of microbial methods• Many have associated computer-related

systems and 21 CFR Part 11 issues• New technology frequently requires new

learning curves• Reluctance to change


What Validation is Not

• Method/technology selection

• Evaluation of feasibility of the method

• Proof of concept testing

• Vendor selection

• Doing it differently for economic considerations


Typical Flow for Selection,Evaluation & Validation for a Rapid Method

Is current compendial or industrystandard method meeting

all of your company’s needs?

Determinewhat test

requirementsand

specifications are (faster,less labor

intensive, etc.)

Look atalternatemethods

and see if themethod can

meet thespecified

requirements

Performsufficientfeasibilityproof ofconcepttesting

Does testing yieldacceptable

results?

Plan andexecute

validationprotocol.

Were results acceptable?

Submit regulatorysupplement if required

Implement test (after approval)

Evaluate other alternate methods,rejecting the unacceptable method

Continue using existing method

No

No

NoNo

Yes

Yes

Yes


Validation Definition

Established documented evidencewhich provides a high degree ofassurance that a specific process will consistently produce a product meeting pre-determined specificationsand quality attributes


The Equipment Validation Model

• Method/Vendor Requirements (Defined)

• Validation Plan Design

• Installation Qualification (IQ)

• Operational Qualification (OQ)

• Performance Qualification (PQ)– Verify and document the system performs as specified

using selected compendial organisms and environmental isolates for routine testing of batches of product.


Intentions of Validation

Demonstration data which shows: Accuracy of method (appropriateness) Reproducibility of method (continuity)


Unique Concerns for Developing ValidationRequirements for Rapid Microbiological Methods

• What type of validation support does the vendor have? (e.g., DMF, sample protocols)

• Is the new method “equivalent” to existing method• If side-by-side studies are used, what is the impact

of the rapid method being “out-of-specification”, when the traditional method is within limits?

• Is the new technology/method subject to different variabilities than traditional methods?

• Can the method handle the expected laboratory throughput?


Unique Concerns for Developing ValidationRequirements for Rapid Microbiological Methods (continued)

• USP definitions for validation terms(e.g., accuracy, precision, ruggedness . . . ) are written specifically for chemistry applications while automated microbiological methods probably should take these definitions into account {USP has drafted a chapter to resolve this}

• Are there other human interface concerns that are not present in the compendial methods?

• Difficulties in setting limits for the “unknown”


Establishing Validation Criteria

• USP Chapter <1225> Validation of Compendial Methods gives criteria, but definitions are in terms of chemical methods

• Draft revision in Pharmacopeial Forum with criteria for Alternative Methods.

• PDA Technical Report 33 has issued microbiological definitions of these same criteria

• Since some tests are quantitative and others are qualitative (e.g., limits tests), not all criteria are applicable to each test type


Validation Criteria

ApplicabilityValidation Criteria

Quantitative Test Qualitative Test

Accuracy Yes No

Precision Yes No

Specificity Yes Yes

Limit of Quantitation/Limit of Detection

Yes/Yes No/Yes

Linearity Yes No

Range Yes No

Ruggedness/Repeatability

Yes/Yes Yes/Yes

Equivalence Yes Yes


Accuracy

• Definition: A measure of the exactness of the microbiological method that is true for all practical purposes

• Determination: Analyze at least five suspensions across the range of test. Calculate each suspension as a percentage dilution of the original.

• Acceptance criteria: Should be equivalent or better than existing methods (±30% are considered equivalent)


Precision

• Definition: Precision may be a measure of either the degree of reproducibility of the microbiological method under normal operating conditions

Repeatability refers to use of the method within the same laboratory over a short period of time using the same analysts and same equipment

Reproducibility uses different analysts with the same equipment


Precision (continued)

• Determination: at least five suspensions across the range of the test should be analyzed. Each suspension should be analyzed for at least ten replicates. Standard deviation or coefficient of variation should be calculated.

• Acceptance criteria: generally, a coefficient of variation in the 15-30% range is acceptable


Precision (continued)

• For comparing precision of two methods,the F-test is recommended. If the critical value exceeds the statistical table, a significant difference exists between the methods.


Specificity

• Definition: the ability of a method to detect a range of microorganisms which demonstrate that the method is fit for use

• Determination: screen against representative range of microorganisms and sample types

• Acceptance criteria: all selected organisms are isolated and enumerated from sample matrices


Limit of Detection

• Definition: a parameter of a limit test which identifies the lowest number of microbes which can be detected, but not necessarily quantified in the test conditions

• Determination: multiple replicates, greater than or equal to five, should be tested, since one cannot reliably obtain samples with a count of one


Limit of Detection (continued)

• Acceptance criteria: ideally, if one organism is tested, it will be detected in the time frame of the assay (use of CHI-square test)


Limit of Quantification

• Definition: the lowest number of microbes which can be determined with acceptable precision and accuracy under the slated experimental conditions

• Determination: since there are no reliable methods to get a sample count of one, greater than or equal to five replicates should be used

• Acceptance criteria: if a single organism is present, it will be quantified in the time frame of the assay


Linearity

• Definition: the ability to elicit results thatare proportional to the concentration of microbes present in the sample within a given range

• Determination: at least five replicates across range of assay

• Acceptance criteria: correlation coefficient r2 = 0.9, or better with a slope not diverging more than 20% from 1.0, i.e., r2 = 0.8 - 1.2


Range• Definition: the interval between the upper and lower

levels of microbes that have demonstrated to be determined with precision, accuracy and linearity, using the method as written

• Determination: the range of the method is validated by verifying that the analytical method provides acceptable precision, accuracy and linearity, when applied to samples at the extreme of the range and within the range

• Acceptance criteria: depends upon performance characteristics of the method


Ruggedness

• Definition: degree of reproducibility of test results obtained by samples under variety of normal test conditions, e.g., different analysts, instruments, lots of reagents

• Determination: prepare a suspension of microbes and test at least ten replicates against each assay variable. Calculate standard deviation or coefficient of variation.

• Acceptance criteria: generally, a coefficient of variation in the 10-15% range is acceptable


Robustness

• Definition: a measure of its capacity to remain unaffected by small but deliberate variations in method parameters and provides an indication of its reliability during normal usage

• Determination: the manufacturer investigates changes up to 20% on the critical reagent concentrations, instrument operation parameters and incubation temperatures

• Acceptance criteria: review results needto be reviewed against the manufacturer’s quality requirements and instructions for use


Equivalence

• Definition: a measure of how similar the test results are to the method it is intended to replace

• Determination: should first be demonstrated in pure culture work. Requires the two methods be run in parallel for comparison. It’s important to have randomized experimental design. A large number of replicates shouldbe used.

• Acceptance criteria: generally, a statistical test is used


Microbiological Reference Standards

• Issues associated with identification and determination of material to be used as a reference standard for this type of testing.

• TR33 highlights the need to show equivalence to existing methods, but the technology for many of these systems is superior to the old method and may not show equivalence.

• Provides some guidance on preparing in-house standards.


Method Validation

• Variability of Microbiological Methods

• Sample Distribution Error

• Cell Morphology

• Metabolic Activity


Other Issues in Validation

• Use of multiple identical equipment within the laboratory and company

• Whether or not you are the first or fiftieth company to submit this test method

• Company’s regulatory compliance status• Unique issues with microbiology equipment

– Regulatory requirements, e.g., LAL– Impact on other system, e.g., sterility testing– Etc.

Documents

(c) 2002 Jeanne Moldenhauer1 Validation of Rapid Microbiology Systems Based upon PDA Technical Report No. 33