Method Validation - United Fresh...2019/10/03 · EC 2073/2005 Microbiological criteria states EN ISO methods there are most frequently used. Most be BAM unless tere is no BAM reference

Method ValidationWho cares? How is it done? My Matrices!

UFPA Workshop3 Oct 2019

Topics Covered

• Who Cares? • AOAC and ISO Background• Connection to Regulations/Agencies

• How is it done?• Comparison of AOAC and ISO Validation Processes

• My Matrices!• Are they included in the validation?

• Group Discussion

2

3. All Rights Reserved.30 September 2019© 3M 3M Confidential.

WHO CARES?AOAC and ISO Background

USDA Bureau of Chemistry Standardize methodology used for

composition of fertilizers by state laboratories

Directed by Harvey Wiley who wrote the 1906 law that began the US Food and Drug Administration (FDA)

• 1965 – Association of Official ANALYTICALChemists

• 1980s - microbiologists, nutraceuticals, other food science professionals

• 1991- AOAC INTERNATIONAL(Association of Official Analytical

COMMUNITIES)

AOAC began as Association of Official Agricultural Chemists

1884

4


[Code of Federal Regulations][Title 21, Volume 1]

[Revised as of April 1, 2018][CITE: 21CFR2.19]

TITLE 21--FOOD AND DRUGS

CHAPTER I--FOOD AND DRUG ADMINISTRATIONDEPARTMENT OF HEALTH AND HUMAN SERVICES

SUBCHAPTER A--GENERALPART 2 -- GENERAL ADMINISTRATIVE RULINGS AND DECISIONS Subpart A--General Provisions Sec. 2.19 Methods of analysis.

Where the method of analysis is not prescribed in a regulation, it is the policy of the Food and Drug Administration in its enforcement programs to utilize the methods of analysis of the AOAC INTERNATIONAL (AOAC) as published in the

latest edition (13th Ed., 1980) of their publication "Official Methods of Analysis of the Association of Official Analytical Chemists," and the supplements thereto ("Changes in Methods" as published in the March issues of the "Journal of the

Association of Official Analytical Chemists"), which are incorporated by reference, when available and applicable. Copies areavailable from the AOAC INTERNATIONAL, 481 North Frederick Ave., suite 500, Gaithersburg, MD 20877, or at the National

Archives and Records Administration (NARA).

PART 2 -- GENERAL ADMINISTRATIVE RULINGS AND DECISIONSSubpart A--General Provisions

https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm?CFRPart=2


US FDA Bacteriological Analytical Manual (BAM)

FDA's Bacteriological Analytical Manual (BAM) presents the agency's preferred laboratory procedures for microbiological analyses of foods and cosmetics…the rapid methods listed in Appendix 1: is a listing of different kits that are commercially available. These methods have not necessarily been evaluated by FDA, and listing of a method in this appendix does not constitute a recommendation.

Compendium of Methods used by the AgencyUSDA FSIS Microbiology Lab Guide (MLG)

The Microbiology Laboratory Guidebook (MLG) contains test methods used by the FSIS Field Service Laboratories to support FSIS regulatory activities….FSIS does not specifically endorse any of the products mentioned in the MLG and acknowledges other products may be available for laboratory use.


US FDA Bacteriological Analytical Manual (BAM)

Introduction

To test for an organism or microbial toxin not covered by the BAM, or to analyze a sample that may require special handling or processing, the user is referred to the Official Methods of Analysis of the AOAC International; …FDA works closely with AOAC International, APHA, EPA, the International Dairy Federation (IDF/FIL), and, by way of participation in Codex Alimentarius, the International Organization for Standardization (ISO). However, not all methods appearing in the BAM have been collaboratively evaluated by one or more of these organizations.

Open to Methods Alternative to the ReferenceUSDA FSIS Guidance for Evaluating Test Kit

Performance

General Considerations

The work should be carried out in a laboratory that is independent of the manufacturer’s economic interest. Alternatively, the validation may be performed through an independent organization such as AOAC, AFNOR, ISO, or NordVal. To avoid handling bias, the identity of the samples should be blinded to the analysts. The study design should be reviewed by an outside party before initiating work. FSIS can review and comment on study design. Finally, all study reports as well as the associated raw data should be available for review by FSIS.


FDA BAM Appendix 3:


FDA BAM Appendix 3

International Standards Organization (1947)

Equal footing• No matter size or strength of a country economy, each participating member has

ONE vote

Voluntary• ISO standards are voluntary• ISO has no legal authority to enforce their implementation • Some ISO standards are adopted in countries as part of their regulatory framework• Created only when a market need – 5 year review : keep, toss, maintain

10

ISO Standards impact everyone, everywhere

COMMISSION REGULATION (EC) No 2073/2005 of 15 November 2005 on microbiological criteria for foodstuffs

2.5 Vegetables, fruits and products thereof

Food Category

Micro-organisms/their

toxins, metabolites

Sampling plan Limits Analytical

reference method

Stage where the criterion applies

n c m N

2.5.1 Precut fruit and vegetables (ready-to-eat) E. coli 5 2 100

cfu/g1000 cfu/g

ISO 16649-1 or 2

Manufacturing process

2.5.2 Unpasteurized fruit and vegetable juices (ready-

to-eat)

E. coli 5 2 100 cfu/g

1000 cfu/g

ISO 16649-1 or 2

Manufacturing process

Need: Validated Food Microbiology MethodsRecognized > 30+ years ago - proprietary methods:

• Generally cheaper

• Provide faster results versus traditional culture methods

• Often easier (less technical skill needed)

Acceptance by regulatory authorities?

Central European Norm (CEN) Eureka project started (now called MicroVal):• Developed technical rules for validation

• Technical rules were transformed into a standardised (ISO) protocol

ISO 16140 (2003) - Protocol for the validation of alternative methods

12

ISO 16140 within European Commission Regulations

“Commission Regulation No. 2073/2005 on microbiological criteria for foodstuffs”:

Article 5 (Section 5) says:

ISPAM – Microbiology Method Harmonization

• Compare validation guidelines• Identified areas of difference• Promote harmonization

Working Group on Microbiology

Guidelines

International Stakeholder Panel on Alternative Methods = ISPAM

14

Participating Stakeholders = >60 industry, Gov, AcademiaGovernment

•Health Canada•Canadian Food Inspection

Agency•US Food & Drug

Administration•US Dept of Agriculture•Meat and Livestock

Australia•Netherlands Food &

Consumer Product Safety Authority

•ACHIPIA – Chile Ministry of Agriculture

•ANSES – French Agency for Food, Environmental and Occupational Health & Safety

•Maryland Department of Agriculture

•Florida Dept of Agriculture

Method Developers

• 3M Food Safety• bioMérieux• BioControl• Bio-Rad• Crystal Diagnostics• Elution

Technologies• Hygiena (Qualicon

Diagnostics)• Morinaga• Neogen• QIAGEN• R-Biopharm• Romer Labs

Food Companies

• Abbott Nutrition• Cargill• Nestle• General Mills• Hershey

Company• McCormick• Quaker Oats• Grain Millers

• AFNOR• MicroVal• NMKL/NordVal• ISO • GFCO-GIG• Allergen Control

Group

Contract Laboratories

• Q Laboratories• Adria Laboratory• AsureQuality• Mérieux

NutriSciences (Silliker)

• Eurofins• Microbac• Vanguard

CertificationBodies/NGOs

15

Comparison of Method Validation GuidelinesCriteria ISO 16140 AOAC Health Canada NordVal US FDA USDA

QualitativeMethods

ISO 16140 Doc. N 1199 (ISO CD 16140-2)PIVC2011-04-06Pending revision of Part 2

OMA, Appendix XDraft revision document dated 3/24/2011

Health CanadaDraft Part 4 dated March 2011

NordValProtocol for the validation of alternative microbiology methods March 2009

FDA’s Qualitative Microbiology Methods Validation (ORA-LAB-7 verstion1.2), pending revision (proposed revision marked in red).

Draft GuidelinesDisclaimer: The use of the term “validation” is not intended to have any application to the implementation of 9 CFR 417 4(a)(1) on initial validation of HACCP plans.

Reference Method

Defined in ISO16140-1- 1st priority is

ISO method, 2nd priority is CEN method if neither exists, then 3rd

priority is other recognized methods

Note – definition still under discussion to open up for non ISO/CEN th d

Can use various existing recognized analytical methods (e.g., AOAC, OMA, ISO, FDA BAM, FSIS MLG, Health Canada) If no appropriate Ref can indicate “NA” in summary tables for POD

Acceptable Ref published by EC(Part 1)May include any methods from methods organizations (i.e., AOAC, FDA, APHA, ICMSF, IDF, ISO, etc…)Where no Ref exists, MMC assess on case by case basis

ISO, CEN, NMKL, BAM,etc… It is up to the applicant; however, as the EU regulation in EC 2073/2005 Microbiological criteria states EN ISO methods there are most frequently used.

Most be BAMunless tere is no BAM reference method.In case of no Bam, then FSIS MLG, AOAC, ISO, and Health Canada are all potential reference methods. APHA, ICMSF, and IDF methods may also be used as reference methods

FSIS Microbiology Laboratory Guidebook (MLG) cultural methods is the is used for validating methods used by FSIS regulated establishments. FDA BAM or methods referenced by ISO or Codex. Non cultural methods applicable in some cases

16

ISPAM – Microbiology Method Harmonization

• Compare validation guidelines• Identified areas of difference• Promote harmonization

Working Group on Microbiology

Guidelines

International Stakeholder Panel on Alternative Methods = ISPAM

17

COMMISSION REGULATION (EU) 2019/229 of 7 February 2019 amending Regulation (EC) No 2073/2005 on microbiological criteria for foodstuffs

Article 1 Amendments to Regulation (EC) No 2073/2005

(1) in Article 2 the following points are inserted after point (m):

‘(n) “a broad range of foods”, as referred to in EN ISO 16140-2, means food as defined by the first subparagraph of Article 2 of Regulation (EC) No 178/2002 of the European Parliament and of the Council (*);

(o) “independent certification body” means a body which is independent from the organisation that manufactures or distributes the alternative method and which provides a written assurance, in the form of a certificate, testifying that the validated alternative method meets the requirements of EN ISO 16140-2;

(p) “production process assurance of the manufacturer” means a production process whose management system guarantees that the validated alternative method remains conform to the characteristics required by EN ISO 16140-2 and ensures that mistakes and defects in the alternative method are prevented;

(*)Regulation (EC) No 178/2002 of the European Parliament and of the Council of 28 January 2002 laying down the general principles and requirements of food law, establishing the European Food Safety Authority and laying down procedures in matters of food safety (OJ L 31, 1.2.2002, p. 1).’;


ISO Members

Microbiology Methods allowed for Official Testing20

ISO methods – online, for a FEEAOAC OMA methods – for a FEE

US FDA Bacteriological Analytical Manual – online, FREEUSDS FSIS Microbiological Laboratory Guidebook – online, FREE

Reference Methods - published


HOW IS IT DONE?AOAC vs ISO Validation

1. Performance characteristics are defined, then…2. compared to a reference method, and then…3. statistically evaluated to determine “equivalence”

A method is validated for the matrices [& sample sizes] included in the validation

Validation is:


Certification Bodies and Validation Guidelines

AOAC Research Institute AOAC INTERNATIONAL Methods Committee Guidelines for Validation of Microbiological Methods for Food and Environmental Surfaces (2012)

• Performance Tested MethodSM

• Official Method of AnalysisSM

• Harmonized Method

Certification to ISO 16140-2:2016Protocol for the validation of alternative (proprietary)

methods against a reference method

• NordVal Certification• MicroVal Certification• NF Validation via AFNOR Certification

Requirements for Certification:

1. Validation - Method comparison study against the reference method

2. Quality System of the manufacturer must be in conformity with quality assurance requirements

3. Renewal of the certified methods – post certification

25

• All foods? (Broad Range)• Specific foods?• Environmental?• Primary production?• Animal feed?

First determine Method SCOPE

CATEGORIESRaw Milk & Dairy Products

Heat Processed Milk & Dairy Products

Raw meat & Ready-to-cook meat products (except poultry)

Ready-to-eat, ready-to-reheatmeat products(except poultry)

Raw Poultry & ready-to-cook poultry products

Ready-to-eat, ready-to-reheat meat poultryproducts

Eggs & egg products (derivatives)

Raw & ready-to-cook fish & seafoods(unprocessed

Ready-to-eat, ready-to-reheat fishery products

Fresh produce & fruits

Processed fruits & vegetables

Dried cereals, fruits, nuts, seeds and vegetables

Infant formula & infant cereals

Chocolate, bakery products & confectionary

Multi-component foods or mealcomponents

Primary production samples

Pet food & animal feed

Environmentalsamples (food or feed production

Method Scope: Matrices ISO 16140-2Table A. 1: Classification of sample types & suggested target combinations for validation studies

There are 18 CATEGORIES recognized / harmonized between ISO & AOAC

“Broad Range of Foods” claim = 5 CATEGORIES of food:• At least 3 food types within EACH category• Test 20 samples (not portions) of EACH food type within the category • = 60 results from different samples within EACH category

ISO 16140: “ALL FOODS” Claim

28

Example of Categories & Food types

Category Types Matrices (samples)

Meat products

Raw Meat cuts, carpaccio’s, minced meat… 20 items (samples)

Heatprocessed

Cooked ham, cooked meat preparations…20 items

Cured Cured ham, bacon, etc…20 items

= 60 different samples tested per Meat Products Category

29

Performance Characteristics – AOAC and ISO Validation

Brodsky, AOAC International, 2013 Annual Meeting

30

Method Comparison Study

AOAC Research InstituteMethod Developer• Matrix Study – inoculated

Independent Laboratory• 20% Foods /surfaces repeated

ISO Certification BodyExpert Laboratory• Matrix study – natural contamination

• Robustness• Stability• Lot-to-Lot Variation• Instrument Variation

31

Potential Method Interference

• Sample matrix

• Sanitizer / Disinfectant (chemistry, concentration)

• Neutralizer

• Sponge / swab type

• Competing organisms (cross reactivity)

• Environmental contaminants (organics and inorganics)

• Deviation from instructions (sampling protocol, sample hold / processing, detection assay interpretation, etc.)

32


AOAC OMA• 1 + matrices (depending on claims)• 3 levels of contamination• Samples sent in blind duplicate

ISO Certification Body• 1 Food matrix• 3 levels of contamination• Samples send in blind duplicate

Method Comparison – Collaborative study

Collaborative study – 10 labs QUALitative / 8 labs QUANTitative

Repeatability


Presentation to Technical Committees

AFNORTechnical

Committee:• Meets 5 times/year

• Must present in person

• Grants Certificate

MicroValTechnical

Committee:• Meets as needed

• Review can place via Skype or in person


NordValTechnical

Committee:• Meets several

times /year

• Review takes place in person/telecon


AOACExpert Review Panel

(ERP):• Meets ~ quarterly

• Must present in person


35. All Rights Reserved.30 September 2019© 3M 3M Confidential.35

Certificates – NF Validation via AFNOR Certification

36

MicroVal Certification

Summary Report available online attached to the Certificate

37

38

Summary Report available online attached to the

Certificate

39

AOAC® Performance Tested MethodsSM

AOAC® Official Method of AnalysisSM

40


MY MATRICES!Method Extension

41

PTM vs OMA - WHY?

• A “fully” validated method per ISO 16140-2:2016 = ILS• A “fully” validated method per AOAC OMA = ILS• EU Directive 2073:2005 = ILS• FDA BAM = AOAC OMA (ILS)• New Zealand, Philippines, Central America, Brazil, Chile, etc. = ILS

Government Agencies - require methods that have been collaborativelystudied (InterLaboratory Study = ILS)

Methods requiring validation:

• New or original methods

• Modifications to validated methods

• Extension of scope - additional analytes, matrices, or changes in use

• Changes involving new technology or automation

• Significant parameter changes: reagents, time/temperature incubation

It is possible that small changes/matrix extensions to a related product etc would not need a ‘full validation’ but may need some level of testing to assess for unintended differences.

43


VerificationDemonstrate in the hands of the USER that the method can be performed to meet the defined method criteria

Distinguishing Validation and Verification

ValidationDemonstrate the METHOD is “equivalent” to the reference method [for the matrices validated] based on defined method criteria

ISO 16140 Series

ISO has recognized several ISO 16140 documents as “high profile” because they believe the global food industry has a great need for these documents:

• ISO 16140-2 Method Validation - Published August 2016

• ISO 16140-3 Method Verification – to Publish ~April 2020

Reviewed by USER LABORATORIES - vs only WG3 Experts

45

Overlap Between Scopes

“Broad Range of Foods”Or – only specific categories?

Categories Validated Routinely Tested in User Lab

Review the results of the validation study.

Compare Verification results to Validation results

47

TWO Kinds of Verification

Implementation (food) item


DISCUSSION

Selecting a Method that meets your needs:

• Is the method validated for YOUR matrix and sample size?

• Was it validated through a recognized validation scheme?• What was the reference used in the validation study?• If is an AOAC Method, is it a PTM method or an OMA method?• Does the lab have the validation data and official method on file?

• Does the method provide the appropriate level of sensitivity?

• Has the lab verified they can perform the method correctly?• Do they participate in a proficiency test program?

49


Documents

Method Validation - United Fresh...2019/10/03 · EC 2073/2005 Microbiological criteria states EN ISO methods there are most frequently used. Most be BAM unless tere is no BAM reference