1

AOAC – NMKL - NordVal

International Symposium 21/22 Mai 2015 in Stockholm

Harmonization of the NordVal International

Validation Protocol with ISO 16140:2015

for the validation of alternative methods.

Sven Qvist

Chairman

NordVal International

www.nmkl.org

2

Recognised international validation organisations have participated in the elaboration of the new ISO/FDIS 16140-2:2015.

AFNOR Certification

AOAC

MicroVal

NordVal International

ISO 16140 method validation

program consists of 6 parts 1. Vocabulary

2. Protocol for validation of alternative methods

3. Protocol for verification of reference and

alternative methods in a single laboratory

4. Protocol for in-house method validation

5. Protocol for interlaboratory validation

6. Protocol for validation of confirmation methods

3

4

Objective of an international protocol for validation of alternative methods

Guidance for applicants, kit producers, expert laboratories, steering groups and technical committees on the procedure of validation of alternative microbiological methods

Obtain international recognition of validations carried out according to ISO 16140:2015

5

Present legal basis for the use of alternative microbiological methods.

EC 2073/2005 states that the use of alternative methods are acceptabel when validated against an authorised reference method and certified in accordance with ISO 16140:2003 or other internationally similar protocol.

6

Importance of validation

• To ensure reliable analytical results

• To ensure a high level of food safety

• Demonstrate equivalence with ref. method

• Demonstrate fit for purpose

• To be in accordance with EU regulations

• To be in compliance with the manufacturer's

claims

7

Validation protocol contain sections for both qualitative and quantitative methods - both divided into:

comparison study of the alternative method

against a reference method

(performed by an expert laboratory)

collaborative study of the alternative method.

Valid results from at least 10 laboratories

(inclusion of 12-15 labs is advisable).

NordVal International criteria for

approval of expert laboratories

• Economical independent of the applicant.

• Accredited to perform the ref. method.

• Be familiar with the principle of the

method to be validated.

•A competent person shall be in charge.

•A detailed protocol shall be approved.

8

9

Method Comparison Study - Qualitative methods

For food: at least 5 relevant food matrices

For other categories: a relevant number of matrices.

For each matrix 60 samples are analysed

Inclusivity, exclusivity, sensitivity, accuracy,

specificity, limit of detection and agreement

between methods are studied

Method comparison study on

inclusivity and exclusivity

Inclusivity: Select 50 pure cultures of the

target microorganisms relevant to the

alternative method and the food matrices

(for Salmonella select 100 pure cultures)

Exclusivity: Select 30 pure cultures of

non-target microorganisms known to

cause interference with the target

microorganism

10

11

Method comparison study for accuracy, sensitivity and specificity Meat matrices in the present NordVal protocol

comprises one main category and 5 submatrices:

1.1 Raw red meat

1.2 Raw white meat

1.3 Raw smoked and salted products

1.4 Heat treated products

1.5 Fermented products

12

Method Comparison Study – Matrices for meat

New ISO 16140:2015 validation protocol – 4

categories

1. Raw meat and ready-to- cook meat products

2. Ready-to-eat and ready-to-reheat meat products

3. Raw poultry and ready-to-cook poultry products

4. Ready-to-eat and ready-to-reheat poultry products

13

Method Comparison Study – Matrices for meat ISO 16140:2015 validation protocol - Category 2:

2.1 Cooked meat products

2.2 Fermented or dried meat products

2.3 Raw cured (smoked) products (aw > 0,92)

2.4 Raw cured (smoked) products (aw < 0,92)

ISO Interpretation of results

between ref. and alt. method

Reference

Method

Alternative

Method

+

+

-

-

+

-

-

+

-

+

Confirmed

Alt. Method

Interpretatio

n

Not needed

PA

Not needed

NA

Not needed

ND

+

PD

-

NA 14

15

Method Comparison Study - Qualitative methods - Screening

Responses Reference method

positive (+/ )

Reference method

Negative (-/)

Alternative

method

Positive

+/+ positive

agreement

(PA)

-/+ positive

deviation

(PD)

Alternative

method

Negative

+/- negative

deviation (ND)

-/- negative

agreement (NA)

16

Method Comparison Study - Qualitative methods - Confirmation

Responses Reference method

positive (+/ )

Reference method

Negative (-/)

Alternative

method

Positive

+/+ positive

agreement

(PA)

-/+ true positive

(TP)

Alternative

method

Negative

+/- false negative

(FN)

-/- negative

agreement (NA) +

false positives (FP)

17

Estimation of accuracy, sensitivity, specificity and agreement - Confirmation Matrix PA

NA

ND

PD

Sum Relative Accuracy

AC (%)

Relative

Sensitivity SE (%)

Relative

Specificity SP

(%)

N (PA+NA+FP) x 100

N

PA + TP x 100

PA+FN

NA x 100

NA+FP

Food cat 1

Food cat 2

Food cat 3

Food cat 4

Food cat 5

Other

18

NordVal International acceptance

criteria for qualitative methods

The overall sensitivity should 95%

The agreement between method, kappa

should indicate very good agreement,

i.e. κ > 0.80.

19

Relative level of detection study of the alternative method against the reference method- ISO 16140:2015

Level 1 Level 2 Level 3

Concentration negative low higher

No of replicates 5 20 5

No of positives 0 25-75% 100%

RLOD =

LOD - alt.

LOD - ref.

Method collaborative study

of a qualitative method

Determine the variability of the results obtained by the alternative method in different laboratories using identical samples

The expert and 10 participating laboratories are using both the alternative and the reference method for the study (10 valid results required)

One food matrix, 8 replicates and three different levels of contamination are analysed in the study

20

Organisations and locations of

the participating laboratories

The 10 collaborators shall come from a minimum of 5 different organisations (advisable with at least 12 collaborators)

A maximum of 3 data sets can be produced by one organisation with laboratories in different locations

T

21

22

Performance characteristics for

the study of quantitative methods

Inclusivity

Exclusivity

Reliability

Repeatability

Reproducibility

Uncertainty

23

Comparison Study-Quantitative methods

For each matrix, calculate the mean and the SD of

the results obtained by the reference method and

its confidence level 2 ∙ SD

Calculate the mean and the SD of the results

obtained by the alternative method.

Plot the results into a designed diagram.

24

Quantitative Methods

Milk powder

0

1

2

3

4

5

6

0 1 2 3 4 5 6

levels

log

cfu

/g

reference method

alternative method

25

Acceptance criteria for

quantitative methods

If the mean of the alternative method is included in the confidence interval of the reference method (i.e. refmean ± 2 SD), there is no significant deviation between the methods.

Repeatability of less than 0.4 log cfu/g illustrates satisfactory precision.

26

NordVal International Steering Group

Denmark: Sven Qvist (chairman)

Finland: Annika Pihjalasaari

Iceland: Franklin Georgsson,

Norway: Kjell Hauge

Sweden: Charlotta Engdahl Axelsson

Secretary: Hilde Skår Norli, NMKL

NordVal International Organisation

• NMKL national committees appoint the steering

group members - one representative from each

of the Nordic countries.

• NordVal International appoints experts in ad

hoc technical committees.

27

NordVal International policy

• The persons involved in the NordVal

International validation programme shall be

independent and cannot have special

interests in the production and distribution

of test kits, special reagents or instruments.

• The members must have the required

competence in microbiology and validation.

28

29

30

31

32

33

Future status of validated and

certified alternative methods

• Capacity, rapidity, cost effectiveness

and reliability have resulted in

increased and dominating role as

preferred methods in the food industry.

• Acceptance by authorities as official

standard reference methods should be

considered as a possibility in future

legislation. 34

NordVal International policy

• The persons involved in the NordVal

International validation programme shall be

independent and cannot have special

interests in the production and distribution

of test kits, special reagents or instruments.

• The members must have the required

competence in microbiology and validation.

35

Increased scope for items to be

covered by ISO 16140-2:2015

• Present validation protocol: Food, animal feed,

environmental samples and samples from the

primary production stage.

• Future items for consideration: Water, virus,

confirmation methods, microtoxins, parasites.

36