CPA is a UKAS

company

Martin Stearn, UKAS Senior Assessment Manager

ISO 15189:2012 and Susceptibility Testing

BSAC User Day 22 June 2016

© UKAS June 2016

CPA is a UKAS

company

• Discuss the various clauses within the ISO 15189:2012 Standard

relevant to antimicrobial susceptibility testing

• Provide some clarity regarding UKAS assessment process

• Not to tell you what you need to do or how to do it

Objectives

2

CPA is a UKAS

company

• Discuss the various clauses within the ISO 15189:2012 Standard

relevant to antimicrobial susceptibility testing

5.5 Examination processes

ISO 15189:2012 Requirements

3

CPA is a UKAS

company

4.3 Document control

4.5 Examination by referral laboratories

4.6 External services and supplies

4.7 Advisory services

4.9 Identification and control of non-conformities

4.10 & 4.11 Corrective & preventive action

4.12 Continual improvement

4.13 Control of records

4.14 Evaluation and audits

4.14.2 Periodic review of procedures

4.14.3 Assessment of user feedback

4.14.4 Staff suggestions

4.14.5 Internal audit

4.16.6 Risk management

ISO 15189:2012

Management requirements

4

CPA is a UKAS

company

5.1.5 Training

5.1.6 Competence assessment

5.1.8 Continuing education and professional development

5.2.3 Storage facilities

5.3.1 Equipment

5.3.1.2 Acceptance testing

5.3.1.3 Instructions for use

5.3.1.4 Calibration and metrological traceability

5.3.1.5 Maintenance and repair

5.3.1.6 Incident reporting

5.3.1.7 Equipment records

5

ISO 15189:2012

Technical requirements

CPA is a UKAS

company

5.3.2 Reagents and consumables

5.3.2.2 Reception and storage

5.3.2.3 Acceptance testing

5.3.2.4 Inventory management

5.3.2.5 Instructions for use

5.3.2.6 Adverse incident reporting

5.3.2.7 Records

5.4.3 Request form information

5.5 Examination procedures

5.5.1.1 General

5.5.1.2 Verification of examination procedures

5.5.1.3 Validation of examination procedures

5.5.1.4 Measurement uncertainty of measured quantity values

6

ISO 15189:2012

Technical requirements

CPA is a UKAS

company

5.5.3 Documentation of examination procedures

5.6 Ensuring the quality of examination results

5.6.2 Quality control (IQC)

5.6.3 Interlaboratory comparisons (EQA)

5.6.4 Comparability of examination results

(procedures, equipment, sites as applicable)

5.7.1 Review of results

5.8 Reporting of results

5.8.2 Report attributes

5.8.3 Report content

5.9 Release of results

5.9.2 Automated selection and release of results

5.9.3 Revised reports

5.10.3 Information system management

7

ISO 15189:2012

Technical requirements

CPA is a UKAS

company

• Laboratories are free to use whatever method they select.

• ISO 15189:2012 does not prescribe any particular method or the

use of a ‘gold standard’ or the need to change to the latest method.

• Neither do UKAS, we’ll assess your methods and your competence

to use them.

• ISO 15189:2012 does specify the need to select methods which

have been validated and verified to achieve the right result.

Method selection

8

CPA is a UKAS

company

5.5.1 Selection, verification and validation of examination processes

5.5.1.1 General

The laboratory shall select examination procedures which have been

validated for their intended use.

‘Shall’ = you must

(only 321 ‘shalls’ in the Standard!)

‘Should’ = you shall unless you can justify not doing so

(24)

NOTE: Preferred procedures are those specified in the instructions for use

of in vitro medical devices or those that have been published in

established/authoritative textbooks, peer-reviewed texts or

journals, or in international consensus standards or guidelines, or

national or regional regulations. (Standard methods)

ISO 15189:2012

9

CPA is a UKAS

company

Confirmation, through the provision of objective evidence, that the

requirements for a specific intended use or application have been

fulfilled.

Validation

10

The method achieves the intended outcome because;

• Evidence A

• Evidence B

• Evidence C

• Evidence D

The method works

The zones of growth inhibition achieved by the AST method

accurately demonstrate susceptibility or resistance for that organism

CPA is a UKAS

company

Confirmation, through the provision of objective evidence, that the

requirements for a specific intended use or application have been

fulfilled.

Validation (again)

The method achieves the intended outcome because;

• Evidence A; published in established/authoritative textbooks, peer-

reviewed texts or journals,

• Evidence B; international consensus standards or guidelines

• Evidence C; national or regional regulations.

• Evidence D: records of who has validated the procedure

The method works

The zones of growth inhibition achieved by the AST method accurately

demonstrate susceptibility or resistance for that organism

11

CPA is a UKAS

company

5.5.1 Selection, verification and validation of examination processes

5.5.1.3 Validation of examination procedures

The laboratory shall validate examination procedures derived from

the following sources:

a) Non-standard methods

b) Laboratory designed or developed methods

c) Standard methods used outside their intended scope

• Applying breakpoint criteria to related or even unrelated organisms

a) Validated methods subsequently modified

• Alteration of the method in any way (e.g. incubation

time/temperature, disc concentration, media, inoculum density)

ISO 15189:2012

12

CPA is a UKAS

company

Verification

We (the laboratory) can achieve the intended outcome because;

• Evidence X

• Evidence Y

• Evidence Z

The method works in our hands

The zones of growth inhibition we achieve by using the validated AST

method accurately demonstrate the susceptibility or resistance of our

isolates to which we have applied the AST method

Confirmation, through provision of objective evidence, that specified

requirements (as demonstrated by validation) have been fulfilled.

13

CPA is a UKAS

company

Verification (again)

We (the laboratory) can achieve the intended outcome because;

• Evidence X; performance characteristics (we can do) demonstrate

the performance claims (what is expected) have been met in our lab

• Evidence Y; documented verification procedure

• Evidence Z; records of review & authorisation of the verification data

The method works in our hands

The zones of growth inhibition we achieve by the AST method

accurately demonstrate the susceptibility or resistance of the isolates to

which the method has been applied

Confirmation, through provision of objective evidence, that specified

requirements have been fulfilled.

14

CPA is a UKAS

company

5.5.1 Selection, verification and validation of examination processes

5.5.1.2 Verification of examination procedures

The laboratory shall obtain information from the manufacturer/method

developer for confirming the performance characteristics (expected

zone diameters, MICs) of the procedure.

What are you aiming to achieve? What are the intended outcomes of

the method being used?

Appropriate control strains (ATCC/NCTC) subjected to the method

should achieve the intended result (zone diameter, MIC) when we do

the test.

ISO 15189:2012

15

CPA is a UKAS

company

5.5.1 Selection, verification and validation of examination processes

5.5.1.2 Verification of examination procedures

Validated examination procedures used without modification shall be

subject to independent verification by the laboratory before being

introduced into routine use.

Generally acceptable for retrospective verification of historical methods

(inc. BSAC), e.g. demonstrable ability to meet performance claims

through quality assurance.

What evidence do you have to demonstrate new methods (inc.

EUCAST) have been verified as fit for purpose before being applied to

clinical isolates and informing clinical decisions.

ISO 15189:2012

16

CPA is a UKAS

company

5.5.1 Selection, verification and validation of examination processes

5.5.1.2 Verification of examination procedures

The independent verification by the laboratory shall confirm, through

obtaining objective evidence (in the form of performance

characteristics) that the performance claims for the examination

procedure have been met. The performance claims for the

examination procedure confirmed during the verification process shall

be those relevant to the intended use of the examination results.

How can you show the assessors that you can accurately determine

the susceptibility of your isolates when using the validated AST

method?

ISO 15189:2012

17

CPA is a UKAS

company

5.5.1 Selection, verification and validation of examination processes

5.5.1.2 Verification of examination procedures

The laboratory shall document the procedure used for the

verification and record the results obtained.

Show us your verification procedure. Is the technical approach to

verification sufficient and suitable for the intended use of the method?

Staff with the appropriate authority shall review the verification

results and record the review.

Who has reviewed the data and signed off the verification? Are they

competent to do so?

ISO 15189:2012

18

CPA is a UKAS

company

uncertainty (of measurement)

parameter, associated with the result of a measurement, that

characterizes the dispersion of the values that could reasonably be

attributed to the measurand (VIM definition)

VIM:

International Vocabulary of Basic and General Terms in Metrology

(ISO)

Martin:

How sure are you that the measurement is correct?

Measurement uncertainty

19

CPA is a UKAS

company

5.5.1.4 Measurement uncertainty of measured quantity values

The laboratory shall determine measurement uncertainty for each

measurement procedure in the examination phase used to report

measured quantity values on patients’ samples.

Where examinations include a measurement step but do not report a

measured quantity value, the laboratory should calculate the

uncertainty of the measurement step where it has utility in assessing

the reliability of the examination procedure or has influence on the

reported result

ISO 15189:2012

20

CPA is a UKAS

company

Metrological Traceability

Property of a measurement result whereby the result can be related to

a reference through a documented unbroken chain of calibrations, each

contributing to the measurement uncertainty (VIM)

Applicable to:

• Calibration of callipers, rulers or whatever else is measuring zone

diameters

• How sure are you of thee diameters?

• Can you demonstrate traceability?

Traceability

21

CPA is a UKAS

company

5.3.1.4 Equipment calibration and metrological traceability

The laboratory shall have a documented procedure for the calibration of

equipment that directly or indirectly affects examination results

Where this is not possible or relevant, other means of providing

confidence in the results shall be applied

- use of certified reference materials

ISO 15189:2012

22

CPA is a UKAS

company

• Entirely up to you

• Not applicable (e.g. MU)

– OK, so how have you come to that conclusion?

• Often see overarching/standalone procedures on MU, validation, verification & traceability/calibration

– OK, so how have you applied that to your AST method

• Need to consider how each applies to AST, do the work and document it

23

Documentation

CPA is a UKAS

company

• All of the above should be covered during the ISO 15189:2012 initial

assessment

• Highly likely that findings of non-conformity will be raised in relation to

AST

• Laboratory is expected to propose suitable improvement action at the

assessment to address and clear the findings

• Evidence of improvement action needs to be submitted (12 weeks) for

review by the Technical Assessor

• All findings must be cleared before grant of accreditation can be made

Initial assessment

24

CPA is a UKAS

company

Schedule of Accreditation

25

CPA is a UKAS

company

Schedule of Accreditation

26

CPA is a UKAS

company

Extension to Scope (ETS)

27

• Proposed changes to your assessed and accredited scope need to be

communicated to your UKAS Assessment Manager

• ETS process is required to add or change tests or methods on your

published Schedule of Accreditation

• New AC6 required detailing the additions / changes plus supporting

documentation, must include:

• SOPs

• Validation & verification data and summary

• Uncertainty of measurement estimation

• Measurement traceability chain

CPA is a UKAS

company

Extension to Scope (ETS)

28

• Many ETS applications run parallel to surveillance / reassessments

• Might be additional site time, often accommodated in existing effort

• If appropriate can be done by remote assessment, if so need:

• IQC performance data and summary

• EQA performance summary

• Training / competency records

• System suitability / evaluation summary

• Any other specific documentation

CPA is a UKAS

company

Extension to Scope (ETS)

29

CPA is a UKAS

company

Extension to Scope (ETS)

30

• UKAS has assessed your competence

to use BSAC methodology

• Your susceptibility testing using BSAC

methodology will be accredited

• UKAS has not assessed your competence

to use EUCAST methodology

• Your susceptibility testing using EUCAST

methodology will not be accredited until

ETS for this has been granted

CPA is a UKAS

company

Extension to Scope (ETS)

31

Inform UKAS early in the

change control process

• UKAS can arrange to assess your

competence to use EUCAST

methodology through ETS process

• Grant of ETS before clinical use

• Your susceptibility testing using

EUCAST methodology will be

accredited on the day you go live

CPA is a UKAS

company

Martin Stearn, UKAS Senior Assessment Manager

Thanks for listening

Any questions?

martin.stearn@ukas.com

© UKAS June 2016