The importance of Proficiency Testing (PT) & why take part

T&M 2011 Workshops 19 – 20 September 2011

Steve Sidney

Why measure ?

"When you can measure what you are speaking about and express it in numbers you know something about it; but when you cannot express it in numbers your knowledge is of a meagre and unsatisfactory kind: it may be the beginning of knowledge but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be."

British scientist Lord Kelvin (1824-1907)

Quality Measurements

QC / QA

Assuring quality of test & calibration results

• Lab shall have QC procedures

Use of CRM

Participation in PT

Replicate tests

Retesting or recalibration

Correlation of results for different characteristics

• What is Proficiency Testing (PT)

• Goal of PT

• Who provides PT

• Success of PT Programs

• Types of PT

• Educational role of PT

Proficiency Testing (1)

Proficiency Testing (2)

• Features of well designed PTS

• Evaluation of Data

• Why PT is important for labs

• The NLA & PT

Definitions

proficiency testing – evaluation of participant performance against pre-established criteria by means of interlaboratory comparisons

participant – laboratory, organization or individual that receives proficiency test items and submits results for review by the PT provider

proficiency testing provider – organization which takes responsibility for all tasks in the development and operation of a proficiency testing scheme

Goal of PT

• Mechanism to evaluate laboratory performance

• Sample evaluated by labs who participate

• Results submitted and statistically evaluated

• Reports generated and sent to participants

Who provides PT

• Independent 3rd Party

• Regulatory body

• Laboratories who conduct an ILC amongst themselves

Success of PT

• Reduction of non-acceptable results

• Evidence

– CAEL ( Canadian Association of Environmental Analytical Laboratories)

(1995 – 2005 )

– Wisconsin State Laboratory of Hygiene

(1996)

CAEL ( Canadian Association of Environmental Analytical Laboratories) 1995 – 2005

Wisconsin State Laboratory of Hygiene (1996)

Types of PT (1)

• Quantitative

– Numeric

– Statistical result

• Qualitative

– Identify presence (YES/NO)

– Statistics usually not appropriate

Types of PT (2)

• Split sample

– Small groups of labs

• Split level

• Partial process

– Not necessarily only a measurement

Educational Role of PT

• Disadvantage of Pass/Fail approach

• PT should deliver what industry requires

• Should be part of QA & Continuous Improvement

• Educational value has been a by-product

Features of well designed PT (1)

• Covers methodology , testing details & approach

• More than one sample • Series of rounds to cover all aspects of

lab’s capabilities • Excess samples can be used as Ref

Materials • Allow more than one analyst to

participate

Features of well designed PT (2)

• Reports to include sufficient data analysis

• Reports provided timeously

• Sufficient frequency

• Provider to have sufficient resources to provide assistance

• Provide guidelines on estimating measurement uncertainty

Features of well designed PT (3)

• PT Program should cover as much of process as possible

• Conduct follow-up workshops

Evaluation of Results

• Lab submits results

• Data is evaluated against

– All other data submitted

– To ease the burden a score is assigned

– Lab takes action dependent on score

How scoring is performed (1)

• Where the ref value and uncertainties are available make use of

𝐸𝑛 = 𝑥 −𝑋

𝑈 where 𝑈 = expanded uncertainty of

𝑥 − 𝑋

If x and X are uncorrelated then 𝑈 can be replaced

with 𝑘 𝑢2 + 𝑈2 (u= lab and U = ref)

How scoring is performed (2)

• The mean is established

• All results are normalised

• Z scores

SD

uxz

X=lab result, u=mean,

SD=standard deviation

Z-score evaluation

• Z – Scores categorised as follows

3z3

3z2

2z2

Pass

Investigate

Fail

Importance of PT (1)

• Wide difference in lab capabilities

• Skill & capacity challenges

• How to achieve & maintain quality of output

• Many initiatives

• Role of PT

Importance of PT (2)

• Will enable lab to monitor it’s own improvement

• Different objectives can be achieved

• Provides a measure of confidence for customers

• Should not be viewed in isolation of other measures

PT & NLA

• Brief History

• Metrology ILC

• PT

• Pesticide Formulations

• Food

• Water Microbiology

References

• ISO 17025 : 2005

• ISO 17043 : 2010

• ILAC Papers

– Educational Aspects of PT Programs (Flemming, Wruck – 2006)

– Discussion of PT Benefits & Progam Strategies (Middlebrook – 2006)

Questions

For further info regarding the NLA’s PT Programmes contact

Hanlie Badenhorst

hanlieb@nla.org.za Tel +(27) 012 349 1500