Accred Qual Assur (1998) 3 :224–226Q Springer-Verlag 1998 GENERAL PAPER

Hans Olav Folke Andersson The rational use of proficiency tests and

intercomparisons

Abstract Interlaboratory compari-sons (in the following abbreviatedas intercomparisons) regardingtests, analyses or measurements areamong the most worthwhile meas-ures a laboratory can take in orderto confirm that its services toclients include the provision of cor-rect results within a stated uncer-tainty. They give a picture of thewhole performance of the labora-tory, and they should be usedmuch more than at present. Unfor-tunately such intercomparisons are,and are made, expensive and pres-

tigious by the formalisms employedin many cases. The connection be-tween intercomparisons and profi-ciency tests and their use for differ-ent purposes is briefly discussed.Some suggestions are made onhow to improve the present stateof the art, i.e. how to increase theuse of intercomparisons, how toperform them efficiently and howto make optimal use of the results.

Key words Proficiency test 7Intercomparison 7 Accreditation 7Method validation

Received: 6 December 1997Accepted: 30 January 1998

Presented at: EUROLAB Workshop onConfidence in Testing – CustomersNeeds, Copenhagen, 11 September 1997

H.O.F. Andersson (Y)SP - Swedish National Testing andResearch Institute,P O Box 857, S-501 15 Boras, SwedenTel.: c46-33-165221; Fax: c46-33-165010e-mail: hans.andersson6sp.se

Intercomparisons and proficiency tests

A procedure whereby a number of laboratories per-form an experiment (test, analysis or measurement) onnominally equivalent materials or objects and comparetheir results is called an intercomparison in this article.The purpose may be to investigate the properties of themethod or methods employed (characterisation as apart of the validation) or to compare the performanceof the laboratories. In the former case it is of impor-tance that the laboratories have an equivalent state ofcompetence, so that it is in fact the properties of themethod and/or the samples that are evaluated.

For clients of laboratories it is of course of the ut-most importance that the methods give correct and fairresults, e.g. for legal approval or certification proce-dures. This is not the case as often as may be expected,and an increased use of intercomparisons for character-isation and validation of methods is recommended fornew but also for existing methods in internationalstandards.

When laboratory performance is studied, and if theexercise is performed by a third party knowing the rightanswer, the intercomparison is called a proficiency test.It is then of importance to know the homogeneity ofthe test samples and the properties of the method verywelI. A proficiency test may be performed in variousfashions with regard to how samples are distributedand evaluated. The main purpose is to demonstratethat the working routines and the competence of thelaboratory are in order.

Unfortunately the concepts are often mixed togeth-er, and an intercomparison or proficiency is involun-tarily measuring a combination of laboratory proficien-cy and method validity. An intercomparison can still bevery valuable for clearing up technical difficulties if thismixture of concepts is recognised and acknowledged.Of course, one should try to separate the effects asmuch as possible by the use of statistical methods.

225

The framework for intercomparisons and proficiency

tests

The statistical background for intercomparisons is wellestablished and the main features are summarised in in-ternational standards such as ISO 5725. A general ex-perience is, however, that even experienced test engi-neers and analysts are unfamiliar with the fundamen-tals, and that planning, performance and evaluation ofintercomparisons are often carried out in a less satisfac-tory way.

It is therefore recommended that courses should berun in laboratories to provide education on the statisti-cal background to the planning and technically correctperformance of intercomparisons.

The procedures for intercomparisons and proficien-cy tests are described in the ISO Guide 43 “Proficiencytesting by laboratory intercomparisons”. Although thisdocument is intended for proficiency testing, the firstpart “Development and operation” can be very usefulas a basis for intercomparisons in general, if used withflexibility. A multitude of documents are being pro-duced and issued by regional bodies, and one shouldtherefore try to stick to the central and internationalones.

The need for flexibility is caused by the need, andusefulness of simplifying the procedures in some cases.One example is when the test specimens are so expen-sive that only a few can be used to get an indication onthe properties of the test; another is when the numberof laboratories is so small that the intercomparison forpractical reasons is performed as a cooperation ratherthan with a formal coordinator (method evaluation). Inboth cases, valuable information can be gained, makingit possible to use methods to the benefit of clients andsociety, although the strict adherence to the Guide maynot be possible or economically defensible. However,the Guide can be used as a common basis, and devia-tions can be declared and assessed as they occur.

In order to enhance the use of intercomparisons,both for method evaluation and for proficiency, it isrecommended that the number of documents should beminimised, and that they are used with flexibility, stat-ing the deviations in each case.

Intercomparisons and proficiency tests in support of

accreditation and method validation

From the client’s point of view, accreditation meansthat results from accredited laboratories may be usedinternationally and in dealings with authorities for ap-proval of materials and products. Accreditation shouldalso provide evidence of technical competence and re-liability in general. In this last respect there is also oth-

er evidence, such as the R&D activities of the laborato-ry, its records, international contacts, participation instandardisation, and goodwill.

The accreditation body should make full use of allintercomparisons in which the laboratory has takenpart, not only those performed as proficiency tests. Alltechnical evidence is of value, providing an accumula-tion of examples of demonstrated competence, even ifnot all the formal requirements of a proficiency test arefulfilled. If, for example, there has been successful par-ticipation in one or two types of intercomparisons re-garding mass-spectroscopic analysis, there is strong evi-dence that the laboratory is competent in related exam-ples of such analyses. Further, if a laboratory has takenpart in an R&D project of the Fourth Framework Pro-gramme of the Commission with regard to noise meas-urements, it is probably competent in this area.

Care should be taken not to accept as an axiom thataccredited laboratories are “better” than non-accre-dited ones. Extensive proficiency tests made in the areaof chemical analysis show that there is no correlation atall in performance between accredited and non-accre-dited labratories. There are also indications that labo-ratories having taken part in many proficiency testsachieve better results than those which are less experi-enced in this respect. All in all this indicates that whatreally happens in these cases is that the investigation isin fact a series of intercomparisons to validate themethod, or rather to have everybody make the sameinterpretation of it.

From another point of view one could state that pro-ficiencies should not really be necessary in accreditedlaboratories! If one looks at the requirements of e.g.the draft revised ISO/IEC Guide 25 and presumes thatall of them are checked in the accreditation assessment,the result from the laboratory should of necessity becorrect within stated uncertainties. This is not so, andthere are at least three reasons. First, the methods arenot validated to a desirable extent, second, the accredi-tation procedure is a sampling process not covering alltypes of behaviour (although technical experts are usedto assess the technical procedures), and third, estimatesof uncertainty components are consistently underesti-mated.

Therefore it is recommended that methods arelooked upon critically with regard to validity if they areused in proficiency tests, that the accreditation proce-dure is simplified even more with regard to technicalassessment if proficiency tests are required (the labora-tory should not have to pay twice), and there should becourses in various technical areas to make realistic un-certainty estimates (it should not be considered presti-gious to have a very small uncertainty which is not real-istic).

With regard to the last point, it is acknowledged thatthere may be commercial interests in stating low or

226

high uncertainties, but these must be overruled by engi-neering common sense. It may also be said that in manyproficiency tests it is the coordinator who should be re-sponsible for the assessment of uncertainties, asso-ciated with sampling, method etc.

How to increase the use of intercomparisons and make

better use of the results

To conclude, some of the main items to be recom-mended to enhance the use and usefulness of intercom-parisons are the following.– At the start of an intercomparison it should be estab-

lished whether it is a proficiency test or a methodevaluation, or a mixture of both. During the planningand evaluation phases there should be efforts to sep-arate the effects of the laboratory from those of themethod if there is any doubt.

– As far as possible, the basic documents, i.e. the inter-national standards and guides, should be used, andregional documents should be avoided, in order tohave a common basis for training, understanding andnomenclature.

– Proficiency tests should be used as instruments forlearning and correction rather than judgement. Inthis way, acceptance and common use can be en-hanced in areas where this is possible. It is also easierto have the proficiency tests (or studies?) performedas part of the normal activity of the laboratory, andnot as special events.

– In the accreditation process, the technical assessment(the ordinary intercomparisons made by the labora-tory) and special proficiency tests (studies) should bebalanced and used in an optimal way. If for instancethe technical assessment is thorough and the meth-ods well established, the need for proficiency tests issmaller. If the method or methodology is new or cap-able of being interpreted in different fashions, itmight be worthwhile to perform an intercomparisonor a proficiency test.

– In chemical analysis the metrological principles oftraceability and uncertainty should be introduced asfar as possible, to be compatible with the principlesof proficiency testing and intercomparisons.

– Courses in statistics and in the assessment of uncer-tainty should be increased. Evidently many uncer-tainty budgets are giving much too small values oftotal uncertainty.

– Co-operation,as exemplified by the EEE groups(EA, EURACHEM and EUROLAB) on proficiencytesting and reference materials, should be increasedto promote the exchange of knowledge and harmon-isation between laboratories and accreditation bod-ies. Likewise, the standardisation and the R&D pro-grams of the Commission should be used to promoteco-operation with regard to validation of methods insupport of directives, presently through the so-calledSTAR group.