Operational de¢nitions of uncertainty

TRAC 2688 15-8-01

trendsOperational de¢nitions of uncertaintyEdelgard Hund, D. Luc Massart, Johanna Smeyers-Verbeke*ChemoAC, Farmaceutisch Instituut, Vrije Universiteit Brussel, Laarbeeklaan 103, B-1090 Brussels, Belgium

Very different approaches for the estimation of theuncertainty related to measurement results arefound in the literature and in published guidelines.This article analyses and compares them. It is clearthat ùncertainty' is not, and should not be, thesame in all situations. As a consequence, opera-tional de¢nitions of uncertainty are proposedthat take into account the differences in the waysin which truth, uncertainty and error are con-ceived. z2001 Elsevier Science B.V. All rightsreserved.

Keywords: Measurement uncertainty; Measurement error;Traceability; Bias

1. Uncertainties about uncertainty

Analytical chemists accept that a measurementcannot be interpreted properly without knowledgeof its uncertainty. EURACHEM [ 1 ] de¢nes uncer-tainty as `̀ a parameter associated with the result ofa measurement, that characterizes the dispersion ofthe values that could reasonably be attributed to themeasurand''. This de¢nition follows the originalde¢nition of the Guide to the Expression of Uncer-tainty in Measurement (also known as the GUM)[ 2,3 ].

The uncertainty of the result of a set of measure-ments, e.g., a concentration, x, can be expressed inthe form of a standard deviation, the so-calledstandard uncertainty, abbreviated as u(x). Theexpanded uncertainty U(x) de¢nes an intervalaround the result of a measurement, x þ U(x),with U(x) = ku(x). The constant k is called the cov-erage factor, and for k = 2, the expanded uncer-tainty is roughly equivalent to half the length of a95% con¢dence interval. Thus, the probability that

the mean value is included in the expanded uncer-tainty is about 95%.

The problem for analytical chemists is how todetermine uncertainty in their speci¢c situationsand, unfortunately, there is much confusion anduncertainty (uncertainties about uncertainty!)about how to proceed. Very different approachesare found in the literature and in published guide-lines. There are different reasons for this. The ¢rstlies in the way errors are treated. It is clear thatuncertainty is related to measurement errors, butliterature reports and guidelines differ in the sour-ces of error that they take into account or to whichthey attribute uncertainty. In particular, the treat-ment of systematic error is a point of discordance.Errors refer to a divergence from the truth, and asecond reason for the confusion in the literature liesin what is considered to be the truth: is one lookingfor the absolute truth or for a relative truth? In theformer case, the reference is the true value, W, whilein the latter case the reference is a consensus value,e.g., one agreed upon for the purpose of compara-tive measurements. A third and very important rea-son is related to the way error is determined inpractice. Analytical chemists are used to determin-ing error by what they call `method validation', andwould like to use this for determining uncertainty.Metrologists have a different approach, which theyapply to physical methods and would like to seeapplied also in analytical chemistry.

In this article, we try to analyse and compare thedifferent approaches. We conclude that uncertaintyis not, and should not be, the same to all practi-tioners and that de¢nitions are therefore requiredthat take into account the differences in the waystruth, uncertainty and error are conceived.

We only consider here the uncertainty of themeasurement process. In several situations, theuncertainty related to, for example, sampling,homogeneity or stability of the samples also playsan important role, and should be taken into consid-eration.

0165-9936/01/$ ^ see front matter ß 2001 Elsevier Science B.V. All rights reserved.PII: S 0 1 6 5 - 9 9 3 6 ( 0 1 ) 0 0 0 8 9 - 9

*Corresponding author. Tel.: +32 (2) 477 4737;Fax: +32 (2) 477 4735.E-mail: [email protected]

394 trends in analytical chemistry, vol. 20, no. 8, 2001

TRAC 2688 15-8-01

2. Measurement errors

When an analytical chemist measures the con-centration of a substance, the result should be con-sidered an estimate of the true value. The error isthen the difference between a stated result and thetrue value. There is a single error value for eachresult and part of it, the systematic error, can becorrected for. In contrast, the uncertainty derivedfrom the errors is a range and no part of the uncer-tainty can be corrected for [ 4 ].

The measurement result obtained deviates fromthe true value because of the existence of a numberof systematic and random errors. The following canbe distinguished [ 5,6 ]:

õ The method bias, a systematic error owing to themethod used.

õ The laboratory bias, which according to one'spoint of view is a systematic error (for anindividual laboratory) or a random error (whenthe laboratory is viewed as a part of a populationof laboratories, as is the case in an inter-laboratory study). In the latter case it is acomponent of the reproducibility of the methodused.

õ The run error, which is a random error owing to,among other factors, time effects, and is includedin the intermediate precision estimate.

õ The repeatability error, which is a random erroroccurring between replicate determinations per-formed within one run.

This list is sometimes called the `ladder of errors'[ 6 ], because there is a hierarchy involved: themethod as such; the method as it is applied by acertain laboratory; the method as it is applied by acertain laboratory on a certain day; and, ¢nally, theerror within that day for an individual determina-tion.

A measurement result can therefore be de-composed as follows: Measurement result = truevalue + method bias + laboratory bias + run error +repeatability error.

Each of these steps on the ladder adds its ownuncertainty. This is usually well understood for therandom errors but less for the method bias. Oneguideline [ 7 ], for example, states that if systematicerror occurs this should be corrected for, and fromthen on assumes that this has been done and theerror does not need to be taken into account inestablishing the uncertainty. However, the estima-

tion of the systematic error is itself the result ofmeasurements and is therefore subject to a degreeof uncertainty. This uncertainty should then betaken into account in an uncertainty statement.When it is concluded that there is no bias, this infact means in most practical cases that the bias issmaller than a certain limit, below which it may existbut cannot be detected. This, too, is a source ofuncertainty and should be included in an exhaus-tive uncertainty statement.

For the uncertainties we can write:Uncertainty = uncertainty associated with themethod bias + uncertainty associated with the labo-ratory bias + uncertainty owing to the run effect +repeatability uncertainty.

Translated into variances this becomes for asingle measurement:

u x ��u2

meth:bias � c2lab � c2

run � c2r

q�1�

Making use of the standard error of the mean[ 5,11 ], the standard uncertainty for the mean ofqUpUn measurements performed as n replicates( repeatability conditions) in each of p runs in eachof q laboratories can be then calculated as:

u x ��u2

meth:bias � c2lab=q � c2

run=qp � c2r =qpn

q�2�

2.1. Traceability and method bias

One of the most dif¢cult problems in chemicalmeasurement is to determine method bias. It is use-ful to consider two types of method bias, namelyabsolute or constant method bias, and relative orproportional method bias. There is a constantbias if a method leads to measurement resultsthat deviate by a constant value from the corre-sponding true value. The observation of 110, 210and 310 instead of 100, 200, 300 would be anexample of a constant method bias. Often, prob-lems with an inappropriate blank correctionresult in a constant bias. If the difference betweenthe measurement result obtained with a certainmethod and the corresponding true value is pro-portional to the concentration of the analyte, a pro-portional method bias occurs. With a proportionalbias, 110, 220 and 330 might, for example, beobserved in the example mentioned above. A pro-portional bias is often caused by a matrix interfer-ence.

trends in analytical chemistry, vol. 20, no. 8, 2001 395

TRAC 2688 15-8-01

Method bias refers explicitly or implicitly to areference or a standard that is considered to repre-sent the truth. In the metrological vocabulary it issaid that the result obtained can then be traced to astated reference. `Traceability' is de¢ned as `̀ theproperty of the result of a measurement or thevalue of a standard, whereby it can be related tostated references, usually national or internationalstandards, through an unbroken chain of compar-isons all having stated uncertainties'' [ 2 ]. The sec-ond edition of the EURACHEM guideline [ 1 ] men-tions four types of reference, which are importantfor chemical measurements. One of them requiresthe use of so-called primary methods. The CCQM[ 8 ] has identi¢ed isotope dilution with mass spec-trometry ( IDMS), coulometry, gravimetry andtitrimetry as methods that have the potential to beprimary methods. Such methods are in principletraceable to SI units, but it is not evident that thisis also the case when these methods are appliedunder the conditions of routine laboratories. Formost other methods the main possibilities are:

õ using the method to make measurements on anappropriate certified reference material (CRM).The method is then traceable to that CRM;

õ making measurements using defined proce-dures. In this case the method is traceable tothe reference method;

õ using the analytical procedure to make measure-ments on a known quantity of pure analyte. Inpractice, this usually means that recovery studiesor standard addition methods are carried out.

Thus, there are always at least two components inthe uncertainty associated with the method bias.The ¢rst is the uncertainty of the reference (whichwe will call here u�traceability� ), the second is theuncertainty associated with the estimated bias:

umeth:bias ��u2

traceability � u2est:bias

q�3�

The ultimate reference in the chemical analyticalworld is the value of the mole. A practical examplewould be the use of a pure standard substance asthe reference. The uncertainty in that reference isdue to the uncertainty of the purity of that standardsubstance. Very often, the use of such a reference isnot practical and one uses as reference the concen-tration of a certain substance as determined by agiven method in a given material. There is then anuncertainty owing to that determination.

3. Error budgets for determining theuncertainty

In the error budget method which, in analyticalchemistry, seems to be used more in Europe thanelsewhere, the variance is estimated of each varia-ble that contributes to the total variance. This is alsoreferred to as the `bottom-up' approach. A simpleexample for an acid^base titration is given in thenew version of the EURACHEM guideline [ 1 ]. Theexample treats the standardisation of a NaOH sol-ution against the titrimetric standard potassiumhydrogen phthalate (KHP). The concentration ofNaOH is obtained from the following formula,with mKHP the mass of KHP, PKHP the purity ofKHP, FKHP the formula weight of KHP and VT thevolume of NaOH used for the titration of KHP:

cNaOH � 1000WmKHPWPKHP

FKHPWVT�4�

Eq. 4 describes a multiplicative relationship.While the variance of a sum or a difference is thesum of the variances of its components, the relativevariance of a product or division is the sum of therelative variances of its components. Eq. 4 is of thetype:

x � kaWbcWd

�5�

where k is a numerical constant. The relative var-iance

c�x�x

� �2

is given by:

c�x�x

� �2

� c�a�a

� �2

� c�b�b

� �2

� c�c�c

� �2

�"

c�d�d

�2 #

�6�

or

�c�x��2 � kaWbcWd

� �2

Wc�a�

a

� �2

� c�b�b

� �2

�"

c�c�c

�2

� c�d�d

� �2 #

�7�


TRAC 2688 15-8-01

or

�c�x��2 � kb

cWdc�a�

� �2

� ka

cWdc�b�

� �2�

kaWbc2Wd

c�c�� 2

� kaWbcWd2

c�d�� 2

�8�

Notice that the coef¢cient for each variancec(yi )is the partial derivative of x with respect to yi,

DxDyi;

so that:

�c�cNaOH��2 � �1000�2W PKHP

FKHPWVTc�mKHP�

� �2

�"

mKHP

FKHPWVTc�b�

� �2

� mKHPWPKHP

F 2KHPWVT

c�FKHP�� 2

�

mKHPWPKHP

FKHPWV 2T

c�VT��2

#�9�

It is more fashionable to write that the errorbudget approach models the uncertainty of a meas-urement result as a ¢rst-order Taylor series:

u�x� ��Xn

i�1

DxDyi

Wu�yi�� 2

� 2Xn31

i�1

Xn

j�i�1

DxDyi

DxDyj

u�yi; yj�vuut

�10�with yi the value considered, and u(yi ) the standarduncertainty related to this value. The second sum-mand under the square root sign refers to the uncer-tainties related to the covariances. The ¢rst-ordermodel is based on the assumption that the uncer-tainty of the uncertainty itself is negligible. In theexample used by EURACHEM, the covariances arenot taken into account. This yields the followingerror budget:

This equation is equivalent to Eq. 9. In the exam-ple, the uncertainty of the purity of the standard isobtained from the speci¢cations of the provider;the other uncertainty sources were evaluated sep-arately. For the uncertainty of the weight, both thefull and the emptied container have to be consid-ered. The uncertainty of each of the weights isderived from a balance repeatability and a contri-bution from the uncertainty of the balance's linear-ity. The other two sources of uncertainty require acomprehensive evaluation as well. The corre-sponding values are given in Table 1. The concen-tration of the NaOH solution is determined as0.1021 mol l31. From Eq. 9 or Eq. 11 and using thedata of Table 1, the corresponding standard uncer-tainty is computed as 0.0001 mol l31.

The error budget method as described in theGUM was developed by metrologists and physi-cists. It has indeed proved its value for physicalmeasurements and is probably suitable for primaryanalytical methods. However, the example of Eq.10 shows that, even for a primary method ( titrim-etry ), the error budget approach becomes quitecomplex. To derive a complete uncertainty esti-mate requires that all relevant parameters be con-sidered. For another primary method (IDMS), Dob-ney et al. [ 9 ] state that this leads to unwieldyexpressions so that strict application of the uncer-tainty propagation law seems a daunting project.For methods with more uncertainty components,it is hardly feasible to construct an error budget.The error budget approach assumes that no impor-tant error is overlooked, and in many practical ana-lytical situations this is not evident because, in prac-tice, it is impossible to predict, for example, theeffect of unanticipated matrix interferences. To esti-mate such sources of error requires systematicmethod validation.

For more complex analytical methods, it seemspreferable to use the information obtained frommethod validation. Method validation leads to esti-mates of random and of systematic errors thatencompass many different sources of uncertainty.From a practical point of view, it requires much lesswork, because the analytical chemist has to validatemethods anyway. From method validation to

u�cNaOH� � 1000W

��PKHP

FKHPWVTWu�mKHP�

� �2

� mKHP

FKHPWVTWu�PKPH�

� �2

� mKHPWPKHP

�3FKHP�2WVT

Wu�FKHP� !2

� mKHPWPKHP

3FKHPW�VT�2Wu�VT�

!2" #vuut�11�


TRAC 2688 15-8-01

uncertainty estimation is not an enormous step, andshould overcome the mixture of scepticism andawe with which the introduction of uncertaintymeasurement and its metrological terminologyhas been greeted. Wood et al. [ 10 ] give a compar-ison of these two approaches, which was initiatedby the UK Ministry of Agriculture, Fisheries andFood. For a variety of examples, such as the deter-mination of total nitrogen in meat products usingthe Kjeldahl method, or the GFAA analysis of lead inwine, it was shown that both approaches lead tocomparable results.

The discussion above does not mean that theerror budget approach has no advantages at all.The fact that one has to consider all sources ofuncertainty allows the largest contributions to beidenti¢ed. These can then be considered particu-larly if the uncertainty has to be reduced. Methodvalidation, on the other hand, focuses on the errorsof the analytical procedure as such, but has a ten-dency to underestimate the importance of othersources of errors such as those related to samplingor some pre-treatment steps.

4. Should all sources of uncertaintyalways be included?

In many practical cases the analyst is not trying todetermine the absolute truth. This is the case, forexample, when standard methods are being used.The method has then been validated by a largegroup of laboratories known to be pro¢cient inthat method. In such a situation, the analyst canassume that the method bias is reduced to anacceptably small value. If a laboratory considersitself a member of the population of pro¢cient lab-oratories, it can also regard the uncertainty relatedto the bias as negligible. From a metrological pointof view, this means that one regards the problem in

a relative way. Even though such an approachmight seem a little too optimistic, it is acceptablein practice. The uncertainty to be considered isthen the uncertainty associated with measurementsperformed with the standard method in the ana-lyst's laboratory. This can be evaluated at anacceptable expense.

5. Operational de¢nitions of uncertainty

Many years ago, analytical chemists describedrandom errors using the general term `precision'.They have now learned that there are differentkinds of precision, such as reproducibility, repeat-ability, and intermediate forms, which should beused in certain well-de¢ned situations. While theterms used indicate immediately what sources ofvariance are included in a precision statement,this is unfortunately not the case for uncertaintystatements. The term ùncertainty' can encompassdifferent sources of uncertainty according to thesituation considered. It is our opinion that differentnames should be given, or different quali¢cationsadded, to the term ùncertainty' depending on theconditions under which an analyst is operating. Inwhat follows, we will consider the following opera-tional de¢nitions of uncertainty (see also Table 2).

Within-laboratory uncertainty only considersthe intermediate precision and therefore accountsfor the repeatability and the run effect.

Reproducibility uncertainty additionally con-siders the reproducibility variance and thereforeaccounts for the within-laboratory uncertainty andthe laboratory bias.

Bias-included uncertainty as well as absoluteuncertainty additionally consider the uncertaintyowing to the method bias. They also take intoaccount the uncertainty owing to the estimatedmethod bias and to the traceability. The bias-

Table 1Values and uncertainties in the standardization of an NaOH solution

Description Value xi Standard uncertaintyu(yi )

Relative standard uncertaintyu(yi )/yi

mKHP Weight of KHP 0.3888 g 0.00013 g 0.00033PKHP Purity of KHP 1.0 0.00029 0.00029VT Volume of NaOH for KHP titration 18.64 ml 0.013 ml 0.00070FKHP Formula weight of KHP 204.2212 g mol31 0.0038 g mol31 0.000019cNaOH Concentration of NaOH solution 0.10214 mol l31 0.0001 mol l31 0.00097


TRAC 2688 15-8-01

included uncertainty and the absolute uncertaintydiffer in the level of traceability of the measurementresults.

We do not propose that these terms, as such,should be generally used, but we do propose thatan international body of analytical chemists shouldconsider the situation and de¢ne generally recog-nised operational de¢nitions of uncertainty. Weshall now consider the references to which the

measurement is traced in each of those cases,what sources of error are considered, and howuncertainty could be determined.

5.1. Within-laboratory uncertainty

A guideline prepared by the NMKL [ 7 ] states thatit wants to de-dramatise the subject of measure-ment uncertainty (which describes very well how

Table 2Some operational de¢nitions of uncertainty

Type of uncertainty Precisionestimatesconsidered

Uncertainty contributions accounted for Uncertainty for the mean ofn measurements performed underrepeatability conditions ux

Within-laboratoryuncertainty

Intermediateprecision

Repeatability, run effect (e.g., analyst and time)��s2analyst � s2

time � s2r =n

qReproducibilityuncertainty

Reproducibility Repeatability, run effect (e.g., analyst and time),lab effect

��s2BL � s2

r =np

Bias-includeduncertainty

Intermediateprecision

Repeatability, run effect (e.g., analyst and time),bias ( lab and method)

��u2

est:bias � s2run � s2

r =np

a

e.g.,��s2d � s2

run � s2r =n

pb

��s2obs=m1 � s2

run � s2r =n

pc

��s2obs=m1 � s2

native=m2 � s2run � s2

r =np

d��est:bias=k�2 � u2

est:bias � s2run � s2

r =nq

e

Absolute uncertainty Intermediateprecision

Repeatability, run effect (e.g., analyst and time),bias ( lab and method), traceability to CRM

��u2

CRM � s2x

p � s2run � s2

r =n

aGeneral expression if the bias is not signi¢cant or a signi¢cant bias is corrected for.bBias estimated from a comparison with the reference method; the uncertainty in the reference method is considered negligible. If a t-test isused in the evaluation of the bias, the pooled variance of both methods is used in s2

d.cBias estimated from recovery experiments with a blank.dBias estimated from recovery experiments; the uncertainty of the spiked concentration is considered negligible. If a t-test is used in theevaluation of the bias, s2

obs and s2native are pooled.

eGeneral expression if a signi¢cant bias is not corrected for.Notations used:s2r , repeatability variance.

n, number of experiments performed under repeatability conditions.s2time, variance component between days.

s2analyst, variance component between analysts.

s2BL, variance component between laboratories.

s2run, variance component between runs (e.g., different days and /or analysts).

s2d, variance of the difference of the mean results of the two methods.

s2est:bias, uncertainty related to an estimated bias.

s2obs, variance observed for the analyses of a spiked sample.

m1, number of determinations performed on a spiked sample.s2native, variance observed for the analyses of a native sample.

m2, number of determinations performed on the native sample.s2x , variance of the mean concentration observed for the CRM (e.g., repeatability or intermediate precision conditions).

u2CRM, uncertainty related to a certi¢ed reference material, speci¢ed on the certi¢cate.

k, coverage factor.


TRAC 2688 15-8-01

many analytical chemists react when confrontedwith the subject ). It tries to do so by using a simpleprocedure based on the determination of a con¢-dence interval with a precision measure. The pre-cision statement is preferably what is called in theNMKL guideline the within-laboratory reproduci-bility. In ISO terms, this is the intermediate preci-sion. If this precision estimate is not available, thenit is suggested that repeatability can be used as theprecision statement, but the guideline warns that, inthat way, the uncertainty is underestimated. Thereference in this guideline is the mean of thelaboratory for that determination. Only the lowerrungs of the ladder of errors are included, namelythose owing to the repeatability and the run errors.Laboratory and method bias are not considered,and there is no traceability to SI units or at least tostated references. In our opinion, this is perfectlyacceptable as long as the limitations in the level ofreference and sources of uncertainty are recog-nised.

A minor remark which could be made about thisguideline is that it would have been preferable toseparate the intermediate precision into its compo-nents, i.e., the repeatability and the between-run(such as between time+between analysts ) compo-nents, by using a designed experiment in the sameway as for the repeatability and between-laboratorycomponents in Section 5.2. If this is done, it is pos-sible to compute the uncertainty when the analystafterwards performs n replicate measurementsunder repeatability conditions, and uses the meanof the replicates as the stated result. The within-laboratory uncertainty on the mean becomes:

u x ��s2run � s2

r =nq

��s2analyst � s2

time � s2r =n

q�12�

For the mean of n replicate measurements per-formed by a single analyst on each of p days thewithin-laboratory uncertainty would then be:

u x ��s2analyst � s2

time=p � s2r =pn

q�13�

The NMKL guideline states that if systematic erroroccurs, this should be corrected for, and from thenon, it assumes that this has been done. The biasestimate is not considered to constitute a sourceof uncertainty. However, as explained earlier,even if the bias is determined to be zero (since itis lower than a certain limit ) an uncertainty isrelated to this factor. Therefore, the NMKL guide-line is somewhat misleading because a less expertuser might consider that the reference level ishigher than it is for this guideline. When no uncer-tainty related to the bias is included in the un-certainty estimate, this should be clear, e.g., fromthe term used in identifying the uncertainty esti-mate.

5.2. Reproducibility uncertainty

In reference [ 11 ] the authors describe an inter-comparison experiment for polyunsaturated fattyacids in oil. Each participating laboratory carriedout duplicate analyses on the sample. The resultsof these determinations after removal of outliers aregiven in Table 3.

The analysis of variance yields two meansquares. By dividing them by the appropriate num-ber of degrees of freedom one obtains s2

r and(s2

r � 2s2BL) as estimates of c2

r and (c2r � 2c2

BL),from which cr and cBL can be computed. Theyare respectively the repeatability variance and thebetween-laboratory variance. The reproducibility

Table 4Results of the analysis of variance (ANOVA) for the inter-labo-ratory study

Variance estimate Variance Corresponding standarddeviation

Within laboratory c2WL 1.9 1.4

Between laboratory c2BL 2.1 1.4

Reproducibility c2R 4.0 2.0

Table 3Results of the inter-laboratory study

Laboratory xi1 xi2 xi

1 7.9 12.0 9.952 9.1 10.4 9.753 9.7 10.2 9.954 12.0 9.9 10.955 10.6 12.1 11.356 10.4 8.0 9.27 9.4 7.7 8.558 10.1 10.5 10.309 12.0 13.0 12.5010 8.1 7.3 7.7011 10.3 11.6 10.9512 10.9 12.5 11.7013 15.0 14.0 14.5014 10.7 7.1 8.90


TRAC 2688 15-8-01

variance can then be obtained as the sum of thetwo variance estimates: s2

R � s2r � s2

BL. Notice thats2BL � s2

run � s2lab. The experimental set-up does

not, however, allow one to separately estimates2run and s2

lab. The variance estimates obtained forthe example are summarised in Table 4.

The uncertainty of an individual analysis is then:

u x ��s2r � s2

BL

q�14�

If n replicate determinations under repeatabilityconditions have been carried out, then this uncer-tainty becomes:

u x ��s2r =n � s2

BL

q�15�

The within- and between-laboratory compo-nents are included in the uncertainty estimate. Itshould be noted that the repeatability is the onecomputed from the inter-laboratory experimentand not the repeatability obtained in an individuallaboratory, which means that individual laborato-ries should use precision estimates of an inter-lab-oratory study for computing uncertainties onlywhen they are suf¢ciently pro¢cient in the particu-lar procedure. This implies, of course, that theyhave access to the results of the inter-laboratorystudy. The sources of uncertainty included in theuncertainty statement are the repeatability var-iance, the run effect, and the laboratory bias, butnot the method bias. If the result is de¢ned by astandard procedure applied, such as would be thecase, for example, for the determination of crude¢bre, then the highest level of reference is reached,since in this empirical method there is, by de¢ni-tion, no method bias. If the standard proceduredetermines the concentration of a well-de¢ned sub-stance, as was the case for the fatty acids, then it ispossible in principle to try tracing the result back toa higher reference level. It should therefore beunderstood that the reference in the method ofdetermining uncertainty described in the exampleis the mean value of the result of a large group ofquali¢ed laboratories, using the standard proce-dure concerned, for a given matrix and level of con-centration.

A frequently asked question is how to perform aninter-laboratory precision study when there arefewer than eight laboratories, the number requiredby ISO for an inter-laboratory precision experi-ment. It has been suggested [ 6 ] that in that situationone should estimate the reproducibility and the

repeatability from the Horwitz function [ 12 ]. Thisfunction was deduced from a consideration of morethan 7500 method performance studies. It relatesthe relative reproducibility standard deviation tothe concentration. The corresponding repeatabilitystandard deviation is considered to be one-half totwo-thirds of the reproducibility, and s2

BL is thenabout 0.5^0.75 of s2

R. Another possibility, when aninter-laboratory experiment cannot be set up, con-sists in the consideration of data obtained fromrobustness tests. Since robustness tests simulatethe changes that can be expected when transferringan analytical method between laboratories (orinstruments, or operators ), the variances observedfor adequately chosen modi¢cations of the methodparameters should give an indication of the repro-ducibility variance.

5.3. Bias-included uncertainty

In the previous sections, only random errors( intermediate precision and reproducibility ) havebeen considered in the uncertainty statements. Sys-tematic error or bias additionally adds uncertaintyto a measurement result.

We will consider here the uncertainty estimationusing information from the in-house validation ofthe analytical method. This implies ( i ) that usuallyno estimate of the reproducibility, but only an esti-mate of the intermediate precision is available and( ii ) that the bias estimated is the overall bias, whichis a combination of the laboratory bias and themethod bias. As mentioned earlier inter-laboratorystudies are required to separate the method biasfrom the laboratory bias.

Eq. 2 can then be written as follows:

ux ��u2

bias � s2run � s2

r =nq

�16�

in which ubias combines the uncertainty in themethod bias with the uncertainty in the laboratorybias.

5.3.1. Method comparisonIf the bias for a new routine method is evaluated

by a comparison with a reference method, theuncertainty associated with the bias (see Eq. 3) is:

ubias ��u2

ref:meth � u2est:bias

q�17�

Indeed the traceability uncertainty, u2traceability,

here is the uncertainty in the reference method. If


TRAC 2688 15-8-01

the latter is a reference method as de¢ned by ISO[ 13 ] it has been evaluated in an inter-laboratorystudy and therefore the reproducibility variance isknown. Most often, the reference method will beconsidered to be not biased and the uncertainty inthe reference method to be negligible. Traceabilityto the reference method is achieved by comparingthe results obtained with the routine and referenceprocedure.

If the bias of the routine method as compared tothe reference method is not signi¢cant, which infact means that it is smaller than a certain limit,this does not mean that there is no uncertainty asso-ciated with the estimated bias. The uncertainty inthe estimated bias, uest:bias, corresponds to theuncertainty associated with the measured differ-ence between the mean results obtained by usingboth methods. As speci¢ed by EURACHEM [ 1 ] andIUPAC [ 14 ] this corresponds to the standard devia-tion term, sd � s�xA3xB�, that appears in the t-testapplied to test whether the difference is statisticallysigni¢cant:

t � MxA3xBM

sd� MxA3xBM

sp

��1

nA� 1

nB

r �18�

with

sp ��nA31�s2

A � �nB31�s2B

nA � nB32

s

the pooled standard deviation of both methods, nA

and nB, being the number of measurements per-formed with the reference and routine method,respectively.

x́A and x́B are the mean results obtained in thelaboratory for the reference and routine method,respectively, and s2

A and s2B are the variances

observed in the laboratory for the reference andthe routine method, respectively. Most often, meas-urements are performed under repeatability orintermediate precision conditions [ 15 ].

This of course implies that s2A and s2

B are estimatesof the samec so that they can be pooled. If this is notthe case,

sd ��s2A

nA� s2

B

nB

s

and an appropriate test such as Cochran's test has tobe used [ 5 ].

If the test reveals that the bias of the methodunder study is not signi¢cant, Eq. 16 thereforebecomes:

u x ��u2

ref:meth � s2d � s2

run � s2r =n

q�19�

which reduces to:

u x ��s2d � s2

run � s2r =n

q�20�

if, as mentioned before, the uncertainty in the refer-ence method is considered negligible.

EURACHEM [ 1 ] recommends the same approachfor the uncertainty statement if the bias is found tobe signi¢cant and if, as required by ISO, the meas-urement result is corrected for the bias. Thisapproach is also preferred by IUPAC [ 14 ]. If nocorrection is applied, the observed bias and its asso-ciated uncertainty are reported in addition to theresult [ 1 ]. When a signi¢cant bias is not consideredrelevant, and therefore is not corrected for, theapproach proposed by IUPAC [ 14 ] for recoveryexperiments (explained in Section 5.3.2) couldalso be applied.

One should note that in the uncertaintyexpressed as in Eq. 20 some terms appear morethan once. Indeed, if in the evaluation of the biasthe measurements performed with the routinemethod are obtained, e.g., under repeatability con-ditions, s2

A is equal to s2r . Some documents, such as

that by the BCR concerning the application of me-trology in chemistry and biology [ 16 ], do not thenadditionally include the repeatability uncertaintyinto the uncertainty statement, since it has alreadybeen considered in the process of assessing thebias. This does not, however, seem to be correct,since the bias is to be considered a source of uncer-tainty additional to the random errors.

5.3.2. RecoveryAnother approach to evaluating the bias is by a

recovery experiment. The IUPAC report discussesthe problems related to recovery estimation and thepossibilities of corrections [ 14 ]. Barwick and Elli-son [ 17 ] focus on the evaluation of the uncertaintiesassociated with recovery. Even though recovery isusually regarded in a relative way, absolute expres-sions are used in the following, since this corre-sponds with the other expressions used. In therecovery experiment, a known amount of analyteis added to the sample matrix. The bias can then beestimated as the difference between the concentra-


TRAC 2688 15-8-01

tion recovered, x́rec, and the known spiked concen-tration, cspike:

est:bias � x rec3cspike �21�

If blank sample material is available, x́rec is theconcentration observed for the spiked sample. Theuncertainty associated with the bias can then becalculated as:

ubias ��s2obs=m � u2

spike

q�22�

where s2obs represents the variance of the replicate

analyses performed on the spiked sample, and m isthe number of replicate analyses performed on thespiked sample. This means that the uncertainty ofthe traceability of Eq. 3 is here the uncertainty of thespiked concentration. However, this uncertaintywill usually be negligible.

If no blank material is available the concentrationrecovered is obtained from the measurement of thesample before and after the addition of the analyte,x́rec = x́obs3x́native. The bias is estimated accordingto Eq. 21. In this situation, the uncertainty associ-ated with the bias contains a contribution from therepeated measurements of the sample after theaddition of the analyte as well as before the addi-tion:

ubias ��s2obs=m1 � s2

native=m2

q�23�

where s2obs again represents the variance of the rep-

licate analyses performed on the sample after theaddition, and m1 is the number of replicate analysesperformed; s2

native is the variance of the replicateanalyses performed before the addition and m2

is the number of replicates. As already mentioned,the uncertainty of the spiking is considered negli-gible.

If s2obs and s2

native are estimates of the same var-iance, they can be pooled. The uncertainty associ-ated with the bias

ubias ��m131�s2

obs � �m231�s2native

m1 �m232

1

m1� 1

m2

� �s�24�

is then again the standard deviation that appears inthe t-test applied to test whether the spiked concen-tration has been recovered.

As for the method comparison, the uncertainty

associated with a non-signi¢cant bias should beconsidered in the uncertainty statement. If thebias is not signi¢cant, or if a signi¢cant bias is cor-rected for, the uncertainty of the mean result of nreplicated measurements is expressed by Eq. 16:

u x ��u2

bias � s2run � s2

r =nq

IUPAC [ 14 ] mentions a pragmatic approach forthe situation when a signi¢cant bias is not consid-ered relevant, and therefore is not corrected for. Itconsists in adding the absolute value of the uncor-rected bias to the expanded uncertainty, the latterbeing calculated assuming that the bias is zero. Thisapproach is also followed by Barwick and Ellison[ 17 ], but they include the recovery (expressed in arelative way) in the standard uncertainty by takinginto account the coverage factor, k, that will be usedin the calculation of the expanded uncertainty. Withthe same approach applied here, Eq. 23, for exam-ple, would become:

ubias ��

est:bias

k

� �2

� s2obs

m1� s2

native

m2

s�25�

However, according to the GUM [ 2 ], correctionsshould always be applied. Consequently, the laterIUPAC document [ 6 ] as well as EURACHEM [ 1 ] donot recommend this approach.

Alternatively, when a signi¢cant bias is not con-sidered relevant and is not corrected for, IUPAC[ 14 ] proposes that one should increase ubias andproceed as if the bias was not signi¢cant. Theincreased ubias is calculated as Mest.biasM / tcrit withtcrit the tabulated t value used in the signi¢cancetest. This increased uncertainty, ubias, thus corre-sponds to the uncertainty that in the signi¢cancetest would just lead to the conclusion that the biasis not signi¢cant. However, according to IUPAC, allthese approaches lead to an overstatement of theuncertainty. Therefore, as already mentioned ear-lier, IUPAC [ 6 ] as well as EURACHEM [ 1 ] recom-mend that one should always correct for the bias, orreport the observed bias and its uncertainty in addi-tion to the result.

5.4. Absolute uncertainty

The way in which bias was assessed in Section5.3.2 does not always provide full traceability to theSI or to the highest metrological level possible. Pri-mary methods allow for a direct traceability to the SI


TRAC 2688 15-8-01

[ 16 ]. A direct traceability to the SI is also possiblewith primary standards [ 18 ]. However, these pri-mary standards only comprise pure chemicals,and therefore they cannot be used as direct refer-ence for analyses with complicated matrices. Thereference materials with the highest level in thehierarchy that is suitable for matrix analysis areCRMs [ 18 ]. In a method-independent context, Val-caèrcel and Rios [ 19 ] also consider the value of aCRM as the highest achievable real reference inthe metrological hierarchy. The high reliability ofthese materials stems from the fact that their contentis determined by different laboratories using differ-ent methods, so that laboratory and method biasesare eliminated to the highest degree possible. If thebias of a certain method is assessed by the analysisof a CRM with a matrix similar to the one understudy, the most reliable estimate of the bias isobtained. If a precision estimate of the routinemethod is available, the uncertainty can be esti-mated using Eq. 16. The uncertainty associatedwith the bias again splits up into the uncertaintyof the estimated bias and the traceability, whichhere is the uncertainty speci¢ed for the CRM bythe certi¢cation organisation:

ubias ��u2

CRM � u2est:bias

q�26�

The uncertainty in the estimated bias, uest:bias,corresponds to the uncertainty associated withthe difference between the concentration meas-ured for the CRM and the certi¢ed value. Thisagain corresponds to the standard deviation termsd = s�x3W� = sx that appears in the t-test oftenapplied to test whether the difference is statisticallysigni¢cant:

t � Mx3WMsx

�27�

with W being the certi¢ed value, x́ the mean resultobtained in the laboratory for the CRM and s2

x thevariance of the mean concentration observed forthe CRM. Most often, measurements are performedunder repeatability or intermediate precision con-ditions.

Notice that in this approach the uncertainty in thecerti¢ed value is considered negligible comparedwith the method precision and, therefore, it is nottaken into account in the signi¢cance test.

Alternative approaches that take the uncertaintyin the certi¢ed value into account have been pro-

posed [ 16,20,21 ]. The following formula is thenused as the criterion for acceptance:

32��u2

CRM � c2

q9 x3W 9 2

��u2

CRM � c2

q�28�

with c2 being the precision of the measurementresults which correspond to s2

x de¢ned beforehand.The uncertainty in the CRM, uCRM, has to beobtained from the certi¢cate. As pointed out byJorhem [ 22 ], the uncertainty intervals described inthe certi¢cates may have different meanings andare not always easy to understand. Jorhem [ 22 ]also shows that the often reported 95% con¢denceinterval calculated on the basis of the mean valuefor each participating laboratory is suitable for thecharacterisation of the CRM but not for the evalua-tion of individual laboratory measurements. Moredetailed certi¢cation reports including informationon how to use the CRM are required for an accept-able evaluation of individual results [ 23 ].

For some analytical problems, the highest level oftraceability can also be reached by using a primarymethod. The EURACHEM guideline [ 1 ] gives a cur-rent de¢nition of a primary method: `̀ A primarymethod of measurement is a method having thehighest metrological qualities, whose operation iscompletely described and understood in terms of SIunits and whose results are accepted without refer-ence to a standard of the same quality''.

As already mentioned in an earlier section, meth-ods accepted as having these properties are IDMS,coulometry, gravimetry, and titrimetry [ 8 ]. Theresults of these methods are usually directly trace-able to the SI and, as a consequence, they shouldprovide the smallest uncertainty achievable. How-ever, for a large number of analytical problems,other techniques are used routinely. Nevertheless,primary methods are used for standardisation pur-poses by National Measurement Institutes. In orderto reduce the uncertainty of the traceability, theycan be used as reference methods to estimate thebias of a routine method. If primary methods and /or primary standards are used in the certi¢cationstudy for CRMs, the latter are considered secondarystandards and have a higher traceability than otherCRMs [ 18 ].

6. Conclusions

To compute uncertainty statements, it is possibleand preferable that one should use as much as pos-


TRAC 2688 15-8-01

sible information obtained from method validationand other quality assurance procedures. In thisway, the uncertainty expression becomes a naturalextension of the validation of methods, which is, foranalytical chemists, a much better understood con-cept than the component-by-component approachand therefore will be adopted much more easily.The ¢rst edition of the EURACHEM Guide for`Quantifying Uncertainty in Analytical Measure-ment' [ 24 ] insisted very much on the component-by-component approach and, in this way, hascaused unnecessary confusion. The second edition[ 1 ] follows closely a document prepared by IUPAC,AOAC, FAO and IAEA [ 6 ] and stresses to a muchlarger extent that `̀Measurement uncertainty mustbe integrated with its existing quality assurancemeasurements, with these measures themselvesproviding much of the information required to eval-uate the measurement uncertainty''. In its draft ver-sion [ 25 ], it goes on to say that `̀ It attempts to correctthe impression gained within the Analytical Com-munity that it is only the so-called component-by-component approach that is acceptable...''. The¢nal version refers to the ISO guide 17025 [ 26 ],which also allows one the use of approaches forthe uncertainty evaluation other than the compo-nent-by-component approach. The second editionof the EURACHEM guide is a very useful document^ much more so than the ¢rst one. It is, however,somewhat unfortunate that the examples cited givemuch more room to the component-by-componentapproach than to those based on method validationdata.

We ourselves consider that, when presenting thesubject, it is better to start with equations in whichsums of variances or sums of relative variances aregiven, rather than with the error propagationapproach of Eq. 9. The latter approach certainlyhas its merits and is useful when going furtherinto the subject, but it is preferable, when talkingto analytical chemists, to start with concepts towhich they are used.

Although the documents cited above form a goodworking basis, they suffer from one defect ^ that theterm ùncertainty' is used for many very differentsituations. It is our opinion that operational de¢ni-tions of uncertainty are required so that a distinctionis made according to the reference to which theresult is traced and the sources of uncertainty thatare included.

Acknowledgements

This work was supported by the Research con-tract No. NM/03/24 of the Belgian Government(The Prime Ministers Service ^ Federal Of¢ce forScienti¢c Technical and Cultural Affairs, Standar-disation Program).

References

[ 1 ] EURACHEM/CITAC Guide, Quantifying Uncertainty inAnalytical Measurement, 2nd edition, 2000.

[ 2 ] Guide to the Expression of Uncertainty in Measure-ment, ISO, Geneva, 1995.

[ 3 ] International Vocabulary of Basic and General Terms inMetrology (VIM), ISO, Geneva, 2nd edition, 1993.

[ 4 ] S. Ellison, W. Wegscheider, A. Williams, Anal. Chem. 69(1997) 607A.

[ 5 ] D.L. Massart, B.G.M. Vandeginste, L.M.C. Buydens, S.de Jong, P.J. Lewi, J. Smeyers-Verbeke, Handbook ofChemometrics and Qualimetrics, Part A, Elsevier,Amsterdam, 1997, pp. 56, 97, 395.

[ 6 ] Report on the FAO, IAEA, AOAC Int., IUPAC, Interna-tional Workshop on Principles and Practices of MethodValidation, 4^6 November 1999, Budapest.

[ 7 ] NMKL, Procedure No. 5, Estimation and Expression ofMeasurement Uncertainty in Chemical Analysis, NordicCommittee on Food Analysis, 1997.

[ 8 ] R. Kaarls, T.J. Quinn, The Comiteè Consultatif pour laQuantiteè de Matieére: a Brief Review of its Origin andPresent Activities, Metrologia 34 (1997) 1, cited in BCRReport, EUR 18405 EN, Community Bureau of Refer-ence, 1998.

[ 9 ] A. Dobney, H. Klinkenberg, F. Souren, W. Van Borm,Anal. Chim. Acta 420 (2000) 89.

[ 10 ] R. Wood, A. Nilsson, H. Wallin, Quality in the FoodAnalysis Laboratory, Royal Society of Chemistry Mono-graphs, Cambridge, 1998.

[ 11 ] G.T. Wernimont, Use of Statistics to Develop and Eval-uate Analytical Methods, AOAC, Arlington, VA, 1985, p.104.

[ 12 ] W. Horwitz, L.R. Kamps, K.W. Boyer, J. Assoc. Off.Anal. Chem. 63 (1980) 1344.

[ 13 ] International Standard, Accuracy (Trueness and Preci-sion) of Measurement Methods and Results, ISO 5725-6-1994, Geneva, 1994, especially paragraph 8.1.

[ 14 ] Harmonised Guidelines for the Use of Recovery Infor-mation in Analytical Measurement, Technical ReportResulting from the Symposium on Harmonization ofQuality Assurance systems for Analytical Laboratories,IUPAC, ISO, AOAC Int. and EURACHEM, Orlando, FL,4^5 September 1996.

[ 15 ] S. Kuttatharmmakul, D.L. Massart, J. Smeyers-Verbeke,Anal. Chim. Acta 391 (1999) 203.

[ 16 ] BCR Report, EUR 18405 EN, Community Bureau ofReference, 1998.


TRAC 2688 15-8-01

[ 17 ] V.J. Barwick, S.L.R. Ellison, Analyst 124 (1999) 981.[ 18 ] R. Walker, I. Lumley, Trends Anal. Chem. 18 (1999)

594.[ 19 ] M. Valcaèrcel, A. Rios, Trends Anal. Chem. 18 (1999) 68.[ 20 ] ISO Guide 35:1989 (E), Certi¢cation of Reference

Materials ^ General and Statistical Principles, ISO, Ge-neva, 1989.

[ 21 ] R. Walker, The selection and use of reference materials:some examples produced by LGC, in: A. Fajelj, M. Par-kany (Editors ), The Use of Matrix Reference Materialsin Environmental Analytical Processes, Royal Society ofChemistry, Cambridge, 1999, p. 155.

[ 22 ] L. Jorhem, Fresenius' J. Anal. Chem. 360 (1998) 370.[ 23 ] J. Pauwels, How to use matrix certi¢ed reference mate-

rial, in: A. Fajelj, M. Parkany (Editors ), The Use ofMatrix Reference Materials in Environmental AnalyticalProcesses, Royal Society of Chemistry, Cambridge,1999, p. 41.

[ 24 ] Quantifying Uncertainty in Analytical Measurement;published on behalf of EURACHEM by the LGC, Lon-don, 1995.

[ 25 ] EURACHEM/CITAC Guide, Quantifying Uncertainty inAnalytical Measurement, 2nd editionn, Draft, June1999.

[ 26 ] ISO/IEC Standard 17025, General Requirements forthe Competence of Testing and Calibration Laborato-ries, ISO, Geneva, 1999.

Edelgard Hund graduated in chemistry from the University ofRegensburg, Germany, in 1997. She is now preparing a Ph.D. thesisin analytical chemistry at the Vrije Universiteit Brussel (VUB),Brussels, Belgium, in the laboratory of analytical chemistry of thePharmaceutical Institute. Professor Johanna (An) Smeyers-Verbeke obtained her Ph.D. in chemistry in 1977 at the VUB.She is now responsible for parts of the courses in AnalyticalChemistry in the Pharmaceutical Institute of the VUB. ProfessorDeèsireè Luc Massart is Head of Department of the Laboratory ofAnalytical Chemistry of the Pharmaceutical Institute of the VUB.He obtained his Ph.D. in chemistry at the University of Ghent, andin 1968 moved to the University of Brussels where he is mainlyinterested in chemometrics.


Documents

Operational de¢nitions of uncertainty