Proficiency Testing of (Gamma) Non-Destructive Assay Laboratories by NPL
Julian Dean
National Physical Laboratory, Hampton Road, Teddington, Middlesex, UK TW11 0LW
The National Physical Laboratory (NPL), as the UK’s National Measurement Institute, is a multidisciplinary
research laboratory which develops and disseminates primary standards of measurement and provides
measurement services and technology transfer across a wide range of technical areas. NPL’s Radioactivity Group
carries out these functions for the radioactivity measurement community; amongst its many areas of expertise, the
group has built up many years’ experience of running Proficiency Test Exercises (PTEs) in fields such as
environmental radioactivity monitoring. In 2007, in response to feedback from the nuclear site decommissioning
sector, NPL began running PTEs for laboratories who need to measure low levels of gamma-emitters in large
waste containers, such as 200 litre drums.
To date, NPL has run four PTEs for waste measurement laboratories. In each exercise, (i) a 200 litre drum
containing ‘simulated’ waste spiked with gamma-emitting radionuclides (with activity concentrations traceable to
primary standards) was prepared and certificated by NPL, (ii) the drum was sent to each participating laboratory in
turn as a 'blind sample’ for assay, (iii) the participants reported their results to NPL, and (iv) all the results were
summarised by NPL in a single report. Each participant’s results were assigned a code number in the report for
confidentiality. The PTEs to date have highlighted possible needs for training in the use of mathematical modelling
software. Also, there is some evidence that results obtained using Segmented Gamma Scanners are biased low
(compared with 'open' measurement geometries) when layers or hotspots are present. The PTEs are proving
valuable to the waste measurement sector by providing them with confidence in waste assay at low activity levels
and enabling laboratories to demonstrate measurement proficiency to third parties.
1. INTRODUCTION
The National Physical Laboratory (NPL) is the UK’s National Measurement Institute (NMI) and a
multidisciplinary research laboratory serving some 20 generic areas of science and engineering. NPL develops
and maintains primary standards of measurement, disseminates them to the user base and helps users improve
measurement accuracy (for example, via training courses, workshops and Measurement Good Practice Guides).
The laboratory works in collaboration with NMIs in other countries (e.g. NIST in the USA and PTB in
Germany) to ensure that primary standards of measurement are harmonised across the globe.
In the field of radioactivity measurement, NPL’s Radioactivity Group addresses users’ needs in a variety of
sectors (e.g. environmental analysis, nuclear medicine and academia). There are needs not only for standards
and reference materials but also for some independent verification that users’ measurement techniques are
accurate. NPL began running Proficiency Test Exercises (PTEs) for radioactivity in the late 1980s, in particular
for the environmental analysis sector where the laboratory has run 19 such exercises over the last 24 years; in so
doing, NPL has acquired considerable experience in areas such as low-level radioactive source preparation and
validation, data analysis methods and the transport of radioactive sources. This expertise is now being applied to
address the needs of a growing sector in the UK – nuclear site decommissioning.
In the early 2000s, many of the older nuclear facilities and sites in the UK moved into the decommissioning
phase of the nuclear cycle. Very large volumes of potential radioactive waste such as lightly-contaminated
building materials, soils and general laboratory wastes were being generated, and this continues to be a problem
to this day. Such materials must be measured for their radioactive content so they can be consigned to the
correct waste streams. Unfortunately, many wastes are poorly defined (e.g. the material is heterogeneous, or
contains ‘hotspots’, or there are no site records available to indicate which radionuclides may be present). This
can lead to conservative overestimates of activity, leading in turn to unnecessary disposal costs and usage of
valuable LLW repository space. These problems led many laboratories to turn to NPL for reference materials
and for advice on how to carry out the measurements more accurately.
In response, NPL ran a user forum in 2005 to identify the specific needs. The highest priority at that time was
for large-volume standards for -emitters in concrete and ‘soft wastes’ with activity concentrations of
approximately 0.4 Bq g-1
(the upper limit for ‘Exempt waste’ in UK legislation at that time [1, 2]). In response,
NPL prepared a prototype 200 litre ‘standard drum’ spiked at this activity concentration level, and ran a PTE [3]
by circulating the drum to laboratories, on a voluntary basis, as a blind sample. Three further PTEs [4, 5, 6]
followed. This paper reviews the PTEs so far, including how the standard drums were prepared and circulated,
how the participants measured them, how the data were analysed, and the conclusions drawn.
2. PREPARATION OF THE DRUMS
The preparation of the drum for the first PTE (2007) has been described previously [7]. A mixed radionuclide
standard solution of 241
Am, 60
Co and 137
Cs was prepared from stock solutions of the individual radionuclides
previously measured in a secondary standard ionisation chamber. The chamber itself had been calibrated using
primary standards of radioactivity defined by absolute counting techniques [8]. High-resolution -spectrometry
was used to check for -emitting impurities. To form the matrix, 240 plastic bottles (500 ml volume each) were
each part-loaded with dried ion-exchange resin; each bottle was then spiked with the mixed standard solution
before being sealed and homogenised. The bottles were then loaded into a 200 L drum in 5 layers of 48 (see
Figure 1), and an overall contents density of approximately 300 kg m-3
(typical of ‘soft waste’) was achieved.
The activity concentration of each radionuclide was calculated and was designated the ‘Assigned Value’ for that
radionuclide.
A second PTE was run in 2009. In response to feedback from the participants, two drums (Drum A and Drum B,
both containing the same radionuclides as the 2007 drum) were prepared. Again, layers of bottles were used, but
each drum contained only one bottle of ion-exchange resin (containing all the activity present); all other bottles
and voids were filled with vermiculite as a low-density inactive filling matrix. Drum A was similar in activity to
the 2007 drum but Drum B was an order of magnitude higher (approximately 10 Bq g-1
).
A problem with preparing PTE sources of this type is that they should ideally resemble closely a ‘real’ waste
form and at the same time be dimensionally stable so that all participants are measuring the same artefact. With
this in mind, the standard drum for the third PTE (2011) contained sheets of steel and plastic designed to fit
inside the drum as a rigid box-like structure (see Figure 2). Again, 241
Am, 60
Co and 137
Cs were included, but this
time stock solutions of the individual nuclides were dispensed to filter paper samples which were then taped to
the steel sheets to mimic surface contamination. The steel sheets were loaded into a pre-weighed drum along
with (inactive) sheets of Nylon 6-6 and ‘Air Cap’ (Sealed Air Limited, Kettering, UK).
In the most recent PTE (2012), the 2011 drum was modified by adding bottles of inactive coarse aggregate into
the base of the drum and also a bottle of ion-exchange resin containing all three of the radionuclides (see Figure
3). ‘Air Cap’ was excluded from this drum.
Table 1 summarises the specifications of the standard drums used in all four PTEs run so far.
3. TRANSPORT AND DATA REPORTING
Prior to each PTE, a timetable for the transport of the drum between the various sites was agreed well in
advance with all the participants, and it was ensured that all associated documentation (e.g. export licenses and
local site documentation) was in place prior to transport. The timetable also provided sufficient time for each
laboratory to measure the drum and included contingency transport time to accommodate unforeseen delays.
Table 1 includes a breakdown of participant numbers.
Prior to delivery of the drum to each participant, NPL provided forms for measurement results and any related
information (e.g. detector types, calibration methods and measurement uncertainties). The radionuclides present
and the approximate overall activity concentrations were also declared, along with other details (see Table 1).
In the 2009, 2011 and 2012 PTEs, to make the test more realistic, certain details (e.g. the location of the activity
within the drum and details of the matrices present) were initially withheld by NPL but were disclosed after an
‘initial’ results deadline had passed. Participants were then invited to submit additional results (before a second
deadline) using this additional information. Not all participants chose to do this. Results submitted before and
after the initial deadline were treated as separate data sets (designated ‘pre-disclosure’ and ‘post-disclosure’). In
the 2007 PTE, a single deadline was imposed; revisions to data were accepted after this deadline, but any such
revisions were made clear in the PTE report.
4. DATA ANALYSIS
All data submitted were analysed using the method described by Harms [9, 10]. Analysis was carried out as
follows:
(i) The deviation ‘D’ of a participant’s result ‘L’ from the Assigned Value ‘N’ was calculated from:
[1]
and the standard uncertainty ‘uD’ of the deviation was calculated from:
[2]
where ‘uL’ and ‘uN’ are the standard uncertainties of the participant’s result and the Assigned Value,
respectively.
(ii) Three parameters (the zeta score ‘ζ’, the relative uncertainty ‘RL’ and the z-score ‘z’) were calculated for
each result, these being defined as:
[3]
[4]
pmed
NL
NR
NLz
[5]
where Rmed is the median value of RL and p is the standard deviation for proficiency assessment.
n.b. In the 2012 PTE, due to a revision of the NPL data analysis procedure, p was defined as 0.05823N, which
corresponds to a deviation ‘D’ of 15% at the 99% confidence level. This value was chosen ‘by perception’ (as
per ISO 13528:2005 [11]); it corresponds to a level of performance that NPL would wish laboratories to be able
to achieve.
In cases where ζ and z were both ≤ 2.58 and RL was not significantly larger than the other values in the data set,
the result was designated as ‘in agreement’ with the Assigned Value. When RL was significantly larger than the
other values, or either ζ or z (but not both) was > 2.58, the result was designated ‘questionable’. If both ζ and z
were > 2.58, the result was designated ‘discrepant’.
All reported results, the results of the above analyses and any supplementary information submitted by the
participants were summarised in a report issued to the participants. All results were coded for confidentiality.
Figure 4 gives an example of a ‘deviation plot’ (for 137
Cs in the 2007 PTE [3]).
The 2007, 2009 and 2011 PTEs included post-exercise workshops to which all participants were invited. The
results were formally presented and discussed, and many participants took the opportunity to present details of
their NDA methods and to provide NPL with feedback.
5. GAMMA NDA METHODS USED BY PARTICIPANTS
The participants between them used a variety of gamma NDA instruments to measure the drums. Most were
high-resolution gamma spectrometers based one or more hyperpure Ge detector crystals with ancillary signal
processing and data analysis systems, often with a sample loading platforms adjacent to the detector(s). Several
participants used Segmented Gamma Scanners (SGSs). A few participants used low-resolution gamma detectors
(e.g. plastic scintillators and NaI(Tl) crystals). In most cases, detection efficiencies were derived using computer
1100100
N
L
N
NLD
2
N
2
LD 100
N
u
L
u
N
Lu
2
N
2
L uu
NL
L
uR L
L
modelling techniques relying on a knowledge of the drum’s dimensions and some knowledge of the
composition of the sample; in many cases, commercially-available software was used. Some laboratories used
other calibration methods (e.g. using ‘in-house’ standard drums containing materials such as spiked
vermiculite), but this was less common.
6. DISCUSSION
Firstly, it is interesting to look at the number of results in agreement with the Assigned Values in each of the
PTEs (see Figure 5). The black columns represent the performance level for data submitted (from all
participants) before the initial deadline whereas the white columns represent performance for data submitted
after that deadline (i.e. when the participants had more information on the drum’s internal structure). The grey
columns represent the initial performance level for the subset of participants who submitted data after the initial
deadline.
It was clear that in all the PTEs there were significant numbers of results which were not in agreement with the
Assigned Value and that an improved knowledge of a drum’s composition did not necessarily improve the
performance of the participants as a whole. It was interesting to note that, after the first deadline, some
laboratories modified their calibration model in an attempt to better match the model with the true ‘internal
structure’ of the standard drum. This did not always result in a more accurate result, possibly because the
participant was changing the procedure they would normally use. Moreover, it was also clear that there were
differences between the results of laboratories using the same commercial modelling software package. All this
is evidence of possible needs for training in modelling.
Also, many of the results derived from SGS measurements were either questionable or discrepant. Figure 6
illustrates the differences (for each PTE) between the mean results from SGS measurements and the mean result
from systems with uncollimated detectors. Although in most cases there is no statistical difference between the
two means, it is of concern that the bias is low in most cases (although the 2012 PTE results show the opposite
trend). This is possibly due to the heterogeneous nature of the standard drums. In all three PTEs, the activity has
been present as either ‘layers’ or ‘hotspots’. In either case, it is possible that the activity in one or more
segments may be small and therefore below the system’s limit of detection. Consequently, the system’s response
to these segments might not have been included in the summation of all the segment responses, leading to an
overall underestimation of the activity in the drum. This effect, if real, could result in consignment to a waste
stream of packages too high in activity for that stream, with legal and public safety implications.
7. CONCLUSIONS
NPL has run a series of four voluntary Proficiency Test Exercises to enable laboratories measuring low levels of
-emitters in bulk materials to test their gamma NDA procedures and calibrations. In each exercise, one or two
‘standard drums’ were prepared, each containing quantities of radioactivity traceable to national standards.
These ‘standard drums’ have been made progressively more realistic by the introduction of hotspots and
materials commonly found in wastes, such as plastics, metals and building materials.
Many participants submitted data which did not agree statistically with NPL’s values and there is some evidence
that the use of Segmented Gamma Scanners for the assay of heterogeneous wastes can lead to underestimates of
activity. There was also evidence of issues with the modelling of detection efficiencies.
These PTEs are proving invaluable to participating laboratories by either flagging up possible measurement
problems or by providing confidence in the assay of gamma-emitting wastes. This is increasingly important in
that it helps reduce volumes of LLW and potentially reduces costs to the operators; moreover, in the UK, it aids
compliance with the UK Low Level Waste Strategy [12]. The PTEs also benefit participants by enabling them
to demonstrate measurement proficiency to regulators, quality assessors and customers.
8. ACKNOWLEDGMENTS
The author thanks present and former colleagues Daniel Ainsworth, Pete Burgess, Arvic Harms, Andy Fenwick,
Cyrus Larijani, Lynsey Keightley, Andy Pearce, Hilary Phillips, Andy Stroak and Jean Wong for their help in
preparing, certificating and dispatching the drums. He also gratefully acknowledges the financial support of the
National Measurement System.
9. REFERENCES
[1] SI 1986 No. 1002. The Radioactive Substances (Substances of Low Activity) Exemption Order 1986.
[2] SI 1992 No. 647. The Radioactive Substances (Substances of Low Activity) Exemption (Amendment)
Order 1992.
[3] Dean, J. C. J. A Comparison of Procedures Used at UK Nuclear Sites for Gamma Assays of Potentially
Contaminated or Activated Materials. NPL Report IR 2 (2007).
[4] Dean, J. C. J. A Second Comparison of Procedures for the Assay of Low Levels of Gamma-emitters in
Nuclear Site Waste. NPL Report IR 19 (2010).
[5] Dean, J. C. J. NPL Nuclear Industry Proficiency Test Exercise 2011. NPL Report IR 25 (2012).
[6] Dean, J. C. J. NPL Nuclear Industry Proficiency Test Exercise 2012. NPL Report IR 30 (2013).
[7] Dean, J. C. J. A UK comparison for measurements of low levels of gamma-emitters in waste drums.
Appl. Radiat. Isotopes 67 (2009) 678-682.
[8] NCRP Report 58. Fundamental or Direct Measurement of Activity in Radioactive Decay. A Handbook
of Radioactivity Measurements. NCRP, Bethesda, National Council on Radiation Protection and
Measurements, 1985 (2nd
edition), 74-139.
[9] Harms, A.V. A New Approach for Proficiency Test Exercise Data Evaluation. Accreditation and
Quality Assurance, 14, 253-261, 2009(a).
[10] Harms, A.V. Visualisation of Proficiency Test Exercise Results in Kiri plots. Accreditation and Quality
Assurance, 14, 307-311, 2009(b).
[11] ISO Standard 13528:2005, ‘Statistical Methods for use in Proficiency Testing by Interlaboratory
Comparisons’, ISO, Geneva (2005).
[12] UK Strategy for the Management of Solid Low Level Radioactive Waste from the Nuclear Industry
(2010).
Figure 1 – 2007 PTE drum during loading.
Figure 2. Schematic of steel and plastic ‘insert’ for 2011 PTE drum.
Figure 3 – Schematic of 2011 PTE drum modified with bottles of aggregate and resin (resin in Bottle ‘A’ only).
Figure 4 – Deviation plot for 137
Cs in 2007 PTE [3]. Black squares denote results ‘in agreement’, white squares are ‘questionable’ and black triangles are ‘discrepant’. The
smaller dotted line represents the percentage relative uncertainty on the Assigned Value multiplied by 2.58. The larger dotted line represents the pass/fail limits of the z-score.
-100
0
100
De
via
tio
n (
%)
Participant code
Figure 5. Percentages of results ‘in agreement’ with the Assigned Value. Black columns represent data
submitted before the initial results deadline, white columns represent data submitted after that deadline and grey
columns represent data submitted initially by the subset of participants submitting data after the initial deadline.
Figure 6. Comparison of means for SGS and ‘uncollimated’ measurements. Black columns denote 241
Am data, dark grey columns denote 60
Co and light grey columns denote 137
Cs.
0
10
20
30
40
50
60
70
80
90
100%
'In
agr
eem
ent'
Data set
Year: 2007 2009 (Drum A) 2009 (Drum B) 2011 2012
Radionuclides: 241
Am, 60
Co and 137
Cs
Approximate activity
concentration (Bq g-1
)
0.4 0.4 10 10 10
Matrix: Ion-exchange resin Ion-exchange resin
Vermiculite
Ion-exchange resin
Vermiculite
Stainless steel
Nylon 6-6
‘Air Cap’
Stainless steel
Nylon 6-6
Coarse aggregate
Ion exchange resin
Activity distribution: Homogeneous Heterogeneous Heterogeneous Heterogeneous Heterogeneous
Matrix distribution: Homogeneous Homogeneous Homogeneous Heterogeneous Heterogeneous
Initial data provided to
participants:
Dimensions of the drum
itself (e.g. wall thickness)
Radionuclides present
Activity concentration
range
Nature of activity
distribution
Description of matrix and
its containment (i.e. type
of bottle, number of layers)
As 2007
As 2007
Dimensions of the drum
itself (e.g. wall thickness)
Radionuclides present
Activity concentration
range
Nature of activity
distribution
Generic description of
matrices and percentage by
mass of each
As 2011
Data provided after
initial deadline:
n/a Size and location of
‘hotspot’
Size and location of
‘hotspot’
Detailed description of
matrices and activity
distribution
As 2011
Number of UK
participants:
18 8 12 16 11
Number of overseas
participants:
None 1 2 4 3
Table 1. Summary of drum specifications, initial data disclosure and participant levels for the four PTEs.