Applying Trend Analysis Techniques to

Quality Control Measurements in

Radionuclide Metrology

Simon Jerome, NPL

LSUF, NPL, 24th January 2006

Event and date 2

LSUF 24th January 2006

Quality Control – ISO 17025:2005

5.9 Assuring the quality of test and calibration results

5.9.1 The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following:

a) regular use of certified reference materials and/or internal quality control using secondary reference materials…

NOTE The selected methods should be appropriate for the type and volume of the work undertaken.

Event and date 3

LSUF 24th January 2006

Two Techniques Commonly Used (More Are Available)

• Shewhart – more easily interpreted

• CuSum (Cumulative Summation) – more sensitive to change

Event and date 4

LSUF 24th January 2006

Use of Shewhart charts

• Common usage is to set 2σ and 3σ limits, based on 11-31 replicate results

• Action taken if 3σ limit is exceeded – expected 3 times in 1000

• Warning is registered if 2σ but not 3σ limit is exceeded – expected 5 times

in 100

• Straightforward, but no ‘warning signals’ before warning or action levels

breached

Event and date 5

LSUF 24th January 2006

• 3σ exceeded – action required

immediately, since the

procedure is out of control.

Measurement should stop

until the cause of the action

limit violation is identified

and corrected.

Shewhart Control Chart - RMG Ionisation Chambers Radium Check

Source Measurement

50.3150

50.3200

50.3250

50.3300

50.3350

50.3400

50.3450

50.3500

50.3550

50.3600

50.3650

1-Aug 3-Aug 5-Aug 7-Aug 9-Aug 11-Aug 13-Aug 15-Aug 17-Aug 19-Aug 21-Aug 23-Aug 25-Aug 27-Aug 29-Aug 31-Aug 2-Sep 4-Sep

Date

Ion

Ch

am

ber

Cu

rren

t/p

A

Event and date 6

LSUF 24th January 2006

• 2σ exceeded – situation needs

to be closely monitored

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source

Measurement

50 .3 15 0

50 .3 20 0

50 .3 25 0

50 .3 30 0

50 .3 35 0

50 .3 40 0

50 .3 45 0

50 .3 50 0

50 .3 55 0

50 .3 60 0

50 .3 65 0

1-Au g 3 -Au g 5 -Aug 7-Aug 9-Au g 11-Au g 13 -Au g 15 -Aug 17 -Aug 19-Au g 2 1-Au g 23 -

Au g

25 -

Au g

2 7-

Aug

2 9-

Aug

3 1-Au g 2-S e p 4-S e p

Date

Ion

Ch

am

ber C

urren

t/p

A

Event and date 7

LSUF 24th January 2006

ISO 8258:1991 – Shewhart Control Charts

• Introduces the concept of Trend Analysis

• Relies on the range observed in measurement:

rj= |x

(j+1)– x

j|

X =

R =

'

'

N

xj

j∑

'

'

)1( −

∑

N

rj

j

Event and date 8

LSUF 24th January 2006

ISO 8258:1991 – Shewhart Control Charts

• Action limit set at: X ± R×2.66

• Warning limit set at: X ± R×1.77

• ‘Information’ limit set at: X ± R×0.89

Based on a series of single observations; limits vary depending on the batch

size and subgroup

Event and date 9

LSUF 24th January 2006

• Warning Limit Exceeded

twice in under 42

measurements. Measurement

should stop until the cause of

the action limit violations are

identified and corrected. (ISO

8259:1991 suggests 2 out of 3

is an action point)

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source

Measurement

5 0.3150

50 .3 2 00

50 .3 2 50

50 .3 3 00

50 .3 3 50

50 .3 4 00

50 .3 4 50

50 .3 5 00

50 .3 5 50

50 .3 6 00

50 .3 6 50

1-Aug 3 -Aug 5-Aug 7-Aug 9-Au g 11-Au g 13-Au g 15-Au g 17 -Au g 19 -Au g 21-Au g 23 -

Au g

25 -

Au g

2 7 -

Au g

2 9 -

Aug

3 1-Aug 2 -S e p 4 -S e p

Date

Ion

Ch

am

ber C

urren

t/p

A

Event and date 10

LSUF 24th January 2006

• Nine points in a row on one

side of the mean. This may

indicate that there has been a

shift in the mean; work can

continue, however the cause

of the shift needs to be

identified and the limits re-set

if justified

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source

Measurement

5 0 .3 15 0

5 0 .3 2 0 0

5 0 .3 2 5 0

5 0 .3 3 0 0

5 0 .3 3 5 0

5 0 .3 4 0 0

5 0 .3 4 5 0

5 0 .3 5 0 0

5 0 .3 5 5 0

5 0 .3 6 0 0

5 0 .3 6 5 0

1-Au g 3 -Au g 5 -Au g 7 -Au g 9 -Au g 11-Au g 13 -Au g 15 -Au g 17 -Au g 19 -Au g 2 1-Au g 2 3 -

Au g

2 5 -

Au g

2 7 -

Au g

2 9 -

Au g

3 1-Au g 2 -S e p 4 -S e p

Date

Ion

Ch

am

ber C

urren

t/p

A

Event and date 11

LSUF 24th January 2006

• Six points in a row steadily

increasing or decreasing.

Suggests that the

measurements are shifting

with time; needs investigation,

since the action and warning

limits will eventually be

exceeded

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source

Measurement

50 .3150

50 .3200

50 .3250

50 .3300

50 .3350

50 .3400

50 .3450

50 .3500

50 .3550

50 .3600

50 .3650

1-Aug 3-Aug 5-Aug 7-Aug 9-Aug 11-Aug 13-Aug 15-Aug 17-Aug 19-Aug 21-Aug 23-

Aug

25-

Aug

27-

Aug

29-

Aug

31-Aug 2-S e p 4 -S e p

Date

Ion

Ch

am

ber C

urren

t/p

A

Event and date 12

LSUF 24th January 2006

• Fourteen points in a row

alternating up and down.

Suggests a cyclical change in

conditions; may be due to two

different operators

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source

Measurement

50 .3150

50 .3200

50 .3250

50 .3300

50 .3350

50 .3400

50 .3450

50 .3500

50 .3550

50 .3600

50 .3650

1-Aug 3-Aug 5-Aug 7-Aug 9-Aug 11-Aug 13-Aug 15-Aug 17-Aug 19 -Aug 21-Aug 23-

Aug

25-

Aug

27-

Aug

29-

Aug

31-Aug 2-S e p 4 -S e p

Date

Ion

Ch

am

ber C

urren

t/p

A

Event and date 13

LSUF 24th January 2006

• Four out of five points in a

row exceeding the information

level. This may indicate a

mean shift

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source

Measurement

5 0 .3 15 0

5 0 .3 2 0 0

5 0 .3 2 5 0

5 0 .3 3 0 0

5 0 .3 3 5 0

5 0 .3 4 0 0

5 0 .3 4 5 0

5 0 .3 5 0 0

5 0 .3 5 5 0

5 0 .3 6 0 0

5 0 .3 6 5 0

1-Au g 3 -Au g 5 -Au g 7 -Au g 9 -Au g 11-Au g 13 -Au g 15 -Au g 17 -Au g 19 -Au g 2 1-Au g 2 3 -

Au g

2 5 -

Au g

2 7 -

Au g

2 9 -

Au g

3 1-Au g 2 -S e p 4 -S e p

Date

Ion

Ch

am

ber C

urren

t/p

A

Event and date 14

LSUF 24th January 2006

• Fifteen points in a row

between the lower and upper

information levels. This may

arise after the procedure has

been operated for some time

and may require the limits to

be re-set

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source

Measurement

50 .3150

50 .3200

50 .3250

50 .3300

50 .3350

50 .3400

50 .3450

50 .3500

50 .3550

50 .3600

50 .3650

1-Aug 3-Aug 5-Aug 7-Aug 9-Aug 11-Aug 13-Aug 15-Aug 17-Aug 19-Aug 21-Aug 23-

Aug

25-

Aug

27-

Aug

29-

Aug

31-Aug 2-S e p 4 -S e p

Date

Ion

Ch

am

ber C

urren

t/p

A

Event and date 15

LSUF 24th January 2006

• Eight points in a row either

side of the mean with none

between the upper and lower

information levels. This may

indicate a mean shift.

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source

Measurement

5 0 .3 15 0

5 0 .3 2 0 0

5 0 .3 2 5 0

5 0 .3 3 0 0

5 0 .3 3 5 0

5 0 .3 4 0 0

5 0 .3 4 5 0

5 0 .3 5 0 0

5 0 .3 5 5 0

5 0 .3 6 0 0

5 0 .3 6 5 0

1-Au g 3 -Au g 5 -Au g 7 -Au g 9 -Au g 11-Au g 13 -Au g 15 -Au g 17 -Au g 19 -Au g 2 1-Au g 2 3 -

Au g

2 5 -

Au g

2 7 -

Au g

2 9 -

Au g

3 1-Au g 2 -S e p 4 -S e p

Date

Ion

Ch

am

ber C

urren

t/p

A

Event and date 16

LSUF 24th January 2006

In practice.

• Use software to record and present the data

• Require operators to manually carry out a check (use a paper-based check

list) for the above conditions

• Carry out investigations as necessary and institute improvement actions

However, it is incorrect simply to:

• Repeat a measurement and find the chart to be ‘in control’

• Reset the chart parameters without at least seeking a reason

• Rely on the software to do it for you

• Rely on manufacturer’s software

Event and date 17

LSUF 24th January 2006

In addition:

• Use a CuSum Chart:

c1 = x1 – X

c2 = c1 + x2 – X

c3 = c2 + x3 – X

.

.

.

ci = ci-1 + xi – X

• Deviation from control become clear very rapidly

Event and date 18

LSUF 24th January 2006

Shewhart Value Control Chart for TriCarb

64.3

64.5

64.7

64.9

65.1

65.3

65.5

65.7

65.9

66.1

30-Ju

n-03

19-A

ug-0

3

08-O

ct-0

3

27-N

ov-0

3

16-Ja

n-04

06-M

ar-0

4

25-A

pr-0

4

14-Ju

n-04

03-A

ug-0

4

22-Sep

-04

11-N

ov-0

4

31-D

ec-0

4

19-Feb

-05

10-A

pr-0

5

30-M

ay-0

5

19-Ju

l-05

Date

Tri

tiu

m c

ou

nti

ng

eff

icie

ncy

(%

)

Ev

ent an

d d

ate1

9

LS

UF

24

thJ

an

ua

ry 2

006

Cu

Su

m fo

r Tritiu

m e

fficie

ncy

-25.0

000

-20.0

000

-15.0

000

-10.0

000

-5.0

000

0.0

000

5.0

000

30-Jun-03

19-Aug-03

08-Oct-03

27-Nov-03

16-Jan-04

06-Mar-04

25-Apr-04

14-Jun-04

03-Aug-04

22-Sep-04

11-Nov-04

31-Dec-04

19-Feb-05

10-Apr-05

30-May-05

19-Jul-05

Date

CuSum Tritium efficiency

Event and date 20

LSUF 24th January 2006

Tritium Check Source for LSC

• Shewhart Chart gives first warning on 6th November – 9 points in a row,

one side of the mean – (and many thereafter)

• CuSum Chart gives first warning around 1st November – problem is clear

from this point on

• On-board Q/C software gave first fault message on 30th January – ‘3σ’

limit violation

Event and date 21

LSUF 24th January 2006

Shewhart Value Control Chart for TriCarb

8.600

9.100

9.600

10.100

10.600

11.100

11.600

12.100

12.600

13.100

13.600

30-Ju

n-03

19-A

ug-0

3

08-O

ct-0

3

27-N

ov-0

3

16-Ja

n-04

06-M

ar-0

4

25-A

pr-0

4

14-Ju

n-04

03-A

ug-0

4

22-Sep

-04

11-N

ov-0

4

31-D

ec-0

4

19-Feb

-05

10-A

pr-0

5

30-M

ay-0

5

19-Ju

l-05

Date

Tri

tiu

m b

ack

gro

un

d

(co

un

ts/s

eco

nd

)

Ev

ent an

d d

ate2

2

LS

UF

24

thJ

an

ua

ry 2

006

Cu

Su

m fo

r T

ritium

Ba

ck

gro

un

d

-10.0

000

0.0

000

10.0

000

20.0

000

30.0

000

40.0

000

50.0

000

60.0

000

30-Jun-03

19-Aug-03

08-Oct-03

27-Nov-03

16-Jan-04

06-Mar-04

25-Apr-04

14-Jun-04

03-Aug-04

22-Sep-04

11-Nov-04

31-Dec-04

19-Feb-05

10-Apr-05

30-May-05

19-Jul-05

Date

CuSum Tritium Background

Event and date 23

LSUF 24th January 2006

Tritium Background for LSC

• Shewhart Chart gives first warning on 15th December – action limit

violation – (event on 8th October assigned to 99Mo/99mTc source in

vicinity)

• CuSum Chart gives first warning around 18th December – problem is clear

from this point on (event on 8th October does not affect CuSum)

• On-board Q/C software gave first fault message on 15th February – ‘3σ’

limit violation

Event and date 24

LSUF 24th January 2006

Analysis of data

• Engineer called out in February

• Deteriorating Photo-Multiplier tube identified

• 14C check source did not return comparable data, apart from on action limit

violation that did not have an assignable cause

• Subsequent measurements with other systems indicate general instability

of 3H measurements – observed at other establishments

• Further investigations reveal ‘hidden’ parameters that low energy β-

emitters are very sensitive to (such as DBB)

• Essential to know the system well before use

Event and date 25

LSUF 24th January 2006

Conclusions

• Full implementation of ISO 8258:1991 allows operators to far better

understand measurement systems

• Potential faults can be trapped by preventative actions rather than

corrective actions

• Simple to use and interpret

• Applicable to similar systems – α-spectrometers, Germanium γ-

spectrometers, NaI(Tl) γ-spectrometers, ionisation chambers &c

• Customer calibrations and source generation can be carried out with

greater confidence

Questions and Feedback