ITRI Industrial Technology Research Institute 2014 NCSL International Workshop and Symposium 1 Evaluation of Proficiency Testing Results with a Drifting

ITRIIndustrial TechnologyResearch Institute

2014 NCSL International Workshop and Symposium 1

Evaluation of Proficiency Testing Results with a Drifting Artifact –

An Example of Standard Resistor

Chen-Yun Hung

Center for Measurement Standards (CMS),Industrial Technology Research Institute (ITRI)

TaiwanJuly 31, 2014


Contents Introduction Stability Testing Performance Evaluation Confidentiality Conclusions


Introduction Proficiency testing (PT) is an effective way to verify a

laboratory’s measurement capability Through PT activities, participants can understand the

differences between their measurement capabilities and those of other laboratories

PT often leads to improvement in measurement competence and quality control

In Taiwan, CMS/ITRI organizes the PT which focuses on calibration laboratories


Why CMS/ITRI? Industrial Technology Research Institute

The biggest non-profit organization in Taiwan Half budget supported by the government About 6,000 employees in most industrial fields

Center for Measurement Standards CMS operates the National Measurement Laboratory NML is the NMI in Taiwan (like a small NIST) Maintain 120 national measurement standards Quality Engineering Department organizes quality

management system in CMS QED is also a PT Provider operated in accordance with ISO

17043, accredited by TAF (P002)

2014 NCSL International Workshop and Symposium

Certificate of accreditation

5

Accreditation NumberP002


Introduction (Cont.) This paper will present a PT scheme for standard

resistors to demonstrate the evaluation of PT results with a drifting artifact A PT of standard resistors numbered “PT2012-KF01” was

performed from October 2012 to February 2013 A total of 16 laboratories participated Two standard resistors with nominal values of 1 Ω and 10 kΩ

were chosen as PT items

6

Nominal Value Manufacturer Model Serial No.

1 Ω iET SRL-1 B2-942510510 kΩ iET SRL-10k B2-9425117


The flowchart of the PT scheme

7

Start

PT Parameter

PT Item

Stability Testing

PT Plan

PT Instruction

PT Orientation

Sample Monitoring

Data Retrieval

Data Analysis

Final Report

End


Stability Testing Ensure the PT items will not undergo any significant

changes during the PT process If not, the stability should be quantified and regarded

as an additional component of the uncertainty The criterion for allowing stability in PT2012-KF01

whereusta is the standard uncertainty of stability testingulab, min is the minimum standard uncertainty of the participants’ resultsuref is the standard uncertainty of the reference laboratory’s result

8

2ref

2min lab,sta 3.0 uuu

Based on the guidelines for limiting the uncertainty of the assigned value in

ISO 13528


Stability Testing (Cont.) The criterion for allowing stability in PT2012-KF01

If the criterion is metThe stability uncertainty will not affect the evaluation of the

participants’ performance

If the criterion is not metThe stability should be regarded as an additional component of

the uncertainty associated with the assigned value

More than one assigned values would be provided to limit the effect of the uncertainty on the performance evaluation

In some cases, PT items would be replaced based on technical expert judgment

9

2ref

2min lab,sta 3.0 uuu


10.000749

10.000750

10.000751

10.000752

10.000753

10.000754

10.000755

20

12

/8/1

20

12

/8/3

20

12

/8/5

20

12

/8/7

20

12

/8/9

20

12

/8/1

1

20

12

/8/1

3

20

12

/8/1

5

20

12

/8/1

7

20

12

/8/1

9

20

12

/8/2

1

20

12

/8/2

3

20

12

/8/2

5

20

12

/8/2

7

20

12

/8/2

9

20

12

/8/3

1

20

12

/9/2

20

12

/9/4

20

12

/9/6

20

12

/9/8

20

12

/9/1

0

20

12

/9/1

2

20

12

/9/1

4

20

12

/9/1

6

20

12

/9/1

8

20

12

/9/2

0

20

12

/9/2

2

20

12

/9/2

4

20

12

/9/2

6

20

12

/9/2

8

20

12

/9/3

0

Measured Value(kΩ)

Date

Nominal Value(10 kΩ)

Measured ValueMax = 10.000753 kΩmin = 10.000751 kΩ

0.9991778

0.9991782

0.9991786

0.9991790

0.9991794

0.9991798

20

12

/8/1

20

12

/8/3

20

12

/8/5

20

12

/8/7

20

12

/8/9

20

12

/8/1

1

20

12

/8/1

3

20

12

/8/1

5

20

12

/8/1

7

20

12

/8/1

9

20

12

/8/2

1

20

12

/8/2

3

20

12

/8/2

5

20

12

/8/2

7

20

12

/8/2

9

20

12

/8/3

1

20

12

/9/2

20

12

/9/4

20

12

/9/6

20

12

/9/8

20

12

/9/1

0

20

12

/9/1

2

20

12

/9/1

4

20

12

/9/1

6

20

12

/9/1

8

20

12

/9/2

0

20

12

/9/2

2

20

12

/9/2

4

20

12

/9/2

6

20

12

/9/2

8

20

12

/9/3

0

Measured Value(Ω)

Date

Nominal Value(1 Ω)

Measured ValueMax = 0.9991793 Ωmin = 0.9991784 Ω

Stability Testing for PT2012-KF01 Two months for stability testing before the PT was initiated

10

Uncertainties

Nominal Valueusta ulab, min uref

1 Ω 0.00000020 Ω 0.000008 Ω 0.00000015 Ω10 kΩ 0.0000004 kΩ 0.00005 kΩ 0.0000015 kΩ

1 Ω 10 kΩ

Criterion for Allowing Stability

PASS


Assigned Value Calibration values reported by NMI were used as

assigned values in this PT scheme National Measurement Laboratory (NML)

PT items were calibrated for three times by NML to ensure their stability If the differences between the three calibration values are all

within the expanded uncertaintyAssigned Value: Median of three calibration values

If notAssigned Value: Average of the calibration values


0.9991784

0.9991788

0.9991792

0.9991796

0.9991800

0.9991804

0.9991808

0.9991812

2012/10/01E120519A

2012/12/07E120641A

2013/03/08E130072A

Measured Value(Ω)

Date/Report No.

Nominal Value(1 Ω)

12

Assigned Value for PT2012-KF01

Assigned Value (X): Average of the calibration values Two assigned values were provided

One was the average of the first and second calibration values

One was the average of the second and third calibration values

Expanded Uncertainty of Assigned Value (Uref) Combine the reported uncertainty of the reference laboratory and

the uncertainty of the difference between two calibration values

1 Ω

The differences between the three calibration values were larger than the expanded uncertainty


10.000744

10.000746

10.000748

10.000750

10.000752

10.000754

10.000756

10.000758

10.000760

2012/10/01E120520A

2012/12/07E120642A

2013/03/08E130073A

Measured Value(kΩ)

Date/Report No.


13

Assigned Value for PT2012-KF01 (Cont.)

Assigned Value (X): Median of three calibration values Expanded Uncertainty of Assigned Value (Uref)

The reported expanded uncertainty of the median

10 kΩ

The differences between the three calibration values were smaller than the expanded uncertainty

Group I II

Nominal Value X Uref X Uref

1 Ω 0.9991795 Ω 0.0000004 Ω 0.9991801 Ω 0.0000005 Ω

10 kΩ 10.000752 kΩ 0.000003 kΩ 10.000752 kΩ 0.000003 kΩ


Performance Statistic En was chosen as the performance statistic

wherex is the participant’s resultX is the assigned valueUlab is the expanded uncertainty of the participant’s resultUref is the expanded uncertainty of the reference laboratory’s assigned value

Criteria for performance evaluation |En| 1, “satisfactory” performance

|En| > 1, “unsatisfactory” performance

2ref

2lab

n

)(

UU

XxE


PT results of the 1 Ω standard resistor in group INominal Value(1 Ω)

0.9950

0.9960

0.9970

0.9980

0.9990

1.0000

1.0010

1.0020

1.0030

Ref Lab-I C D E G K L O

Lab Code

Measured Value(Ω)

Measured Value

Assigned Value = 0.9991795 Ω

15

PT Results for PT2012-KF01

|En| > 1

(x – X) & Ulabis large


PT results of the 1 Ω standard resistor in group II

16

PT Results for PT2012-KF01 (Cont.)

Nominal Value(1 Ω)

0.9984

0.9986

0.9988

0.9990

0.9992

0.9994

0.9996

0.9998

1.0000

Ref Lab-

II

A B F H I J M N P

Lab Code

Measured Value(Ω)

Measured Value

Assigned Value = 0.9991801 Ω

Note: The value of Lab J is not available.


PT results of the 10 kΩ standard resistor

17

PT Results for PT2012-KF01 (Cont.)


9.9960

9.9970

9.9980

9.9990

10.0000

10.0010

10.0020

10.0030

10.0040

10.0050

10.0060

Ref A B C D E F G H I J K L M N O P

Lab Code

Measured Value(kΩ)

Measured Value

Assigned Value = 10.000752 kΩ

• CMS/ITRI only provided the PT results and suggestions

• Did not judge whether the participants were qualified laboratories

|En| > 1


Confidentiality Each laboratory is given a code that is mailed to the laboratory

in a sealed envelope Only knows its own laboratory code

The confidentiality should especially be considered when the assigned value of a drifting artifact is determined More than one assigned values are provided

The number of assigned values should be controlled to avoid fewer participants in one group

Using linear regression to predict the assigned value on a given dayThe transfer schedule is not publicly available

PT items should be transferred to participants by PT provider

18

ISO/IEC 17043 4.10.1The identity of participants shall be confidential and known only to persons involved in the operation of the PT scheme


Conclusions CMS/ITRI has performed PTs in Taiwan for more than 10

years with the focus on calibration laboratories Ensuring the stability of artifacts always has been given a

high priority CMS/ITRI shares a PT of standard resistors with quantified

criterion for stability testing using statistical methods Most of standard resistor calibration laboratories in Taiwan

have good measurement competence Although the PT items drift over time, it will not affect the

evaluation of the participants’ performance by a well-designed method

ITRIIndustrial TechnologyResearch Institute


Thanks for your attention

Chen-Yun HungEmail: [email protected]

Documents

ITRI Industrial Technology Research Institute 2014 NCSL International Workshop and Symposium 1 Evaluation of Proficiency Testing Results with a Drifting