Equivalence Tests: Principles and the PM · PDF fileequivalence tests: principles and the ......

Equivalence Tests:Principles and the PM Issue

Ulrich Pfeffer, Jutta Geiger, Thorsten Zang

Conference: Current and Future Air Qualiy Monitoring14 – 15 December 2010, London

RSC aamg Conference London, 14 - 15 December 2010ulrich.pfeffer@lanuv.nrw.de2

• Monitoring requirements

• Standardization

• Reference and equivalent methods

• Basic principles of equivalence testing

• Comparisons NRW – national – international

• Conclusions, outlook and future demands

Overview

• Information of the public (Annex XVI; at least daily; hourly whereever practicable)

• Definition of reference methods (Annex VI):– PM10: EN 12341 (under revision; merge with EN 14907)– PM2,5: EN 14907– Use of different methods if equivalent

Requirements of Directive 2008/50/EC

• Gravimetry– LVS: „Kleinfiltergeräte“ (single filter, SEQ, Partisol)

– HVS: Digitel DHA-80 (filter changer)

• AMS– Beta-Absorption

– Oszillating Microbalance (TEOM, TEOM-FDMS)

– Optical Instruments (light scattering), particle counters

? AMS equivalent to reference method?

Instruments in European Practise

What is an

equivalent method?

EU Guidance on Equivalence

New version: January 2010

Definition of Equivalence

• Laboratory test only relevant for modifications of RM

• Series of field tests (candidate vs reference method)

• Test conditions must be representative for practical use

• Minimum 4 comparison at a minimum of 2 sites

Test Programme 3 for PM (1)

• Between-sampler uncertainty: RM 2 µg/m³, CM 2,5 µg/m³

• Split data sets at 30 µg/m³ for PM10, at 18 µg/m³ for PM2.5

• Perform orthogonal regression

• Perform calibration if necessary

• Calculate uncertainty for candidate method

• Compare results with data quality objective of Directive

Test Programme 3 for PM (2)

Basic Equations

(random term)² (bias)²

PM Data Evaluation

PM10-Comparison LVS3 - Digitel DHA-80

REGRESSION OUTPUT slope b 0,986 uncertainty of b 0,004 sign intercept a -0,06 uncertainty of a 0,16 number of data pairs 790 EQUIVALENCE TEST RESULTS random term 2,4 µg/m³ bias at LV -0,8 µg/m³ combined uncertainty 2,5 µg/m³ relative uncertainty 5,0% pass ref sampler uncertainty 0,67 µg/m³ limit value 50 µg/m³

Annex C of GDE

2003, February – September; approx. 100 – 120 data pairs

LUA Materialien No. 66, 175 pages (2005) www.lanuv.nrw.de

Wiesbaden Experiment of German Networks

PM10 Reference Method

Data Quality

Expanded Uncertainties of Gravimetric Methods - Campaign Wiesbaden(EU Equivalence Test)

5 6 7 8 9 10 11 12 13 14 15 16

Digitel SEQ

Reference Method: Single Filter LVS, ubs = 0.8 µg/m³

JRC Scientific and Technical Reports,EUR 23216 EN – 2008, pp. 21-22, pp. 54-60

Continuous Methods PM10

Equivalence Tests

Expanded Uncertainties of Corrected PM10-Data - Year 2005(EU Equivalence Test)

IM2 All

ALS All

TEOM SES FH 62 IR

Uncertainty of Corrected Data (NRW)

• ‚Synchronised Hybrid Ambient Real-time ParticulateMonitor‘

• Combination of light scattering photometry and beta radiation attenuation

Thermo Model 5030 SHARP Monitor

Parameter 4 Sites in Cologne / Brühl / Titz / CologneType Urban background / gravel mining / rural / trafficPeriod 2006: Jan-May / May-Jul / Jul-Sep / Sep-NovReference Method Leckel LVS3 or SEQ47/50 / QFFNr of Data Pairs 67 / 55 / 52 / 54Mean of RM Data (µg/m³) 30,8 / 26,2 / 25,1 / 25,7Range of RM Data (µg/m³) 9-120/ 8-79 / 9-83 / 13-51Between-RM Uncertainty (µg/m³) 0,76 / 0,59 / 0,76 / 1,27Between-CM Uncertainty (µg/m³) 0,57 / 1,20 / 1,11 / 1,23Relation RM – CM CM=1,00*RM–0,29 / CM=1,00*RM–1,21

CM=0,98*RM–0,53 / CM=1,04*RM–2,82 Expanded Uncertainty CM 12,8% / 12,0% / 14,0% / 9,3%

After Correction (S-I) 13,5% / 12,4% / 16,8% / 11,8%

TÜV Results SHARP Monitor PM10

Parameter 4 Sites in Cologne / Brühl / Titz / CologneType Urban background / gravel mining / rural / trafficPeriod 2006: Jan-May / May-Jul / Jul-Sep / Sep-NovReference Method Leckel LVS3 or SEQ47/50 / QFFNr of Data Pairs 54 / 40 / 47 / 40Mean of RM Data (µg/m³) 24,0 / 15,0 / 13,6 / 16,5Range of RM Data (µg/m³) 7-98 / 3-43 / 4-37 / 5-40Between-RM Uncertainty (µg/m³) 1,63 / 0,73 / 0,49 / 0,58Between-CM Uncertainty (µg/m³) 0,58 / 1,17 / 0,87 / 0,68Relation RM – CM CM=0,92*RM+1,04 / CM=1,08*RM+0,23

CM=1,10*RM+0,05 / CM=0,93*RM–0,05 Expanded Uncertainty CM *) 20,4% / 21,4% / 23,3% / 19,6%

After Correction (S-I) *) 20,6% / 15,7% / 15,5% / 17,7%

TÜV Results SHARP Monitor PM2.5

*) related to 35 µg/m³

UK Equivalence Programme

Test of FDMS

PM10 - LANUV• 2 TEOM FDMS (Type B)

• 2 Digitel DHA-80 HVS

• July 2005 – Jan. 2007

• ≈ 6 - 7 months

Two Sites in Duisburg (VDUI / DUBR)

PM10 Results TEOM FDMS in DuisburgEU Equivalence Test - PM10

Stations VDUI and DUBR (375 Data Pairs)

0 20 40 60 80 100 120 140 160

Digitel DHA-80

Orthogonal Regressiony = 0,98x - 1,28

Expanded uncertainty: 15,7%

Measurement periodsVDUI: 22.07.05 - 28.02.06 (n = 202)DUBR: 14.07.06 - 31.01.07 (n = 173)

• After promising results for PM10 the network in

NRW was equipped with FDMS also for PM2.5

• Firstly only one station with co-located

reference method

Experiences with TEOM FDMS – PM2.5

TEOM FDMS (C) PM2,5 – LANUV ResultsVergleich der PM2,5 Konzentration an der Station DMD2

1.05. bis 18.11.2008

y = 0,98x + 8,85R2 = 0,93

0 10 20 30 40 50 60 70 80 90

SEQ PM2,5 [µg/m³]

TEOM FDMS PM 2,5Linear (TEOM FDMS PM 2,5)

PM2.5 – Station DMD2 - 1.05.2008 – 18.11.2008

TEOM FDMS (C) PM2,5 – LANUV ResultsVergleich der PM2,5-Konzentrationan der Station WULA

22.08. bis 23.11.2008

y = 1,11x + 13,29R2 = 0,84

0 10 20 30 40 50 60

SEQ PM2,5 [µg/m³]

TEOM FDMS PM2,5AusreisserLinear (TEOM FDMS PM2,5)Outlier

PM2.5 – Station WULA - 22.08.2008 – 23.11.2008

• All measurements at AEI stationswith TEOM FDMS for PM2.5

were stopped in the LUQS network in NRW

• 9 AEI stations in NRW operatedwith SEQ gravimetrically only

• Extension of PM2.5 networkfor non-AEI stations postponed

Conclusions – Decisions in NRW

TEOM FDMS

Equivalence TestingResults from Other Countries

Results from Switzerland (PM10)

Results from Sweden (PM10)

Equivalence Testing

Recent Data 2010

Preliminary Data January – October 2010

• Reference method:– PM10: Digitel DHA-80-Geräte (quartz fibre)– PM2.5: Leckel SEQ 47/50 (glass fibre)

• TEOM FDMS– 7 sites for PM10

– 3 sites for PM2.5 (replacement of dryers during spring)

• SHARP– 5 sites or PM10

– 2 sites for PM2.5

• BAM– 1 test station for PM2.5 together with SHARP and RM

LANUV Equivalence Tests 2010

Spreadsheet RIVM_PM_equivalence_v2.6

Calibration setting: Slope through origin

Results for TEOM FDMS

19,5; 23,4; 15,3; 14,6; 12,3; 9,9; 13,8

18,515,51,0PM10

20,3; 18,1; 12,820,316,91,0PM2.5 (refurbisheddryer)

21,0; 14,4; 34,924,022,80,93PM2.5 (old dryer)

Expanded relative uncertainty for thesubsetindividual site

Expanded relative uncertainty for thesubsetconcentrationsPM10 > 30 µg/m3

PM2.5 > 18 µg/m3

Expanded relative uncertainty for thefull data set

Calibrationfactor(slope b)

19,5; 23,4; 15,3; 14,6; 12,3; 9,9; 13,8

18,515,51,0PM10

20,3; 18,1; 12,820,316,91,0PM2.5 (refurbisheddryer)

21,0; 14,4; 34,924,022,80,93PM2.5 (old dryer)

PM2.5 > 18 µg/m3

PM10: no calibration needed; but two sites with significantly different factors (0,83 and 1,11, respectively)

PM2.5: with the refurbished dryer it is possible to meet the data quality objectives; but thefactors at the different sites vary quite a bit (0,93; 1.02 and 1,11)

LANUV Equivalence Tests 2010PM2.5 - Station WULA 01.01.2010 - 23.03.2010 old dryer

y = 1,03x + 6,08R2 = 0,97

0 20 40 60 80 100 120 140SEQ PM2.5 [µg/m3]

LANUV Equivalence Tests 2010PM2.5 - Station WULA 24.03.2010 - 14.10.2010 refurbished dryer

y = 1,06x - 1,38R2 = 0,93

0 20 40 60 80 100 120 140SEQ PM2.5 [µg/m3]

Results for SHARP and BAM

SHARP: pass the equivalence test for PM10 and PM2.5; the calibration factor isidentical for both PM fractions and very similar for all sites

BAM: could pass the test even without calibration factor, but more data is needed to evaluate this method

13,1; 17,3; 14,6; 13,4; 15,1

18,314,31,1PM10(SHARP)

19,620,619,61,03PM2.5 (BAM)

13,5; 12,515,413,31,1PM2.5 (SHARP)

PM2.5 > 18 µg/m3

13,1; 17,3; 14,6; 13,4; 15,1

18,314,31,1PM10(SHARP)

19,620,619,61,03PM2.5 (BAM)

13,5; 12,515,413,31,1PM2.5 (SHARP)

PM2.5 > 18 µg/m3

2008-2009

LANUV Fachbericht No. 26, 136 pages (2010) www.lanuv.nrw.de

Expanded Uncertainties of Gravimetric Samplers for PM2,5

19,6%16,4%

9,0%12,1%

14,7%11,2%10,6%

19,5%23,1%

18,8%22,7%

15,1%12,3%

14,6%16,9%

42,4%38,8%

L1 L2 D1 D2 D3 D4 D5 D6 D7 D8 S1 S2 S3 S4 S5 S6 S7 R3 S8 S9S10

Sampler

Urel (95%)Urel after calibration

D2 = D8 without correction of field blanks

S10 = S2, GF10 only

Expanded Uncertainties of AMS for PM2,5

16,9% 17,3%

23,6%22,3%24,6% 22,4%

22,0%18,6%

K1 K2 K3 K4 K5 K6 K7 K8 K9 K10 K11

Raw Values

After Calibration

K1 + K3 = TEOM FDMS

K10 = Raw Values of K8

K11 = K7 with Corr. Factor

• HiVol sampler (Digitel) is equivalent to the reference method (Uexp = 10 %)

• Gravimetric methods may generally meet the DQO

• In most cases also data of AMS can meet the DQO

• Calibration factors are needed for some AMSResults may vary dramatically from site to siteTransfer of results to unknown sites is critical

Conclusions (1)

What is urgently needed?Demonstration of equivalence for all AMSRigid regime for QA/QC for all PM measurementsRevision / harmonisation of EN 12341 and EN 14907New standard for AMSRe-consider some procedures of GDEShare results of DoESet up validated data base for results of DoE

Conclusions (2)

Equivalence Tests:Principles and the PM Issue

Ulrich Pfeffer, Jutta Geiger, Thorsten Zang

Conference: Current and Future Air Qualiy Monitoring14 – 15 December 2010, London

Thank you for your attention!

Equivalence Tests: Principles and the PM · PDF fileequivalence tests: principles and the ......

Documents

RECOMENZAR Perdí un juguete que me acompañó en mi infancia...Pero gané el recuerdo del amor de quien me hizo ese regalo. VMGR

Equivalence Considerations for Orally Inhaled Products for ... · PDF fileEquivalence Considerations for Orally Inhaled Products for Local Action—ISAM/IPAC-RS European Workshop Report

) MANUAL - Marines.mil 3500.53A.pdf · This Manual is applicable to the Marine Corps Total Force. 6. Certification. Reviewed and approved this date. ... VMGR CORE MODEL …

News & Views - Wainhouse · with guidance from Polycom, the H.264-based switching infrastructure consists of a video MCU (VMCU) and a video manager (VMGR), the latter of which manages

Equivalence Checking of TTCN-3 Test Case Behavior · PDF fileEquivalence Checking of TTCN-3 Test Case Behavior Filip Makedonski am Institut f ur Informatik Gruppe Softwaretechnik f

STEEL ROLLER SHUTTER DOORS - Brisbane - KRGS · PDF fileSTEEL ROLLER SHUTTER DOORS. KRGS Steel Roller Shutters are highly versatile that deliver strength and versatility. CURTAIN Constructed

CLEARSTACK BI FOLDING DOOR - KRGS Doors€¦ · CLEARSTACK BI FOLDING DOOR SYDNEY 8 Enterprise Circuit Prestons NSW 2170 Phone 02 9608 9888 1800 987 822 Fax 02 9608 9333 Email sales@krgsdoors.com.au

Equivalence of ISPF and RDz LPEX Editor Functions - IBM · PDF fileEquivalence of ISPF and RDz LPEX ... 1 Equivalence of ISPF and RDz LPEX Editor Functions ... The “LPEX profile”

きんたい Kintai...2011/07/01 · 2011年4月1日/Apr. 1, 2011 空気は冷たく、空は暗かった。時間は 21時になろうとしていた。3月24 日、海兵第152空中給油飛行隊(VMGR-152)所属の8機のKC-130Jのうち、1機

ALUMINIUM SHUTTER - SERIES III · Supply and install Aluminium Shutters Series III manufactured by KRGS Roller Grilles and Shutters 1800 897 822. A DOOR DESIGN WITH YOU IN MIND ALUMINIUM

CompDoc - Division of Waste Managementwaste.ky.gov/RLA/recycling/Documents/KRGS Packet.pdf · Recycle Right Know Before You Throw The Tricky These items seem like they could be disposed

CMS Issues. Background – RAL Infrastructure TM Nsd Xrd- mgr TM Nsd Xrd- mgr TM Rhd stagerd TGW Rhd stagerd TGW Cupv Vmgr Vdqm nsd Cupv Vmgr Vdqm nsd Cupv

19MM GRILLE - KRGS Doors

KRGS - finansdanmark.dkfinansdanmark.dk/media/19274/krgs-skema.pdfSkema Penge Real FMS IFS Res K K H H Resultatoplysninger Bal K K H H Balanceoplysninger ... Sgb K (1-3,6) Grunde og

Equivalence of computerized versus paper-and-pencil ... · PDF fileEquivalence of computerized versus paper ... Ladyshewsky, 2014; ... The information literacy instruction assessment

PROFUNDIZAR BRECHAS vmgr oct 2005 PROFUNDIZAR BRECHAS: LAS POLÍTICAS DE FINANCIAMIENTO DE LA EDUCACIÓN SUPERIOR victor manuel gonzález romero I nstituto

Equivalence Hypothesis Testing in Experimental · PDF fileEquivalence Hypothesis Testing in Experimental Software Engineering ... (EHT) into the Empirical ... Equivalence Hypothesis

CLEARGLIDE BI FOLDING DOOR...The KRGS Clearglide Bi Folding Door is a most versatile innovative door and wall system free from mullions and restrictive pillars, providing the ultimate

Lincoln Raider Mobil… · Web view* All VMGR history, emblems, patches, and photos are used with permission from the command