Upload
cleenltd
View
344
Download
3
Embed Size (px)
Citation preview
VARIOGRAM-DERIVED MEASURES FOR QC PURPOSES
Markku OhenojaControl Engineering group
University of Oulu
1
05/02/2023Faculty of Technology / Control Engineering / Markku Ohenoja
02.05.2023
2
Faculty of Technology / Control Engineering / Markku [email protected]
Petersen, L., Minkkinen, P. & Esbensen, K.H. 2005, "Representative sampling for reliable data analysis: Theory of Sampling", Chemometrics and Intelligent Laboratory Systems, vol. 77, no. 1–2, pp. 261-277.
Time
Mea
s.
https://s-media-cache-ak0.pinimg.com/236x/64/46/7f/64467fa3382ac08d567d36b6aef0513b.jpg
BACKGROUND
• All measurements retain some amount of uncertainty, but also sampling errors may affect on the result
• Utilization of different measurements collected with very different sampling rates requires evaluation of their information content
• Environmental measurements are often periodic, sparsely collected and from various sources
• Variographical analysis used for evaluating sampling errors and information content of the measurement
02.05.2023
3
Faculty of Technology / Control Engineering / Markku [email protected]
OUTLINE
• What is Variogram and how it is calculated?
• Variogram-derived measures
• Examples within MMEA
02.05.2023
4
Faculty of Technology / Control Engineering / Markku [email protected]
VARIOGRAM
• Tool for empirical estimation of sampling errors incl. analytical error
• Enables optimizing the sampling strategy with respect to variance of the sampling error and number of samples takes
• Provides an estimate of the standard error of the lot mean and the minimum possible error (MPE) of sampling
02.05.2023
5
Faculty of Technology / Control Engineering / Markku [email protected]
Semi-variogramChrono-variogram
Variographical analysis
GeostatisticsKriging Variography Chronostatistics
VARIOGRAM
• Collection of the data• At least 30 samples with systematic sampling• 1/5 smaller sampling interval than routine samples• Flowrate/sample weight should be included
• Calculation of the heterogeneity of the data• Calculation of the experimental variogram v(j)
• Relationship between the samples and the lag distance j
• Estimation of the intercept v(0) (=MPE)• Graphically, separate experiment…
• Auxiliary functions for comparing sampling strategies• Point-to-point calculation, algebraic modeling…
02.05.2023
6
Faculty of Technology / Control Engineering / Markku [email protected]
ℎ𝑛 = 𝑎𝑛 −𝑎𝐿𝑎𝐿 ∙𝑀𝑛𝑀𝑛തതതത
𝑣ሺ𝑗ሻ= 12(𝑁− 𝑗) ൫ℎ𝑛+𝑗 −ℎ𝑗൯2𝑁/2𝑛=1 ≈ 12(𝑁− 𝑗)𝑎𝐿2 ൫𝑎𝑛+𝑗 −𝑎𝑗൯2
𝑁/2𝑛=1
VARIOGRAM
02.05.2023
7
Faculty of Technology / Control Engineering / Markku [email protected]
0 10 20 300
5
10
15
20
25
30Variogram of 24h averaged online data
Sampling interval (days)
Rel
ativ
e st
anda
rd d
evia
tion
of th
e sa
mpl
ing
erro
r (%
)
0 10 20 300
5
10
15
20
25
30Variogram of daily sample
Sampling interval (days)
Rel
ativ
e st
anda
rd d
evia
tion
of th
e sa
mpl
ing
erro
r (%
)
VariogramSystematic samplingRandom sampling
VariogramSystematic samplingRandom sampling
σ2, σ
, 2σ,
...
VARIOGRAM
02.05.2023
8
Faculty of Technology / Control Engineering / Markku [email protected]
0 10 20 300
5
10
15
20
25
30Variogram of 24h averaged online data
Sampling interval (days)
Rel
ativ
e st
anda
rd d
evia
tion
of th
e sa
mpl
ing
erro
r (%
)
0 10 20 300
5
10
15
20
25
30Variogram of daily sample
Sampling interval (days)
Rel
ativ
e st
anda
rd d
evia
tion
of th
e sa
mpl
ing
erro
r (%
)
VariogramSystematic samplingRandom sampling
VariogramSystematic samplingRandom sampling
3x
INDICES
02.05.2023
9
Faculty of Technology / Control Engineering / Markku [email protected]
• Variogram-based indices applied for QC and PAT purposes
• Standard error of the mean• MPE/σProcess• v(1)/σProcess
INDICES
02.05.2023
10
Faculty of Technology / Control Engineering / Markku [email protected]
• Variogram-based indices applied for QC and PAT purposes
• Standard error of the mean• MPE/σProcess• v(1)/σProcess
Process stability measure
Bisgaard & Kulahci, Quality Engineering, 17(2), 2005
Drift estimationPaakkunainen et al.,
Chemometrics and Intelligent Laboratory Systems, 88(1),
2007Fault diagnosisKouadri et al., ISA
Transactions, 51(3), 2012Temporal
uncertainty propagation
Jalbert et al., Journal of Hydrology, 397(1-2), 2011
DQOs for control charts
Minnit & Pitard, Journal of SAIMM, 108(2), 2008
STANDARD ERROR OF THE MEAN
• Variance estimate of the sampling attained from variogram
• Standard error of the mean calculated based on variance estimate and number of samples collected during a selected time frame
• Recursive calculation possible for online measurements moving average and its confidence intervals from the selected time frame
02.05.2023
11
Faculty of Technology / Control Engineering / Markku [email protected]
STANDARD ERROR OF THE MEAN
02.05.2023
12
Month 2M 3M 4M 5M HalfYear Year All0
0.5
1
1.5
2
2.5
3
2M
, %
Time frame for the lot mean
Online 17h averageOnline 12h averageOnline 8h averageOnline 6h averageOnline 4h averageOnline data
Month 2M 3M 4M 5M HalfYear Year All0
5
10
15
20
25
30
35
40
2M
, %
Time frame for the lot mean
LaboratoryCalibrated onlineRaw online x 10
Faculty of Technology / Control Engineering / Markku [email protected]
STANDARD ERROR OF THE MEAN
02.05.2023TIEDEKUNTA TIEDEKUNTA / osasto osasto osaston osasto / Etuniminen Sukuniminen-Sukuniminen
13
23-Nov-2009 12-Jan-2010 03-Mar-2010 22-Apr-2010 11-Jun-2010 31-Jul-2010 19-Sep-2010 08-Nov-2010 28-Dec-2010 16-Feb-20110
20
40
6031-Dec-2010
7.341 7.3415 7.342 7.3425 7.343 7.3435 7.344 7.3445 7.345 7.3455x 105
10
15
20
25Lot mean and 2
M (%) for Three day average
7.341 7.3415 7.342 7.3425 7.343 7.3435 7.344 7.3445 7.345 7.3455x 105
0
10
20
30
23-Nov-2009 12-Jan-2010 03-Mar-2010 22-Apr-2010 11-Jun-2010 31-Jul-2010 19-Sep-2010 08-Nov-2010 28-Dec-2010 16-Feb-20115
10
15
20
25
30Lot mean and confidence intervals for Three day average
DATA COMPARISON
• Multiple measurement sources with different sampling rates
• Data harmonization and comparison• Based on MPE• Comparable averaging of the dense data around sparse
samples,• Variographical analysis for whole averaged dense data
mimicking more densely collected laboratory measurements
• Information content evaluation based on v(1)/σProcess
02.05.2023
14
Faculty of Technology / Control Engineering / Markku [email protected]
WHAT SPARSE CANNOT SEE?
02.05.2023
15
0 5 10 15 20 250.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16Variograms for collective samples
Sampling interval
Varia
nce
Variogram, Sparse meas.Variogram, Av. dense meas.
0 5 10 15 20 250
0.1
0.2Variogram of sparse measurement
Varia
nce
Sampling interval
0 200 400 600 800 10000
0.1
0.2Variogram of averaged dense measurement
Sampling interval
Varia
nce
Faculty of Technology / Control Engineering / Markku [email protected]
WHEN DENSE IS NOT REPRESENTATIVE?
02.05.2023TIEDEKUNTA TIEDEKUNTA / osasto osasto osaston osasto / Etuniminen Sukuniminen-Sukuniminen
16
26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-20130
10
20
30
40
Mea
s.
Time series
Dense meas.Sparse meas.
26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-20130
0.5
1
1.5es/P
Inde
x
Dense meas.Sparse meas.
26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-2013-1
-0.5
0
0.5
1Substracted index
Inde
x
SUMMARY
02.05.2023Faculty of Technology / Control Engineering / Markku [email protected]
17
• Variogram can be utilized for1. Sampling error estimation2. Sampling optimization3. Moving average and confidence interval calculation4. Information content evaluation
• Recursive calculation enables e.g. monitoring, filtering, decision making
• Information content evaluation allows comparison of measurement sources