Rudolf Žitný, Ústav procesní a zpracovatelské techniky ČVUT FS 2010

Error analysisStatisticsRegression

Experimental methods E181101 EXM8

EVALUATION OF EXPERIMENTAL DATAEXM8

Distribution of errors. It is assumed that a true value of a quantity is distorted by n-small effects of the same magnitude (positive or negative). Superposition of these effect results to a random error, having binomial distribution. As soon as the number of effects goes to infinity, this distribution reduces to the normal Gauss distribution of errors

)2

exp(2

1)( 2

2

where is the mean quadratic error called standard deviation.

n

iin 1

22 1

Probability, that an error is somewhere within the range <-,> is the integral distribution

)2

(2

)exp(2

22)

2exp(

2

2)()(

2/

0

2

02

2

erf

dttddP

Example P()=0.68 P(3)=0.997

2xe dx

Gauss integral

()d is the probability that an error is within

interval <,+d>

EVALUATION OF EXPERIMENTAL DATAEXM8

Arithmetic average of repeated measurement

n

iixn

x1

1

this is the best estimate of expected value. Standard deviation of single measurement can be estimated using this average (the best estimate of standard deviation)

1

)()( 1

2

n

xxx

n

ii

Please notice the fact, than n-1 and not just n is used in denominator. This is because we do not know the expected value, estimated as arithmetic average, and therefore number of degrees of freedom is reduced by 1 (n-1).

The set of recorded data x1,…. xn enables to evaluate also standard deviation of

the calculated arithmetic average (which is obviously smaller than the standard deviation of measured data)

)1(

)()( 1

2

nn

xxx

n

ii

Measuring chainEXM8

The measured quantity x (e.g. temperature) is usually measured by a chain of different instruments (e.g. by thermocouple and voltage amplifier), with generally nonlinear characteristics (voltage is not exactly linear function of temperature for thermocouple) and instrument transforms input signal according to its characteristics. There are always some random errors superposed.

f(x)thermocouple

g(x)amplifier

x (actual value) f(x)+fiy=g(f(x)+fi)+ gi

Random noise with normal distribution (f) and zero mean value

Random noise with normal distribution (g) and zero mean value

Measuring chainEXM8

Expected mean value for n repeated experiments

22

21 1

22

21

1 1[ ( ( ) ) ] [ ( ( )) ... ]

2!

1( ( ))

2

n nfi

fi gi fi gii i

n

fii

g gy g f x g f x

n n f f

gg f x

f n

Therefore the mean value (even for a very large number of experiments n) is distorted in the case that the function g(x) is nonlinear and this deviation is proportional to variance of errors applied to instrument f(x) (thermocouple):

22

2( ( ))

2fg

y g f xf

Variance of thermocouple noise

Mean value of noise is zero

Measuring chainEXM8

n

igifi

n

igifiy f

g

nxfgxfg

n 1

22

1

2 ]...[1

))](())(([1

2 2 2 2

1 1 1

2 2 2

1 1 1( ) 2

( )

n n n

y fi gi gi fii i i

f g

g g

f n n f n

gcrosscorrelation

f

Expected variance of y for repeated measurement of the same value x

Variance of thermocouple

noise

Variance of amplifier noise

Taylor expansionEXM8

Taylor expansion of function of M variables

1 1 2 2

2

1 21 1 1

( , ,..., )

1( , ,..., )

2

M M

M M M

M j j kj j kj j k

y f x x x

f ff x x x HOT

x x x

Variance of evaluated propertyEXM8

Variance of property calculated from M measured values (independent variables)

),...,,( 21 Mxxxfy 2

21 1 2 2 1 2

1

2 22 2 21 1 2

1 21 1 1 11 2 1 1 2

1[ ( , ,..., ) ( , ,..., )]

1[ ... ] ( ) .... ( ) 2 ...

n

y i i M Mi Mi

n n n ni Mi i i

i i Mii i i iM M

f x x x f x x xn

f f f f f f f

n x x x x n x n x x n

2 2 2 2 21

1

( ) .... ( )y MM

f f

x x

Variance of evaluated propertyEXM8

Proof of variance of arithmetic average)1(

)()( 1

2

nn

xxx

n

ii

11

22 2 2

1

1( ,...., )

1( )

n

n ii

nx

x xi

x x x xn

n n

Variance of evaluated propertyEXM8

VL

pD128

4

222222222 )()()()( VLpD VLpD

Example related to the project of capillary rheometer ( syringe): Evaluation of viscosity from the following capillary rheometer data

Geometry: D diameter of needle, L-length of needle, p pressure drop, V volumetric flowrate

Variance of individual measured parameters can be estimated from repeated measurement, e.g. from repeated measurement of the needle length L

1

)(1

2

2

n

LLn

ii

L

The variance can be sometimes estimated from instrument data sheets

Hagen Poiseuille relation for

laminar flow

Data RegressionEXM8

Hopper

Data RegressionEXM8

Regression analysis: Approximation of relationship between independent variables x (there can be more than one independent variable) and dependent variable y.

Let us assume that data are arranged in the matrix of observation points (each row describes one point x,y). For example this is a matrix with two columns and N rows if there is one independent variable x and N-pairs of x,y.

),(),....,,( 21 pxfpppxfy M

p

N

i i

ii pxfyp

1

22 )),(

()(

The relationship y(x) is represented by model

where is vector of model parameters.

where i is standard deviation of

dependent variable y at the point x.

Regression analysis looks for the model parameters giving the best approximation of observation points, i.e. minimising the goal function

Chi square criterion

Data RegressionEXM8

A good model f(x,p) (that reasonably approximates the unknown relationship y(x)) should give chi square value of about N-M (N is number of points and M is number of identified parameters p).

Another indicator of quality of the selected regression model is correlation index r

2

2

( ( , ))1

( )i i

i

y f x pr

y y

The correlation index r=1 in the case of absolutely perfect fit (model reproduces all observation points exactly), the worst case is r=0, because than the function f would be better approximated by a constant, the mean value of dependent variable .y

Linear regression analysis EXM8

In this case only the models f(x,p) which are linear with respect to the model parameters pk are used

1 1 1

1

( ) .. ( )

...

...

( ) ( )

M

N M N

g x g x

A

g x g x

M

mmm xgppxf

1

)(),(

gm(x) are design functions, which can be selected more or less arbitrarily, they must be only linearly independent. Example g1=1, g2=x, g3=x2,…

For N observation points the design matrix A is defined as

Aij=gj(xi)

[[ ]][ ] [ ]predictionA p y

Linear regression analysis EXM8

Parameters p are identified in such a way that the sum of squares will be minimized (it corresponds to minimization of chi square criterion for the case, that standard deviation error of all data points is the same).

2 ([ ] [[ ]][ ]) ([ ] [[ ]][ ])

[ ] [ ] [ ] [[ ]] [[ ]][ ] [ ] [[ ]] [ ] [ ] [[ ]] [ ]

[ ] [ ] [ ] [[ ]] [ ] 2[ ] [[ ]] [ ]

T

T T T T T T T

T T T T T

s y A p y A p

y y p A A p p A y y A p

y y p A p p A y

2 2 2

1 1 1 1

( ( )) ( )N M N M

i m m i i m imi m i m

s y p g x y p A

The sum of squares can be expressed also in matrix notation as a scalar product of two vectors (residual vectors of differences between measured values of y and prediction by linear model)

Design matrix (function of xi)

Vector of data yi

Linear regression analysis EXM8

Looking for minimum of sum of squares (zero gradient at minimum)

This is system of linear algebraic equations for unknown vector of model parameter p

Right hand side vector

0]]][[[]][[2]])[[][2(][

2

pAAAyp

s TTT

[[ ]] [[ ]][ ] [[ ]] [ ]T TA A p A y

[[ ]] [[ ]] [[ ]]TC A ASquare matrix M x M

[ ] [[ ]] [ ]TB A y

This system is called NORMAL EQUATIONS and inverted matrix [[C]]-1

is called COVARIANCE MATRIX.

Linear regression analysis EXM8

The covariance matrix C-1 is closely related to probable uncertainties (standard deviations) of calculated parameters:

2 1 2pk kk yC

Variance of measured data

Variance of calculated parameters

2 2 2 2 2

1 1

( ) ( )k

i

N Nk

pk yi yi ii yy

pp

1 1

1 1 1

M M N

k km m km im im m i

p C b C A y

1 1 1

1 1 1

M M Mm m

km km km imm

k

i m mi i

p

y

b bC C C A

y y

2 2 1 1 2 1 1

1 1 1 1 1 1

2 1 1 2 1 2 1

1 1 1

N M M M M N

pk y km im kn in y km kn im ini m n m n i

M M M

y km kn mn y km km y kkm n m

C A C A C C A A

C C C C C

Proof:

NonLinear regressionEXM8

In this case the model can’t be decomposed to linear combination of design functions, ane has a general form y=f(x,p1,…,pM) – this model can be in form of an algebraic expression, but it can be for example solution of differential equation. The parameters p should be again calculated from the requirement, that the sum of squares of deviations (or weighted sum of squares) is the least possible.

The Marquardt Levenberg method is based upon linearisation of optimised model f(xi,p1,…,pM)=fi, where xi are independent variables of the i-th observation point and p1,…,pM are optimised parameters of model. The least squares criterion is used for optimisation

iiii wfys 22 )(

0)(22

ii

j

iii

j

wp

ffy

p

s

0)( 0

i

ij

i

kk

k

iii w

p

fp

p

ffy

Increment of k-th parameter in iteration step

Weight of i-th data point

NonLinear regressionEXM8

jkk

jk BpC

Each iteration of Marquardt Levenberg method consists in solution of linear algebraic equations for vector of parameter increments

i

ik

i

j

ijk w

p

f

p

fC

i

iiij

ij wfy

p

fB )( 0

Concergency of iterations is improved by artificial increase of C matrix diagonal, by adding a constant to C11, C22,…CMM. For very large the algorithm reduces to the steepest discent method (gradient method) – slow, but reliable, while for very small iterations approach Gauss method – faster but sensitive to initial estimate of searched parameters.

Example RegressionEXM8

31

2 3 21

1

( )N

i ii

y p x

s y p x

3

3 3 11 1

61

1

( ) 0

N

i iNi

i i i Ni

ii

y xy p x x p

x

3 21

2 1

3 21

2 11

6

1

( )

1

( )1

1

N

i ii

y

N

i ii

p N

ii

y p x

N

y p x

Nx

Regression model

Example CalibrationEXM8

Simultaneous calibration of multiple thermocouples or pressure transducers

A/D converter

T

Tr

U1U2

UM

Consider linear characteristics of individual channels

for 1, 2,...,j j ju k T t j M

Measured data are represented by matrix of observation points

Reference temperature

Voltage 1 Volage 2 … Voltage M

Tr1 u11 u12 … u1M

Tr2 u21 u22 u2M

… …

TrN uN1 uN2 uNM

1(of 5)

Example CalibrationEXM8

Calibration means identification of constants kj and tj of all transducers. As soon as the reference values Tr are accurate (recorded by a standard instrument with better accuracy than the accuracy of calibrated probes) the problem is quite simple: Parameters kj,tj can be identified by linear regression for each probe separately.

2

1 1 1

1 1

for 1, 2,...,

N N N

ri ri ij riji i i

N Nj

ri iji i

T T u Tk

j Mt

T N u

1 1 1

2 2

1 1

2

1 1 1 1

2 2

1 1

( )

( )

N N N

ij ri ij rii i i

j N N

ri rii i

N N N N

ij ri ij ri rii i i i

j N N

ri rii i

N u T u Tk

N T T

u T u T Tt

N T T

j j

j

u tT

k

Recorded voltage

Evaluated temperature

[[C]][B]

2(of 5)

Example Calibration COVARIANCEEXM8

Covariance matrix C is inverted matrix of normal equations

1

2 2 2 2

1 1 1 11

2

1 1

2 2 2 2

1 1 1 1

( ) ( )

[[C ]]= for 1, 2,...,

( ) ( )

N

rii

N N N N

ri ri ri rii i i i

N N

ri rii i

N N N N

ri ri ri rii i i i

TN

N T T T N T

j M

T T

T N T N T T

2 1 211

2 1 222

k u

t u

C

C

Estimated

variances of transducers

Variances of kj and tj

3(of 5)

Example Calibration SIMULTANEOUSEXM8

Actual temperature Ti in the i-th measurement is not exactly the recorded reference value Tr (due to inaccuracy of standard instrument) but Ti is the same for all probes assuming a good mixing of liquid in the bath (this assumption is fulfiled even better with simultaneous calibration of pressure transducers). Question is how to use this information for improvement of identified constants accuracy?

The best estimate of actual temperature of bath in the i-th measurement is based upon minimisation of deviation with respect Tri and deviations of the predicted temperatures from M-probes (assuming that their characteristics are known)

2 2 2

1

( ) ( ) ( )M

i i ri ij j j jj

s T w T T u k T t

1

2

1

( )M

ri ij j jj

i M

jj

wT u t k

Tw k

Weight of standard instrument (select

high w if accuracy of standard is high)

result

4(of 5)

Example Calibration SIMULTANEOUSEXM8

The best approximation of bath temperature Ti can be used instead of Tri, and the whole procedure repeated until convergency is achieved

2

1 1 1

1 1

N N N

i i ij iji i i

N Nj

i iji i

T T u Tk

tT N u

j=1,2,…, M

i=1,2,…, N

1

2

1

( )M

ri ij j jj

i M

jj

wT u t k

Tw k

2

1 1 1

1 1

N N N

i i ij iji i i

N Nj

i iji

r r r

ri

T T u Tk

tT N u

j=1,2,…, M

convergeyesno

Result kj, tj

Data: w,uij,Tri

5(of 5)

Example Laser scanner (1 of 2)EXM8

How to identify a circle, given set of points xi yi

x

y x0 y0

xi yi

n

iii yyxxRs

1

220

20

2 ))()((

20

20 )()( yyxx

R

ii

i

0))(1(

0))(1(

01

1

10

10

1

n

iii

n

iii

n

i i

yy

xx

But this is a system of 3 nonlinear equations

Example Laser scanner (2 of 2)EXM8

How to identify a circle, given set of points xi yi

x

y x0 y0

x1 y1

x2 y2

x3 y3 3 points define a circle. So you can evaluate

triplets (for n=100 this is 161700 radii)

and estimate radius by average.!3)!3(

!

3

n

nn

]))(())((

))(())(()([

2

1

]))(())((

))(())(()([

2

1

23122312

2331233112210

23122312

2331233112210

xxyyyyxx

xxxxyyyyxxyyy

xxyyyyxx

xxxxyyyyyyxxx