Parameter estimation: basic concepts

• Basic problem: We measure range and phase data that arerelated to the positions of the ground receiver, satellites andother quantities. How do we determine the “best” positionfor the receiver and other quantities.

• What do we mean by “best” estimate?• Inferring parameters from measurements is estimation• Two styles of estimation (appropriate for geodetic type

measurements)– Parametric estimation where the quantities to be estimated are the

unknown variables in equations that express the observables– Condition estimation where conditions can be formulated among the

observations. Rarely used, most common application is leveling wherethe sum of the height differences around closed circuits must be zero

Parameter estimation: basic concepts

All parametric estimation methods can be broken intoa few main steps:

• Observation equations: equations that relate theparameters to be estimated to the observed quantities(observables).

• Example: Relationship between pseudorange, receiver position,satellite position (implicit in r), clocks, atmospheric andionospheric delays.

• Stochastic model: Statistical description that describesthe random fluctuations in the measurements (and maybethe parameters).

• Inversion that determines the parameters values from themathematical model consistent with the statistical model.

Observation model

• Observation model are equations relatingobservables to parameters of model:– Observable = function (parameters)– Observables should not appear on right-hand-side of

equation• Often function is non-linear and most common

method is linearization of function using Taylorseries expansion.

• Sometimes log linearization for f=a.b.c i.e.products of parameters

Taylor’s series expansion• In most common Taylor series approach:

• The estimation is made using the difference between the observations andthe expected values based on a priori values for the parameters.

• The estimation returns adjustments to a priori parameter values• Since the linearization is only an approximation, the estimation should be

iterated until the adjustments to the parameter values are zero.• For GPS estimation: Convergence rate is 100-1000:1 typically (i.e., a 1

meter error in a priori coordinates could result in 1-10 mm of non-linearityerror).

),,,()(

)(

),,,(

4321

4321

xxxxXXX

XfXfyy

xxxxfy

oxo =!"

"+=!+

=

Least-squares estimation

• Originally formulated by Gauss.• Basic equations: L is vector of observations; A is

linear matrix relating parameters to observables;X is vector of parameters; v is vector of residuals

• Minimizing residuals results in the least-squaressolution:

vAXL +=

LAAAXTT 1)( !

=

Weighted least-squares• In standard least squares, nothing is assumed about the

residuals v except that they are zero mean.• One often sees weighted-least-squares in which a

weight matrix is assigned to the residuals: Residualswith larger elements in P are given more weight.

• The least-squares solution becomes:

PLAPAAX

Pvvimize

TT

T

1)(

:)(min

!=

Weighted least squares• The least squares solution is given by:

• P is the weight matrix, defined by:

σo2 = a priori variance

ΣL = covariance matrix of the observations.

• The law of covariance propagation gives the covariance matrix of theunknowns ΣX:

LPAPAAXTT

rr1)( !

=

1

2

1 !"=L

o

P#

11 )( !!"=" AAL

T

X

Processing GPS pseudorange measurements

The pseudorange measurements jRi(t) can be modeled as:t = time of epochjRi = pseudorange measurementjρi = satellite-receiver geometric distancec = speed of lightjδ = satellite clock biasδi = receiver clock biasΔI = ionospheric propagation errorΔT = tropospheric propagation errorMP = multipathε = receiver noise(ranges in meters, time in seconds)

Neglecting the propagation, multipath, and receiver errors, eq.(1) becomes:

The geometric distance between satellite j and receiver i is given by:

with [jX, jY, jZ] = satellite position, [Xi, Yi, Zi] = receiver position in an ECEF coordinate system.

We need to solve for [Xi, Yi, Zi, δi], assuming that we know [jX, jY, jZ, jδ]. A major problem here is that theunknowns [Xi, Yi, Zi] are not linearly related to the observables…

!""# ++$+$+%+= )()()())()(()()( tMPtTtIttcttR i

j

i

j

i

j

))()(()()( ttcttR i

j

i

j

i

j !!" #+=

( ) ( ) ( )

( )iiii

j

i

j

i

j

i

j

i

j

ZYXft

or

ZtZYtYXtXt

,,)(

)()()()(222

=

!+!+!=

"

"

Processing GPS pseudorange measurements

Assuming that we now the approximate coordinates of the receiver [Xo, Yo, Zo], one can write that the actualcoordinates equal initial approximate coordinates (assumed to be known) plus a slight adjustment (unknown):

ΔXi, ΔYi, ΔZi are our new unknowns. We can now write:

Since we know the approximate point [Xo, Yo, Zo], we can now expand f(Xo+ΔXi, Yo+ΔYi , Zo+ΔZi) using aTaylor’s series with respect to that point:

We intentionally truncate the Taylor’s expansion after the linear terms.

ioi

ioi

ioi

ZZZ

YYY

XXX

!+=

!+=

!+=

( ) ( )ioioioiii ZZYYXXfZYXf !+!+!+= ,,,,

...!2

1

),,(),,(),,(

),,(),,(

2

2

+!

!+

"!

!+"

!

!+"

!

!+

=

x

f

ZZ

ZYXfY

Y

ZYXfX

X

ZYXf

ZYXfZYXf

i

o

ooo

i

o

ooo

i

o

ooo

oooiii

Processing GPS pseudorange measurementsRecall from earlier that: ( ) ( ) ( ) )()()()(),,(

222tZtZYtYXtXZYXf o

j

o

j

o

j

o

j

ooo !="+"+"=

dx

dunu

dx

du n

n

1!=

( ) ( ) ( )[ ]

( ) ( ) ( )[ ]

( )

( ) ( )

)(

)(

)1()(

)(

)()(2

)(2

1

)(

)(2

1

)()()(

)(2

1

)()(

2

1

)(

)()()(),,(

2

222

2/1

2/1

2/1222

t

XtX

Xu

XtX

X

XtXXtX

Xu

X

XtX

Xu

X

ZtZYtYXtX

Xu

X

XuXu

X

Xu

X

ZtZYtYXtX

X

ZYXf

o

j

o

j

o

o

j

o

o

j

o

j

o

o

o

j

o

o

o

j

o

j

o

j

o

o

oo

o

o

o

o

j

o

j

o

j

o

ooo

!

""=

""

=

#

"#"=

#

"#=

#

"+"+"#=

#

#=

#

#=

#

"+"+"#=

#

#

"

Recall the chain rule:

Therefore:

Processing GPS pseudorange measurements

The partial derivatives are:

We can now substitute these partial derivatives:

We now have an equation that is linear with respect to the unknowns ΔXi, ΔYi, ΔZi.

)(

)(),,(

)(

)(),,(

)(

)(),,(

t

ZtZ

Z

ZYXf

t

YtY

Y

ZYXf

t

XtX

X

ZYXf

o

j

o

j

o

ooo

o

j

o

j

o

ooo

o

j

o

j

o

ooo

!

!

!

""=

#

#

""=

#

#

""=

#

#

i

o

j

o

j

i

o

j

o

j

i

o

j

o

j

oooiii Zt

ZtZY

t

YtYX

t

XtXZYXfZYXf !

""!

""!

""=

)(

)(

)(

)(

)(

)(),,(),,(

###

Processing GPS pseudorange measurementsNow let us go back to our pseudorange measurements jRi(t) and rewrite our model equation:

We can rearrange the above equation by separating the known and unknown terms of each side (recall that thesatellite clock correction jδ(t) is provided in the navigation message):

We can simplify the notation by assigning:

Let us assume that we have 4 satellites visible simultaneously. We can then write for these 4 satellites:

)()()(

)(

)(

)(

)(

)()()( tctcZ

t

ZtZY

t

YtYX

t

XtXttR i

j

i

o

j

o

j

i

o

j

o

j

i

o

j

o

j

o

j

i

j !!"""

" #+$#

#$#

#$#

#=

)()(

)(

)(

)(

)(

)()()()( tcZ

t

ZtZY

t

YtYX

t

XtXtcttR ii

o

j

o

j

i

o

j

o

j

i

o

j

o

j

j

o

j

i

j !"""

!" #$#

#$#

#$#

#=##

)()()(

)(

)(

)(

)(

)(

)(

tcttRl

t

ZtZa

t

YtYa

t

XtXa

j

o

j

i

jj

o

j

o

j

Zi

j

o

j

o

j

Yi

j

o

j

o

j

Xi

j

!"

"

"

"

##=

##=

##=

##=

iiZiiYiiXi

iiZiiYiiXi

iiZiiYiiXi

iiZiiYiiXi

cZaYaXal

cZaYaXal

cZaYaXal

cZaYaXal

!

!

!

!

"#+#+#=

"#+#+#=

"#+#+#=

"#+#+#=

4444

3333

2222

1111

Processing GPS pseudorange measurementsTired of carrying along all these terms, subscripts, and superscripts? Me too. Let us introduce matrix notation:

L = vector of n observations. Must have at least 4 elements (i.e. 4 satellites), but in reality will have from 4 to 12elements depending on the satellite constellation geometry.X = vector of u unknowns. Four elements in our case.A = matrix of linear functions of the unknowns (= design matrix), n rows by u columns.

Now we can write our problem in a matrix-vector form:

!!!!!

"

#

$$$$$

%

&

=

!!!!

"

#

$$$$

%

&

'

'

'

=

!!!!!

"

#

$$$$$

%

&

(

(

(

(

=

l

l

l

l

L

Z

Y

X

X

caaa

caaa

caaa

caaa

A

i

i

i

i

ZiYiXi

ZiYiXi

ZiYiXi

ZiYiXi

4

3

2

1

444

333

222

111

r

r

)

XAL

rr=

iiZiiYiiXi

iiZiiYiiXi

iiZiiYiiXi

iiZiiYiiXi

cZaYaXal

cZaYaXal

cZaYaXal

cZaYaXal

!

!

!

!

"#+#+#=

"#+#+#=

"#+#+#=

"#+#+#=

4444

3333

2222

1111

Processing GPS pseudorange measurements

Starting from our problem in a matrix-vector form:

The least squares solution is given by:

P is the weight matrix, defined by:

σo2 = a priori variance

ΣL = covariance matrix of the observations.

The law of covariance propagation gives the covariance matrix of the unknowns ΣX:

!

r L = A

r X , P

LPAPAAXTT

rr1)( !

=

1

2

1 !"=L

o

P#

11 )( !!"=" AAL

T

X

Processing GPS pseudorange measurements

The least square solution is used to solve for:

Once ΔXi, ΔYi, ΔZi are found, the antenna coordinates [Xi, Yi, Zi] are obtained using:

The associated covariance matrix of the unknowns Σx is:

!!!!!

"

#

$$$$$

%

&

='=' ((

2

2

2

2

11 )(

ttztytx

ztzzyzx

ytyzyyx

xtxzxyx

L

T

X AA

))))

))))

))))

))))

ioi

ioi

ioi

ZZZ

YYY

XXX

!+=

!+=

!+=

!!!!

"

#

$$$$

%

&

'

'

'

=

i

i

i

i

Z

Y

X

X

(

r

Processing GPS pseudorange measurements

We can then transform Σx from an ECEF frame to a local topocentric frame using the law of variance propagation(disregarding the time-correlated components of Σx):

where R is the rotation matrix:

with ϕ = geodetic latitude of the site, λ = geodetic longitude of the site.

The DOP factors (Dilution Of Precision) are given by:

!!!

"

#

$$$

%

&

='='2

2

2

uueun

eueen

nunen

T

XTRR

(((

(((

(((

!!!

"

#

$$$

%

&

'

''

=

()()(

))

()()(

sinsincoscoscos

0cossin

cossinsincossin

R

2222

222222

22

tuen

t

zyxuen

en

u

GDOP

TDOP

PDOP

HDOP

VDOP

!!!!

!

!!!!!!

!!

!

+++=

=

++=++=

+=

=

Dilution of Precision“Dilution of precision” = DOP:

Quantifies the impact of the satellite constellation geometry on position and time: TDOP = time dilution of precision PDOD = position dilution of precision GDOP = geometric dilution of precision (time + position)

Derived from the diagonal terms of the cofactor matrix ⇒ ~ standard deviations High GDOP ⇒ bad configuration Low GDOP ⇒ good configuration

Dual-frequency receiver ⇒ LC observable Remaining unknowns:

⇒ Antenna position Xi, Yi, Zi⇒ Phase ambiguities: 1 per satellite orbital arc⇒ Tropospheric delay: 1 zenith total delay parameter every 2 hours, for instance.

Data: Static positioning: the GPS antenna is fixed

• 1 hour @ 30 sec w/ 8 satellites => 960 LC observations• Unknowns = 12• Solve for a system of 960 equations and 12 unknowns (Least squares, Kalman)• We can even afford more unknowns, especially if long observation sessions (24 h or

more): horizontal tropospheric gradients, orbital parameters, EOP Kinematic positioning: the GPS antenna is mobile

• Data (assuming 8 satellites) = 8 per epoch• Unknowns:

First epoch = 12 As soon as ambiguities are solved = 4 (3) Solving the ambiguities as fast and early as possible is critical Then we can carry them on as we solve for positions (Kalman)

Processing GPS phase measurements

Error estimation• Covariance matrix associated with least squares solution:

• Formal errors of the least squares inversion given by:– Diagonal terms = variances = (standard deviation)2

– Off-diagonal terms = correlations (-1 to 1)

• Interpretation of the formal errors?

!!!!!

"

#

$$$$$

%

&

==' (

2

2

2

2

1)(

ttztytx

ztzzyzx

ytyzyyx

xtxzxyx

T

X PAA

))))

))))

))))

))))

!!!

"

#

$$$

%

&

='='2

2

2

uueun

eueen

nunen

T

XTRR

(((

(((

(((

Random variables• Observed and estimated values include random errors =

random variables• Random variables are described by a probability

distribution, or probability density, p(x)• The probability P that a random variable X falls between x

and x+dx is found by integrating p(x):

• Of course:

!

P(a " X " b) = p(x)dxa

b

#

!

P("# $ X $ +#) = p(x)dx"#

+#

% =1

Normal distribution• Most important density

function = normal(=Gaussian) distribution:

• Parameters:– Mean value µ– Standard deviation σ

(measure of scatter aroundmean)

!

p(x) =1

" 2#e$ x$µ[ ]

2/ 2" 2

Normal distribution• Probability that a random sample is

below x = cumulative densityfunction F(x):

– At µ−σ: F(x) = 0.16– At µ+σ: F(x) = 0.84– Chance of falling between µ−σ and

µ+σ = 0.84-0.14=0.68%• Similarly:

– 0.95% corresponds to the chance offalling between µ−2σ and µ+2σ

– 0.99% corresponds to the chance offalling between µ−3σ and µ+3σ

!

P(X " x) = p(x)dx#$

x

% = F(x)

A! valid in 1 dimension…!

Chi-square distribution• The sum of n independent and normally distributed random variables x1

2+ x22

+…+ xn2 is a random variable (often called χ2) that follows a chi-square

distribution:

• This distribution can be used to calculate the probability Kn that a randomvariable that follows a chi-square distribution falls within a given interval.

• Assuming independent and normally distributed measurement errors, a least-squares solution (i.e. the fit of N data points yi (i=1,…,N) to a model with Madjustable parameters aj (j=1,…,M)) is equivalent to minimizing:

• Sum of random variables independent and normally distributed ⇒ adjustmentsfollows a χ2 distribution (with (N-M) degrees of freedom)

2

2/

2/1)2/(

2)2/()(

nn

xn

nn

exxp !=

"=

##

!=

"#

$%&

' (N

i i

Mii aaxyy

1

2

1 )...;(

)

Confidence ellipse• Geodetic LS problem: estimates (X,Y) of a particular

network point, with the associated covariance matrix:

• Relation with covariance matrix:

ϕ ab

x

y

contour of equal probability p

( )[ ]2

12

2

2

2

1

22

2

2

1

2

2

2

1

2

1

2

1

2

2

2

1

12

4)(2

1

2)2tan(

!!!!!!!"

"

"

"

!!

!#

$$+±+=%&'

=

=

$=

cb

ca

1)(

'

)(

'2

2

2

2

1

2

=+!! c

y

c

x

x’

y’

• In the rotated (x’,y’) coordinate system, the equation of the ellipse becomes:

!"

#$%

&='

2

221

12

2

1

((

((

Confidence ellipse• Probability that the estimated point lies within this ellipse? The chi-square

cumulative probability K can be used to estimate the probability for the followinginequality:

• Geometrical interpretation of the chi-square: the confidence ellipse (or error ellipse):x

yEstimates from LS fit

(= GPS positions)

Each estimate insidethis contour has a

probability K% to occur

0.98893σ0.86472σ0.39351σK2(c2)c

3.0350.992.4480.952.1460.90

cK2(c2)

2

2

2

1

2''

cyx

!+""

In 2 dimensions, 1-sigma = 39% confidence…!

Precision Least-squares fitting is a maximum likelihood estimator if the measurement

errors are:– Independent– Normally distributed

In the case of real (GPS) data:– Measurement errors do not necessarily follow a normal distribution…– Outliers: data points that are “way off”

• Least-squares adjustment is still going to try to fit them with a model…• Need for careful data editing before inversion (delete data if error > 3σ)

– Systematic errors:• Do not average out if enough data is taken! (≠ statistical, or random error)• Usually VERY difficult to deal with• Ex.: tribrach calibration, monument deformation

– Errors are correlated in time: cf. daily estimates and atmosphere Conclusion on formal errors:

– They are not a realistic representation of the true errors– They usually underestimate the true error