Variational Quality Control

DA/SAT Training Course, March 2006

Variational Quality Control

Erik AnderssonRoom: 302 Extension: 2627

[email protected]


• Introduction

• VarQC formalism

• Rejection limits and tuning

• Minimisation aspects

• Examples

• Summary

Plan of Lecture


Introduction (1)

Assuming Gaussian statistics, the maximum likelihood solution to the linear estimation problem results in observation analysis weights (w) that are independent of the observed value.

22

2

)(

bo

b

bba

w

Hxywxx

Outliers will be given the same weight as good data, potentially corrupting the analysis


Introduction (2)

Real histogram distributions of departures (y-Hxb) can show significant deviations from the pure Gaussian form.

They often reveal a more frequent occurrence of large departures than expected from the corresponding Gaussian (normal) distribution with the same mean and standard deviation -

showing as wide “Tails”.

Observed departure from the background

“Tail”QC-rejection?

Actual distribution

Gaussian

K

Airep temperature observations


The Normal Jo (1)

The normal observation cost function Jo(x) has a quadratic form, which is consistent with the assumption that the errors are Gaussian in nature.

)()(2

1)( 1 HxyHxyxJ T

o R

If observations errors are uncorrelated then this simplifies to:

2

1 2

1)(

io

N

io

HxyxJ

y: array of observations of length=Nx: represents the model/analysis variablesH: observation operatorsR: observation error covariance matrixσo: observation error standard deviation

Normalized departure


The Normal Jo (2)

The general expression for the observation cost function is based on the probability density function (the pdf) of the observation error distribution (see Lorenc 1986):

constln pJo

p is the probability density function of observation error

arbitrary constant,chosen such thatJo=0 when y=Hx


The Normal Jo (3)

When for p we insert the normal (Gaussian) distribution (N):

2

N0 2

1constln

o

HxyNJ

2

2

1exp

2

1

oo

HxyN

we obtain the usual expression

In VarQC a non-Gaussian pdf will be used,

resulting in a non-quadratic expression for Jo.


VarQC formulation (1)

In an attempt to better describe the tails of the observed distributions, Ingleby and Lorenc (1993) suggested a modified pdf, written as a sum of two distinct distributions:

GApNAp )1(QC

A is the prior probability of gross error

Normal distribution (pdf),as appropriate for

‘good’ data

pdf for data affected bygross errors



Thus, a pdf for the data affected by gross errors (pG) needs to be specified. Several different forms could be considered.

In the ECMWF implementation (Andersson and Järvinen 1999, QJRMS) a flat distribution was chosen.

o

G

dDp

2

11

The reasons for this choice and its consequences will become clear in the following

D is the width of the distribution

D is here written as a multiple d of the observation error, either

side of zero.



Inserting pQC for p in the expression Jo=-ln p + const, we obtain:

dA

2A

JJJ

JJ

ooo

oo

2)1( :as defined with γ

)exp(1

1

)exp(ln

NNQC

NQC

We can see how the

presence of γ modifies the normal cost function and its gradient


Probability of gross error

The term modifying the gradient (on the previous slide) can be shown to be equal to:

)exp( NoJ

P

PW 1NQCoo JWJ

the a-posteriori probability of gross error P, given x and assuming that Hx is correct (see Ingleby and Lorenc 1993)

Furthermore, we can define a VarQC weight W:

It is the factor by which the gradient’s magnitude is reduced.

• Data which are found likely to be incorrect (P≈1) are given reduced weight in the analysis.

• Data which are found likely to be correct (P ≈ 0) are given the weight they would have had using purely Gaussian observation error pdf.


Illustrationsflat pQC (left), wide Gaussian pQC (right)

GradientGradient

QC WeightQC Weight


Application

In the case of many observations, all with uncorrelated errors, JoQC is computed as a sum (over the observations i) of independent cost function contributions:

The global set of observational data includes a variety of observed quantities, as used by the variational scheme through their respective observation operators. All are quality controlled together, as part of the main 4D-Var estimation.

The application of VarQC is always in terms of the observed quantity.

i

oii

ii

io JppJ QCQCQCQC const ln-constln


Minimisation

VarQC requires a

good‘preliminary

analysis’. Otherwise incorrect

QC-decisions will occur.

In operations VarQC is therefore

switched on after 40

iterations.(Now 40)


Multiple minima

The probability of gross error

depends on the size of the

departure. When the probability for rejection is

close to 50/50 this will be reflected in the cost function as two distinct

minima of roughly equal

magnitude.


Rejection limits

VarQC does not require the specification of threshold values at which rejections occur - so called rejection limits. Rejections occur gradually.

If, for example, we classify as rejected those data that have P>0.75, then we can obtain an analytical relationship for the effective rejection limit as a function of the two VarQC input parameters A and d (or γ) only:

)/3ln(2 i The first implementation of VarQC was thereby tuned to roughly reproduce the rejections of the old scheme (OIQC).


Tuning the rejection limit

The histogram on the left has been transformed (right) such that the

Gaussian part appears as a pair of straight lines forming a ‘V’ at zero.

The slope of the lines gives the Std deviation of the Gaussian.

The rejection limit can be chosen to be where the actual distribution is some distance away from the ‘V’ - around 6 to 7 K in this case, would

be appropriate.


Tuning exampleBgQC too tough

BgQC and VarQC correctly tuned

The shading reflects the value of P,the probability of gross error


Example (1)

For given values of the VarQC input parameters

(A and d), the QC result(i.e. P), is a

function of the normalized departure

(y-Hx)/σo, only.


Example (2)

VarQC checks all data and all data types simultaneously. In this Australian example the presence of aircraft data has led to the rejection of a PILOT wind.


Example (3) - a difficult one

Observations of intense and small-scale features may be rejected although the measurements are correct.

The problem occurs when the resolution of the analysis system (as determined by the B-matrix) is insufficient.


Summary

• VarQC provides a satisfactory and very efficient quality control mechanism - consistent with 3D/4D-Var.

• The implementation can be very straight forward.

• VarQC does not replace the pre-analysis checks - the checks against the background for example.

• All observational data from all data types are quality control simultaneously, as part of the general 3D/4D-Var minimisation.

• The setting of VarQC parameters needs regular revision.

A good description of background errors is essentialfor effective, flow-dependent QC

Documents

Variational Quality Control