CSDA Conference, Limassol, 2005 University of Medicine and Pharmacy “Gr. T. Popa” Iasi Department of Mathematics and Informatics Gabriel Dimitriu University

CSDA Conference, Limassol, 2005

University of Medicine and Pharmacy “Gr. T. Popa” IasiDepartment of Mathematics and Informatics

Gabriel DimitriuUniversity of Medicine and Pharmacy of IasiDepartment of Mathematics and Informatics

700115 Iasi, Romania, email: [email protected]

Gabriel DimitriuUniversity of Medicine and Pharmacy of IasiDepartment of Mathematics and Informatics

700115 Iasi, Romania, email: [email protected]

Implementation issues related to data assimilation using Kalman

filtering

Implementation issues related to data assimilation using Kalman

filtering

100100100111101011010010101001100110101001110010

f x

ContentsContents

• Introduction

• Kalman filter with full covariance matrix

• Factorization of the covariance matrix

• Kalman filter in square root form

• Examples of factorized filters (RRSQRT filter,

Ensemble filter and POEnK filter)

• Implementation issues

• Conclusions

• Introduction

• Kalman filter with full covariance matrix

• Factorization of the covariance matrix

• Kalman filter in square root form

• Examples of factorized filters (RRSQRT filter,

Ensemble filter and POEnK filter)

• Implementation issues

• Conclusions


Introduction

The purpose of data assimilation is to incorporate measured observations into a dynamical system model in order to produce accurate estimates of all the current (and future) state variables of the system.

In the atmospheric chemistry field, for example the knowledge of initial conditions (and sometimes boundary conditions) of chemical concentrations is a very challenging task.


Introduction

Both the Kalman filter and the variational methods are suitable to be used in online forecast applications.

In the variational data assimilation, information provided by the observations is used to find an optimal set of model parameters through a minimization process.

There is a tendency to extend the assimilation procedure with Kalman filter techniques, for example to obtain useful background covariances for the cost function.


Introduction

For this study, the Kalman filter was chosen as a data assimilation tool. Not needing to build an adjoint model is seen as a major advantage here, since an adjoint of the chemistry model is complicated.

Other advantages of the Kalman filter which are considered are the availability of an analyzed covariance, describing the quality of the result, and the simple introduction of uncertainties in model parameters.

Numerical results from some investigations (based mainly on running simple tests using Matlab routines) are presented.


Kalman filter with full covariance matrix






















Factorization of the covariance matrix














Kalman filter in square root form








Examples of factorized filters

• The filters are based on the concept that, although the degree of freedom in the state is very large, the errors in the state are described very well by a limited number of directions, typically less than 100.

• Whether these directions are called singular vectors, modes, the basic equations in all filter implementations remain the same.


RRSQRT filter

• The Reduced Rank Square RooT (RRSQRT) filter was developed for asimilation of water level measurements in a shallow water models.

• In the RRSQRT formulation of the Kalman filter, the covariance matrix is expressed in a limited number of (orthogonal) modes, which are re-orthogonalized and truncated to a fixed number during each time step.


Ensemble filter

• In comparison with RRSQRT approach, the ENsemble Kalman Filter (ENKF) is based on convergence of large numbers.

• Both approaches lead to a low-rank approximation of the covariance matrix. The ensemble filter was introduced by Evensen (1994) for assimilation of data in oceanographic models.

• The basic idea behind the ensemble filter is to express the probability function of the state in an ensemble of possible states.


POEnK filter

• A new direction in implementation of low-rank filters is the use of two filters next to each other. The combination should compensate for errors in one or both of the individual filters.

• The Partially Orthogonal Ensemble Kalman Filter (POENKF) proposed in (Heemink et al., 2001) runs a RRSQRT filter next to an ENKF.

• The basic idea is to let the RRSQRT part compute the bulk of the covariance structure, described in the first modes.


POEnK filter

• The ENKF part should account for the truncation error, by introducing directions in the covariance matrix that have been lost during reduction.

• This procedure incorporates the advantages of both filter types, and accounts for their major disadvantages. Ensemble filters suffer from a lack of convergence; many ensembles are required before sample mean and correlations are stable. An ensemble filter is able to estimate and maintain any correlation introduced by the stochastic model, however.

• The reverse holds for the RRSQRT filter: a few modes are sufficient to describe the main part of the covariance structure, but some of the correlation structure is lost during the reduction.


Implementation issues








Conclusions

• In this study, the background, implementation and some numerical results in data assimilation of some common low-rank filters have been discussed.

• Low-rank filters are either based on factorizations of the covariance matrix (e.g. RRSQRT filter), or approximation of statistics from a finite ensemble (ENKF).

• A new direction in filter implementation is to use of two filters next to each other of the same form or hybrid (POENKF).


Conclusions


• The factorization approch is often based on the linear Kalman filter which has been extended towards nonlinear models; the ensemble technique is a reformulation of the filter problem in a statistic approach.

• In spite of the different philosophies, all low-rank filters turn out to have a similar implementation. Evolution of mean and covariance is in each of the filters performed by propagation of an ensemble of state vectors by the model.

• The propagation of the forecast ensemble is the most expensive part of the filter.

Conclusions


• There are several approaches for the analysis of measurements, based on whether the gain will lead to a minimal variance or not, and whether the filter is based on the factorization or the ensemble approach.

• The forms with a minimum variance gain are in practice most often used, and differ hardly from each other in computational costs.

• The main data structure in all filters is the covariance square root: a large low-rank matrix, with state vectors stored in the columns.

Conclusions


• The covariance square root needs to be transfomed at least one time during each time step, which is an expensive operation.

• In addition to forecast and analysis, the filters based on factorization require a singular value decomposition or re-orthogonalization of the covariance square root.

• For comparable costs, the RRSQRT filter produces stable and more accurate results than POENKF.

Documents

CSDA Conference, Limassol, 2005 University of Medicine and Pharmacy “Gr. T. Popa” Iasi Department of Mathematics and Informatics Gabriel Dimitriu University