CHIMERE regional forecasting system and performances...This report documents the CHIMERE regional forecasting system and its statistical performances against in-situ surface observations

Grant agreement n°633080

MACC-III Deliverable D_46.1.1 and 50.1.1

CHIMERE regional forecasting system and performances Dossier #6 June-Jul.-Aug. 2014 Sept.-Oct.-Nov. 2014

Date: 02/2015 Lead Beneficiary: MF-CNRM (#23) Nature: R Dissemination level: PU

File: MACCIII_ENS_DEL_D_46.1.1_D_50.1.1_JJA-SON2014_CHIMERE_Dossier6.docx.doc/.pdf

Work-package 46 and 50 (ENS, model forecasts and verification) Deliverable D46.1.1 and D50.1.1 Title Dossiers documenting regional forecasting systems and

their performances & Verification part of model dossiers commenting in particular skill scores by trimestrial periods

Nature R Dissemination PU Lead Beneficiary MF-CNRM (#23) Date 02/2015 Status Final version Authors Laurence Rouïl, Bertrand Bessagnet, Anthony Ung,

Frédérik Meleux (INERIS,#17) Matthias Beekmann, Gilles Foret, Adriana Coman, Benjamin Gaubert (CNRS-LISA, #10) Laurent Menut (CNRS-LMD, #10)

Approved by Virginie Marécal Contact [email protected]

[In case the deliverable is not a report: provide a description of it inside this box.]

This document has been produced in the context of the MACC-III project (Monitoring Atmospheric Composition and Climate). The research leading to these results has received funding from the European Community's Horizon 2020 Programme under grant agreement n° 633080. All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission has no liability in respect of this document, which is merely representing the authors view.

2 / 42


Executive Summary / Abstract The MACC III (Modelling Atmospheric Composition and Climate, www.copernicus-atmosphere.eu) project is establishing the core global and regional atmospheric environmental service delivered as a component of Europe's GMES/Copernicus (Global Monitoring for Environment and Security) initiative. The regional forecasting service provides daily 4-days forecasts of the main air quality species (ozone, NO2, and PM10) from 7 state-of-the-art atmospheric chemistry models and from the median ensemble calculated from the 7 model forecasts. This report documents the CHIMERE regional forecasting system and its statistical performances against in-situ surface observations for quarter#21 (June, July, August 2014) and quarter#22 (September, October, November 2014). The Chimere version remains the same for these periods to the one used one year before. During quarter #21, the operationalization of the forecast and analyses chains required significant changes. The forecast files are now produced earlier than previously: the first 48-hours of the forecast are now produced before 7h UTC everyday. To achieve this target, the ensemble production chain has been split into slots of 24-hours terms. These important changes may explain why the availability statistics of some individual models were degraded during quarters #21 and #22. Verification is achieved using the up-to-date methods described in the MACC-II dossiers covering quarters #15 and #16. In this dossier, the dataset of surface observations used for verification is collected from the EEA/EIONET NRT database. Experience from the MACC-II/OBS subproject (Deliverable D_16.3) has shown that this pre-operational service is enough reliable and it offers a better geographical coverage than the previous database that was used for MACC-II verification. As for the past three years, the verification statistics are based on the use of only representative sites selected from the objective classification proposed by Joly and Peuch (Atmos. Env. 2012). During these two quarters, the CHIMERE performances compared to the Ensemble and also to the results of CHIMERE for previous periods show different features. For ozone, the scores decreased with a different time profile. It is very complex to understand why as the version of CHIMERE does not change over the last few years. The station data from the EEA database which are partly different from the set used previously may explain part of the changes. The scores for NO2 are stable. And a significant improvement for PM10 is shown which could be due to the insertion of the near real time boundary conditions for aerosols in the forecasting chain compared to the set-up of last year.

3 / 42

http://www.copernicus-atmosphere.eu/http://www.copernicus-atmosphere.eu/


Table of Contents

1. CHIMERE facts sheet .............................................................................................................. 6

1.1. Products portfolio (at the end of quarter#22) ............................................................ 6

1.2. Performance statistics ..................................................................................................... 6

1.3. Availability statistics ........................................................................................................ 6

1.4 Assimilation and forecasts system: synthesis of main characteristics ............................ 7

2. Evolutions in the CHIMERE suite............................................................................................ 9

3. CHIMERE background information ...................................................................................... 10

3.1 Forward model ............................................................................................................... 10

3.1.1 Model Geometry ......................................................................................................... 10

3.1.2 Forcings and boundary values ..................................................................................... 11

3.1.2.1 Meteorology ............................................................................................................. 11

3.1.2.2 Chemistry ................................................................................................................. 11

3.1.2.3 Landuse .................................................................................................................... 11

3.1.2.4 Surface emissions ..................................................................................................... 11

3.1.3 Dynamical core ............................................................................................................ 12

3.1.4 Physical Parametisations ............................................................................................. 12

3.1.4.1 Turbulence ............................................................................................................... 12

3.1.4.2 Deposition ................................................................................................................ 12

3.1.5 Chemistry .................................................................................................................... 12

3.2 Assimilation system ........................................................................................................ 12

3.2.1 Optimal Interpolation .................................................................................................. 13

3.2.2 Ensemble Kalman Filter (EnKF) ................................................................................... 13

3.2.2.1 Model version coupled to the EnKF ......................................................................... 13

3.2.2.2 Assimilation method: Ensemble Kalman Filter ........................................................ 14

3.2.2.3 Covariance Modelling ............................................................................................... 16

3.3 Developments achieved and plans ................................................................................ 18

ANNEX A: Verification report for quarter#21 .......................................................................... 23

4 / 42


ANNEX B: Verification report for quarter#22 .......................................................................... 33

5 / 42


1. CHIMERE facts sheet

1.1. Products portfolio (at the end of quarter#22)

Name Description Freq. Available for users at

Species Time span

FRC Forecast at surface,50m,250m, 500m,1000m,2000m, 3000m, 5000m above ground

Daily 9:00 UTC

O3, NO2, CO, SO2,PM2.5, PM10, NO, NH3, NMVOC, PANs, Birch pollen at surface during season

0-96h, hourly

ANA Analysis at the surface

Daily 11:00 UTC for the day before

O3 and PM10 0-24h of the day before, hourly

1.2. Performance statistics See annexes.

1.3. Availability statistics The statistics below describe the ratio of days for which the MATCH model outputs were available on time to be included in the ensemble fields (analyses and forecasts) that are computed at Météo-France. The following labels are used referring to the reason of the problem causing unavailability: (P) if the failure comes from the individual regional model production chain (T) if this is related to a failure of the data transmission from the partners to Météo-France central site (C) if this is a failure due to the central processing at Météo-France (MF) Quarter 21 (June, July, August 2014). The ratio of days on which CHIMERE forecasts and analyses were provided on time is:

Terms Analyses 0-96h frc Availability 53 % 79 %

To be included in the ensemble, the model outputs should be received at Météo-France before: 14:00 UTC for the analysis and 11:00 UTC for the forecast. CHIMERE forecasts were missing from 18 to 23(P), on 29(P) June 2014, on 1(P), 5(P), 7(P), 11(P), 19(T), 22(P) and from 28(P) to 31(P) July 2014, from 05 to 06(P), on 16(P) and 22(P) August 2014.

6 / 42


CHIMERE analyses were missing on 3(P), from 17 to 24(P), from 29 to 30(P) June 2014, on 5(P), 7(P), 8(P), 11(P), 19(P) and from 24 to 31(P) July 2014, on 01(P), from 5 to 8(P), from 13 to 25(P) August 2014. During quarter #21, we had some difficulties to produce due to instability on our computing system which makes our computing time too long to meet with the daily deadline of production. Quarter 22 (September, October, November 2014) : the ratio of days on which CHIMERE forecasts and analyses were provided in time is: Terms Analyses 0-24h frc 25-48h frc 49-72h frc 73-96h frc Availability 54 % 68 % 65 % 46 % 24 %

To be included in the ensemble, the model outputs should be received at Météo-France before: 11:30 UTC for the analysis, 5:00 UTC for the 0-24h forecast, 6:00 UTC for the 25-48h forecast, 6:45 UTC for the 49-72h forecast and 7:30 UTC for the 73-96h forecast. Availability of CHIMERE forecasts was incomplete on 3(P), 7(P), from 10 to 18(P), from 20 to 30(P) September 2014, from 1 to 5(P), from 7 to 12(P), from 14 to 31(P) October 2014, from 1 to 5(P), from 10 to 14(P), from 16(P) to 19(P), on 21(P), from 23 to 24(P), from 26 to 30(P) November 2014. CHIMERE analyses were missing on 3(P), 7(P), from 10 to 18(P), on 21(P), from 24 to 30(P) September 2014, from 3 to 5(P), on 7(P), from 9 to 12 (P), from 14 to 16(P), on 21(P), from 24 to 29 (P) October 2014, and from 3 to 4 (T), on 21 (P) and on 28(P) November 2014. During quarter #22, we had some difficulties to produce due to instability on our computing system which makes our computing time too long to meet with the daily deadline of production.

1.4 Assimilation and forecasts system: synthesis of main characteristics Assimilation and Forecast System Horizontal resolution 0.1° Vertical resolution Variable, 8 levels from the surface up to

500 hPa Gas phase chemistry MELCHIOR2, comprising 44 species and 120

reactions (Derognat, 2003) Heterogeneous chemistry Aerosol size distribution 8 bins from 10 nm to 40 μm Inorganic aerosols Primary particle material, nitrate, sulphate,

ammonium Secondary organic aerosols Biogenic, anthropogenic Aqueous phase chemistry Dry deposition/sedimentation Classical resistance approach

7 / 42


Mineral dust Dusts are considered Sea Salt Inert sea salt Boundary values Values provided by MACC-GRG Initial values 24h forecast from the day before Anthropogenic emissions MACC-TNO inventory Biogenic emissions Forecast System Meteorological driver 00:00 UTC operational IFS forecast from

the day before Assimilation System (not yet activated for daily operations) Assimilation method Optimal Interpolation, Ensemble Kalman

filter Observations Surface ozone (rural) and PM10 Frequency of assimilation Every hour over the day before Meteorological driver 00:00 UTC operational IFS forecast for the

day before

8 / 42


2. Evolutions in the CHIMERE suite 2014/04: additional levels and species in the daily grib2 file provision 2014/01: New grib2 file 2013/07/20: Boundary conditions from NRT global forecasts (Gas + aer) – or MACCII climatology 2013/03/01: implementation of the birch pollen module 2012/10/10: Extension of the MACCII domain in CHIMERE and calculation of D+3. 2012/06/20: Bug fixed in the emission processing which can have severe effects on the model performances. 2012/01/01: New version of CHIMERE running with horizontal resolution of 0.1°. 2010/01/06: The system is running robustly at INERIS. 2009/12/01: Operational delivery of the new CHIMERE version runs for the MACC AQ forecasts with discontinuities during December. 2009/10/01: Interruption of the CHIMERE forecast daily delivery due to the system migration from LISA (ECMWF) to INERIS. 2009/06/01: start of MACC pre-operational forecasts.

9 / 42


3. CHIMERE background information

3.1 Forward model The CHIMERE multi-scale model is primarily designed to produce daily forecasts of ozone, aerosols and other pollutants and make long-term simulations for emission control scenarios. CHIMERE runs over a range of spatial scale from the regional scale (several thousand kilometres) to the urban scale (100-200 Km) with resolutions from 1-2 Km to 100 Km. On this server, documentation and source codes are proposed for the complete multi-scale model. However most data are valid only for Europe and should be revisited for applications on other continents. CHIMERE proposes many different options for simulations which make it also a powerful research tool for testing parameterizations The chemical mechanism (MELCHIOR) is adapted from the original EMEP mechanism. Photolytic rates are attenuated using liquid water or relative humidity Boundary layer turbulence is represented as a diffusion (Troen and Mahrt, 1986, BLM) Vertical wind is diagnosed through a bottom-up mass balance scheme. Dry deposition is as in Wesely (1989). Wet deposition is included Six aerosol sizes represented as bins in the model. Aerosol thermodynamic equilibrium is achieved using the ISORROPIA model. Several aqueous-phase reactions considered Secondary organic aerosols formation considered Advection is performed by the PPM (Piecewise Parabolic Method) 3d order scheme for slow species. The numerical time solver is the TWOSTEP method. Its use is relatively simple provided input data is correctly provided. It can run with several vertical resolutions, and with a wide range of complexity. It can run with several chemical mechanisms, simplified or more complete, with or without aerosols. CHIMERE is a parallel model that has been tested on machines ranging from desktop PCs running the GNU/Linux operating system, to massively parallel supercomputers (HPCD at ECMWF). CHIMERE is a French national CNRS tool meaning that source code and documentation are freely available on a dedicated web site, training courses are organized twice a year for users. More than 120 users, from 30 institutes, are registered on the model e-mail list.

3.1.1 Model Geometry CHIMERE is an eulerian deterministic model, using variable resolution in time and space (for cartesian grids). The model uses any number of vertical layers, described in hybrid sigma-p coordinates. The model runs over the GEMS-MACC domain with a 0.1° resolution and 8 vertical levels extending from the surface up to 500 hPa.

10 / 42


3.1.2 Forcings and boundary values The model is off-line and has to be forced for meteorology and boundary conditions. Two interfaces has been develop to connect CHIMERE with the chemical boundary conditions deliver by the Mozart global forecasts. The first one dedicated to gaseous species is active in MACC since the boundary conditions are provided operationally. The second one for aerosols (Sea salt, dust, black carbon...) is not used yet waiting for the daily provision of boundary conditions for particulate matter.

3.1.2.1 Meteorology CHIMERE can use many meteorological models and interfaces are provided for the following models: MM5, WRF, IFS/ECMWF. For most studies done with CHIMERE, the MM5 model was used forced by the National Centers for Environmental Prediction (NCEP) global meteorological data. MM5 was configured with the PBL option MRF (Option 5), based on the Troen and Mahrt (1986) parameterization (the most consistent with the CHIMERE mixing formulation). The Schultz (Option 8) microphysics parameterization has also been tested with CHIMERE and is recommended. Within, MACC, CHIMERE is directly forced by the IFS forecasts from the daily operational products delivered at 00 UTC.

3.1.2.2 Chemistry Boundary conditions can be either "external" or given by a coarse resolution CHIMERE simulation. In case of "external" forcing, the model is provided with several databases: The LMDz-INCA model [Hauglustaine et al., 2005] for gas-phase chemical species. The global aerosol model GOCART for mineral aerosols [Chin et al., 2004] or the CHIMERE-DUST outputs. The 3-hourly GRG global forecast is used to provide boundary conditions for a set of pollutants in MACC.

3.1.2.3 Landuse The proposed domain interface is based on the Global Land Cover Facility (GLCF: http://glcf.umiacs.umd.edu/data/landcover 1kmx1km resolution database from the University of Maryland, following the methodology of Hansen et al. (2000, J. Remote Sensing).

3.1.2.4 Surface emissions The model provides an interface combining several emissions sources such as EMEP (Yearly totals), IER (Time variations), TNO (Aerosol emissions), UK Dept of Environment (VOC speciation, Passant, 2002). The MACC emissions (TNO) are used in CHIMERE for MACC.

11 / 42


3.1.3 Dynamical core Three advection schemes are implemented: The Parabolic Piecewise Method (PPM, a three-order horizontal scheme, after Colella and Woodward, 1984), the Godunov scheme (Van Leer, 1979) and the simple upwind first-order scheme.

3.1.4 Physical Parametisations

3.1.4.1 Turbulence Vertical turbulent mixing takes place only in the boundary-layer. The formulation uses K-diffusion following the parameterization of [Troen and Mahrt, 1986], without counter-gradient term.

3.1.4.2 Deposition Dry deposition is considered for model gas species i and is parameterized as a downward flux F(d,i)= -v(d,i) c(i) out of the lowest model layer with c(i) being the concentration of species i. The deposition velocity is, as commonly, described through a resistance analogy [Wesely, 1989]. The wet deposition follows the scheme proposed by [Loosmore, 2004]

3.1.5 Chemistry

CHIMERE offers the option to include different gas phase chemical mechanisms. The original, complete scheme [Lattuati, 1997], hereafter called MELCHIOR1, describes more than 300 reactions of 80 gaseous species.

The hydrocarbon degradation is fairly similar to the EMEP gas phase mechanism [Simpson, 1992]. Adaptations are made in particular for low NOx conditions and NOx-nitrate chemistry. All rate constants are updated according to [Atkinson, 1997] and [De More, 1997]. Heterogeneous formation of HONO from deposition of NO2 on wet surfaces is now considered, using the formulation of [Aumont, 2003]. In order to reduce the computing time a reduced mechanism with 44 species and about 120 reactions is derived from MELCHIOR [Derognat, 2003], following the concept of chemical operators [Carter, 1990]. This reduced mechanism is called MELCHIOR2 hereafter.

MACC CHIMERE version consists in the baseline gas-phase version with MELCHIOR2 chemistry, together with a sectional aerosol module. This module accounts for 7 species (primary particle material, nitrate, sulfate, ammonium, biogenic secondary organic aerosol SOA, anthropogenic SOA and water). Potentially, Chloride et Sodium can be included (high computing time). In its initial version the module uses 6 bins from 10 nm to 40 μm. Now the module moves to 8 bins from 10 nm to 10 μm.

3.2 Assimilation system

12 / 42


During a first stage, waiting for the development of the ensemble kalman filter, we will use an optimal interpolation method to assimilate daily concentration values for correcting the raw forecasts of CHIMERE. This method has been widely evaluated in the Prev’Air system for ozone and PM10.

3.2.1 Optimal Interpolation

The analysis method is designed to assess, as accurately as possible, near–real time surface concentration fields of ozone and PM10. CHIMERE analysis is carried out over France and since summer 2009 over Europe. The observations retrieved in near–real time are used in combination with D-1 daily maxima for ozone, D-1 daily mean for PM10 and D-1 hourly values for both. We use one of the methods proposed by Blond et al [2003], based on the kriging of the differences between simulated and observed values, often called innovations in meteorology. Kriging methods have the advantage of providing spatial interpolations that necessitate few assumptions and give robust results. Few assumptions are needed in kriging methods; a sensitivity analysis on the kriging parameters has been performed, enabling to select the most appropriate parameters. The choice of the measurement sites is a crucial stage in the analysis procedure: the monitoring stations selected must deliver concentrations representative of the gridded concentrations. Rural stations are selected in priority, then suburban stations and urban stations, provided that the influence of local sources of pollution and local meteorology is minor. At a given location s, the analyzed concentration is calculated from the following equation:

( ) ( ) ( ) ( ) ( )( )kbkp

kkkba sZsYswsZsZ −+= ∑

=0

1

where Za(s) refers to the analyzed concentration at site s; Zb(s) refers to the corresponding simulated value; Yo(sk) is the measured concentration at site sk and wk(sk) are the weights derived from the kriging constraints (see Blond et al. [2003] for more details about the method). Innovations Yo(sk) - Zb(sk) are estimated at each monitoring site sk. The kriging method used here is ‘‘exact’’: at the measurement sites, the analyzed concentration is equal to the observed concentration.

3.2.2 Ensemble Kalman Filter (EnKF) The ensemble Kalman Filter is now coupled to the CHIMERE model. It allows to assimilate (separately at the moment) ozone measurements from ground based stations and from the IASI instrument on board the METOP platform.

3.2.2.1 Model version coupled to the EnKF The CHIMERE model (http://www.lmd.polytechnique.fr/chimere/) is a regional Chemistry-Transport Model. It is used operationally since 2006 for Air Quality monitoring and forecasting by the French platform PREVAIR (http://www.prevair.org/fr/index.php) for main

13 / 42

http://www.lmd.polytechnique.fr/chimere/http://www.prevair.org/fr/index.php


atmospheric pollutants (O3, NO2, PM10). In the framework of MACC, forecast and analysis are produced using this model either for R-ENS either for R-EVA. The version of the model used in R-EDA is slightly different and the aim of the work made in this sub-project is to develop and evaluate an Ensemble Kalman Filter (EnKF) coupled with this model to eventually replace the current OI algorithm used operationally in R-ENS and R-EVA. Nevertheless, most of model features remain unchanged. We use pre-processors to compute various forcing needed by the core model. The IFS 12-hourly analysis and complementary 3-hourly forecast are used as off-line meteorological forcing. A meteorological module (diagmet) is used to diagnose additional meteorological variables needed such as boundary layer height. The anthropogenic emission pre-processor ingests TNO emissions (Visschedijk et al., 2007) to determine hourly emission fluxes available for the model. Biogenic emissions are calculated using the MEGAN module (Guenther et al, 2006) fed by hourly meteorological variables. We also use 3-hourly chemical forcing at top and boundaries of the domain from the IFS-MOZART system (Fleming et al., 2009). In this version, the horizontal resolution to cover the GEMS domain is 0.5°x0.5°. 20 hybrid (σ,p)vertical levels are used on the vertical covering the 1000hPa-200hPa range. The model is coupled with a sequential EnKF.

3.2.2.2 Assimilation method: Ensemble Kalman Filter

An advanced sequential data assimilation method (EnKF) has been set-up for the purpose of 3D data assimilation. We use ensembles, generated by using Monte Carlo methods, to calculate spatially and temporally varying forecast-error covariances for the purpose of performing data assimilation.

In mathematical terms, the general data assimilation problem is defined by the computation of the probability density function (PDF) of the model solution, conditioned on the measured observations, (i.e. following the Bayes theorem, we have to estimate a posterior PDF). This PDF is usually represented using statistical moments or an ensemble of model states and searching for estimators like mean, mode or maximum likelihood. In the case of the EnKF, since the size of the ensemble is limited, it is difficult to obtain a very accurate representation of the PDF in high dimensional problems (Evensen, 2007). We restrict ourselves to finding a good estimate for the mean of the PDF. In the case of the EnKF (with Gaussian hypothesis for the errors), the ensemble mean and covariance describe the PDF of the assimilated fields because Gaussian PDFs are fully determined by their mean and variance; thus the solution becomes computationally feasible.

The analysis equation which allows us to up-date each ensemble member is written as:

( ) ( )fiTfeTfefiai HdRHHPHP Ψ−++Ψ=Ψ −1 (1) where fiΨ represents an ensemble member i (model state) (“f” stands for forecast, “a” for analysis), d is the vector of observations available at the time of analysis, H represents the linear version of the observation operator which permits the projection from the model space onto the observation space, feP is the forecast covariance error matrix, R is the observation covariance error matrix and

( ) 1−+= RHHPHPK TfeTfee (2)

14 / 42


is known as the Kalman gain matrix, where “T” refers to the transposed matrix. The “best estimate” is calculated as a mean over the ensemble members using the formula (with N the ensemble size):

∑=

Ψ=ΨN

i

ai

a

N 11 (3)

And the analysed covariance error matrix Pa as the covariance over the ensemble:

( )( )TaaiN

i

aai

ae N

P Ψ−ΨΨ−Ψ−

= ∑=11

1 (4)

The same formula is used for the forecast covariance error matrix ( feP ) using f

iΨ insteadaiΨ

(and fΨ instead aΨ ).

The initial ensemble ( ) Nifi ,1=Ψ , whose mean is the current state estimate, is updated in the analysis step (Eq. 1) taking into account all knowledge about the error statistics (in the model, feP , and in the measurements, R). The key of this method is the transformation of the forecast ensemble into an analysis ensemble ( ) Niai ,1=Ψ with appropriate statistics. They are two ways to treat uncertainty in observations: one consists in adding perturbations to them according to the observational error in order to obtain N vectors of measurements ( ) Niid ,1= (N is the ensemble size). In this case, in Equation 1, we will use id instead of d to update each member of the ensemble (details in Burgers et al. 1998). In this manner, we avoid the loss of the ensemble spread after assimilation. A second alternative is to use a Square Root Filter formulation (Maybeck, 1979). This formulation avoids the loss of positive definiteness of the error covariance matrices. It was demonstrated that the elimination of the sampling error associated with the perturbed observations makes the EnSRF (Ensemble Square Root Filter) more accurate than the EnKF for the same ensemble size (Whitaker and Hamill, 2002, Sakov and Oke, 2008). This is the reason for selecting square root formulation in our study (we use the same formulas and notations as in Evensen, 2004).

Knowing that usually the ensemble size tested is up to one hundred members, and that the number of observations associated with AQ is at least ten times larger, it seems, when considering the size of the matrix involved in inversion, that we have to solve an ill-conditioned problem. It may be extremely difficult to accurately evaluate the inverse of a matrix when the largest eigenvalue may be many orders of magnitude larger than its smallest eigenvalue. An ensemble with a limited number of members cannot estimate accurately the background error across the entire state space due to spurious error correlations; therefore it is better to restrict the new information provided by the measurement to a local neighbourhood. Thus, “covariance localisation” has become a very widely used technique to filter out the spurious long-range correlations, and increase the rank of the background covariance matrix. In the Houtekamer and Mitchell (1998) formulation, the size and ill-conditioning problems are simultaneously solved by using a cutoff radius beyond which covariances between variables are assumed to be zero. This is the choice made in our assimilation system.

To assimilate satellite data, we have to be able to project the vectors from the model space onto the observation space, calculate the innovations (differences between the

15 / 42


observations and the simulated fields projected in the observational space) and reproject this information onto the forecast model space. All these operations are achieved by constructing H, the observation operator. The formula used is:

( ) ( )fifi LASH ψψ ⋅⋅= (5) Making observations and simulated fields comparable first implies performing a vertical interpolation L (in order to have the same number of vertical layers for the model and for the IASI retrieval, one layer for each km up to 12 km). The second operation is a convolution by the averaging kernel A. As already mentioned, the averaging kernel matrix provides the information which, if properly applied to a particular in situ profile data, transforms that profile in order to have the same resolution and a priori dependence as the IASI retrievals (see the equation below from Rodgers, 2000):

( ) ( ) amama xAIAxxxAxx −+=−+=ˆ (6) where xm represents the model simulated profile, xa is the a priori profile (here issued from the McPeters climatology), I is the identity matrix and A is the averaging kernel; in this way we transform the model profile into a pseudo retrieved profile. Note that in the assimilation case, adding the a priori profile is not needed (Rodgers, 2000) because the a priori was removed from the IASI columns before, therefore only the first term in Equation 6 ( )mAx is required. The last step for constructing H is the integration on the vertical (S), up to 6 km, in order to obtain a scalar value corresponding to the column value.

For surface observations, the H allows a simple “interpolation” of model’s values in the grid cell corresponding to the station’s geographical localisation.

3.2.2.3 Covariance Modelling

At the moment, both types of observations are assimilated separately with two different configurations of the system. Distinction is made in the following for both systems. Nevertheless, the basic idea remain the same, at least for Background error covariance matrix: the ensemble formulation is used to describe time varying error of the model. As described below, the way of building the ensemble is different with the nature of observation used. It should be noted that this aspects will be changed for further versions of the system for which we are currently building a unified version.

Background Error Covariance Matrix

Satellite observations (ozone) (Coman et al, 2011)

There are several methods to set-up an ensemble, but no unified theory has been developed yet, at least for chemistry-transport simulations (Galmarini et al., 2004). Ensembles can be derived from a single model while perturbing model parameters (Beekmann and Derognat, 2003), numerical and physical parameterisations (Mallet and Sportisse, 2006) or more basically just perturbing a set of initial conditions. In this case, the initial ensemble was created by applying 3-dimensional “pseudo-random” perturbations to a reference run. These perturbations were taken from a Gaussian distribution with zero mean, unitary variance and a Gaussian spatial covariance with a fixed decorrelation length (Evensen,

16 / 42


1994) in order to obtain a 2-dimensional field. Given two such pseudo-random fields for two distinct layers, a new couple of fields vertically correlated with a specific covariance between layers can be generated (Eq. A13-A14 from Evensen, 1994). This procedure is applied for the 20 correlated perturbations fields corresponding to the model layers. The 3-dimensional perturbation field obtained after this procedure is characterised by a decorrelation length fixed at 200 km in the horizontal and at 1 km in the vertical, following the comparisons between a model reference run and ground-based/MOZAIC observations, presented in Boynard et al. (2010). The amplitude of perturbations was fixed at 10% of the simulated ozone concentrations in each grid cell. Perturbations were applied each 3 hours during the spin-up period of 24 h, and then during the whole assimilation period. During the one day forecast periods, between two analyses, these perturbations accumulated to give a dispersion of the ensemble from the mean varying between 17% and 25%. This is consistent with the model error statistics established by comparison with surface and free tropospheric ozone observations (Honoré et al., 2008). No temporal correlation was used in this configuration (i.e. white noise was assumed).

Surface observations (ozone)

First, we follow the same methodology as for the satellite. An additive inflation was added hourly for the ozone field of each ensemble members, in particular a normally distributed perturbation (characterized by a zero mean and a decorrelation length fixed at 200 km). On the vertical, the correlation of this additive noise is one in the boundary layer and no perturbations above. In addition, a physically sound ensemble was created by perturbing model parameters and input fields which explain most of ozone variability (Hanea et al., 2004). The main parameters that affect the ozone variability were fixed following the sensitivity tests presented in Boynard et al. (2010). Perturbations of the parameters are Log-normally distributed with fixed standard deviation; spatial correlations are also taken into account (mostly characteristic of the synoptic scale). Consequently, although we obtain a better spatial structure of the error field, the dispersion of the ensemble associated to the parameter perturbations broadly underestimates the error.

Finally, the perturbation of the ozone state largely dominates. In the initial set-up, perturbations of the ozone field were prescribed using time-varying perturbations during the day (between 15 and 25%). Now, we are using the Desroziers diagnostic (Desroziers et al. 2005, Schwinger et al. 2011) of the previous day to adjust the standard deviation of the perturbations (cf. Equation 5). In this case, the ensemble spread is weaker, but the temporal structure is better giving similar results than previous system (with a more stable algorithm).

(5)

The term yjo denote ozone measurements made at station j over p stations, yja and yjb are the corresponding values in observation space of the analyzed and background state respectively.

Observation Error Covariance Matrix

∑ =

−

−= p

j

b

j

o

j

b

j

a

jb yyyyp 1

1σ

17 / 42


Satellite observations (ozone) (Coman et al, 2011)

The errors in the observations were regrouped (representativeness and instrumental error) in the R matrix, whose diagonal is filled with the results obtained during the inversion procedure (Eremenko et al., 2008). We consider that there is no error correlation between different satellite observations used simultaneously in the assimilation (R is diagonal). This is certainly a simplification, but the degree of horizontal error correlation is unknown. However, only a limited set of satellite observations is used within the adopted localisation procedure (see below). We see that in our case, the observation error of the 0–6 km ozone column is about 16% on the average over the pixels available for the whole month. In the retrieval procedure the diagnosed error is not temporally correlated, thus we do not consider such a correlation in our system.

We apply a local analysis in order to avoid spurious correlations in the background ensemble, which are introduced by the perturbation method for finite ensemble sizes, and which do not have any geophysical reality. The basic idea of localisation is to perform the analysis at a given grid point using the observations within a local region centred at that point. The radius of this region was fixed at 200 km, corresponding to the decorrelation length in the horizontal perturbations applied (following Boynard et al., 2010). The maximum number of observations to be assimilated was limited to 30 pixels. This parameter was chosen after sensitivity tests. No vertical localisation was applied.

Surface observations (ozone)

The observation error covariance matrix is build as the sum of the measurement error (fairly reliable in the case of ozone: ~1-2%) and of the representativeness error. This representativeness error is more difficult to estimate because it should express spatial representativeness of a single measurement which is depending of the local environment of the station (land-use, emissions) and the meteorology (boundary layer height, synoptic and local winds). These parameters are difficult to characterise and often varying in time (at diurnal and/or seasonal scales).

The methodology used in this case was to define a representative set of stations (compared to the model resolution, i.e 0.5°x0.5°) following the classification of Fleming et al. (2005). Thus, we obtain clusters of stations namely MOU (for mountain or at least remote sites), RUR (for rural sites) and PUR (U1) (for peri-urban sites) that are eligible for data assimilation. The classification allows an objective clustering; i.e based on the measurements themselves more precisely on the mean diurnal cycle observed at each station. Moreover, we wanted to derive representativeness errors for assimilation purposes. Considering that those representativeness errors are unbiased, we have calculated the standard deviation between different sites contained in the same model grid cell and we found values about 4-5 ppb that are consistent with the results obtained by Fleming et al. 2004. Thus, assuming that observations errors are uncorrelated, diagonals terms of the observation error covariance matrix are finally set to 25ppb².

3.3 Developments achieved and plans

18 / 42


Future works will concern the development of the fire emission module and the implementation of a new CHIMERE version which is less time consuming.

19 / 42


References Beekmann M., and Derognat C.: Monte Carlo uncertainty analysis of a regional-scale transport

chemistry model constrained by measurements from the atmospheric pollution over the Paris area (ESQUIF) campaign, J. Geophys. Res., 108(D17), 8559, doi:10.1029/2003JD003391, 2003.

Bessagnet B., L. Menut, G. Aymoz, H. Chepfer and R. Vautard, Modelling dust emissions and transport within Europe: the Ukraine March 2007 event, J. Geophys. Res., 113, D15202, doi:10.1029/2007JD009541, 2008.

Bessagnet B., L.Menut, G.Curci, A.Hodzic, B.Guillaume, C.Liousse, S.Moukhtar, B.Pun,

C.Seigneur, M.Schulz, Regional modeling of carbonaceous aerosols over Europe - Focus on Secondary Organic Aerosols, J. Atmos. Chem., in press, 2009.

Boynard, A., Beekmann, M., Foret, G., Ung, A., Szopa, S., Schmechtig, C. and Coman, A.: Assessment of regional ozone model uncertainty with a modelling ensemble using an explicit error representation, Atm. Env., 45, 784-793, 2011.

Burgers, G., Van Leeuwen, P.J. and Evensen, G.: Analysis Scheme in the Ensemble Kalman Filter, Mon. Weather Rev., 126, 1719-1724, 1998.

Coman, A., Foret, G., Beekmann, M., Eremenko, M., Dufour, G., Gaubert, B., Ung, A., Schmechtig, C., Flaud, J.-M., and G. Bergametti, Assimilation of IASI partial tropospheric columns with an Ensemble Kalman Filter over Europe, 26943-26997, 11, ACPD, 2011.

de Meij A., Gzella, A., Cuvelier, C., Thunis, P., Bessagnet, B., Vinuesa, J.F., Menut, L., Kelder H.,

The impact of MM5 and WRF meteorology over complex terrain on CHIMERE model calculations, Atmos. Chem. Phys. , in press, 2009.

Desroziers, G., L. Berre, B. Chapnik and P. Poli, Diagnosis of observation, background and analysis-error statistics in observation space, Q. J. R. Meteorol. Soc., 131, 3385–3396 doi: 10.1256/qj.05.108, 2005.

Eremenko, M., Dufour, G., Foret, G., Keim, C., Orphal, J., Beekmann, M., Bergametti, G., and Flaud, J.-M.: Tropospheric ozone distributions over Europe during the heat wave in July 2007 observed from infrared nadir spectra recorded by IASI, Geophys. Res. Lett., 35, L18805,doi:10.1029/2008GL034803, 2008.

20 / 42


Evensen, G., Sequential data assimilation with a nonlinear quasigeostrophic model using Monte Carlo methods to forecast error statistics, J. Geophys. Res., 99 (C5), 10143-10162, 1994.

Evensen, G.: Sampling strategies and square root analysis schemes for the EnKF, Ocean Dynamics, 54, 539-560, DOI 10.1007/s10236-004-0099-2, 2004.

Evensen, G.: Data assimilation: The Ensemble Kalman Filter, Springer-Verlag Berlin Heidelberg, 2007.

Flemming, J., van Loon, M., Stern, R., 2004. Data assimilation for CTM based on optimum interpolation and KALMAN filter. In: Borrego, C., Incecik, S. (Eds.), Air Pollution Modeling and its Application, vol. XVI. Kluwer Academic/Plenum Publishers, New York.

Flemming, J., A., Inness, H., Flentje, V., Huijnen, P., Moinat, M. G., Schultz, and O. Stein, Coupling global chemistry transport models to ECMWF’s integrated forecast system, Geosci. Model Dev., 2, 253-265, 2009.

Galmarini S., Bianconib R., Klug W. et al.: Ensemble dispersion forecasting – part I: Concept, approach and indicators, Atmos. Environ., 38, 4607– 4617, 2004.

Guenther, A., Karl, T., Harley, P., Wiedinmyer, C., Palmer, P.I. and Geron, C.: Estimates of

global terrestrial isoprene emissions using MEGAN (Model of Emissions of Gases and Aerosols from Nature), Atmos. Chem. Phys., 6, 3181-3210, 2006.

Hanea, R., Velders, G. and Heemink, Data assimilation of ground level ozone in Europe with a

kalman filter and chemistry transport model, J. Geophys. Res., 109, 5183-5198, 2004. Honoré C., L. Rouil, R. Vautard, M. Beekmann, B. Bessagnet, A. Dufour, C. Elichegaray , J.-M.

Flaud, L. Malherbe, F. Meleux, L. Menut, D. Martin, A. Peuch, V.-H. Peuch, N. Poisson, Predictability of European air quality: the assessment of three years of operational forecasts and analyses by the PREV'AIR system, J. Geophys. Res., 113, D04301, doi: 10.1029/2007JD008761, 2008.

Houtekamer, P.L. and Mitchell, H.L.: Data assimilation using an Ensemble Kalman Filter technique, Mon. Weather Rev.,126, 796-811, 1998.

Mallet, V., and Sportisse B.: Uncertainty in a chemistry-transport model due to physical parameterizations and numerical approximations: An ensemble approach applied to ozone modeling, J. Geophys. Res., 111, D01302, doi:10.1029/2005JD006149, 2006.

21 / 42


Maybeck, P.: Stochastic models, estimation, and control, Academic Press, London, 1979.

Rodgers, C.D.: Inverse Methods for Atmospheric Sounding: Theory and Practice, World Scientific, Series on Atmospheric, Oceanic and Planetary Physics, 2, Hackensack, N. J., 2000.

Rouil L., C. Honore, R. Vautard, M. Beekmann, B. Bessagnet, L. Malherbe, F. Meleux, A. Dufour,

C. Elichegaray, J.-M. Flaud, L. Menut, D. Martin, A. Peuch, V.-H. Peuch, N. Poisson, PREV'AIR : an operational forecasting and mapping system for air quality in Europe, Bull. Am. Meteor. Soc., doi: 10.1175/2008BAMS2390.1, 2009.

Sakov, P. and Oke, P.R.: Implications of the form of the ensemble transformation in the ensemble square root filters, Monthly Weather Review, 136, 1042-1053, 2008.

Schwinger, J., and H. Elbern, Chemical state estimation for the middle atmosphere by four-dimensional variational data assimilation: A posteriori validation of error statistics in observation space, J. Geophys. Res., 115, doi:10.1029/2009JD013115, 2010.

Szopa S., G. Foret, L. Menut, A. Cozic, Impact of large scale circulation on European summer

surface ozone: consequences for modeling, Atmospheric Environment, 43(6), Pages 1189-1195, doi:10.1016/j.atmosenv.2008.10.039, 2009.

Valari M. and L. Menut, Does increase in air quality models resolution bring surface ozone concentrations closer to reality?, J. Atmos. Ocean. Tech., doi: 10.1175/2008JTECH A1123.1, 2008.

Vivanco M. G., Palomino I., Vautard R., Bessagnet R., Martin F., Menut L., Jimenez S., Multi-year assessment of photochemical air quality simulation over Spain, Env. Mod. and Software, doi:10.1016/j.envsoft.2008.05.004, 2008.

Visschedijk, A.J.H., Zandveld, P.Y.J., and Denier van der Gon, H.A.C.A., High resolution gridded European database for the EU Integrate Project GEMS, TNO-report 2007-A-R0233/B.

Whitaker, J.S. and Hamill, T.M.: Ensemble Data assimilation without perturbed observations, Mon. Weather Rev., 130, 1913-1924, 2002.

22 / 42


ANNEX A: Verification report for quarter#21 This verification report covers the period June/July/August 2014. The CHIMERE skill scores are successively presented for three pollutants: ozone, NO2 and PM10. The skill is shown for the entire forecast horizon from 0 to 96h (hourly values), allowing to evaluate the entire diurnal cycle and the evolution of performance from day 0 to day 3. Since November 2012, the forecast has been extended to 96h range and over a larger European domain (25°W-45°E, 30°N-70°N) than previously. According to VAL subproject recommendations (D85.2), the five following statistical indicators are used for model skill evaluation: mean bias, root-mean square error, modified normalized mean bias, fractional gross error and the correlation. Quarter #21 is the first one for which the observation dataset for verification is collected from the Up-To-Date (UTD) data stream of the European Environmental Agency (EEA). In MACC, verifications were done against all available Near-Real-Time (NRT) data. Since MACC-II, verifications have been performed against selected data among the NRT dataset from the different countries in Europe. Data provision to MACC relied on ad hoc bilateral agreement with Environment Agencies in 14 different countries. In MACC, D-INSITU work showed a considerable number of differences, the EEA dataset reporting more sites for ozone and less for the other species. Efforts were done to merge the two data sources, by helping EEA to get access to data in the countries with which GEMS-MACC has been in contact and which do not provide all their NRT data to EEA. In MACC-II, the work to merge EEA and data currently received in NRT advanced well. There were some technical issues on data formats and availability times of the EEA dataset, that have been mostly solved, in collaboration with OBS subproject (see MACC-II/D-16.3).

The observations from EEA/EIONET are downloaded and are stored in an operational database at MF-CNRM. As in MACC-II, the observations are selected in order to take into account the typology of sites, follows the work that has been carried out in MACC [Joly and Peuch, 2012] to build an objective classification of sites, based on the past measurements available in Airbase (EEA) (see MACC D_R-ENS_5.1 for more details). This objective approach is necessary because there is no uniform and reliable metadata currently for all regions and countries, which have all different approaches to this documentation. Verification is thus restricted to the sites that have a sufficient spatial representativeness with respect to the model resolution (10-20 km). The statistical approach using only representative sites -according to the objective classification- is clearly the way forward (as it does not also thin too much the NRT data available), leading to a general significant improvement of the overall skill scores (see MACC-II D_102.1_1/D106.1_1 for more details). Filtering stations on the EEA/EIONET NRT data leads to a mean numbers of: ~500 sites for ozone, ~400 sites for NO2, ~100 sites for SO2, ~10 sites for CO, ~300 sites for PM10. These numbers are lower than the number of stations available for previous years, but we expect an improvement of data collection and delivery from the EEA in the coming months. Note also that stations from Germany were missing during the whole quarter #21, decreasing even more these numbers. Checking the daily observation datasets revealed some inconsistencies that needed to be addressed in relation with EEA, such as undesirable zero concentration values and unrealistic time series at some stations. Some ad hoc treatments of the observations have been introduced at MF-CNRM.

23 / 42


Figure 1 :coverage of surface observations used for verification, collected from the EEA, and

after filtering.

The usage of the observation dataset is twofold: for verification of the forecasts and also for assimilation in the regional models. To be used for data assimilation, downloading the observations at 7h UTC is a reasonable compromise between the amount of data and the desired early time of production of the analyses. It will give the possibility to produce soon the regional analyses earlier, around 11h UTC. However, the number of observations at the end of the day decreases rapidly (Figure 2), due to the fact that some countries do not report observations to the EEA during the night. For forecast verification, observations are thus downloaded later, at 23h UTC, which leads to a more homogeneous distribution over the day (Figure 2). Similarly to forecast verification, MF-CNRM plans to set up procedures for verification of the NRT analyses. To get prepared, MF-CNRM has set up a sorting of observations, so that some stations are not distributed for assimilation, but kept for future verification scores of NRT analyses.

24 / 42


Figure 2 : mean number of observations available per hour of the day, during November 2014, for assimilation, for forecast verification and for future analysis verification.

Joly, M. and V.-H. Peuch, 2012: Objective Classification of air quality monitoring sites over Europe, Atmos. Env., 47, 111-123.

25 / 42


CHIMERE: ozone skill scores against data from representative sites, period #21 (June, July, August 2014)

CHIMERE has a higher bias than the Ensemble. The lowest bias occurs when ozone concentration are high in the mid-afternoon. Diurnal cycles are similar for both and quite stable for all time-lags. Compared to previous year, the results are quite worst during the morning with 6 µg/m3 more. This high inter-annual variability is difficult to understand as well as the change in the temporal profile. Otherwise, the minimum bias is slightly better than last year.

CHIMERE RMSE is higher than the ensemble one. Minimum RMSE occurred during daytime when ozone concentration is maximum. The difference between CHI and ENS looks stable from one day to another, even if the score becomes worse. The scores seem to be worse than last year with 2 µg/m3 more and the gap between CHI and ENS RMSE bigger. The time profile looks also a bit different.

As for the mean bias, the maximum correlation occurs during the mid-afternoon. The CHIMERE correlation is close to ENS correlation. The score is far better than last year. Once again. such improvement is difficult to explain.

26 / 42


The time profile of the modified mean bias of CHIMERE is very similar to the Ensemble one. A slight difference occurs during daytime when values are lowest. More or less the same values as last year .

The time profile of the fractional gross error of CHIMERE is very similar to the Ensemble one. A slight difference occurs during daytime when values are lowest. More or less the same values as last year.

27 / 42


CHIMERE: NO2 skill scores against data from representative sites, period #21 (June, July, August 2014)

The CHIMERE mean bias is positive during nighttimes and becomes negatives at daytime. The daily variability is higher in CHIMERE than in the Ensemble. Very close to the results of last year with a temporal profile slightly different.

The diurnal cycle is a bit different between CHIMERE and the Ensemble. CHIMERE RMSE is higher than the Ensemble during the evening peak (traffic rush) and both show similar rmse during the morning traffic rush. Compared to last year, the rmse of CHIMERE evening peak decreases by 3-4 µg/m3.

The correlations are similar for both. High values at nighttimes and low values in daytimes. Lower values than last year.

28 / 42


The negative modified mean bias of CHIMERE is lower than the Ensemble with a similar diurnal cycle. The underestimation of CHIMERE appears less important than for the Ensemble. Same conclusions as last year.

The over/under estimation is higher in the Ensemble than in Chimere during daytime. Similar to last year.

29 / 42


CHIMERE: PM10 skill scores against data from representative sites, period #21 (June, July, August 2014)

CHIMERE bias depicts a lower underestimation than the ENS one. The diurnal cycle is similar for both. The improvement of CHIMERE (especially compared to ENS) is significantly important compared to last year. This could be due to the insertion of BC conditions for aerosols in the CHIMERE forecasts.

CHIMERE and ENSEMBLE have similar RMSE with the same temporal profile which shows a high variability and stability regarding from one day to another. As for the bias, the improvement comparing to last year is significant.

The Chimere correlation is lower than that of the Ensemble with the same diurnal cycle. The correlations are very low and lower than last year.

30 / 42


CHIMERE underestimation is lower than the ENSEMBLE one.

The fractionnal gross error of CHIMERE is lower than the Ensemble with a similar time profile. It’s the opposite of last year.

31 / 42


Analysis of CHIMERE performances for quarter#21 The comparison of CHIMERE against observations for this quarter (June 2014 to August 2014) shows.

1) About ozone: The Chimere scores decrease compared to last year without possible explanation as the model version is still the same and the input data as well. The temporal profile has changed also and the temporal variability is quite high. The daily variability is stable from one day to another. The scores modifications could therefore be due to different meteorological conditions and to the use of a different dataset for verification (EEA database versus files acquired country by country). The future version of CHIMERE would provide improvement of process for ozone like the on-line calculation of photolysis rates and more efficient model settings for chemistry.

2) About NO2 The scores are more or less stable compared to last year with performances showing a large intra-day variability, higher than the Ensemble. The profile is similar from one day to another.

3) PM10 Compared to the previous year, the score show significant improvements regarding the bias and RMSE but not correlation. This could be due to the insertion of boundary conditions for aerosols which should be responsible for a decrease of the bias. Further possible improvements would be to activate the computation of the secondary inorganic aerosol online.

32 / 42


ANNEX B: Verification report for quarter#22 This verification report covers the period September/October/November 2014. The CHIMERE skill scores are successively presented for three pollutants: ozone, NO2 and PM10. The skill is shown for the entire forecast horizon from 0 to 96h (hourly values), allowing to evaluate the entire diurnal cycle and the evolution of performance from day 0 to day 3. Since November 2012, the forecast has been extended to 96h range and over a larger European domain (25°W-45°E, 30°N-70°N) than previously. According to VAL subproject recommendations (D85.2), the five following statistical indicators are used for model skill evaluation: mean bias, root-mean square error, modified normalized mean bias, fractional gross error and the correlation. Quarter #21 was the first one for which the observation dataset for verification is collected from the Up-To-Date (UTD) data stream of the European Environmental Agency (EEA). In MACC, verifications were done against all available Near-Real-Time (NRT) data. Since MACC-II, verifications have been performed against selected data among the NRT dataset from the different countries in Europe. Data provision to MACC relied on ad hoc bilateral agreement with Environment Agencies in 14 different countries. In MACC, D-INSITU work showed a considerable number of differences, the EEA dataset reporting more sites for ozone and less for the other species. Efforts were done to merge the two data sources, by helping EEA to get access to data in the countries with which GEMS-MACC has been in contact and which do not provide all their NRT data to EEA. In MACC-II, the work to merge EEA and data currently received in NRT advanced well. There were some technical issues on data formats and availability times of the EEA dataset, that have been mostly solved, in collaboration with OBS subproject (see MACC-II/D-16.3).

The observations from EEA/EIONET are downloaded and are stored in an operational database at MF-CNRM. As in MACC-II, the observations are selected in order to take into account the typology of sites, follows the work that has been carried out in MACC [Joly and Peuch, 2012] to build an objective classification of sites, based on the past measurements available in Airbase (EEA) (see MACC D_R-ENS_5.1 for more details). This objective approach is necessary because there is no uniform and reliable metadata currently for all regions and countries, which have all different approaches to this documentation. Verification is thus restricted to the sites that have a sufficient spatial representativeness with respect to the model resolution (10-20 km). The statistical approach using only representative sites -according to the objective classification- is clearly the way forward (as it does not also thin too much the NRT data available), leading to a general significant improvement of the overall skill scores (see MACC-II D_102.1_1/D106.1_1 for more details). Filtering stations on the EEA/EIONET NRT data leads to a mean numbers of: ~500 sites for ozone, ~400 sites for NO2, ~100 sites for SO2, ~10 sites for CO, ~300 sites for PM10. These numbers are lower than the number of stations available for previous years, but we expect an improvement of data collection and delivery from the EEA in the coming months. Note also that stations from Germany were missing during September and October, decreasing even more these numbers. Checking the daily observation datasets revealed some inconsistencies that needed to be addressed in relation with EEA, such as undesirable zero concentration values and unrealistic time series at some stations. Some ad hoc treatments of the observations have been introduced at MF-CNRM.

33 / 42


Figure 1 :coverage of surface observations used for verification, collected from the EEA, and

after filtering.

The usage of the observation dataset is twofold: for verification of the forecasts and also for assimilation in the regional models. To be used for data assimilation, downloading the observations at 7h UTC for the day before is a reasonable compromise between the amount of data and the desired early time of production of the analyses. It will give the possibility to produce soon the regional analyses earlier, around 11h UTC. However, the number of observations at the end of the day decreases rapidly (Figure 2), due to the fact that some countries do not report observations to the EEA during the night. For forecast verification, observations are thus downloaded later, at 23h UTC, which leads to a more homogeneous distribution over the day (Figure 2). Similarly to forecast verification, MF-CNRM plans to set up procedures for verification of the NRT analyses. To get prepared, MF-CNRM has set up a sorting of observations, so that some stations are not distributed for assimilation, but kept for future verification scores of NRT analyses.

34 / 42


Figure 2 : mean number of observations available per hour of the day, during November 2014, for assimilation, for forecast verification and for future analysis verification.

Joly, M. and V.-H. Peuch, 2012: Objective Classification of air quality monitoring sites over Europe, Atmos. Env., 47, 111-123.

35 / 42


CHIMERE: ozone skill scores against data from representative sites, period #22 (September, October, November 2014)

CHIMERE has a higher bias than the Ensemble. The lowest bias occurs when ozone concentration are high in the mid-afternoon. Diurnal cycles are similar for both and quite stable for all time-lags. Compared to previous year, the results are worst during the morning with 2 µg/m3 more. Otherwise, the minimum bias is similar to last year. The CHIMERE bias is also similar to the previous period.

CHIMERE RMSE is higher than the ensemble one. Minimum RMSE occurred during daytime when ozone concentration is maximum. The difference between CHI and ENS looks stable from one day to another. The scores seem to be worse than last year with 2 µg/m3 more and the gap between CHI and ENS RMSE bigger. The time profile looks also a bit different with less intra-day variability now.

The CHIMERE correlation is close to ENS correlation. The score is better than last year and close to the one of the previous period.

36 / 42


The time profile of the modified mean bias of CHIMERE is very similar to the Ensemble one. A difference occurs during daytime when values are lowest. A slight improvement compared to last year.

The time profile of the fractional gross error of CHIMERE is very similar to the Ensemble one. A slight difference occurs during daytime when values are lowest. A slight improvement compared to last year.

37 / 42


CHIMERE: NO2 skill scores against data from representative sites, period #22 (September, October, November 2014)

The CHIMERE mean bias is negative except for the morning hours corresponding to the traffic rush. The daily variability is a bit higher in CHIMERE than in the Ensemble. Very close to the results of last year with a temporal profile slightly different.

CHIMERE RMSE is very close to the ENSEMBLE RMSE with the same time profile. A small gap between both happens at lowest values. Compared to last year, the score is similar

The correlations are similar for both. High values at nighttimes and low values in daytimes. The variability is high. Same values as last year.

38 / 42


The negative modified mean bias of CHIMERE is similar to the Ensemble with a similar diurnal cycle. A small period occurs every for CHIMERE with over-estimation during morning rush traffic hours. The underestimation of CHIMERE appears less important than for the Ensemble. Similar to last year.

Almost the same score for the Ensemble and CHIMERE except during nighttime.

39 / 42


CHIMERE: PM10 skill scores against data from representative sites, period #22 (September, October, November 2014)

CHIMERE bias depicts a lower underestimation than the ENS one. The diurnal cycle is similar for both. Compared to last year, CHIMERE is now continuously underestimated (opposite to last year with a permanent overestimation). The difference between ENS and CHI is lower.

CHIMERE and ENSEMBLE have similar RMSE with the same temporal profile which shows a high variability and stability regarding from one day to another. The improvement comparing to last year is very significant.

The Chimere correlation is lower than that of the Ensemble with the same diurnal cycle. The chimere performance is slightly better than last year.

40 / 42


The negative modified mean bias of CHIMERE is lower than the Ensemble with a similar diurnal cycle. The underestimation of CHIMERE appears less important than for the Ensemble. Same conclusions as last year.

The CHIMERE fractional gross error displays close values to ENS but decrease earlier in the mid-afternoon.

41 / 42


Analysis of CHIMERE performances for quarter#22 The comparison of CHIMERE against observations for this quarter (Sep 2014 to Nov 2014) shows.

1) About ozone: The Chimere scores decrease compared to last year without possible explanation as the model version is still the same and the input data as well. The temporal profile of scores has changed also and the temporal variability is quite high. The daily variability is stable from one day to another. The performances looks similar to the one we have got for the previous period. The future version of CHIMERE would provide improvement of process for ozone like the on-line calculation of photolysis rates and more efficient model settings for chemistry.

2) About NO2 The scores are more or less stable compared to last year and also last period with performances showing a large intra-day variability, higher than the Ensemble. The profile is similar from one day to another.

3) PM10 Compared to previous year, the score show significant improvements regarding the bias and RMSE but not correlation. The characteristics are similar to the previous period. This could be due to the insertion of boundary conditions for aerosols which should be responsible for a decrease of the bias. Further possible improvements would be to activate the computation of the secondary inorganic aerosol online.

42 / 42

1. CHIMERE facts sheet1.1. Products portfolio (at the end of quarter#22)1.2. Performance statistics1.3. Availability statistics1.4 Assimilation and forecasts system: synthesis of main characteristics

2. Evolutions in the CHIMERE suite3. CHIMERE background information3.1 Forward model3.1.1 Model Geometry3.1.2 Forcings and boundary values3.1.2.1 Meteorology3.1.2.2 Chemistry3.1.2.3 Landuse3.1.2.4 Surface emissions3.1.3 Dynamical core3.1.4 Physical Parametisations3.1.4.1 Turbulence3.1.4.2 Deposition3.1.5 Chemistry3.2 Assimilation system3.2.1 Optimal Interpolation3.2.2 Ensemble Kalman Filter (EnKF)3.2.2.1 Model version coupled to the EnKF3.2.2.2 Assimilation method: Ensemble Kalman Filter3.2.2.3 Covariance Modelling3.3 Developments achieved and plans

ANNEX A: Verification report for quarter#21ANNEX B: Verification report for quarter#22

Documents

CHIMERE regional forecasting system and performances...This report documents the CHIMERE regional forecasting system and its statistical performances against in-situ surface observations