Highlights from Operational Verification in COSMO

Highlights from Operational Highlights from Operational Verification in COSMOVerification in COSMO

WG5 COSMO General Meeting, Rome 2011

Authors: ALL

Presented by Adriano Raspanti


• Quick look to some common plots

• Intercomparison between driving model and high resolution

• Conditional verification

• Fuzzy verification

• Long term trends (mainly precipitation)

TEMPERATURE AT 2 M - SON 2010 - MAM 2011

MEAN SEA LEVEL PRESSURE - SON 2010 – MAM 2011

Grafikinhalte

BC from GME CEU,CPL,CRU !!!

BC from IFS C7,CI7,CGR !!!

BC from GME ???

BC from IFS ???

WIND SPEED AT 10 M - SON 2010 - MAM 2011

TOTAL CLOUD COVER - SON 2010 - MAM 2011







COSMO General Meeting – Roma 05-09 Sept 2011

COSMOME vs ECMWF Temperature

SON

MAM

DJF

JJA


COSMOME vs ECMWF Wind Speed

SON

JJA

MAM

DJF


COSMOI7 vs ECMWF Temperature

SON

JJA

MAM

DJF


COSMOI7 vs ECMWF Wind Speed

SON

JJA

MAM

DJF


TemperatureCOSMOME vs COSMOIT

SON

JJA

MAM

DJF


Wind SpeedCOSMOME vs COSMOIT

SON

JJA

MAM

DJF

Temp 2m - 7km vs 3kmTemp 2m - 7km vs 3km


Fall Winter

Spring Summer

Underestimation of Temp, mainly in winter. error ~2o, worse with 7km by ~0.5o

Clear diurnal cycle

Wind Speed - 7km vs 3kmWind Speed - 7km vs 3km

Fall Winter

Spring Summer

Overestimation of wind (DJF,SON)2-2.5deg bias

similar attitude of 2 models


17 Verification results at MeteoSwiss in 2011

COSMO GM / WG5 Parallel Session, 05.09.2011

Precipitation (12h-sums +36 to +48h):Spring 2011 over Switzerland (SYNOP‘s)frequency bias: COSMO-7 & IFS

observed frequency

V. Stauch



Precipitation (12h-sums +12 to +24h):Spring 2011 over Switzerland (SYNOP‘s)COSMO-7 & COSMO-2

V. Stauch

for both models mean over 9 gridpoints foreach station

T2m COSMO-RU 2.2 and 7 km, Sochi, station Krasnaya Polyana

19

2.2 km –Less overestimating

7 km

T2m in COSMO-RU 7 and 2.2 km, Krasnaya Polyana

20

Method: 1) nearest point 3D optimized !

COSMO-RU 2.2 km is better than COSMO-RU 7 km for Krasnaya Polyana

T2m in COSMO-RU 7 and 2.2 km, Moscow

21

Method: 1) nearest point 3D optimized !

COSMO-RU 2.2 km RMSE is even slightly higher than that of COSMO-RU 7 km for Moscow


PERFORMANCE DIAGRAM

Period March 2010 - April 2011


50% of points (median) > 1 mm/24h








1 Point (maximum) > 1 mm/24h








50% of points (median) > 1 mm/24h&

Maximum > 25 mm/24h



Maximum > 50 mm/24h



Maximum > 75 mm/24h



Maximum > 100 mm/24h



Maximum > 25 mm/24h



Maximum > 50 mm/24h



Maximum > 75 mm/24h






Maximum > 25 mm/24h



Maximum > 50 mm/24h



Maximum > 75 mm/24h






Maximum > 25 mm/24h



Maximum > 50 mm/24h



Maximum > 75 mm/24h



Maximum > 75 mm/24h








Conditional Verification Temp – TCC obs <=25%

SON

MAM

DJF

JJA

Better behaviour for all the seasonsCompare to no condition model


Conditional Verification Temp – TCC obs >=75%&Wind Speed (obs) <=2 m/s

SON

MAM

DJF

JJA

Similar. Differences in bias

COSMO General Meeting – Roma 05-09 Sept 2011 COSMO General Meeting – Roma 05-09 Sept 2011

Conditional Verification Temp – TCC obs <=25%&Wind Speed (obs) <=2

m/s

SON

MAM

DJF DJF

JJA

Similar. Differences in bias

2mT 2mT vsvs 2mT in overcast conditions2mT in overcast conditions 2mT overcast/no wind2mT overcast/no wind

Fall Winter

Spring Summer

2mT 2mT vsvs 2mT in skyclear conditions2mT in skyclear conditions 2mT skyclear/no wind2mT skyclear/no wind

Fall Winter

Spring Summer

CAPE>50

CAPE<50

Very high POD values for unstable conditions, FAR not so different


WS - Z0>0.25

WS - Z0<0.25

Wind Speed with respect to Roughness Length

Strong underestimation of wind in positions with small roughness length and increased error


Td, 2mT – dry soil

Td, 2mT – wet soil

W_SO Water content of first soil layer(kg/m2) 1cm.

Td: Higher error in dry soil and larger underestimation2mT: Higher error in wet soil and larger understimation









results for 20103h accumulated precipitation sumsover the domain of the swiss radar composit

models: COSMO-2 and COSMO-7for all 8 daily forecast runs, precipitation sums from +3 to +6h

observation precipitation estimates of the swiss radar composit

in case of a missing value, the full date will not be evaluated

Neighborhood verification for precipitation



FSS, COSMO-2 minus COSMO-7, 2010

Winter Spring

AutumnSummer

numbers = FSS-Score of COSMO-2

colours = differences COSMO-2 minus COSMO-7

COSMO-2 better

COSMO-7 better

• COSMO-2: similar skill in all seasons

• COSMO-2 better than COSMO-7 on almost all scales

• COSMO-2 better than COSMO-7 especially in Winter and Summer

Tanja Weusthoff



3. „Fuzzy in Time“

• Extension of the spatial window with a window in time volume (dx * dy * dt)

• Evaluation of the forecasts in this volume• Time-window ntm = [1h,3h,5h,7h,9h]

dt

dx

dy

T. Weusthoff



dxdy

dx

dymodel

observation

e.g. FSS

fraction blue pxiels model = 12/75 (dt=3), 5/25 (dt=1)

fraction blue pxiels obs = 12/75 (dt=3), 3/25 (dt=1)

t0

t0-1

t0+1

t0

t0-1

t0+1

T. Weusthoff



00 01 02 03 04 05 06 07 08 09 10 11 12 13 1420 21 22 23

00-02

01-03

02-04

03-05

04-06

05-07

…

23-01

22-00

21-23

20-22

hourly accumulated precipitation

Evaluation of 3h accumulations, time window ntm = 5 (t0 +-2h)

00 UTC 03 UTC 06 UTC 08 UTC

22 UTC 02 UTC

01 UTC 05 UTC

3 hourly accumulated precipitation

Evaluation of 3h accumulations, time window ntm = 1

09 UTC00 UTC 03 UTC 06 UTC

23 UTC 01 UTC 02 UTC 04 UTC

T. Weusthoff



FSS for different time-windowsCOSMO-2, July 2010

T. Weusthoff


COSMO GM / WG5 Parallel Session, 05.09.2011 T. Weusthoff

FSS for different time-windowsCOSMO-7, July 2010



FSS for different time-windows COSMO-2 minus COSMO-7, July 2010

T. Weusthoff



Summary „fuzzy in time“

• FSS increases on all scales with increasing time-window• greatest effect for small spatical scales• lowest effect for high threshods

• Both models show a similar increase• difference COSMO-2 minus COSMO-7 stays equal, resp.

becomes littler for high time-windows

• For Upscaling the influence of a time tolerance is relatively low und restricted on low thresholds ( effect of the avergaing)

Application of time-windows on the gridscale would make sense; simultaneous application with space tolerance brings no great change

T. Weusthoff

Ulrich Damrath: Fuzzy and standard verification for COSMO-EU and COSMO-DE, COSMO GM, Rome September 2011

Configuration of precipitation verification with FUZZY-methods

Up to May 2011: Observation data: Radar data prepared by assimilation scheme

Model data: GME-, CEU- and CDE-GRIBS interpolated to CDE-grid (nearest gridpoint)

Run: 00 UTC

Forecast times: GME, CEU: 06-18, 06-30, CDE: 06-18 hours

Verification area: part of CDE that is covered by radar data

Since May 2011: Observation data as before, modell data: CEU- and CDE-GRIBS interpolated to CDE-grid

(nearest gridpoint)

Run: 00, 03, 06, 09, 12, 15, 18, 21

Forecast times: 01-04, 03-06, 06-12, 12-15, 15-18, 18-21 hours

Verification aread : CDE, Northern part of Germany, Southern part of Germany, North-Western part of Germany, North-Eastern part of Germany , South-Western part of Germany, South-Eastern part of Germany


Application of Fuzzy-methods

Calculation of all Fuzzy-scores with the IDL-Program by Beth Ebert.

Monthly evaluation of data for Fractions Skill Score and Upscaling ETS

Generation of results for 8 (forecast runs)

* 7 (forecast intervals)

* 3 (2 models and one difference)

* 7 (regions)

* 2 (scores)

---------------------------------------------2352 Plots per time interval

Necessity to have a fast access to the data


Some examples:ETS upscaling July 2011, Run: 00 UTC, forecast time 01-04 hours


Some examples :FSS July 2011, Run: 00 UTC, forecast time 01-04 hours


Some examples :ETS upscaling July 2011, Run: 00 UTC, forecast time 12-15 hours




Some examples :ETS upscaling July 2011, Run: 00 UTC, forecast time 18-21 hours










COSI @ DWD


COSI @ DWD


COSI @ DWD


COSI @ DWD

Seasonal trend 0.2mm/24h + ECMWF

All the versions present a seasonal cycle with an overestimation during summertime (except COSMO-7 and I2) COSMO-7 and I2 underestimate Overestimation error decreases in D+2 (spin-up effect vanished)Latest summer worsening EU and I2

Dataset: high resolution network of rain gauges coming from COSMO dataset and Civil Protection Department 1300 stationsMethod: 24h/6h averaged cumulated precipitation value over 90 meteo-hydrological basins

Seasonal trend 20mm/24h + ECMWF

• Slight bias reduction during latest seasons

• winter 2010: all the versions overestimate (probably due to lack of representativeness of the rain gauges over the plain during snowfall)

• Strong COSMO-7 underestimation BUT slight improvement during latest seasons

•General underestimation during latest seasons exc. I7

Seasonal trend 0.2mm/24h + ECMWF

• Very light improvement trend• Seasonal error cycle: lower ets during winter and summertime• no significant differences between D+1 and D+2 winter 2010 (very snowy particularly in Northern Italy): low ets value (D+1 and D+2) model error or lack of representativeness of the rain gauges over the plain during snowfall ?

Seasonal trend 20mm/24h + ECMWF

• Low values during summertime (in general)

•All the versions present two “big jump” at jja08 and jja09, after the values increase and become quite stationary

•Skill decreases with forecast time

Documents

Highlights from Operational Verification in COSMO