Joint 8th COPS Workshop and CSIP Meeting 2009, 26-28 October 2009

Water Vapour Intercomparison Effort in the Frame of the Convective and Orographically-Induced Precipitation Study:

Airborne-to-Ground-based and airborne-to-airborne Lidar Systems

R. Bhawar, P. Di Girolamo, D. Summa, C. Flamant, D. Althausen, A. Behrendt,P. Bosser, M. Cacciani, C. Champollion, T. Di Iorio, G. Ehret, C. Kiemle, C.

Herold, S. D. Mueller, S. Pal, M. Radlach, A. Riede, P. Seifert, M. Shiler, M. Wirth, V. Wulfmeyer

Joint 8th COPS Workshop and CSIP Meeting 2009, 26-28 October 2009

Water vapour lidar inter-comparison effort

error estimates for the different water vapour lidars based on mutual inter-comparison.main objective

Different Instruments

airborne and ground-based water vapour lidar systems2 airborne DIALs: DLR DIAL & CNRS DIAL4 ground-based: 3 Raman (BASIL, BERTHA, ING) 1 DIAL (UHOH)

Simultaneous and co-located data for all lidar pairs

to compute relative bias and root-mean square (RMS) deviations

Needcomplete and comprehensive inter-comparison

tables

Sample from the intercomparison table for IOP-9c on 20 July 2007

Approach to identify the airborne lidar profiles to be compared with the ground-based lidar profiles


Distance between the aircraft footprint and the ground-based system not exceeding 10 km.


6078.1

10

22

2

3

TRnm

px

OHOH

OH

Comparisons between DIAL systems have been performed in terms of water vapour number concentration (water vapour molecules per m³), while comparisons between DIAL and Raman lidars are carried out in terms of water vapour mixing ratio.

In order to do so, DIAL results expressed in terms of water vapour number concentration must be converted into water vapour mixing ratio.

where mH2O is the water molecular mass (18.01508 molecular mass units), R is the gas constant of dry air (0.28704 J/(gK)), p is pressure and T is temperature.


2

1

2

1

21

21

21,

2

,z

zz

z

zzrelativei

zqzq

zqzq

zzBIAS

2

1

2

1

21

221

21,

2

,z

zz

z

zzz

relativei

zqzq

zqzqN

zzRMS

Relative bias

Relative root-mean square deviation

Absolute bias and root-mean square(obt. through the multiplication by the mean of the data of the two instr.)

z

z

zz

Nzqzq

2

1

212

Overall mean bias and RMS over the whole inter-comparison range

(weighted mean)

N

ii

N

iii

topbott

w

xwzzx

1

1,

A total of 25 profile-to-profile inter-comparisons between BASIL and CNRS DIAL

18 inter-comparisons possible with a minimum distance not exceeding 5 km 7 inter-comparisons with a minimum distance between 5 and 10 km

Example of comparison at 20:08 UTC on 31 July 07

• minimum distance between the two sensors of 1.8 km

• The two profiles show a very good agreement, with deviations not exceeding 0.25 g/kg.


0

500

1000

1500

2000

2500

3000

3500

-3 -2 -1 0 1 2 -100 -50 0 50 100

2

3

4

6

7

B

C

D

E

F

G

b

c

d

e

f

2

3

4

6

7

B

C

D

E

F

G

b

c

d

e

f

1607 06:09 1607 06:24 1607 06:38 1607 06:53 1607 07:07 1607 07:28 2507 19:02 2507 19:24 2507 19:37 2507 19:50 2507 20:16 2507 20:36 3107 19:26 3107 19:40 3107 19:54 3107 20:08

1 3107 20:21A 3107 20:42a 1407 11:18

1507 06:30 1907 13:04 3007 09:34 0108 08:08 0108 10:32 0108 13:44

Bias (g/kg)

Hei

ght a

.s.l.

(m)

Bias (%)

0

500

1000

1500

2000

2500

3000

3500

-3 -2 -1 0 1 2 3 -50 0 50 100

2

3

4

6

7

B

C

D

E

F

G

b

c

d

e

f

2

3

4

6

7

B

C

D

E

F

G

b

c

d

e

f

Hei

ght (

m)

1607 06:09 1607 06:24 1607 06:38 1607 06:53 1607 07:07 1607 07:28 2507 19:02 2507 19:24 2507 19:37 2507 19:50 2507 20:16 2507 20:36 3107 19:26 3107 19:40 3107 19:54 3107 20:08

1 3107 20:21A 3107 20:42a 1407 11:18

1507 06:30 1907 13:04 3007 09:34 0108 08:08 0108 10:32 0108 13:44

RMS (g/kg)

RMS(%)

All profiles of bias and RMS deviation between BASIL and CNRS DIAL

With the exception of a few points,

• bias within ± 1 g/kg (or ± 20 %), • RMS < 1.5 g/kg (or 30 %).

High bias values are often found in coincidence with high values of RMS, especially in daytime comparisons, which may be associated with the large random error affecting BASIL measurements.

0

500

1000

1500

2000

2500

3000

3500

5 10 15 20

0,50 0,75 1,00

RMS (%)

RMS (g/kg)

RMS (%)H

eigh

t (m

)

RMS (g/kg)

Profiles of mean bias and mean RMS deviation between BASIL and CNRS DIAL obtained considering the 25 profile-to-profile intercomparisons

Mean bias increase with altitude from -1 % (-0.1 g/kg) to +10 % (+0.1 g/kg), with an intermediate maximum of +5 % (+0.25 g/kg) around 2 km (i.e. up to the top of the boundary layer). Larger bias values are found at the top of the boundary layer, where the effect of in-homogeneities may be larger.

Overall mean bias: 2.1 % (or 0.12 g/kg) Overall RMS deviation: 13.7 % (0.82 g/kg) in the altitude region 0.5–3.5 km a.s.l.


Overall mean bias: 1.13 %Overall RMS deviation: 10.1 % in the alt. region 0.5–3.0 km a.s.l. (cut-off dist. 5 km)

8 night-time and 17 daytime.When considering only the night-time comparisons, the analysis can be extended up to higher heights, i.e. up to the CNRS DIAL flight altitude (~4.5 km)


N. Date Time (UTC) Min. distance (km) BIAS (%) RMS dev. (%)

1 14 July 2007 11:18 5.43 -4.79 22.1

2 15 July 2007 6:30 9.70 0.89 22.8

3 16 July 2007 06:09 2.24 1.42 11.74

4 16 July 2007 06:24 2.15 -7.21 18.05

5 16 July 2007 06:38 2.03 -4.47 16.34

6 16 July 2007 06:53 1.066 -4.18 13.16

7 16 July 2007 07:07 2.40 8.20 27.30

8 16 July 2007 07:28 7.51 -7.37 15.15

9 19 July 2007 13:04 3.19 -12.1 27.5

10 25 July 2007 19:02 7.12 -1.09 20.43

11 25 July 2007 19:24 2.21 11.05 18.8

12 25 July 2007 19:37 1.30 8.91 14.39

13 25 July 2007 20:04 2.66 7.74 11.97

14 25 July 2007 20:16 2.706 9.62 18.87

15 25 July 2007 20:36 4.43 9.34 14.92

16 30 July 2007 9:34 9.5 9.26 29.1

17 31 July 2007 19:26 2.011 1.91 9.62

18 31 July 2007 19:40 1.61 5.6 6.21

19 31 July 2007 19:54 1.64 1.75 5.38

20 31 July 2007 20:08 1.842 1.52 6.50

21 31 July 2007 20:21 3.01 -4.11 7.12

22 31 July 2007 20:42 3.017 -1.93 10.67

23 1 August 2007 08:08 1.62 -7.0 15.0

24 1 August 2007 10:32 8.8 -3.75 17.66

25 1 August 2007 13:44 5.22 -4.51 18.92

0 2 4 6 8 10-15

-10

-5

0

5

10

15

BIA

S (

%)

Distance (km)

A total of 3 profile-to-profile inter-comparisons between BASIL and DLR DIAL





0

500

1000

1500

2000

2500

3000

3500

-15 -12 -9 -6 -3 0 3 6 9 12 15

-0,50 -0,25 0,00 0,25 0,50

BIAS (%)

BIAS (g/kg)

BIAS (%)

Hei

ght (

m)

BIAS (g/kg)

0

500

1000

1500

2000

2500

3000

3500

0 20 40 60 80 100

0 1 2 3 4 5

RMS (%)

RMS (g/kg)

RMS (%)

Hei

ght (

m)

RMS (g/kg) The mean bias is found to be within ± 3 % up to 3 km or ± 0.3 g/kg all the way up to 3 km.

Profiles of mean bias and mean RMS deviation between BASIL

and DLR DIAL

All profiles of bias and RMS deviation between BASIL and DLR DIAL

Overall mean bias: -1.85 % (or 0.037 g/kg) Overall RMS deviation: 15.2 % (0.59 g/kg) in the altitude region 0.5–3.5 km a.s.l.

Overall mean bias: -1.04 % (or -0.018 g/kg) Overall RMS deviation: 10.03 % (0.4 g/kg) in the altitude region 0.5–3.0 km a.s.l.

A total of 6 profile-to-profile inter-comparisons between BERTHA and CNRS DIAL (only night-time)




0 2 4 6 8 100

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

-2 0 2

31 July 07, 1926 UTC

Hei

ght (

m)

Mixing ratio (g/kg))

CNRS DIAL Bertha Raman lidar

Bias (g/kg))


Profiles of mean bias and mean RMS deviation between

BERTHA and DLR DIAL

All profiles of bias and RMS deviation between BERTHA

and CNRS DIAL

Overall mean bias: -4.37 % (or 0.123 g/kg) Overall RMS deviation: 23 % (0.662 g/kg) in the altitude region 0.5–4.5 km a.s.l.

The mean relative bias is found to vary with altitude from 20 % in the lower height interval to -30 % in the 2.5-3.0 km interval (i.e. up to the top of the boundary layer). The mean absolute bias shows a lower altitude variability, with values in the range -0.3÷0.2 g/kg. Here and above negative values indicate that CNRS DIAL is dried than BERTHA.

(Only night-time)

A total of 9 profile-to-profile inter-comparisons between ING Raman Lidar and CNRS DIAL (6 night-time, 3

daytime)



• The two profiles show a very good agreement, with deviations not exceeding 1 g/kg.

0

1000

2000

3000

4000

5000

0 2 4 6 8 10 12 -1 0 1 2

(a)

Water Vapour mixing ratio (g/kg)

Hei

ght (

m)

CNRS DIAL IGN Lidar

31 July 07, 20:56 UTC

Bias (g/kg)

(b)

0

500

1000

1500

2000

2500

3000

3500

4000

4500

0 1 2 3 4 5 50 100

250707_19:18 250707_20:26 250707_20:50 310707_19:20 310707_20:35 310707_20:56

RMS (g/kg)

2507 19:18 2507 20:26 2507 20:50 3107 19:20 3107 20:35 3107 20:56 1507 07:13 1607 07:43 2607 08:04

Hei

ght(

m)

RMS (%)

0

500

1000

1500

2000

2500

3000

3500

4000

4500

-40 -30 -20 -10 0 10 20 30 40

-2 -1 0 1 2

BIAS (%)

BIAS (g/kg)

BIAS (%)

Hei

ght (

m)

BIAS (g/kg)

0

500

1000

1500

2000

2500

3000

3500

4000

4500

0 10 20 30 40 50 60 70 80

0 1 2

RMS (%)

RMS (g/kg)

RMS (%)

Hei

ght (

m)

RMS (g/kg)

Profiles of mean bias and mean RMS deviation between ING Raman Lidar and DLR DIAL

All profiles of bias and RMS deviation between ING

Raman Lidar and CNRS DIAL

Mean relative bias: max. variability in the 2-3 km interval (i.e. up to the top of the boundary layer), up to ± 15 %.

Mean absolute bias: -0.2÷1 g/kg.

9 inter-comparisonsOverall mean bias: 10.86 % (or 0.69 g/kg) Overall RMS deviation: 19.35 % (1.74 g/kg) in the altitude region 0.5–2.0 km a.s.l.

6 night-time inter-comparisonsOverall mean bias: 3.18 % (or 0.27 g/kg) in the altitude region 0.5–4.5 km a.s.l.

0 1x1023 2x1023

0

1000

2000

3000

4000

5000

-1,0x1023-5,0x1022 0,0 5,0x1022 1,0x1023

HE

IGH

T (m

)

NUMBER DENSITY

DLR UHOH

BIAS (NUMBER DENSITY)

BIAS

A total of 4 profile-to-profile inter-comparisons between UHOH DIAL and DLR DIAL

Example of compariosn at 07:22 on 15 July 2007

• min. distance between the two sensors of 1.5 km

• The two profiles show a very good agreement, with deviations not exceeding ± 2x1022 m-3.

4 inter-comparisonsOverall mean bias: 2.9 % in the alti

UHOH DIALCNRS DIAL

1.8.2007, 13:46 UTC

A total of 5 profile-to-profile inter-comparisons between CNRS DIAL and DLR DIAL

Example of compariosn at 11:53 on 30 July 2007

• min. distance between the two sensors of 2.48 km

• The two profiles show a very good agreement, with deviations not exceeding ± 0.25 g/kg or ± 1.5x1022 m-3.

N.

Date Time (UTC)

Min. distance (km)

BIAS (%)

RMS dev. (%)

tracks

1 18 July 2007

16:01 3.67 9.2 10.94 coincident, opp.

direction

2 18 July 2007

16:17 9.82 7.46 23.55 cross

3 30 July 2007

10:41 6.62 9.97 44.30 cross

4 30 July 2007

11:53 2.48 -6.99 14.59 Parallel, same

direction

5 30 July 2007

12:09 3.57 -0.184 %

19.63 parallel, same

direction

Water vapour heterogeneity plays a major role in the interpretation of the airborne-to-airborne inter-comparisons, with effects being generally more marked than for the ground-based-to-airborne lidar inter-comparisons illustrated before.These effects are mainly due to horizontal averaging and are more severe for isolated crossing points of the flight tracks.6 night-time inter-comparisonsOverall mean bias: 3.93 % (or 0.168 g/kg or 1.1x1022 m-3 ) in the altitude region 0.5–4.0 km a.s.l.

Smaller bias values are present when comparing data collected along parallel flight tracks and coincident directions for the two air-borne systems

0 5,89%-4.77% -0.05%

BASIL BERTHADLR DIAL CNRS DIAL

1.52%

UHOH DIAL

-1.83 %

ING

-1.06 %

• Derive the overall bias values for all the lidar systems from mutual bias values.

• This is possible when there is at least one instrument that carried out measurements that are comparable with those of all other lidar systems.

• This was the CNRS DIAL that, thanks to the several flights performed in the frame of the EUFAR Project H2OLidar, was able to guarantee multiple overpasses over all Supersites equipped with ground-based lidar systems.

• We attribute equal weight on the data reliability of each instrument and impose the summation of all mutual bias between lidar pairs to be zero.

• Overall relative values for UHOH DIAL, DLR DIAL, IGN Raman lidar, BASIL, CNRS DIAL and BERTHA are found to be -4.77 %, -1.83 %, -1.06 %, 0.05 %, 1.52 % and 5.89 %, respectively.

•All sensors are found to be characterized by a overall bias not exceeding 5 %.

An intensive inter-comparison effort involving 6 water vapour lidar systems was carried out in the frame of COPS with the goal of providing accurate error estimates for these systems.

A total of 57 profile-to-profile inter-comparisons involving all possible lidar pairs were considered.

Results reveal the presence of low systematic errors (bias) – not exceeding 5 % - in the measurements carried out by all lidar systems operated during COPS.

Specifically, overall relative values for the involved lidar systems are found to be: -4.77 %, -1.83 %, -1.06 %, 0.05 %, 1.52 % and 5.89 % for UHOH DIAL, DLR DIAL, IGN Raman lidar, BASIL, CNRS DIAL and BERTHA, respectively.

For what concerns the airborne-to-ground-based inter-comparisons, there appear to be no evident dependency of the bias and RMS deviation on spatial distance between the different lidar pairs.

Concerning to the airborne-to-airborne inter-comparisons, results indicate that smaller bias values are present when comparing data collected along parallel flight tracks and coincident directions for the two air-borne systems.

Summary


Airborne DIAL vs Ground-based Lidar

In order to reduce statistical fluctuations, for the purpose of the present inter-comparison, we considered:

• for the CNRS DIAL an integration time of 80 sec, corresponding to an horizontal integration length of 12-15 km (vert. res.: 250 m).

• for the DLR DIAL an integration time of 50 sec, corresponding to an horizontal integration length of 7.5-10 km (vert. res.: 300 m; step: 25 m).

• for BASIL an integration time of 1 min for night-time measurements and of 5 min for daytime measurements (vert. res.: 150 m; step: 30 m).

• for the UHOH DIAL an integration time of 5 min.

• for the BERTHA Raman Lidar an integration time of 3 min (vert. res.: 150 m; step: 60 m).

• for the ING Raman Lidar an integration time of 15 min.


Coming to the airborne-to-airborne inter-comparisons, results indicate that smaller bias values are present when comparing data collected along parallel flight tracks and coincident directions for the two air-borne systems.Comparison of data from airborne and ground-based lidars has the potential to allow assessing the representativeness error of vertically pointing ground-based lidar systems, and in general of ground-based remote sensors, used for satellite data validation, and assess the sub-grid scale variability of water vapour usually parameterized in mesoscale atmospheric models.

Documents

Joint 8th COPS Workshop and CSIP Meeting 2009, 26-28 October 2009