21
ROM SAF Report 36 Ref: SAF/ROM/METO/REP/RSR/036 Web: www.romsaf.org Date: 23 Jun 2020 ROM SAF Report 36 An initial assessment of the quality of RO data from PAZ N E Bowler Met Office, Exeter, UK

An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36Ref: SAF/ROM/METO/REP/RSR/036Web: www.romsaf.orgDate: 23 Jun 2020

ROM SAF Report 36

An initial assessment of the quality of RO data from PAZ

N E Bowler

Met Office, Exeter, UK

Page 2: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

Document Author Table

Name Function Date CommentsPrepared by: N E Bowler ROM SAF Project Team 23 Jun 2020Reviewed by: M Forsythe Met Office review 27 Jan 2020Reviewed by: J Eyre Met Office review 08 Feb 2020Reviewed by: S Healy ROM SAF Science Coordinator 09 Mar 2020Approved by: K B Lauritsen ROM SAF Project Manager 18 Jun 2020

Document Change Record

Issue/Revision Date By Description0.1 14 Jan 2020 NEB 1st draft

0.2 04 Feb 2020 NEB 2nd draft, following Mary’s review

0.3 12 Feb 2020 NEB 3rd draft, following John’s review

0.4 12 Mar 2020 NEB 4th draft, following Sean’s review

1.0 23 Jun 2020 NEB Final version

ROM SAF

The Radio Occultation Meteorology Satellite Application Facility (ROM SAF) is a decentralised pro-cessing centre under EUMETSAT which is responsible for operational processing of radio occultation(RO) data from the Metop and Metop-SG satellites and radio occultation data from other missions.The ROM SAF delivers bending angle, refractivity, temperature, pressure, humidity, and other geo-physical variables in near real-time for NWP users, as well as reprocessed Climate Data Records(CDRs) and Interim Climate Data Records (ICDRs) for users requiring a higher degree of homogene-ity of the RO data sets. The CDRs and ICDRs are further processed into globally gridded monthly-mean data for use in climate monitoring and climate science applications.The ROM SAF also maintains the Radio Occultation Processing Package (ROPP) which containssoftware modules that aid users wishing to process, quality-control and assimilate radio occultationdata from any radio occultation mission into NWP and other models.The ROM SAF Leading Entity is the Danish Meteorological Institute (DMI), with Cooperating Entities:i) European Centre for Medium-Range Weather Forecasts (ECMWF) in Reading, United Kingdom,ii) Institut D’Estudis Espacials de Catalunya (IEEC) in Barcelona, Spain, and iii) Met Office in Exeter,United Kingdom. To get access to our products or to read more about the ROM SAF please go to:https://www.romsaf.org

Intellectual Property Rights

All intellectual property rights of the ROM SAF products belong to EUMETSAT. The use of theseproducts is granted to every interested user, free of charge. If you wish to use these products, EU-METSAT’s copyright credit must be shown by displaying the words "copyright (year) EUMETSAT" oneach of the products used.

Page 3: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

Abstract

The PAZ satellite was launched on 22nd February 2018 into a sun-synchronous polar orbit. Thissatellite carries a polarimetric instrument for taking radio-occultation measurements of the atmo-sphere. The instrument, known as ROHPP (Radio Occultations and Heavy Precipitation with PAZ)is a variant of an earlier IGOR (Integrated GPS Occultation Receiver) instrument which has beenadapted to measure horizontally and vertically polarised signals. Only one radio-occultation instru-ment is carried by the satellite, and therefore it is only able to measure setting occultations. Due to thedual-polarisation nature of the instrument, the calculation of bending angles follows a slightly differentpath from conventional instruments.

Overall the performance of the bending angle data is similar to that from other operational instru-ments. There are differences for the bending angle data in the troposphere, but this is most likelyto be due to differences between processing centres, rather than instrumental differences. The re-fractivity data above 35 km is more accurate than similar data from FY-3C, even though the bendingangle accuracy is similar at these levels. This is believed to be due to differences in the choice ofclimatology used in the processing.

The data have recently been made available in near-real-time. Due to problems with connectionbetween the satellite to the ground infrastructure, the data is experiencing long delays. Improvementshave been made recently, but more work in this area is still needed.

3

Page 4: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

Contents

1 Bending angle evaluation 51.1 Bias and standard deviation characteristics . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Vertical correlations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Refractivity assessment 9

3 Assimilation test 13

4 Other notable features 16

5 Conclusion 18

Bibliography 21

4

Page 5: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

1 Bending angle evaluation

The PAZ satellite is operated by Hisdesat, and the radio occultation data are downloaded to NOAA’sFairbanks station and then transferred to UCAR (University Corporation for Atmospheric Research)for processing. Past data have been processed by UCAR from the 10th May 2018 onwards, and havebeen made available on the CDAAC (COSMIC Data Analysis and Archive Center) website. Near-realtime GNSS-RO data from the PAZ satellite has been available via the Global TelecommunicationSystem (GTS) since 19th December 2019. To assess the performance of the instrument, data wasextracted from CDAAC for between 1 December 2018 and 28th February 2019.

The ROHPP instrument onboard PAZ is intended to measure precipitation by examining the po-larisation of the received signal. Measuring the signal due to precipitation has been the subject ofan extended study [3] and is not considered here. This report focusses on the conventional bend-ing angle and refractivity statistics. Considerable work has been required to process these data intostandard products [5].

1.1 Bias and standard deviation characteristics

Figure 1.1 shows the normalised difference between the bending angle observation and the back-ground forecast from the Met Office’s operational global numerical weather prediction (NWP) model.These differences are known as the innovations. The mean and standard deviation are calculated as

µ =1N

N

∑i=1

Oi −Bi

Bi(1.1)

σ =

√1N

N

∑i=1

(Oi −Bi

Bi−µ

)2

(1.2)

where Oi and Bi are the observed and background values for occultation i in the period, and thereare N occultations overall. Figure 1.1 compares the statistics of PAZ with FY-3C and with Metop-A. PAZ provides fewer observations than either Metop-A or FY-3C. This is expected, since there isonly one receiver onboard the satellite, and it therefore only measures setting occultations. Above10 km the mean statistics from each satellite are very similar. Below this level PAZ shows a negativevalue for the mean, FY-3C shows a positive value, and Metop-A is negative above 5 km and positivebelow. Other satellites processed by UCAR also show a negative mean value in this region, so thesedifferences are likely to be a result of processing differences and not related to the instruments.

Above 35 km the standard deviations of the normalised innovations are smallest for Metop-A, andsimilar for FY-3C and PAZ. Between 10 km and 35 km PAZ and Metop-A have similar standarddeviations with FY-3C larger. Below 10 km PAZ and FY-3C have similar standard deviations, withMetop-A larger.

Figure 1.2 shows the statistics for the normalised innovations for PAZ, before and after variousquality-control steps have been applied. Approximately 20% of the data have been flagged as havingproblems by UCAR. In addition to this a small number of occultations are rejected by the Met Officequality control. The number of observations rejected by the data provider is rather higher than onemight hope for, but perhaps illustrates the difficulties in generating bending angles from polarimetricdata.

5

Page 6: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

−10 −8 −6 −4 −2 0 2 4 6 8 10(O-B)/B [%]

0

5

10

15

20

25

30

35

40

45

50

55

60

Impa

ct h

eigh

t [km

]

QC appliedNo. of occultations: 10893, 36573, 33683Data from 01/12/18 to 28/02/19

BA Global O-B statistics for Paz, Metop-A, FY-3Cprovided by UCAR, DMI, CMA

0 5000 10000 15000 20000 25000 30000 35000Number of occultations

Mean (Paz)Std dev (Paz)Mean (Metop-A)Std dev (Metop-A)Mean (FY-3C)Std dev (FY-3C)

1

10

100

1000

Ap

prox

imat

e pr

essu

re [h

Pa]

Figure 1.1: The bias and standard deviation of the normalised difference between theobservation and the NWP model background forecast (O−B)/B for bending angle. Com-parison between statistics for PAZ with Metop-A and FY-3C.

−10 −8 −6 −4 −2 0 2 4 6 8 10(O-B)/B [%]

0

5

10

15

20

25

30

35

40

45

50

55

60

Impa

ct h

eigh

t [km

]

0 2000 4000 6000 8000 10000All dataBAngle not flaggedSummary bit not flaggedData assimilated

1

10

100

1000

Ap

prox

imat

e pr

essu

re [h

Pa]

No. of occultations: 11530Data from 03/12/18 to 21/01/19

Plotted at 10:38, 08 Jan 2020

BA Pre- and post-QC statistics for PAZ provided by UCAR

Figure 1.2: The bias and standard deviation of the normalised difference between the ob-servation and the NWP model background forecast (O−B)/B for bending angle. Statisticsshown for various points in the quality-control process.

6

Page 7: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

−10 −8 −6 −4 −2 0 2 4 6 8 10(O-B)/B [%]

0

5

10

15

20

25

30

35

40

45

50

55

60

Impa

ct h

eigh

t [km

]

0 500 1000 1500 2000 2500 3000Number of occultations

High lat: |lat|>60degMid lat: 30deg<|lat|<60degLow lat: |lat|<30deg

1

10

100

1000

Appr

oxim

ate

pres

sure

[hPa

]

QC appliedNo. of occultations: 11244 (8703 after QC)Data from 03/12/18 to 18/01/19

Plotted at 09:10, 08 Jan 2020

BA O-B statistics by latitude for PAZ provided by UCAR

−10 −8 −6 −4 −2 0 2 4 6 8 10(O-B)/B [%]

0

5

10

15

20

25

30

35

40

45

50

55

60

Impa

ct h

eigh

t [km

]

0 2000 4000 6000 8000 10000 12000 14000Number of occultations

High lat: |lat|>60degMid lat: 30deg<|lat|<60degLow lat: |lat|<30deg

1

10

100

1000

Ap

prox

imat

e pr

essu

re [h

Pa]

QC appliedNo. of occultations: 56052 (36573 after QC)Data from 01/12/18 to 28/02/19

Plotted at 09:13, 08 Jan 2020

BA O-B statistics by latitude for Metop-A provided by DMI

Figure 1.3: The bias and standard deviation of the normalised difference between theobservation and the NWP model background forecast (O−B)/B, separated by differentlatitudes for bending angle. Statistics for PAZ (left) and Metop-A (right).

Figure 1.3 shows the bias and standard deviation of the normalised innovations, as Figure 1.1,but separated by different latitude ranges. In the tropical region the standard deviations appear tohave a sharper peak in the data for PAZ. Additionally, the standard deviation at high latitudes in thelower stratosphere (7-20 km) appears to be larger for PAZ than Metop-A. This pattern of differencesis somewhat similar to that seen for KOMPSAT-5 [1].

1.2 Vertical correlations

When bending angle data is assimilated into the Met Office’s NWP system it is assumed that the errorin each bending angle measurement is independent of the errors in every other measurement. There-fore we would like the vertical observation-error covariance matrix R to be diagonal, and the verticalcorrelations of O−B (which corresponds to the covariance matrix B+R) to be close to diagonal,containing only the correlations from the background-error covariance matrix. Figure 1.4 shows thevertical correlation of the normalised innovations for PAZ and Metop-A. Due to the way that bendingangle is calculated as a smoothed difference between Doppler shifts, we expect a region of positivecorrelations near the diagonal, and negative correlations at further distances.

The vertical correlations for PAZ are generally somewhat similar to that seen for Metop-A. Above20 km the core of the positive correlations is broader in the PAZ data than for Metop, indicating thata greater level of vertical smoothing is used for PAZ. Below 20 km there is a sharp decrease in thevertical correlations for PAZ which is not seen in the Metop data. Below 10 km the vertical correlationsfor Metop data become larger, which is not seen in PAZ data.

7

Page 8: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

0 10 20 30 40 50 60Impact height [km]

0

10

20

30

40

50

60

Impa

ct h

eigh

t [km

]

QC appliedNo. of occultations: 12237 (9461 after QC)Data from 03/12/18 to 24/01/19

Plotted at 11:03, 08 Jan 2020

BA Vertical (O-B)/B correlations for PAZprovided by UCAR

−1.00

−0.75

−0.50

−0.25

0.00

0.25

0.50

0.75

1.00

0 10 20 30 40 50 60Impact height [km]

0

10

20

30

40

50

60

Impa

ct h

eigh

t [km

]QC appliedNo. of occultations: 56052 (36573 after QC)Data from 01/12/18 to 28/02/19

Plotted at 11:05, 08 Jan 2020

BA Vertical (O-B)/B correlations for Metop-Aprovided by DMI

−1.00

−0.75

−0.50

−0.25

0.00

0.25

0.50

0.75

1.00

Figure 1.4: Vertical correlations of normalised differences between the observation andthe NWP model background for bending angle.

8

Page 9: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

2 Refractivity assessment

Measurements of refractivity are calculated from the bending angle, using an Abel integral and as-sumptions about the spherical symmetry of the atmosphere. Figure 2.1 shows the mean and standarddeviation of normalised differences between the observed refractivity and that produced by the NWPmodel forecast (i.e. the normalised innovations for refractivity). The mean normalised innovation isvery similar between all three satellites between 10 km and 45 km. Above 45 km the mean inno-vation for FY-3C is larger than for the other two satellites. In the troposphere there are small meandifferences between the three satellites, mirroring the differences seen for bending angle.

The standard deviation of the refractivity innovations from PAZ are smaller than those from FY-3Cthroughout the atmosphere. They are larger than those from Metop-A above 40 km, but are smallerbelow 10 km. Given that the standard deviations of bending angle innovations above 40 km are similarbetween PAZ and FY-3C, it is remarkable that the refractivity standard deviations at high levels shouldbe much smaller. This is presumably due to UCAR using a more sophisticated climatology [4] in thestatistical opimisation of the refractivity data.

The vertical correlation of the difference between modelled and observed refractivity are shownin Figure 2.2 for PAZ, FY-3C and Metop-A. Since refractivity is derived from the vertical integral ofthe bending angles, it is expected that there are long-range correlations in this quantity. In the datafor Metop-A there are negative correlations off-diagonal around 30 km. A similar region of smallcorrelations is seen in the data for PAZ, but is absent for FY-3C. Above this region all the data arepositively correlated, with Metop-A having the smaller long-range correlations. Between 10 km and20 km the data from PAZ is positively correlated. 20 km corresponds with the point at which thevertical correlations for bending angle showed a sharp increase (Figure 1.4).

The spatial distribution of the average normalised innovation is shown in Figure 2.3 for PAZ andMetop-A. Although the global average of the refractivity difference for PAZ is similar to that for Metop-A (Fig. 2.1) this plot shows that there are large regional differences, particularly around the northpole. Presumably this is driven by the differences in the climatological data used in the statisticaloptimisation.

9

Page 10: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

−10 −8 −6 −4 −2 0 2 4 6 8 10(O-B)/B [%]

0

5

10

15

20

25

30

35

40

45

50

55

60

Geop

oten

tial H

eigh

t [km

]

QC appliedNo. of occultations: 10075, 34097, 33091Data from 01/12/18 to 28/02/19

REF Global O-B statistics for Paz, Metop-A, FY-3Cprovided by UCAR, DMI, CMA

0 5000 10000 15000 20000 25000 30000Number of occultations

Mean (Paz)Std dev (Paz)Mean (Metop-A)Std dev (Metop-A)Mean (FY-3C)Std dev (FY-3C)

1

10

100

1000

Ap

prox

imat

e pr

essu

re [h

Pa]

Figure 2.1: The bias and standard deviation of the normalised difference between theobservation and the NWP model background forecast (O−B)/B for refractivity.

10

Page 11: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

0 10 20 30 40 50 60Geopotential Height [km]

0

10

20

30

40

50

60

Geop

oten

tial H

eigh

t [km

]

QC appliedNo. of occultations: 20961 (16570 after QC)Data from 03/12/18 to 01/03/19

Plotted at 14:19, 09 Jan 2020

REF Vertical (O-B)/B correlations for PAZprovided by UCAR

−1.00

−0.75

−0.50

−0.25

0.00

0.25

0.50

0.75

1.00

0 10 20 30 40 50 60Geopotential Height [km]

0

10

20

30

40

50

60

Geop

oten

tial H

eigh

t [km

]

QC appliedNo. of occultations: 34595 (33091 after QC)Data from 01/12/18 to 28/02/19

Plotted at 14:21, 09 Jan 2020

REF Vertical (O-B)/B correlations for FY-3Cprovided by CMA

−1.00

−0.75

−0.50

−0.25

0.00

0.25

0.50

0.75

1.00

0 10 20 30 40 50 60Geopotential Height [km]

0

10

20

30

40

50

60

Geop

oten

tial H

eigh

t [km

]

QC appliedNo. of occultations: 56052 (34097 after QC)Data from 01/12/18 to 28/02/19

Plotted at 14:21, 09 Jan 2020

REF Vertical (O-B)/B correlations for Metop-Aprovided by DMI

−1.00

−0.75

−0.50

−0.25

0.00

0.25

0.50

0.75

1.00

Figure 2.2: Vertical correlations of normalised differences between the observation andthe NWP model background for refractivity.

11

Page 12: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

QC appliedNo. of occultations: 20961 (16570 after QC)Data from 03/12/18 to 01/03/19

Plotted at 16:02, 09 Jan 2020

REF Mean (O-B)/B REF from 50.0 to 55.0 km: PAZprovided by UCAR

Backgrounds from Met Office

−15

−10

−5

0

5

10

15

QC appliedNo. of occultations: 56052 (34097 after QC)Data from 01/12/18 to 28/02/19

Plotted at 16:02, 09 Jan 2020

REF Mean (O-B)/B REF from 50.0 to 55.0 km: Metop-Aprovided by DMI

Backgrounds from Met Office

−15

−10

−5

0

5

10

15

Figure 2.3: The mean value of the normalised difference between the observation andthe NWP model background forecast (O−B)/B, averaged into 5 degree bins of latitudeand longitude.

12

Page 13: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

3 Assimilation test

In order to test the impact that adding these data would have on the operational NWP system anassimilation test was run. This test mimics the operational system at low resolution, adding bendingangle observations from PAZ to the data assimilation. Since the test was run on data from the CDAACarchive, all PAZ data were considered for assimilation with no consideration of the timeliness of thedata. This is compared with a similar low-resolution run without the additional observations. This runalso did not include observations from FY-3D or KOMPSAT, as these satellites have only recentlybeen accepted into operations. The test was run between 1st December 2018 and 28th February2019 using a global forecast model at N320 resolution (640x480 grid-points). 7-day forecasts arelaunched every 12h, and these are verified against ECMWF analyses and against observations.Verification results are shown in Figure 3.1. These show that the forecasts which include PAZ obser-vations have smaller errors for many variables. The change appears to be particularly beneficial whenverifying against ECMWF analyses. There are no variables for which there is a clear degradation inperformance.

It will be noted that these improvements are somewhat similar to those that were seen when in-troducing observations from FY-3D [2]. Since both satellites provide good quality RO data, it is not asurprise that their impact on the Met Office’s NWP system is similar. When verifying impacts usingthe root-mean-square error (RMSE) as in Figure 3.1 the benefit seen is from a combination of the re-duction in bias as well as a reduction in the time-varying forecast errors. To help separate these two,we can examine the impact of the change using the difference in the standard deviation of the fore-cast error — this is shown in Figure 3.2. Although some of the individual results change, especiallyfor verification against ECMWF analyses, the overall message conclusion that the new observationsbring benefit is unchanged.

When assessing forecast performance, we also consider assimilation statistics. This is the root-mean-square (RMS) difference between the forecast from the previous data assimilation cycle (6hago) and the observations. The change in the RMS difference to satellite sounding channels (mi-crowave and infra-red) provides useful information on the behaviour of the assimilation with the newobservations. For this test the change in the RMS difference is generally neutral (with some channelsshowing a larger RMS difference and some showing a smaller RMS).

13

Page 14: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

T+0

T+6

T+12

T+24

T+36

T+48

T+60

T+72

T+84

T+96

T+10

8T+

120

T+13

2T+

144

T+16

8

NH_T50NH_T100NH_T250NH_T500NH_T850NH_T_2mNH_W50

NH_W100NH_W250NH_W500NH_W850

NH_W10mNH_Z250NH_Z500NH_Z850

TR_T50TR_T100TR_T250TR_T500TR_T850TR_T_2mTR_W50

TR_W100TR_W250TR_W500TR_W850

TR_W10mSH_T50

SH_T100SH_T250SH_T500SH_T850SH_T_2mSH_W50

SH_W100SH_W250SH_W500SH_W850

SH_W10mSH_Z250SH_Z500SH_Z850

anlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanl

max = 10

% Difference (Add PAZ vs.Control) - overall 0.15%

RMSE against ecanal for 20181201 to 20190228

T+0

T+6

T+12

T+24

T+36

T+48

T+60

T+72

T+84

T+96

T+10

8T+

120

T+13

2T+

144

T+16

8

NH_PMSLNH_T50

NH_T100NH_T250NH_T500NH_T850NH_T_2mNH_W50

NH_W100NH_W250NH_W500NH_W850

NH_W10mNH_Z250NH_Z500NH_Z850

TR_T50TR_T100TR_T250TR_T500TR_T850TR_T_2mTR_W50

TR_W100TR_W250TR_W500TR_W850

TR_W10mSH_PMSL

SH_T50SH_T100SH_T250SH_T500SH_T850SH_T_2mSH_W50

SH_W100SH_W250SH_W500SH_W850

SH_W10mSH_Z250SH_Z500SH_Z850

surfsondessondessondessondessondessurfsondessondesAMDARSsondesSatwindsurfsondessondessondessondessondessondessondessondessurfsondessondesAMDARSsondesSatwindsurfsurfsondessondessondessondessondessurfsondessondesAMDARSsondesSatwindsurfsondessondessondes

max = 10

% Difference (Add PAZ vs.Control) - overall 0.07%

RMSE against observations for 20181201 to 20190228

Figure 3.1: Verification results for a test using observations with PAZ, compared to a base-line system using the root-mean-square error (RMSE). (left) Verification against ECMWFanalyses and (right) verification against observations. The triangles show the change inthe root-mean-square (RMS) error of the forecast, with green (blue) triangles indicatingthat the test has smaller (larger) errors. Where the change is statistically significant thebox surrounding the triangle is shaded.

14

Page 15: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

T+0

T+6

T+12

T+24

T+36

T+48

T+60

T+72

T+84

T+96

T+10

8T+

120

T+13

2T+

144

T+16

8

NH_T50NH_T100NH_T250NH_T500NH_T850NH_T_2mNH_W50

NH_W100NH_W250NH_W500NH_W850

NH_W10mNH_Z250NH_Z500NH_Z850

TR_T50TR_T100TR_T250TR_T500TR_T850TR_T_2mTR_W50

TR_W100TR_W250TR_W500TR_W850

TR_W10mSH_T50

SH_T100SH_T250SH_T500SH_T850SH_T_2mSH_W50

SH_W100SH_W250SH_W500SH_W850

SH_W10mSH_Z250SH_Z500SH_Z850

anlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanlanl

max = 10

% Difference (Add PAZ vs.Control) - overall 0.12%

sd against ecanal for 20181201 to 20190228

T+0

T+6

T+12

T+24

T+36

T+48

T+60

T+72

T+84

T+96

T+10

8T+

120

T+13

2T+

144

T+16

8

NH_PMSLNH_T50

NH_T100NH_T250NH_T500NH_T850NH_T_2mNH_W50

NH_W100NH_W250NH_W500NH_W850

NH_W10mNH_Z250NH_Z500NH_Z850

TR_T50TR_T100TR_T250TR_T500TR_T850TR_T_2mTR_W50

TR_W100TR_W250TR_W500TR_W850

TR_W10mSH_PMSL

SH_T50SH_T100SH_T250SH_T500SH_T850SH_T_2mSH_W50

SH_W100SH_W250SH_W500SH_W850

SH_W10mSH_Z250SH_Z500SH_Z850

surfsondessondessondessondessondessurfsondessondesAMDARSsondesSatwindsurfsondessondessondessondessondessondessondessondessurfsondessondesAMDARSsondesSatwindsurfsurfsondessondessondessondessondessurfsondessondesAMDARSsondesSatwindsurfsondessondessondes

max = 10

% Difference (Add PAZ vs.Control) - overall 0.06%

sd against observations for 20181201 to 20190228

Figure 3.2: Verification results for a test using observations with PAZ, compared to abaseline system using the standard deviation. (left) Verification against ECMWF analysesand (right) verification against observations. Figure format as Figure 3.1

15

Page 16: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

4 Other notable features

The data from PAZ have only been available on the GTS since 19th December 2019. The timelinessof the data, as received via GTS is shown in Figure 4.1. The data is suffering from long delays, andlittle of the data is being received within three hours. An investigation into the cause of the delaysis ongoing, and recent changes by Hisdesat have reduced the delays somewhat. It is understood (ECardellach, IEEC, personal communication, 2020) that the satellite is unable to make contact withthe Fairbanks ground station on every orbit. It is planned to allow PAZ to make use of a secondground station to reduce these problems. The “main” run of the Met Office forecast system takesplace at around 2 hours and 42 minutes after the nominal time of the data assimilation window, andthe “update” run occurs at approximately 6 hours and 19 minutes after the nominal time. We estimatethat approximately 11% of observations from PAZ would be used by the “main” run, and around 36%would be used by the “update” run. Given these low percentages it is not considered that PAZ isready for use in operational NWP.

16

Page 17: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

Figure 4.1: Time delay in receiving the occultations, as calculated from the receipt time inthe Met Office’s observations data base.

17

Page 18: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

5 Conclusion

Archived and near-real-time data from PAZ have been made available to centres around the world.The data are of similar quality to operational satellites. The following list highlights the main differ-ences that are of note:

• The vertical correlations of normalised bending angle innovations are noticeably longer-rangeabove 20 km than below this height.

• The standard deviations of normalised refractivity innovations are clearly smaller than thosefrom FY-3C above 35 km. The standard deviations of bending angle innovations in this heightrange are similar.

• The normalised refractivity innovations are strongly correlated between 10 km and 20 km.

• NWP trials show small but clear benefit from assimilating bending angles from PAZ.

• The near-real-time data suffers from substantial delays.

18

Page 19: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

ROM SAF (and earlier GRAS SAF) Reports

SAF/GRAS/METO/REP/GSR/001 Mono-dimensional thinning for GPS Radio Occulation

SAF/GRAS/METO/REP/GSR/002 Geodesy calculations in ROPP

SAF/GRAS/METO/REP/GSR/003 ROPP minimiser - minROPP

SAF/GRAS/METO/REP/GSR/004 Error function calculation in ROPP

SAF/GRAS/METO/REP/GSR/005 Refractivity calculations in ROPP

SAF/GRAS/METO/REP/GSR/006 Levenberg-Marquardt minimisation in ROPP

SAF/GRAS/METO/REP/GSR/007 Abel integral calculations in ROPP

SAF/GRAS/METO/REP/GSR/008 ROPP thinner algorithm

SAF/GRAS/METO/REP/GSR/009 Refractivity coefficients used in the assimilationof GPS radio occultation measurements

SAF/GRAS/METO/REP/GSR/010 Latitudinal Binning and Area-Weighted Averaging ofIrregularly Distributed Radio Occultation Data

SAF/GRAS/METO/REP/GSR/011 ROPP 1dVar validation

SAF/GRAS/METO/REP/GSR/012 Assimilation of Global Positioning System Radio OccultationData in the ECMWF ERA-Interim Re-analysis

SAF/GRAS/METO/REP/GSR/013 ROPP PP validation

SAF/ROM/METO/REP/RSR/014 A review of the geodesy calculations in ROPP

SAF/ROM/METO/REP/RSR/015 Improvements to the ROPP refractivity andbending angle operators

SAF/ROM/METO/REP/RSR/016 Simplifying EGM96 undulation calculations in ROPP

SAF/ROM/METO/REP/RSR/017 Simulation of L1 and L2 bending angles witha model ionosphere

SAF/ROM/METO/REP/RSR/018 Single Frequency Radio Occultation Retrievals:Impact on Numerical Weather Prediction

SAF/ROM/METO/REP/RSR/019 Implementation of the ROPP two-dimensional bendingangle observation operator in an NWP system

SAF/ROM/METO/REP/RSR/020 Interpolation artefact in ECMWF monthlystandard deviation plots

SAF/ROM/METO/REP/RSR/021 5th ROM SAF User Workshop on Applications ofGPS radio occultation measurements

SAF/ROM/METO/REP/RSR/022 The use of the GPS radio occultation reflection flagfor NWP applications

SAF/ROM/METO/REP/RSR/023 Assessment of a potential reflection flag product

SAF/ROM/METO/REP/RSR/024 The calculation of planetary boundary layer heights in ROPP

SAF/ROM/METO/REP/RSR/025 Survey on user requirements for potential ionosphericproducts from EPS-SG radio occultation measurements

19

Page 20: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

Bowler: Quality of PAZ ROM SAF Report 36

ROM SAF (and earlier GRAS SAF) Reports (cont.)

SAF/ROM/METO/REP/RSR/026 Estimates of GNSS radio occultation bending angleand refractivity error statistics

SAF/ROM/METO/REP/RSR/027 Recent forecast impact experiments with GPSradio occultation measurements

SAF/ROM/METO/REP/RSR/028 Description of wave optics modelling in ROPP-9 andsuggested improvements for ROPP-9.1

SAF/ROM/METO/REP/RSR/029 Testing reprocessed GPS radio occultation datasetsin a reanalysis system

SAF/ROM/METO/REP/RSR/030 A first look at the feasibility of assimilating singleand dual frequency bending angles

SAF/ROM/METO/REP/RSR/031 Sensitivity of some RO measurements to the shape ofthe ionospheric electron density profile

SAF/ROM/METO/REP/RSR/032 An initial assessment of the quality of RO data from KOMPSAT-5

SAF/ROM/METO/REP/RSR/033 Some science changes in ROPP-9.1

SAF/ROM/METO/REP/RSR/034 An initial assessment of the quality of RO data from Metop-C

SAF/ROM/METO/REP/RSR/035 An initial assessment of the quality of RO data from FY-3D

ROM SAF Reports are accessible via the ROM SAF website: http://www.romsaf.org

20

Page 21: An initial assessment of the quality of RO data from PAZ · time GNSS-RO data from the PAZ satellite has been available via the Global Telecommunication System (GTS) since 19th December

ROM SAF Report 36 Bowler: Quality of PAZ

Bibliography

[1] N. E. Bowler. An initial assessment of the quality of RO data from KOMPSAT-5, 2018. ROM SAFreport 32, available at https://www.romsaf.org/general-documents/rsr/rsr_32.pdf.

[2] Neill E. Bowler. An initial assessment of the quality of RO data from FY-3D, 2019. ROM SAFreport 35, available at https://www.romsaf.org/general-documents/rsr/rsr_35.pdf.

[3] E. Cardellach, S. Oliveras, A. Rius, S. Tomas, C. O. Ao, G. W. Franklin, B. A. Iijima, D. Kuang,T. K. Meehan, R. Padulles, M. de la Torre Juarez, F. J. Turk, D. C. Hunt, W. S. Schreiner, S. V.Sokolovskiy, T. Van Hove, J. P. Weiss, Y. Yoon, Z. Zeng, J. Clapp, W. Xia-Serafino, and F. Cerezo.Sensing Heavy Precipitation With GNSS Polarimetric Radio Occultations. Geophys Res Lett,46:1024–1031, 2019.

[4] M. E. Gorbunov and A. V. Shmakov. Statistically average atmospheric bending angle model basedon COSMIC experimental data. Izvestiya Atmospheric and Oceanic Physics, 52:622–628, 2016.

[5] D. Hunt. PAZ Neutral Atmosphere Radio Occultation Retrieval Processing. Pre-sentation given at the ROM SAF - IROWG 2019 Workshop, Elsinore, Den-mark, 19-25 September 2019, obtainable from https://www.romsaf.org/romsaf-irowg-2019/en/open/1570202090.ca527af47f02b56bd1d9cd50f448ec0d.pdf/Hunt__paz_hv_ro_processing_v03.pdf(last accessed 04 Feb 2020).

21