A multi-sensor physically based weather/non-weather radar echo classifier using polarimetric and environmental data in a real-time

national system

Lin TangCooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma,

Norman, Oklahoma

Jian Zhang NOAA/OAR National Severe Storms Laboratory, Norman, Oklahoma

Carrie LangstonCooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma,

Norman, Oklahoma

John KrauseCooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma,

Norman, Oklahoma

Kenneth HowardNOAA/OAR National Severe Storms Laboratory, Norman, Oklahoma

Valliappa LakshmananCooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma,

Norman, Oklahoma

________________________________________________________________________________________________Corresponding author address: Lin Tang, CIMMS, 120 David Boren Dr., Norman, OK 73072 Email: lin.tang@noaa.gov

1

Abstract

Polarimetric radar observations provide information regarding the hydrometeor

shape, size, and phase as well as an improved skill in differentiating radar echoes

of hydrometeors from those of non-hydrometeors. In this work, a multi-sensor

physically based algorithm is designed to classify weather/non-weather radar

echoes. The algorithm uses reflectivity, correlation coefficient, and temperature

sounding data to quality control the reflectivity data by applying a set of explicit

meteorological rules. The proposed methodology are tested with a large number

of real data cases across different geographical regions and seasons and showed

a high accuracy (Heidke Skill Score of 0.83) in segregating weather and non-

weather echoes. Comparing to other quality control methodologies using all

polarimetric variables, this algorithm’s advantage is in its simplicity, effectiveness

and computational efficiency. The current methodology is also demonstrated in a

real-time multi-radar and multi-sensor national mosaic system.

1. Introduction

A significant challenge in radar-derived quantitative precipitation estimation

(QPE) is the separation of precipitation from non hydro-meteorological returns.

Additionally, there are errors from uncertainties associated with where the radar is

sampling precipitation (i.e., the issue of non-uniform vertical profiles of reflectivity,

see Koistinen 1991; Kitchen et al. 1994; Andrieu and Creutin 1995; Smyth and

Illingworth 1998; Krajewski and Vignal 2001; Germann and Joss 2002) and the

conversion of radar reflectivity into rain rate (i.e., the issue of Z-R relationships, see

Marshall et al. 1955; Carbone and Nelson 1978; Rosenfeld et al. 1993; White et al.

2003).

An initial step for accurate radar QPE is the identification and removal of

non-precipitation contaminants including the electromagnetic interference with

2

transmitters (e.g., sun strobes), anomalous propagation (AP) of the radar beam due

to specific atmospheric temperatures or moisture gradients, beam blockage due to

topography (Bebbington et al., 2007), the biological echoes (birds, bats, insects),

power returns from manmade objects on the ground (Bachmann and Tracy, 2009)

, and sea clutter (Gray and Larsen, 2004). Manual quality control (QC) of radar

reflectivity data has been practiced at commercial weather companies and in

operations such as the National Weather Services River Forecast Centers. However,

manual quality control is resource expensive and prohibitive for timely warnings of

severe weather and flash floods on a local to national scale. To address the timely

need for quality radar information, an automated approach is required.

The national mosaic and multi-sensor QPE (NMQ) system is a real-time multi-

radar, multi-sensor system (Zhang et al. 2011). The system integrates 140+ U.S.

Weather Surveillance Radar-1988 Doppler (WSR-88D) radars and 30+ Canadian

weather radars to generate a three-dimensional reflectivity mosaic and suite of

severe weather and QPE products over the conterminous US (CONUS) domain at

1km spatial resolution every 5 minutes. Applying QC to the reflectivity is critical in

assuring the accurate depiction of storms and reliability of precipitation estimates.

The WSR-88D radar network was comprised of single-polarization radars before

2011. The reflectivity QC methodology used in the NMQ was a neural network

approach (QCNN) based on reflectivity, velocity and spectrum width, as well as

atmospheric environmental data (Zhang et al. 2004; Lakshmanan et al. 2007; 2010)

. Evaluations of the single-polarization radar QC in NMQ indicated that the scheme

was effective at removing a significant amount of non-precipitation returns under

various scenarios. However, biological clutter during the peak migratory seasons

of spring and fall remains a major issue, especially during periods when storms

develop and or move into regions of ‘biological blooms’. These ‘blooms’ are not

fully censored by the QCNN because the magnitude, vertical depth, and local texture

of insects/birds/bats returns in single-polarization radar fields are often very

similar to widespread, shallow stratiform rain or snow (Lakshmanan et al., 2010)

. The biological blooms can cause erroneous precipitation in the QPE products.

3

Conversely, light rain and snow echoes were sometimes removed as a result of

minimizing the biological blooms. Additional challenges are present when removing

AP from the ground and sea surface at far ranges. While AP echoes usually have zero

velocities, WSR-88D velocity field does not extend beyond 230km when reflectivity

data extends up to 460km, therefore, AP echoes at far ranges can be confused with

small convective storms in the reflectivity field.

Commencing in 2011, the WSR-88D network has been upgraded with dual

polarimetric capability. Polarimetric radar observations provide additional

information regarding the hydrometeor shape, size, and phase and an improved

skill in differentiating radar echoes of hydrometeors from those of non-

hydrometeors. The hydrometeor classification algorithms (HCAs) with varying

levels of complexity have been developed for radars of different wavelengths

(e.g., Liu and Chandrasekar 2000; Zrnic et al. 2001; Lim et al. 2005; Gourley et al.

2007; Marzano et al. 2008; Park et al. 2009, and etc.) and were shown, by multiple

investigators, to perform well.

Park et al. 2009 developed a fuzzy logic HCA that classifies WSR-88D radar

echoes into 11 different hydrometeor and non-hydrometeor categories, and the

algorithm has been implemented as part of the National Weather Service

operational radar product baseline. By taking the output of this sophisticated

classification scheme and removing the non-meteorological echoes, a dual-

polarization (DP) radar QC algorithm could be easily implemented. However, the

fuzzy logic HCA was applied on a pixel-by-pixel basis. Due to non-exclusive

membership functions in the fuzzy logics for different echo types, individual pixels

associated with biological or other non-hydrometeors could be incorrectly

identified as precipitation and vice versa (e.g., Liu & Chandra 2000). For instance,

there are cases that the HCA misclassifies clutter echoes as category “BD” (big drop)

(see, Figure1) or “UK” (unknown). The HCA also had difficulties in identifying severe

AP or sea clutter especially when the AP is beyond the range where the beam

supposedly intersects the melting layer. Figure2 provides an example of severe sea

clutter observed by WSR-88D KMHX at 23:58:34 UTC on 29 June 2012. As shown in

4

Figure 2(b), the HCA is able to classify most of the clutter from the sea surface

except the portion at far ranges. Therefore further refinements are needed for the

operational WSR-88D HCA algorithm to more accurately segregate hydrometeor

and non-hydrometeor echoes. Further, the operational HCA involves all

polarimetric variables, i.e., reflectivity Z, velocity V, spectrum width SPW,

correlation coefficient RhoHV, differential reflectivity Zdr, and differential phase of

propagation PhiDP. The HCA requires a pre-processing and quality assurance of

each of the variables before the fuzzy logic computations. The computational cost

(memory and CPU) for running such algorithms in real-time can be relatively high,

which poses a challenge for processing a national network of more than 140 radars.

For applications that do not require detailed classifications of different hydrometeor

species, such a sophisticated HCA might be an excessive option given that a simpler

scheme may be sufficient.

Figure1: (a) the original base reflectivity field at the elevation angle (EA) of 0.5 degrees from radar KEWX; (b) the corresponding hydrometeor field derived from HCA at the EA of 0.5 degree at 07:19:46 UTC on 06 May 2012.

Figure2: (a) the original base reflectivity field from radar KMHX at the elevation angle of 0.5 degrees; (b) the corresponding hydrometeor field derived from HCA; (c) the associated RhoHV field. The fields are observed at 23:58:34 UTC on 29 June 2012.

5

Lakshmanan et al. (2013) used a neural network approach (“dpQCNN”

hereafter) to segregate weather and non-weather echoes using all polarimetric

variables and showed a good Heidke Skill Score (HSS) of 0.8 (Lakshmanan et al.,

2013). Due to the statistical scoring process and the “black box” character of a

neural network model, it is not easily determined which variables are the most

important contributors to a particular output, and it may have limited ability to

explicitly identify possible causal relationships (Tu 1996). It requires all of the six

polarimetric radar moments and the computational requirements (especially the

memory) can be relatively high. Given the distinctly different characteristics of

weather vs. non-weather echoes in the correlation coefficient (ρhv or RhoHV) field

and in the 3D structure of reflectivity and atmospheric thermal dynamic fields, there

is potential for a simple and physically based method to segregate weather and non-

weather echoes.

In the current study, we explore a physically based dual-polarization radar

QC (“dpQC”) approach that segregates weather and non-weather echoes using

correlation coefficient, reflectivity, atmospheric sounding data, and a set of

explicit meteorological principles. With the reduced number of input variables,

computations related to the calibration or quality assurance of various polarimetric

radar variables (e.g., Zdr and Kdp) are reduced and the possible sources of error

lessened. The scheme was tested on the CONUS domain in real-time for several

months including the peak biological migrating season in the spring. The results

show that the dpQC scheme was very effective in removing biological echoes

while retaining the light precipitation. It performs equally well at segregating

weather and non-weather echoes as the operational WSR-88D HCA scheme and

the Lakshmanan (2013) scheme but with significantly less memory and CPU

requirements.

2. The Multi-Sensor Physically Based Dual-Polarization Radar QC Method

6

The following section provides details of the dpQC methodology with each

component being presented using real data examples individually, and then

collaboratively.

2.1 The basic RhoHV filter

Correlation coefficient (RhoHV) is one of three DP base moments. It

measures the consistency of the horizontal and vertical returned power and phase

for each pulse. It also provides information on the quality of the dual-polarization

data fields and implies information on the nature of the scatters. RhoHV is

expressed as a number between 0 and 1: generally pure rain or pure snow are

associated with high values close to 1; low RhoHV values, alternatively, represent a

spectrum of scatters that are likely non-meteorological. Using a simple RhoHV

thresholding approach, the clutter with low coherent signals can be censored.

Figure 3 shows an example of a basic RhoHV filter where all echoes with RhoHV ≤

0.90 are removed. Figure 3(a) is the base reflectivity map from the radar KEWX at

05:51:39 UTC on 6 May 2012. With the biological echoes (“blooms”) mixed with the

storm, the conventional QCNN algorithm failed to separate the precipitation

information from the non-precipitation as shown in Figure 3(b). While the power

returns from birds and insects can be as strong as meteorological scatterers, the

correlation coefficient of the former is much lower than that of raindrops (Figure

3(d)). There were some unphysical high RhoHV values (greater than 1) at the edge

of precipitation areas and in blooms, which were associated with very low signal-to-

noise ratios and will be discussed later in this paper. With a simple RhoHV

threshold, the QC’ed reflectivity field (Figure 3(c)) is nearly free of the biological

bloom contamination. This example shows the remarkable effectiveness of a simple

RhoHV filter when separating weather and non-weather echoes in cases of a heavily

mixed precipitation and biological echoes that are normally impossible for single

polarization radar based QC approaches.

7

Figure3: (a). The raw reflectivity field from radar KEWX at 05:51:39 UTC on 06 May 2012; (b) the reflectivity field after the process of QCNN; (c) the QC’ed reflectivity field with the method of QC_DT; (d) the RhoHV field. All images are from 0.5-deg tilt.

Figure 4(a) is a histogram of RhoHV values in a volume scan observed by KARX

on 26 June 2012 at 19:33 UTC. An experienced radar meteorologist reviewed the

data, and manually separated meteorological or non-meteorological echoes. A

histogram of RhoHV values was then constructed for each of the two echo types and

then normalized. The histograms (Figure 4(a)) showed that non-meteorological

pixels (gray bars) had a wide spread value of RhoHVs between 0.3 and 0.95, while

the meteorological echoes (black) are generally associated with RhoHVs greater

than 0.95. However, this RhoHV feature is not persistent across every weather

situation. For a convective storm observed by radar KVNX, at 07:36 UTC on 20 May

2011, the histograms (see Figure 4(b)) of weather and non-weather echoes are

overlapping, with the clutter RhoHV values spreading across the whole range of

0-1. Apparently there were power returns from non-meteorological objects with

high correlation between the horizontal and vertical directions; at the same time,

some portions of the weather echoes were associated with low cross-correlation

8

coefficients smaller than 0.80. In this case and likely others, a fixed RhoHV threshold

would lead to false alarms or misses in clutter detection. Therefore additional steps

are devised to further screen the clutter associated with high RhoHV values and vice

versa.

(a) (b)Figure4: The histogram plots of the RhoHV values of the power returns from the non-meteorological (gray) and meteorological targets (black) observed by radars (a) KARX at 19:33 UTC on 26 June 2012 and (b) KVNX at 07:36 UTC on 20 May 2011, respectively.

2.2 Handling Exceptions to the Simple RhoHV Filter

a). Low RhoHVs in hail

Due to the complex shape of large hailstones, the horizontal and vertical radar

pulses behave differently resulting in low RhoHV (0.80-0.90) in hail regions of

convective storms (e.g., the circled areas in Figure 5(b)). However, hailstorms are

generally associated with intense horizontal (conventional) reflectivity (> 45-

50dBZ) and tall storm echo tops (> 8km). By adjusting the RhoHV filter threshold

in regions with the aforementioned characteristics, the hail cores can be protected

from the filtering. Figure 5(c) shows the QC’ed reflectivity field with the adjusted

RhoHV filter, where the biological echoes close to the radar are completely removed

while weather echoes including the hail are successfully retained.

9

Figure5: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC on a PPI scope at the elevation angle of 0.5 degrees. The precipitation event is observed from radar KAMA at the 02:50:30 UTC on 01 May 2012. The circled areas point out the hail echoes.

b). Low RhoHVs associated with non-uniform beam filling

Non-uniform beam filling (NBF) occurs when significant gradients of hydrometeor

properties (concentration, size, shape, or phase) occur within a pulse volume, e.g.,

near the edge of hail cores in convective storms. These gradients can occur in

either the horizontal or vertical. The simple RhoHV filter can incorrectly remove

precipitation echoes in regions with severe NBF, as shown in a hailstorm example

in Figure 6. The circled area in Figure 6(b) indicates a low RhoHV region due to

the NBF near a strong hail core. The simple RhoHV threshold could have removed

the echoes in this area. However, an examination of high echo tops in this region

indicated that the echoes were associated with storms. Therefore the echo-top

height information was applied to redeem those precipitation echoes with low

cross-correlation coefficients, as demonstrated in Figure 6(c).

10

Figure6: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC at the elevation angle of 0.5 degrees. The precipitation event is observed from radar KVNX at the 03:37:13 UTC on 01 May 2012. The white-circled areas outline regions of severe NBF.

c). Low RhoHVs in wet snow aggregations

Due to their mixed phases, echoes from the wet snow aggregates could have low

RhoHV values (<0.95). Figure 7 is a case of low correlation coefficient observed in

the melting layer during a winter precipitation event. To mitigate the effects from

low correlation coefficient in wet snow aggregations, the temperature sounding

data is employed in the dpQC algorithm. Using the approximate zero-degree height,

the algorithm searches for a ring of low RhoHV (~0.85-0.95), which is associated

with the melting layer. Along each of the azimuth radial direction, if the RhoHV

drop and rise are detected, the threshold of RhoHV filter is adjusted lower for the

melting hydrometeors. As shown in Figure 7(c), the dpQC was able to retain the

weak weather echoes. Retaining weak echoes in a cold environment is critical

for accurate precipitation estimations in cool seasons and for the identification of

potential icing hazards for aviation.

11

Figure7: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC on a PPI scope at the elevation angle of 0.5 degrees. The precipitation event is observed from radar KCLE at the 16:24:29 UTC on 26 January 2012.

d). High RhoHVs in severe AP phenomena

Severe AP is occasionally associated with highly correlated power returns from

horizontal and vertical directions, where RhoHV values can be 0.95 or higher

(Figure 8(b)). The application of a simple RhoHV filter is not able to remove AP in

these situations. To address this issue, the RhoHV filter is followed by a vertical

gradient check (Zhang et al. 2004), which checks the vertical structure of the

echoes. If the strong power returns are only observed in the lowest tilt without

a consistency in the vertical, then the echoes are most likely associated with non-

precipitation scatterers. The vertical gradient test is dependent on the volume

coverage pattern (VCP) and the severity of the anomalous propagation. It is not

effective when the beam is so severely super refracted that large areas of strong

AP echoes are observed on the 2nd (3rd for the VCP 12 or 212) or even higher tilts.

Further, it cannot be applied at very far ranges because the height difference

between the lowest two tilts is too large, and a shallow precipitation cloud may only

be observed in the first tilt. Figure 8 provides an example where the gradient check

successfully censored AP echoes associated with high RhoHV values.

12

Figure8: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC at the elevation angle of 0.5 degrees. The precipitation event is observed from radar KOUN at the 02:08:28 UTC on 11 May 2010. The white-circled areas outline the region of severe AP.

e). High RhoHVs in weak echoes

When the signal-to-noise (SNR) ratio is low, RhoHV is often noisy in appearance

with varying magnitudes. RhoHV values may be greater than 1 for the weak signals

associated with large noise, according to estimate calculations of the co-polar

correlation coefficient (Melnikov and Zrnić, 2007). The biased high value RhoHV

(>1) appears in both meteorological and non-meteorological areas when the signal

to noise ratio is low (Figure 9(b)). To clean the clutter left from the basic RhoHV

filtering, further steps are required to differentiate the biased RhoHV of clutter

contamination from the precipitation echoes. The gradient of echo power along the

vertical height is computed for the data bins of biased RhoHV in the regions close

to the radar at low tilts. Vertical discontinuity is considered a characteristic of the

clutter. At far ranges or high tilts, the pixels of biased RhoHV are first segmented

into groups. The groups located closer to clutter areas (0<RhoHV<0.80) than

precipitation regions (0.95<RhoHV<1.0) are identified as clutter and removed.

Otherwise, they are considered as weak signal from the meteorological targets

(Figure 9 (c)) and are retained.

13

Figure9: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC at the elevation angle of 0.5 degrees. The data is obtained from radar KTLX at the 12:31:51 UTC on 08 March 2013.

f). The clutter at far ranges from radar site and other limitations of RhoHV

Due to the range difference between the polarimetric variables such as RhoHV

(298 km) and reflectivity field (460 km) of the WSR-88D radars, the RhoHV filter is

not able to handle the clutter further than 298 km away from the radars. To address

this issue, a simple image processing technique is used to remove sun strobes and

electronic interferences at ranges greater than 298 km. Different from others,

these two types of echoes often appear as isolated radial spikes originating from

the radar site and ending at the edge of the coverage. When portions of such spikes

are removed within the RhoHV coverage umbrella (due to low RhoHV values), the

rest of the spikes are also removed [see Figure 10 (a, b, c)]. This technique is not

effective on the AP ground or sea clutter at far distances (see Figure 10 (d, e, f)) due

to the fact that those clutter lack of the specific spike geometry. AP at far ranges

does not pose a significant problem for the inland WSR-88D radars because they

are relatively densely distributed with an average spacing of less than 250km. As a

result, inland data beyond 298km are not used in NMQ. Identification of AP echoes

beyond 298km for coastal radars remains an issue and requires additional data

sources (e.g., satellite).

14

Figure10: The (a) (d) raw reflectivity (b) (e) RhoHV and (c) (f) reflectivity field after QC at the elevation angle of 0.5 degrees. The fields are observed from radar KICX (a, b, c) at the 01:31:47 UTC on 11 Mar. 2013 and KJAX (d, e, f) at the 08:43:14 UTC on 23 September 2012.

2.3 Overview of the dpQC algorithm

Based on the detailed case studies presented above, a physically based multi-

sensor dpQC algorithm was assembled consisting of the following seven steps:

2.3.1 Clear air filter

The clear air filter is the first step for the clutter screening in the dpQC

algorithm. When the surface temperature is high (>10°C) and the radar uses a clear

air mode (i.e., VCP 31 or 32), all echoes in the whole volume scan are removed. This

simple step effectively removes biological echoes in the warm season and largely

reduces computations down stream.

2.3.2 Basic RhoHV filter

15

In the basic RhoHV filter, four different thresholds are applied according

to various scenarios predefined through reflectivity intensities and echo top

heights. The filter threshold is automatically adjusted for the situations when low

correlation coefficient values are found in the precipitation events, as discussed in

sections 2.2(a-c).

2.3.3 RhoHV texture filter

The RhoHV texture filter follows to censor weak echoes associated with noisy

local RhoHV texture. The estimate of the RhoHV texture is obtained by averaging

RhoHV data along the radial using a 1-km running-average window and subtracting

the smoothed estimates of RhoHV from the original values. The root mean square

value of the residuals is calculated as the local RhoHV texture (Park et al. 2009). The

texture characterizes the magnitude of the small-scale fluctuations of RhoHV along

the radial. The echo is censored as clutter when its RhoHV texture is larger than a

threshold. After two steps of Basic RhoHV and RhoHV texture filtering, most of non-

weather echoes with the RhoHV less than 0.90 are removed, and the clutter pixels

with high RhoHV values (i.e., those in overlapping region of the histograms in Figure

4) are further processed.

2.3.4 Spike filter

The spike filter was designed to handle spikes from electronic interferences and

sun strobes that extend beyond the maximum range of the RhoHV field (298 km), as

discussed in section 2.2(f).

2.3.5 Speckle filter

A speckle filter follows to remove isolated dots of echoes in the reflectivity fields.

2.3.6 Spatial analyze

After the speckle filter, a couple of spatial analyses are performed to handle the

non-weather echoes with high RhoHV values [i.e., those discussed in sections 2.2(d-

e)]. In regions close to the radar, a vertical gradient test is employed to check the

vertical continuity of any strong echoes that exist in the lowest tilts. If the echo

intensities decrease with height dramatically, then they are identified as clutter (AP)

. In regions far away from the radar, high RhoHV bins with low SNRs are classified

16

as weather or non-weather depending on their proximities to the nearest storm or

clutter centers.

2.3.6 Final clean up

In the final clean up, a median filter is applied to remove residual speckles and to

fill in small voids within weather echoes where sparse low RhoHV values are found

among blocks of meteorological targets.

In the dpQC algorithm, each step can remove clutter echoes when the stringent

criteria are met, leaving less data to process for the next step. This design effectively

increased the computational efficiency. The environmental data is used to remove

clear air and biological echoes in warm season and to help delineating areas of

mixed hydrometeor phases in the basic RhoHV filter. Figure 11 illustrates an

overview flowchart of the dpQC algorithm.

17

Figure11: Flow chart of the dpQC algorithm.

3. Evaluation of the dpQC Methodology

The dpQC algorithm was developed across 25 cases that represented various

weather and non-weather echo regimes from different regions of the US. There

were total of 138-hour of data from 29 polarimetric WSR-88D radars, as shown in

Table 1. The designed method is tested and evaluated in its QC performance and

computational resource.

Table 1: List of cases used for the development and tuning of dpQC.

18

Events Radar Time and date Event summary1 KCCX

(40.92ON, 78.00OW)0600-1100 UTC 11 Dec 2012 Winter storm (weak echoes)

2 KFFC (33.36ON, 84.57OW)

1100-1400 UTC 16 Nov 2012 Winter storm (weak echoes)

3 KJGX (32.68ON, 83.35OW)

1600-2200 UTC 17 Sep 2012 BI + convective

4 KMPX (44.85ON, 93.57OW)

0000-0200 UTC 10 Dec 2012 Winter storm (weak echoes)

5 KMHX (34.78ON, 76.88OW)

1800-2300 UTC 29 Jun 2012 AP + sea clutter

6 KOUN (35.24ON, 97.46OW)

2000 UTC 10 May 2012 –0800 UTC 11 May 2012

BI + AP + convective

7 KAMA (35.23 ON, 101.7 OW)

0100-0500 UTC 1 May 2012 BI + AP + hails + NBF

KVNX (36.74 ON, 98.13 OW)KDDC (37.76 ON, 99.97 OW)

BI + ground clutter (WF) + Hail + NBF

KICT (37.65 ON, 97.44 OW)

BI + Hail + NBF

8 KAMX (25.61 ON, 80.41 OW)

1600-UTC 18 May 20120800 UTC 19 May 2012

Chaff + convective

KBYX (24.60 ON, 81.70 OW)

Chaff + convective

9 KARX (43.83 ON, 91.19 OW)

1700-2100 UTC 26 Jun 2012 BI + stratiform

10 KCLE (41.41 ON, 81.86 OW)

1400 UTC 07 Feb 2012-0500 UTC 08 Feb 2012

Winter storm (weak echoes) + EI

11 KEWX (29.70 ON, 98.03 OW)

0200-1000 06 May 2012 BI + AP + EI + convective

12 KHTX (34.93 ON, 86.08 OW)

0000-0200 UTC 11 Aug 2012 BI + Hail + NBF

13 KLTX (33.99 ON, 78.43 OW)

1500-1700 UTC 17 Sep 2012 BI + convective

14 KMRX (36.17 ON, 83.40 OW)

2100-2300 UTC 17 Sep 2012 BI + Hail

15 KPBZ (40.53 ON, 80.22 OW)

1100-2300 UTC 11 Feb 2012 Winter storm (weak echoes)

16 KPDT (45.69 ON, 118.9 OW)

0900-1100 UTC 20 Jul 2012 AP + stratiform

17 KYUX (32.50 ON, 114.7 OW)

0900-1100 UTC 20 Jul 2012 AP + EI

18 KABR (45.46 ON, 98.41 OW)

0800 UTC 11 Aug 2012 Ground clutter + stratiform

19 KFSD (43.59 ON, 96.73 OW)

0000 UTC 04 Aug 2012 Hail + NBF

19

20 KJAX (30.48 ON, 81.70 OW)

0800 UTC 23 Sep 2012 BI + sea clutter

21 KNQA (35.34 ON, 89.87 OW)

1000 UTC 21 Sep 2012 AP + stratiform

22 KLOT (41.60 ON, 88.08 OW)

2000 UTC 04 Aug 2012 Hail + convective

23 KRLX (38.31 ON, 81.72 OW)

0800 UTC 01 Oct 2012 BI + AP +stratiform

24 KSGF (37.24 ON, 93.40 OW)

0600 UTC 29 Feb 2012 Hail + NBF

25 KRIW (43.07 ON, 108.5 OW)

1300 UTC 25 May 2012 Ground clutter + stratiform

Total 29 radars 138 h 25 events

Note: the abbreviations in the table are AP: anomalous propagation; EI: electromagnetic interference; BI: biological clutter; NBF: Non-uniform beam filling; WF: wind farm.

The dpQC algorithm was implemented on 122 radars in real-time in the NMQ

system (http://nmq.ou.edu/qvs-2012.html) since 19:25:00 UTC on 05 Dec 2012.

Figure 12 shows several examples (e.g., bloom, light precipitation, chaff) of the real-

time national composite reflectivity fields before and after the dpQC. In Figure

12 (a) and (b), the power returns from non-meteorological chaff (outlined with

white circles) cannot be censored with the single-polarization radar QC due to their

similarities to precipitation echoes in the reflectivity field. On the other hand, the

aluminum-coated glass fibers (chaff) are not as correlated in the horizontal and

vertical directions as meteorological objects, therefore the dpQC was able to identify

and remove the clutter (Figure 12(c)). Real time tests also showed the advantages

of the dpQC over the single-polarization QC in removing blooms while retaining

light precipitation (regions outlined with the white ovals in Figs. 12d, e, and f).

Such advantages are a result of distinctly different RhoHV features associated with

blooms (biological echoes) and with light precipitation.

20

Figure12: (a), (d) the raw composite reflectivity before any process of quality control; (b), (e) the composite reflectivity after the quality control of QCNN that is currently used in the NMQ system; (c), (f) the composite reflectivity field after process of dpQC. The images are cropped from the real-time NMQ system.

The dpQC and dpQCNN (Lakshmanan et al. 2013) algorithms were evaluated

on a set of independent cases (Table 2) and the two algorithms performed very

similarly with the HSS scores of 0.83 and 0.79, respectively. However, due to the

simplicity of the dpQC algorithm, it took much less computational resources when

compared with the more sophisticated dpQCNN scheme. Based on the test of

282 radar volume scans on a workstation with 4 quad-core Intel® Xeon® E5520

2.27GHz processors and 12GB DDR3 RAM, the average CPU for processing one radar

volume scan is 49.55s for dpQCNN and 3.15s for dpQC and the average RAM was

172MB for dpQCNN and 83MB for dpQC.

Table 2: Testing cases for evaluating the QC algorithm.

Events Radar Time and date Event summary1 KMHX (34.78ON, 76.88OW) 01 UTC 27 Aug 2011 Hurricane2 KMHX (34.78ON, 76.88OW) 19 UTC 27 Aug 2011 Hurricane3 KOUN (35.24ON, 97.46OW) 05 UTC 11 May 2010 AP + BI4 KOUN (35.24ON, 97.46OW) 11 UTC 19 May 2010 Supercell + BI5 KAMA (35.23 ON, 101.7 OW) 14 UTC 11 Apr 2012 Rain + BI6 KAMA (35.23 ON, 101.7 OW) 12 UTC 12 Apr 2012 Rain + BI7 KAMA (35.23 ON, 101.7 OW) 18 UTC 15 Feb 2013 Rain+ GC

21

8 KVNX (36.74 ON, 98.13 OW) 09 UTC 20 May 2012 Storm + BI9 KVNX (36.74 ON, 98.13 OW) 23 UTC 29 May 2012 ST+BI

10 KVNX (36.74 ON, 98.13 OW) 11 UTC 29 Jan 2013 Storms11 KBMX (33.17 ON, 86.77 OW) 11 UTC 30 Jan 2013 Storms12 KCLE (41.41 ON, 81.86 OW) 17 UTC 15 Feb 2013 Snow13 KDMX (41.73 ON, 93.72 OW) 03 UTC 29 Jan 2013 Convective14 KLVX (37.98 ON, 85.94 OW) 16 UTC 15 Feb 2013 Snow15 KTLH (30.40 ON, 84.33 OW) 11 UTC 26 Feb 2013 Squall line and clutter16 KTLX (35.33 ON, 97.28 OW) 23 UTC 09 Mar 2013 Squall line and clutter

Note: the abbreviations in the table are AP: anomalous propagation; BI: biological clutter; ST: sun strobe; GC: ground clutter.

Table 3: Summary of computational testing.

VCP Radar and Location Time and date

(UTC)

dpQCNN dpQCCPU

(Min: sec)RAM(MB)

CPU(Min: sec)

RAM(MB)

12 KBBX(39.50 ON, 121.63 OW)

1200-1500 04 Apr 2013

47:37 216 2:22 79

212 KBMX(33.17 ON, 86.77 OW)

1700-2000 24 Apr 2013

39:58 211 2:45 105

31 KBUF(42.95 ON, 78.74 OW)

1000-1300 13 Apr 2013

5:36 119 0:37 77

121 KCLE(41.41 ON, 81.86 OW)

0900-1200 19 Apr 2013

22:32 157 1:39 77

11 KCLX(32.66 ON, 81.04 OW)

1700-2000 04 Apr 2013

34:37 210 1:56 88

32 KFWS(32.57 ON, 97.30 OW)

0100-0400 15 Apr 2013

6:26 114 0:38 75

21 KLOT(41.60ON, 88.08 OW)

1000-1300 08 Apr 2013

22:08 157 1:49 86

221 KLRX(41.60ON, 88.08 OW)

0300-0600 20 Apr 2013

16:51 156 1:07 77

211 KTLX(35.33ON, 97.28 OW)

0700-1000 13 Apr 2013

37:09 205 1:54 84

Total 282 vol scans

232:54 14:47

Average Per volume scan

49.55s 172 3.15s 83

4. Conclusions

A multi-sensor physically based weather/non-weather echo classifier was

developed using polarimetric radar observations and atmospheric environmental

22

data. The algorithm applied a set of explicit meteorological principles that segregate

weather and non-weather echoes in the radar reflectivity field based on correlation

coefficient and 3D reflectivity and temperature structure. The new algorithm was

evaluated using 16 independent events and showed a high accuracy (Heidke Skill

Score of 0.83) in segregating weather and non-weather echoes. When compared

with a more sophisticated QC algorithm that uses all polarimetric variables and a

neural network approach, the dpQC showed a similar HSS score (0.83 vs. 0.79), but

with much less CPU (3.15 vs. 49.55 s) and RAM (83 vs. 172 MB) usages for process

a volume scan data on average. Due to the simplicity and transparency of the

algorithm, it is easy to implement and maintain in an operational environment with

a large national radar network.

5. References

Andrieu, H. and J.-D. Creutin, 1995: Identification of vertical profiles of radar reflectivity for hydrological applications using an inverse method. Part I: Formulation. J. Appl. Meteorol. 34, 225– 239.

Bachmann, S.M., and M. D. Tracy, 2009: Data driven adaptive identification and suppression of ground clutter for weather radar, 25th Conference on International Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, 11.B3: Phoenix, AZ.

Bebbington, D., S. Rae, J. Bech, B. Codina, and M. Picanyol, 2007: Modeling of weather radar echoes from anomalous propagation using a hybrid parabolic equation method and NWP model data. Natural Hazards and Earth System Sciences, Vol. 7, pp.391–398, ISSN 1561-8633

Carbone, R. E., and L. D. Nelson, 1978: The evolution of raindrop spectra in warm-based convective storms as observed and numerically modeled. J. Atmos. Sci., 35, 2302–2314.

Germann, U, and J. Joss, 2002: Mesobeta profiles to extrapolate radar precipitation measurements above the Alps to the ground level. J. Appl. Meteorol. 41, 542–557.

Gourley, J. J., P. Tabary, and J. Parent-du-Chatelet, 2007: A fuzzy logic algorithm for the separation of precipitating from nonprecipitating echoes using polarimetric radar observations. J. Atmos. Oceanic Technol., Vol. 24, pp. 1439–1451, ISSN 1520-0426

Gray, W., and H. Larsen, 2005, Radar rainfall estimation in the new zealand context, Atmos. Sci. Let. 6: 31-34.

23

Kitchen, M., R. Brown, and A. G. Davies, 1994: Real- time correction of weather radar data for the effects of bright band, range and orographic growth in widespread precipitation. Quart. J. Roy. Meteor. Soc., 120, 1231–1254.

Koistinen, J., 1991: Operational correction of radar rainfall errors due to the vertical reflectivity profile. Preprints, 25th Int. Conf. on Radar Meteorology, Paris, France, Amer. Meteor. Soc., 91–94.

Krajewski, W. F., B. Vignal, 2001: Evaluation of Anomalous Propagation Echo Detection in WSR-88D Data: A Large Sample Case Study. J. Atmos. Oceanic Technol., 18, 807–814.

Lakshmanan, V., A. Fritz, T. Smith, K. Hondl, and G. J. Stumpf, 2007: An automated technique to quality control radar reflectivity data. J. Appl. Meteor. Climatol., 46, 288–305.

Lakshmanan, V., J. Zhang, and K. Howard, 2010: A technique to censor biological echoes in weather radar images. J. Appl. Meteor. Climatol., 49, 453–462.

Lakshmanan, V., C. Christopher, J. Krause, and L. Tang 2013: Quality control of weather radar data using polarimetric variables. J. Atoms Oceanic Technol., submitted.

Lim, S., V. Chandrasekar, and V. N. Bringi, 2005: Hydrometeor classification system using dual-polarization radar measurements: Model improvements and in situ verification. IEEE Trans. Geosci. Remote Sens., 43, 792–801.

Liu, H., and V. Chandrasekar, 2000: Classification of hydrometeors based on polarimetric radar measurements: Development of fuzzy logic and neuro-fuzzy systems, and in situ verification. J. Atmos. Oceanic Technol., 17, 140–164.

Marshall, J. S., W. McK. Palmer, The distributions of raindrop with size, J. Meteorol., 5, 165–166, 1948.

Marzano, F., D. Scaranari, M. Montopoli, and G. Vulpiani, 2008: Supervised classification and estimation of hydrometeors from C-band dual-polarized radars: A Bayesian approach. IEEE Trans. Geosci. Remote Sens., 46, 85–98.

Melnikov, V. M., D. S. Zrnić, 2007: Autocorrelation and Cross-Correlation Estimators of Polarimetric Variables. J. Atmos. Oceanic Technol., 24, 1337–1350.

Park, H. S., A. V. Ryzhkov, D. S. Zrnic, and K.-E. Kim, 2009: The hydrometeor classification algorithm for the polarimetric WSR-88D: Description and application to an MCS. Wea. Forecasting, 24, 730–748.

Rosenfeld, D., D. B. Wolff, and D. Atlas, 1993: General probability-matched relations between radar reflectivity and rain rate. J. Appl. Meteor., 32, 50–72.

Smyth T.J., A. J. Illingworth, 1998: Radar estimates of rainfall rates at the ground in bright band and non-bright band events. Quart J Royal Meteorol Soc 124:2417–2434

24

Tu, J. V., 1996: Advantage and disadvantage of using artificial neural networks versus logistic regression for predicting medial outcomes, J Clin Epidemiol ,49, No. 11, pp. 1225-1231, 1996

White, A. B., P. J. Neiman, F. M. Ralph, D. E. Kingsmill, and P. O. G. Persson, 2003: Coastal orographic rainfall processes observed by radar during the California Land-Falling Jets Experiment. J. Hydrometeor., 4, 264–282.

Zhang, J., S. Wang, and B. Clarke, 2004: WSR-88D reflectivity quality control using horizontal and vertical reflectivity structure. 11th Conf. on Aviation, Range, and Aerospace Meteorology, 5.4, Hyannis, MA.

Zhang, J, and Coauthors, 2011: National Mosaic and Multi-sensor QPE (NMQ) system: Description, results, and future plans. Bull. Amer. Meteor. Soc., 92, 1321–1338.

Zrnic, D. S., and A. V. Ryzhkov , J. M. Straka, Y. Liu, and J. Vivekanandan, 2001: Testing a procedure for the automatic classification of hydrometeor types. J. Atmos. Oceanic Technol., 18, 892–913.

25