An algorithm developer’s tool

An algorithm developer’s tool

[email protected] Severe Storms LaboratoryNorman OK, USAhttp://www.wdssii.org/

mailto:[email protected]

http://w/

http://w/

April 22, 2023 [email protected] 2

A developer’s tool

The Warning Decision Support System – Integration Information (WDSS-II) A collection of meteorological algorithms for severe

weather analysis, diagnosis and prediction Hail, tornadoes, wind, lightning

An integrated set of loosely coupled tools for: Severe weather diagnosis Image processing Statistical validation Ground-truth verification


WDSS-II applications (algorithms and tools) are just executables. launched on the command line. In deployed systems through scripts.

Can easily change input to filtered form, or accumulate a different product (such as rainfall rate or hail size)

Applications exist for many tasks: Image processing (smoothing, dilating, eroding, etc.) Objective analysis (Cressman, Barnes, Gaussian, etc.) Scoring grids (error statistics) Statistical skill based on ground truth

WDSS-II Applications

w2accumulator –i xmllb:/data/realtime/radar/KTLX/code_index.lb \

-I MaxShear_0-3km \

-o /data/realtime/radar/KTLX/ -r -t “30 60 120”


Creating a new algorithm

An algorithm is essentially a data filter Takes some data as input Produces new data as output

The algorithm developer should be able to specify the scientific processing in the middle Without having to worry about data ingest, data

formats, notification, etc. But provide a library of common computations on

the typical data used.


w2algcreator

W2algcreator is a WDSS-II tool To write the format-independent code for ingesting

data into your application. The algorithm developer writes an XML file

specifying the inputs and adaptable parameters. The algorithm itself is auto-generated!

With a “fill in the blank” for the scientific computation

WDSS-II class libraries can be used for common computations.

Easy to add new input and output formats.


Display

The WDSS-II displays are highly configurable to aid trouble-shooting.Display of intermediate outputs is easy and

convenient.


Example intermediate product

Created with no modification of the display. Just configuration files.

Algorithm developer marks the radar associated with each detection. For easy debugging.


The end-result

So, what kinds of algorithms have been developed in WDSS-II?


Single-radar/Multi-sensor algorithms

Some single-radar (multi-sensor) algorithms in WDSS-II


Multi-radar/multi-sensor algorithms

A typical multi-radar deployment of WDSS-II


Relevance to Q2

Which of those are relevant to Q2?Some existing severe weather algorithms

may be relevantProbability of hail for identifying radar echoes

with potential hail contamination.More likely:

Developing new algorithmsBuilding algorithms as data filtersExisting lower-level tools.


Accumulation algorithm

Six Hour Path of Rotational Shear Accumulation of shear to form rotation tracks.

Accumulation could be as: Maximum Rate Total

This tool could be used for rainfall depth from rainrate for example.


Motion Estimation

Uses K-Means clustering and Kalman filters

Forecast dBZActual dBZ

30 min30 min


Need for new approach

Traditional centroid tracking Accurate at small scales, but not at large scales Inaccurate when storms merge or split Possible to extract trends from the information

Flow-based tracking Cross-correlation, Lagrangian methods, etc. Are accurate at large scales, but not at small scales Not useful in decision support because trends of

storm properties can not be extracted


K-Means clustering

K-Means clustering is a hybrid approachCluster the input data to find clusters

Like centroid-based tracking methodsBut at different scales.

Track the clusters using flow-based methods (minimization of cost-functions)

Like flow-based methodsDoes not involve cluster matching (e.g: Titan)


Example clusters

Two different scales shown

Both scales are tracked


Extrapolation

Smooth the motion estimates spatially using OBAN

techniques (Gaussian kernel)

temporally using a Kalman filter (assuming constant velocity)

Repeat at different scales and choose scale appropriate to extrapolation time period.


Trends

The clusters can be used to extract trends of any gridded field. Configurable to extract minimum, maximum, count,

sum, time-delta, etc. of gridded fields within cluster Even fuzzy combination of multiple fields

Extremely useful for research! Statistical properties of storms Changing drop-size distributions with time Which clusters are convective? Trends in rain-rates … Trends in cloud-top temperatures …


Polygon statistics

Using cluster trends is useful for deriving storm properties.What about extracting statistics around a

fixed location?Maybe around rain gages?

WDSS-II has a tool to provide polygon statistics from any gridded field(s)The polygons can change with time (e.g:

weather service watch areas)


Quality Control Neural Network (QCNN)

Developed for MDA false alarms in non-storm echo. With QCNN, shows over 90%

reduction in the non-storm MDA false alarms and zero change to detections within storm echo.

The same QC technique would be useful in estimating precipitation as well.

Based on local statistics of reflectivity, velocity and spectrum width fields, vertical statistics and morphological image processing

Handles AP/GC, radar artifacts and some biological signals.

Neural network for optimal combination of inputs.

After QC

Before QC


Current uses of WDSS-II in the NWS

WDSS-II is a leading edge system Provides capabilities not yet in the “official” National Weather Service

systems. The Storm Prediction Center

defines daily threat areas launch a WDSS-II domain

automatically configures the data ingest and starts the algorithms. NWS forecast offices

WDSS-II products are converted into AWIPS format and piped the AWIPS displays in several NWS forecast offices.

But the AWIPS display is too restrictive. Therefore … The 4D WDSS-II display is to be implemented as a separate app on

AWIPS but controlled from within D2D. Concept of algorithm development capabilities

Being considered for next redesign of AWIPS


In summary

How can WDSS-II be useful in Q2 As an algorithm development toolkit

Multi-sensor inputs in real-time and for archived cases But limited to user workstations JADE will provide web-based capabilities.

Individual tools Objective Analysis tools and other low-level tools. Image processing filters Quality control of radar data Motion estimation and extrapolation (short-term QPF) Storm statistics Polygon statistics

Please visit this website: http://www.wdssii.org/

http://www.wdssii.org/



Documents

An algorithm developer’s tool