Nowcasting of thunderstorms

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 1

Valliappa.Lakshmanan@noaa.govNational Severe Storms Laboratory & University of OklahomaSeminar at City University of New York CREST programhttp://cimms.ou.edu/~lakshman/

Nowcasting of thunderstorms

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 2

What is nowcasting?

Skilled short-term estimates and predictions Typically 0-60 minutes For emergency managers, transportation, etc. Made by meteorologists

With guidance from automated algorithms Guidance to forecasters involves supplying estimates & predictions for:

Spatial location of thunderstorms Where is the storm now? What is the path the storm has traveled? Where will the storm be in 30 minutes?

Intensity of thunderstorms Weakening? Strengthening?

Potential hazards Hail? Lightning? Tornadoes? Flooding?

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 3

Hazard prediction

This talk will focus on estimating and predicting: Spatial location of thunderstorms Intensity characteristics of thunderstorms

Hazard prediction is carried out by tailored algorithms Hail Detection Algorithm

Looks for high radar reflectivity aloft Cores may descend to cause hail

Flash flood prediction algorithm Estimate rainfall amount based on radar reflectivity Accumulate rainfall in delineated basins Couple with flow model (soil moisture, etc.)

Etc.

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 4

How to do nowcasting

Numerical models Can not be done in real-time Skill of numerical models an area of much research

May be the future Rule-based prediction of growth and decay

Identify boundaries from multiple sensors or human input Extrapolate echoes likely to persist or form Approach of “Auto Nowcaster” from NCAR

Qualitatively: works Quantitatively: similar issues as numerical models

Linear extrapolation of radar echoes Highly skilled in the short term (under 60 minutes) Can be done in real-time Assumption is of steady-state (no growth/decay)

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 5

Methods for estimating movement

Linear extrapolation involves: Estimating movement Extrapolating based on movement

Techniques:

1. Object identification and tracking Find cells and track them

2. Optical flow techniques Find optimal motion between

rectangular subgrids at different times

3. Hybrid technique Find cells and find optimal

motion between cell and previous image

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 6

Some object-based methods

Storm cell identification and tracking (SCIT) Developed at NSSL, now operational on NEXRAD Allows trends of thunderstorm properties

Johnson J. T., P. L. MacKeen, A. Witt, E. D. Mitchell, G. J. Stumpf, M. D. Eilts, and K. W. Thomas, 1998: The Storm Cell Identification and Tracking Algorithm: An enhanced WSR-88D algorithm. Weather & Forecasting, 13, 263–276.

Multi-radar version part of WDSS-II Thunderstorm Identification, Tracking, Analysis, and Nowcasting (TITAN)

Developed at NCAR, part of Autonowcaster Dixon M. J., and G. Weiner, 1993: TITAN: Thunderstorm Identification, Tracking, Analysis,

and Nowcasting—A radar-based methodology. J. Atmos. Oceanic Technol., 10, 785–797

Optimization procedure to associate cells from successive time periods Satellite-based MCS-tracking methods

Association is based on overlap between MCS at different times Morel C. and S. Senesi, 2002: A climatology of mesoscale convective systems over

Europe using satellite infrared imagery. I: Methodology. Q. J. Royal Meteo. Soc., 128, 1953-1971

http://www.ssec.wisc.edu/~rabin/hpcc/storm_tracker.html

MCSs are large, so overlap-based methods work well

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 7

Object-based methods, pros & cons

How object-based methods work: Identify high-intensity clump of pixels as

“cells” Associate cells between time frames

Closest distance/values/overlap, etc. Pros:

Small-scale prediction Can find out history of a thunderstorm

(“trends”) Cons:

Splits and merges hard to keep track of Hard to avoid association errors Most storm cells last only about 20

minutes Large-scale predictions are difficult to

build up

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 8

Optical flow methods

How optical flow methods work Take rectangular region around each

pixel of current image Move rectangular window around

previous image Choose movement that minimizes error

between images Need to ensure that successive pixels

do not have very different movements Do not identify and associate cells

Pro: Removes cell identification and association errors

Con: No trends possible Not affected by splits/merges

Pro: More accurate motion estimates Con: Small-scale tracking not possible

Poor motion estimates where no storms available in current/previous image

Often have to use global movement Or interpolate between storms

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 9

Some optical flow methods

TREC Minimize mean square error within subgrids between images No global motion vector, so can be used in hurricane tracking Results in a very chaotic wind field in other situations

Tuttle, J., and R. Gall, 1999: A single-radar technique for estimating the winds in tropical cyclones. Bull. Amer. Meteor. Soc., 80, 653-668

Large-scale “growth and decay” tracker MIT/Lincoln Lab, used in airport weather tracking Smooth the images with large elliptical filter, limit deviation from global vector Not usable at small scales or for hurricanes

Wolfson, M. M., Forman, B. E., Hallowell, R. G., and M. P. Moore (1999): The Growth and Decay Storm Tracker, 8th Conference on Aviation, Range, and Aerospace Meteorology, Dallas, TX, p58-62

McGill Algorithm of Precipitation by Lagrangian Extrapolation (MAPLE) Variational optimization instead of a global motion vector Tracking for large scales only, but permits hurricanes and smooth fields

Germann, U. and I. Zawadski, 2002: Scale-dependence of the predictability of precipitation from continental radar images. Part I: Description of methodology. Mon. Wea. Rev., 130, 2859-2873

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 10

Need for hybrid technique

Need an algorithm that is capable of Tracking multiple scales: from storm cells to squall lines

Storm cells possible with SCIT (object-identification method) Squall lines possible with LL tracker (elliptical filters + optical flow)

Providing trend information Surveys indicate: most useful guidance information provided by SCIT

Estimating movement accurately Like MAPLE

How?

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 11

Technique

1. Identify storm cells based on reflectivity and its “texture”

2. Merge storm cells into larger scale entities

3. Estimate storm motion for each entity by comparing the entity with the previous image’s pixels

4. Interpolate spatially between the entities

5. Smooth motion estimates in time

6. Use motion vectors to make forecasts

Courtesy: Yang et. al (2006)

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 12

Why it works

Hierarchical clustering sidesteps problems inherent in object-identification and optical-flow based methods

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 13

Advantages of technique

Identify storms at multiple scales Hierarchical texture segmentation

using K-Means clustering Yields nested partitions (storm

cells inside squall lines) No storm-cell association errors

Use optical flow to estimate motion Increased accuracy

Instead of rectangular sub-grids, minimize error within storm cell

Single movement for each cell Chaotic windfields avoided

No global vector Cressman interpolation between

cells to fill out areas spatially Kalman filter at each pixel to

smooth out estimates temporally

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 14

1. Identifying storms: K-Means clustering

Obtain a vector of measurements at each pixel Statistics in neighborhood of each pixel (called “texture”) Can also use multiple sensors or channels

Divide up vector space into K “bands” The bands can be equally spaced by equal-probability Center the clustering algorithm at each of these bands Assign each pixel to the band that it lies in

Perform region growing Pixels in same band adjacent to each other are part of region Compute region properties

Move pixel from one region to another if cost function lowered Cost function lower if pixel moves to region whose mean texture it is closer to Cost function lower if pixel moves to region that it is closer (spatially) to

Iterate until stable

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 15

The cost function

The cost function takes into account Textural similarity between pixel at x,y and the mean texture of kth cluster

Spatial contiguity of pixel to cluster

Weighted appropriately (lambda=0.2 seems to work well)

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 16

Clustering: example

Radar reflectivity K=4 clustering

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 17

2. Hierarchical clustering

At the end of iteration, all pixels have been assigned to their best clusters

Most detailed scale of segmentation

Scale=0 Clusters are typically very small

Combine clusters to form larger regions

Find mean inter-cluster distance Combine regions which are

spatially adjacent whose textural means are close to each other

Repeat to get largest regions

Reflectivity Scale=0

Scale=1 Scale=2

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 18

3. Compute motion estimates

Starting with scale=2, project the current cluster backward Move the cluster around within the previous image Choose the movement that minimizes mean absolute error Minimization based on kernel estimate, to reduce outlier errors

A motion estimate obtained for each cluster Less noisy than pixel-based estimates

Automatic smoothing over region of cluster Scale=0 is the noisiest (fewer pixels)

What about newly developing cells? Limit the search space to maximum expected storm movement If mean absolute error is too large, assume that cell is new

Will take movement based on neighboring cells

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 19

4. Spatially interpolate motion vectors

Need motion estimate between regions Spatially interpolate between regions Weighted by distance from region (Cressman weights) Weighted by size of region Fill out spatial grid

Can use background wind field to fill out domain Constant weight for background wind field (from model) Use scale=2 motion estimate as background field for scale=1

Repeat process to get motion vector for scale=2 Use scale=1 motion estimate as background field for scale=0 Repeat process to get motion vector for scale=1

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 20

5. Kalman filter

Motion estimates are smoothed in time Each pixel runs a Kalman filter (constant acceleration model) Smoothes the motion estimates

Courtesy: Yang et. al (2006)

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 21

6. Use motion estimate to do forecast

Forward Using motion estimate at a pixel, project the point to where it should be Create a spatial Gaussian distribution of the point’s value at that location

Interpolation For fast moving storms, it is possible that there will be gaps in the output field Interpolate between projected points

Use different scales for different time periods, for example: Use scale=0 for forecasting less than 15 minutes Use scale=1 for forecasting 15-45 minutes Use scale=2 for forecasting longer than 45 minutes

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 22

7. Trends

What about trends? Compute properties of current cluster

Min, max, mean, count, histogram, etc. Project cluster backwards onto previous sets of images

Can use fields other than the field being tracked Compute properties of projected cluster Use to diagnose trends

Not used operationally yet

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 23

Example: hurricane (Sep. 18, 2003)

Image Scale=1

Eastward s.ward

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 24

Satellite water vapor (Feb. 28, 2003)

Image 30-min forecast

60-min forecast

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 25

Typhoon Nari (Taiwan, Sep. 16, 2001)

Composite reflectivity and CSI for forecasts > 20 dBZ Large-scale (temporally and spatially)

Courtesy: Yang et. al (2006)

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 26

Tornado case (Apr. 20, 1995)

Complete life-cycle of a storm: CSI at different scales and time periods

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 27

Tornado case (May 8, 2003)

Courtesy: Yang et. al (2006)

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 28

Comparison with other techniques (dBZ)KTLX, May 3 1999

Bias CSI

MAEForecasting reflectivity through

different techniques (30min)

1. Persistence2. TREC (xcorr)3. Same wind-field for all storms4. Hierarchical K-Means + Kalman

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 29

Comparison with other techniques (VIL) KTLX, May 3 1999

Bias CSI

MAEForecasting VIL through different

techniques (30 min)

1. Persistence2. TREC (xcorr)3. Same wind-field for all storms4. Hierarchical K-Means + Kalman

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 30

Forecast loop of VIL (May 3, 1999)

Oct. 23, 2006 Valliappa.Lakshmanan@noaa.gov 31

References

Technique described in this paper: Lakshmanan, V., R. Rabin, and V. DeBrunner, 2003: Multiscale storm

identification and forecast. J. Atm. Res., 67-68, 367-380 http://cimms.ou.edu/~lakshman/Papers/kmeans_motion.pdf

Some of the results shown here are from: Yang, H., J. Zhang, C. Langston, S. Wang (2006): Synchronization of Multiple

Radar Observations in 3-D Radar Mosaic, 12th Conf. on Aviation, Range and Aerospace Meteo. Atlanta, GA, P1.10

http://ams.confex.com/ams/pdfpapers/104386.pdf Software implementation

w2segmotion is one of the algorithms that is part of WDSS-II Lakshmanan, V., T. Smith, G. J. Stumpf, and K. Hondl, 2006 (In Press): The

warning decision support system - integrated information (WDSS-II). Weather and Forecasting.

http://www.wdssii.org/