Intelligent integration for nowcasting Selected slides from a talk given at the 38 th Annual Congress of the Canadian Meteorological and Oceanographic

Intelligent integration for nowcasting

Selected slides from a talk given at the 38th Annual Congressof the Canadian Meteorological and Oceanographic Society.

For the complete powerpoint file see:

A Fuzzy Logic-based Analog Forecasting System for Ceiling and Visibility, 38th Annual Congress of the Canadian Meteorological and Oceanographic Society, May 31-June 3, 2004, Edmonton, Alberta.

http://arxt39.cmc.ec.gc.ca/~armabha/papers_and_presentations

Future Role of Operational Meteorology

Scientific and systematicforecast process

Partnership with technology

How?

Fuzzy LogicIntegrationAlgorithm

ProductGenerator

User

HumanInput

(> 15 min)

SelectiveClimatological

Input

Real-TimeData

Algorithms

ModelOutput

Algorithms

Data AssimilationMesoscale Model

Real-Time DataPreprocessing

QualityControl

SensorSystems

1. RAP, Intelligent Weather Systems, www.rap.ucar.edu/technology/iws/design.htm

Intelligent Weather Systems (RAP/NCAR) 1

WeatherRadar

Nowcasts

RAP, Thunderstorm Auto-Nowcasting, www.rap.ucar.edu/projects/nowcast GUI

IWS Design

• Expert system development framework

• Applies existing knowledge, techniques and algorithms

• Achieves intelligent integration of all relevant, real-time data

• Supports rapid development of useful, maintainable operational applications

Fuzzy logic integration algorithm

For example, a fuzzy rule for forecasting radiation fog: 2

If sky clear and wind light and humidity high and humidity increasing

Then chance of radiation fog is high

Intelligent Weather Systems (RAP/NCAR) 1

1. RAP, Intelligent Weather Systems, www.rap.ucar.edu/technology/iws/design.htm 2. Jim Murtha, 1995: Applications of fuzzy logic in operational meteorology, Scientific Services and Professional Development Newsletter, Canadian Forces Weather Service, 42-54 3. Meteorological applications of fuzzy, http://chebucto.ca/Science/AIMET/applications

Satellite image

Wind speed

Humidity

Humidity trend

Chance of radiation fog(qualitative description)

W1

low med hilow

W2 medhi

Fuzzy Rule Base

Matrix of fuzzyrules coversspace ofall predictors

System canrun continuouslyto give real-time,smart forecastquality control.

For details,see examples. 3

Human input

Decision

For example, choice of

data and fcst technique

Operational MeteorologyA Scientific and Systematic Forecast Process:

a partnership with technology! 1

Technology Meteorologist

Observation Sat, radar, awos… Reports from public

Analyses 4DVAR, AI… Pattern recognition

Diagnoses RDP, AI… Conceptual models

Prognoses GEM, EPS, UMOS… Science, experience,training

Products/Services

PerformanceMeasures

1. Jim Abraham, 2004: Science-Operations Connection workshop, Meteorological Service of Canada, Toronto, 24-26 February 2004.

WORKSTATIONSCRIBE/AVIPADS, etc. Decisions

“Smart Alert” Concept

Impendingproblem

Bust

0 1 2 3 4 5 6 87 9 10 11 12232221

100+603025201510987654321

Search

St. John’s

| | | | | | | | || | | | | | | | || | | | | | | | || | | | | | | | || | | | | | | | |

…| | | | | | | | |

Make

Save Send

WeatherWind

CeilingVisibilityDirectionSpeedTime…Weather

FitLoose Tight| | | | | | | | || | | | | | | | || | | | | | | | || | | | | | | | || | | | | | | | |

…| | | | | | | | |

00h 121501h 131402h 1412...12h 1408

00h R-L-01h R-L-02h L-...12h L-

Search

AMD TAF CYYT 270010Z 270024 1315KT 2SM -RA BR OVC006 TEMPO 0002 1/2SM -DZ FG OVC003 FM0200Z 14010KT 1/2SM -DZ FG OVC002 TEMPO 0224 1/4SM -DZ FG OVC001 RMK NXT FCST BY 06Z=

Search Make

INTEGRATION

CLIMATEARCHIVE

data

PRODUCTDISPLAY(editable)

HEADS-UPALERT &DISPLAY

ACTUALWEATHER

MAP(animated)

GUIDANCEDISPLAY(satellite,

NWP, etc.)

FORECASTER(interacts, intervenes)

awareness and knowledge

PREDICTION

UPPER AIR

SATELLITE

METAR

REAL-TIMEOBSdata

RAW, QC’dWEATHER

data

MODELLEDWEATHER

NWPdata

PRODUCTGENERATION

PRODUCTSinformation

MODELLEDWEATHER

MAP(editable)

DSS(interaction withintegration and

prediction)

PRODUCTSPECIFICATIONS

CONSISTENCYCHECKING

TRANSLATION

FORECAST

EXTRAPOLATION

PROJECTEDOBS

AIknowledge

USER

MODEL-BASEDWEATHERELEMENTS

VERIFICATION

0 time

official forecast

actual trend

!

Graphic interventionFirst resort

Direct interventionLast resort

data and information• up-to-the-minute intelligent data fusion• abstract features• derived fields• intelligently composed “interest fields”

RADAR

DAdata

information• special interests• cost-based decision-making models

POST-PROCESSING

Battleboard raises forecaster’s situational awareness

GUI leverages forecaster’s actions

* Forecaster Workstation User Requirements Working Group meeting notes, 2000: Decision support systems for weather forecasting based on modular design, updated slightly for Aviation Tools Workshop in 2003.

DECISION SUPPORT SYSTEMS *

Decision Support Systems Design

Generic: no-name, conceptual design that could link and

integrate the most useful elements of WIND, AVISA, MultiAlert,

SCRIBE, FPA, URP, and so on in evolving WSP application, NinJo.

Modular: shows where distinct sub-tools / agents can be developed.

Working in this way, individual developers could work on isolated

sub-problems and anticipate how to plug their results into a larger

shared system. As technology inevitably improves, improved modules

can be easily installed and quickly implemented.

User-centered: forecast decision support systems from forecaster's

point of view, designed to increase situational awareness.

Hybrid: combines complementary sources of knowledge, forecasters

and AI, to increase the quality of input data and output information.

Intelligent integration of data, information, and model output, and

use of adaptive forecasting strategies are intrinsic in this design.

Hybrid Forecast Decision Support Systems

Hybrid forecast system development is a current direction of the Aviation Weather

Research Program (AWRP) 1 and the Research Applications Program (RAP), 2

NCAR (the main organizers of AWRP R&D).

“If a statistical / analog forecast disagrees with a model forecast, or if different

sensors disagree about how C&V are measured, what should we do about it?

Fuzzy logic could simulate how humans might apply confidence factors to

different pieces of information in different scenarios.” 3

AWRP Terminal Ceiling and Visibility Product Development Team (PDT) project,

Consensus Forecast System, a combination of:

COBEL, a physical column model 4

Statistical forecast models, local and regional

Satellite statistical forecast model

1. Aviation Weather Research Program, http://www.faa.gov/aua/awr

2. Research Applications Program, http://www.rap.ucar.edu

3. Norbert Driedger, 2004, personal communication.

4. Cobel, 1-D model, http://www.rap.ucar.edu/staff/tardif/COBEL


AWRP National Ceiling and Visibility PDT research initiatives: 1

Data fusion: intelligent integration of output of various models, observational data, and forecaster input using fuzzy logic 2, 3

Data mining, C5.0 pattern recognition software for generating decision trees based on data mining, freeware by Ross Quinlan (http://www.rulequest.com), like CART Analog forecasting using Euclidean distance development of daily climatology for 1500+ continental US (CONUS) sites Incorporate AutoNowcast of weather radar in 2004-2005 4

Incorporate satellite image cloud-type classification algorithms 5

1. Gerry Wiener, personal communication, July 2003.

2. Intelligent Weather Systems, RAP, NCAR, http://www.rap.ucar.edu/technology/iws

3. Shel Gerding and William Myers, 2003: Adaptive data fusion of meteorological forecast modules, 3rd Conference on Artificial Intelligence Applications to Environmental Science, AMS.

4. AutoNowcast, http://www.rap.ucar.edu/projects/nowcast

5. Tag, Paul M., Bankert, Richard L., Brody, L. Robin. 2000: An AVHRR Multiple Cloud- Type Classification Package. Journal of Applied Meteorology: Vol. 39, No. 2, pp. 125-134.

1. Herzegh, P. H., Bankert, R. L., Hansen, B. K., Tryhane, M., and Wiener, G., 2004: Recent progress in the development of automated analysis and forecast products for ceiling and visibility conditions, 20th Conference on Interactive Information and Processing Systems, American Meteorological Society.

National C&V Forecast System

DATABASE OF FORECAST COMPONENT PERFORMANCE VS WEATHER CONDITION.

FORECASTCOMPONENT WEIGHTS BASED ON PERFORMANCE DATABASE.

Eta Model

Augments RUC in CONUS and will support subsequent Alaska product

EXPERT SYSTEM-BASED FORECAST MERGE PROCESS(Weighted Simple Additive Model)

RUC20

C & V values derived from forecast hydrometeor and humidity fields.

Persistence

Statically carries forward current C & V conditions.

CurrentDisplay: NCV web, ADDS,

Cockpit, Other.

Forecast of Ceiling, Visibility & Flight

Category on RUC Grid

FY 04

Improved C&V Translation

Experimental use of data mining for improved translation.

Obs-Based Techniques

First trials of forecasts from historical data using obs inputs.

Rule-Based Methods

Practical forecast methods from operations for targeted locale.

Future

COBEL Column ModelColumn model with initialfocus on fog and low cloud in NE.

Others TBD. Hybrids

Future methods focused on C & V.

Feedback LoopUsing FY03-04 Mods


1. Richard Wagoner, 2001: Background briefing on post processing data fusion technology at NCAR, online presentation, http://www.rap.ucar.edu/general/press/presentations/wagoner_21feb2001.pdf

2. John K. Williams, 2004: Introduction to Fuzzy Logic as Used in the NCAR Research Applications Program, Artificial Intelligence Methods in Atmospheric and Oceanic Sciences: Neural Networks, Fuzzy Logic, and Genetic Algorithms, Short Course, American Meteorological Society, 10-11 January 2004, Seattle, WA. ftp://ftp.rap.ucar.edu/pub/AMS_AI_ShortCourse/Williams_AMS_ShortCourse_11Jan2004.pdf

According to Richard Wagoner, Deputy Director at Research Applications

Program (“Technology Transfer Program”), NCAR: 1

• NCAR / RAP is now a “continuous set theory” [fuzzy set theory]

development center.

• Over 90% of systems developed use fuzzy logic [FL] as the

intelligence integrator. [ … P.S. It is now 100% 2 ]

• [FL offers] unprecedented fidelity and accuracy in systems development.

• Automatic FL-based systems now compete with human forecasts.

Fuzzy Logic at Research Applications Program, NCAR

A graphical illustration to fuzzy logic, http://www.mcu.motsps.com/lit/tutor/fuzzy/fuzzy.html

Since we can assign numeric values to linguistic expressions, it follows that we can also combine such expressions into rules and evaluate them mathematically.A typical fuzzy logic rule might be:

If temperature is warm and pressure is low then set heat to high

Fuzzy logic


How Rules Relate to a Control Surface

A fuzzy associative matrix (FAM) can be helpful to be sure you are not missing any important rules in your system. Figure shows a FAM for a control system with two inputs, each having three labels. Inside each box you write a label of the system output. In this system there are nine possible rules corresponding to the nine boxes in the FAM. The highlighted box corresponds to the rule:

If temperature is warm and pressure is low then set heat to high


The input to output relationship is precise and constant. Many engineers were initially unwilling to embrace fuzzy logic because of a misconception that the results were not repeatable and approximate. The term fuzzy actually refers to the gradual transitions at set boundaries from false to true.

Three Dimensional Control Surface

Intelligent integration for nowcasting

For more information, see:

A Fuzzy Logic-based Analog Forecasting System for Ceiling and Visibility, 38th Annual Congress of the Canadian Meteorological and Oceanographic Society, May 31-June 3, 2004, Edmonton, Alberta.



Documents

Intelligent integration for nowcasting Selected slides from a talk given at the 38 th Annual Congress of the Canadian Meteorological and Oceanographic