Quo Vadit Visual Weather? By Jozef Matula. What is IBL Visual Weather? Meteorological Workstation SW...

Quo Vadit Visual Weather?Quo Vadit Visual Weather?

By Jozef Matula By Jozef Matula

What is IBL Visual Weather?What is IBL Visual Weather?

• Meteorological Workstation SW providing:– Met. data processing and visualisation– Interactive forecasting tools– Forecast production and workflow

management– Batch production– Extensibility with Python API– Web services (WMS, WCS, WFS, JMBL)

• Everything in one box or Client-Server• Highly configurable• 1.2M+ lines of C++, 120k+ lines of Python, 40k lines of

XML, 120k lines of Fortran, 220k lines of configuration…

PAST YEAR OF VW DEVELOPMENTPAST YEAR OF VW DEVELOPMENTDevelopers have worked really hard during…

3, 2, 1, start!3, 2, 1, start!

• 3 maintained development branches:– 3.1.x stable– 3.2.y stabilising– 3.3.z development

• Most significant development areas:– Forecast production and workflow– Grid “dataset” concept– Field Diagnostics and Modification (UK

MetMorph)– Batch production improvements– Tropical cyclone stuff– Web Services

FORECAST PRODUCTIONFORECAST PRODUCTIONThe purpose of life is…

Forecast Production - FeaturesForecast Production - Features

• We refactored our editor of features (mainly for SIGWX requirements)

• Every feature is timestamped and geolocated• Stored and shared in the “Feature Database”• Usability enhancements:

– Single click drawing– Curve optimalisation

(to reduce noise from tablet)– Cooperation with

underlying layers• Work on WFS-T access in progress

Example of a Feature ProductExample of a Feature Product

With courtesy of 21 OWS USAFE, Sembach

Forecast Production – Text forecastsForecast Production – Text forecasts

• XML Message Editor Templates (declarative language)

• User input validation• Preview before issue• Internal production and delivery chain

– Different outputs from one form– Single click for whole production and delivery

• Revision management– Corrections, amendments and cancelations– Access to history of issued forms

• Initialisation of form contents from– VW Mathematical Kernel (Observation, NWP data)– Plugins in Python

Forecast Production - Message Forecast Production - Message Editor 2Editor 2

PDF

AFTN/GTS

Message Editor – Template Message Editor – Template LanguageLanguage

• XML description language, consisting of:– “Model” – hierarchical data structure

definition– “Layout” – mapping of model to GUI

Message Editor – Building BlocksMessage Editor – Building Blocks

Instance of data model can be transformed in multiple ways and delivered to multiple places

Aerodrome Warning

Aerodrome Warning

Text Bulletin

ODT w. all reports

PDF for Station 1

Send to AFTN

Print 2x copies

FTP to WebPDF for Station

2

PDF for Station 3

Copy to Archive

Production Pipeline

Message Editor –Message Editor – Embedded Custom Embedded Custom UIsUIs

Forecast Production – Workflow Forecast Production – Workflow ManagementManagement

• Personal Task Manager provides a role based ToDo list interconnected with production applications

Hyperlink

Access to history of

work

Workflow

actions

State of work

Colour of task can

be used to indicate type or

priority of the task

THE “DATASET” CONCEPTTHE “DATASET” CONCEPTWell I am the recent forecast who is more?

Dataset – What for?Dataset – What for?

• If you modify fields, you might want to see the previous revisions (On Screen Analysis, Field Modification)

• EPS models require completely new dimension(s) in “grid space navigation” (perturbations, statistics fields, probability fields)

Leve

l

Run base time

Fore

cast

Dataset

Datasets – EPS ExampleDatasets – EPS Example

Dataset is Dataset is also a also a

global map global map variablevariable

CONSISTENT FIELD CONSISTENT FIELD MODIFICATION (METMORPH)MODIFICATION (METMORPH)

Tell me that this is not a magic…

Field ModificationField Modification

• We redesigned original simple field modification module into generalised field modification framework

• 2 available implementations:– Old style single-field modification– UK Met Office MetMorph (successor of OSFM)

• Output is stored in GRIB and available to visualisation or batch production

• Comparing to UKMO Horace 4 MetMorph:• Facelifted and integrated• Removed some original limitations• Access to history of changes

MetMorph – Main FeaturesMetMorph – Main Features

• Global and mesoscale model merging• Dynamical field change • Editing of wind• Editing of precipitation (enhancing, reduction,

wind advection)• Merging and advecting precipitation from radar• Orographic modulation• Field spatial and time smoothing and

interpolation• Variation of changes in time

– Shape and change-vector morphing– Variation of change-strenght in time

Met Morph – Dynamical MSLP Met Morph – Dynamical MSLP changechange

MetMorph – Time LinkingMetMorph – Time Linking

BATCH PRODUCTION BATCH PRODUCTION IMPROVEMENTSIMPROVEMENTS

Web server cannot imagine his life without…

Batch production improvementsBatch production improvements

• Parallel execution of batch jobs• Ability to have “user” and “role based” batch jobs

and limit triggers.• Improved triggers for:

– Detection of particular NWP parameter & forecast

– Area/station list based monitoring of observations

– Detecting completion of a Feature Product• New jobs:

– Field Diagnostics – precalculation of grid fields algorithms

– Field Cutting– BatchOSA, Batch Message Editor production

TROPICAL CYCLONE TROPICAL CYCLONE FORECASTING TOOLSFORECASTING TOOLS

Also an umbrella is one of…

Tropical Cyclone Forecasting ToolsTropical Cyclone Forecasting Tools

• Just an experimental internal development – both implemented in Python as a kind of tutorial sandbox example:– TC Storm tracking visualisation (parsing the

free text messages from GTS)– Dvorak helper construction

Tropical Cyclone Forecasting Tools - Tropical Cyclone Forecasting Tools - ExamplesExamples

WEB SERVICESWEB SERVICES

One day if I want to cook an egg I’ll make for it use of…

Present Web Services in Visual Present Web Services in Visual WeatherWeather

• Multi threaded with control of maximum load• Implemented as yet another VW service• WMS:

– Exposes ANY product from visualisation– Configurable dimensions and layer styles

• WCS:– Exposes any grid field (or derived field) from VW

database, including EPS fields.• WFS:

– Access to TAF/METAR reports– We see a lot of potential here (everything can

be seen as a feature!)• JMBL – JMOBS and JMGRID

OGC WS implementation dilemmasOGC WS implementation dilemmas

• The challenge for building web applications is to find the proper border between server and client, and to choose the proper protocol

• Also there are some technical problems:

Visual Weather WMS “specialities”Visual Weather WMS “specialities”

• PROJ4: and IBL: CRS namespaceCRS=PROJ4:+proj=latlong +datum=NAD83CRS=IBL:TEPHIGRAM

• Cross-sections – Do you know CRS:1?– How to pass the “place” information:

• Horizontal route:DIM_PLACE=LZIB EPSG:4326[48;19] EHAM

• Time crosssectionDIM_PLACE=EPSG:4326[48;19]

• JPEG2000 (good compression with alpha transparency) – but lack of support in browsers/Flex – the same with GeoTIFF

Visual Weather W*S dimensionsVisual Weather W*S dimensions

• For WMS naming and dimensions are configurable:– No dimensions at all (latest data use case)

– Single TIME dimension (both for observed or forecasted data from the recent model run)

– DIM_RUN+TIME dimension – doesn’t look to be a good way (non-orthogonal dimension space)

– DIM_RUN+DIM_FORECAST

– Optional ELEVATION dimension with configurable ways how to format the values (850 vs. 850hPa).

Visual Weather W*S dimensions (2)Visual Weather W*S dimensions (2)

• For WCS we have only static dimensions:– DIM_RUN, DIM_FORECAST, TIME, (+ EPS DIM_MEMBER) – time dimensions overlap

• For WFS – noone cares about dimensions anymore – it’s like single table SQL…

• WCS and especially WFS are vulnerable to DoS attacks!

RIA powered by pure VW WMSRIA powered by pure VW WMS

1600 OTHER IMPROVEMENTS, 1600 OTHER IMPROVEMENTS, CHANGES AND FIXES CHANGES AND FIXES

Well and something more…

Thank youThank youfor your attention!for your attention!

Questions?Questions?

Thank youThank youfor your attention!for your attention!

Questions?Questions?

Thanks to KNMI and particularlyThanks to KNMI and particularlyKees and Helen for the hospitality!Kees and Helen for the hospitality!

Thanks to KNMI and particularlyThanks to KNMI and particularlyKees and Helen for the hospitality!Kees and Helen for the hospitality!