Upload
christopher-barrett
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
1
Areas for National and International Collaboration Toward Net-Centric
Weather Information Access
Joint Action Group for XML and Web Services
Omaha, NE
July 2007
2
• Developing a work plan to resolve issues• Version control• Conceptual Data Model (UML vs. ER)• Data dictionary• XML schemas• Access services
– W3C/OGC standards– General access by major data types– Specific access for major customers
• Standards followed– Must be non-proprietary
Areas Addressed for Collaboration
3
WMO/ICAO Issues(Not to be Distributed)
• Overall concept is to reduce risk– Developing the standard is to reduce risks to customers
• Financial – different sets of equipment for US/Europe– Without US position the WMO dictates the standard– But there is still the risk that we develop to the US standard but
then have to readapt to a standard that the WMO puts forth
• Provide leadership to the international community
• Partnership with EUROCONTROL provides leverage with WMO for adoption of standard
4
EUROCONTROL Meeting Issues
• Significant issues for data exchange– Version control– Conceptual Data Model (UML vs. ER)– Data dictionary– XML schema– Access services
• W3C/OGC standards• General access by major data types• Specific access for major customers
– Non-proprietary standards
• Many of these have been addressed to considerable extent by existing US programs
5
Version Control
• Includes software versioning and operational support to backward/forward compatibility
• This is driven by the need for agility over time, especially as standards evolve
• Existing services use XML Schema 1.0– XML Schema 1.1 will provide inherent version support
• Not yet released• Is expected to lower version control costs
• What entity will host and fund version control support?
6
Conceptual Data Model – Data Storage
• A uniform conceptual data model of database design across DOD and non-DOD has not yet been decided upon and may not be necessary or desirable
• DOD has such a conceptual data model– Joint METOC Conceptual Data Model (JMCDM )
• Uses ER representation• Gridded data, observations, climatology, imagery, platforms, space weather,
alphanumeric bulletins, remote sensed observations, catalog, …
• Representation of the model– DOD uses ER representation
• DOD is investigating translation from ER to UML• Report is expected to be available late August 2007
– Some non-DOD agencies and EUROCONTROL use UML
7
Conceptual Data Model – Data Exchange
• A conceptual model of Web Services interface exchange messages is desirable
• Schema exist from which these models could be generated– DOD - JMBL – NOAA - DWML and GML– …
8
Conceptual Data Model - JMBL Experience
• Approach started with “top down” approach– Initial work was development of an ER database model– The data model was then used as a starting point to develop the
JMBL data exchange schema– This interface schema abstracted the database design
• Approach evolved to “bottom up” approach– Examined user need to revise JMBL– Use cases can assist with this approach
9
Data Dictionary
• There are several technologies available to capture a data dictionary– Dictionaries, taxonomies, ontologies, …
• DOD– Master Parameter List (MPL) identifies all parameters available at the
JMBL interface level– Working on a taxonomy
• NOAA– Could develop the equivalent of the Master Parameter List from DWML
• Unidata– Uses netCDF CF convention as a data dictionary– Lacking in sensor, satellite and radar
• A standard should be considered across the community
10
XML Exchange Schema
• DOD – Joint METOC Broker Language (JMBL)– Schema for data request and response– Gridded data, observations, climatology, imagery, platforms, space weather, alphanumeric
bulletins, remote sensed observations, …– Initial operational capability at AFWA, FNMOC and NAVO
• Gridded data, observations, alphanumeric messages, platform information• Phased implementation in progress for additional METOC data types
• NOAA – Digital Weather Markup Language (DWML)– Schema for data request and response– Forecasts and observations– In use at NOAA
• NOAA – Common Alerting Protocol (CAP)– Schema for data response– Watches, warnings and advisories– This is an alerting protocol in use by NWS but not specifically a weather protocol
• FAA/NWS – Aviation Digital Data Service (ADDS)– Schema for data response– NWS text observations, forecasts and advisories specific to aviation
11
Access Services
• Relevant standards– W3C,OGC, … (elaborated on following slide)
• Exchange schema addresses– General access by major data types
– Specific access for major customers
– Finest granularity possible
• Compliance beyond basic W3C standards depends on requirements derived from use-cases– Follow through with work plan will be necessary to evaluate relevant
standards
• Standards followed must be non-proprietary
12
Relevant Standards• W3C Standards
– XML, XML Schema, SOAP, WSDL, RDF, OWL (Semantic Web)
• OASIS– ebXML registry/repository
• Web service profile• OWL profile
– UDDI registry– Common Alerting Protocol (CAP)
• ISO TC/211 Geographic information standards– ISO 19101: Geographic information - Reference model– ISO 19103: Geographic information - Conceptual schema language– ISO 19107: Geographic information - Spatial schema– ISO 19108: Geographic information - Temporal schema– ISO 19109: Geographic information - Rules for application schema– ISO 19110: Geographic information - Methodology for feature cataloging– ISO 19111: Geographic information – Spatial referencing by coordinates– ISO 19115: Geographic information - Metadata– ISO 19118: Geographic information - Encoding– ISO 19123: Geographic information - Schema for coverage geometry and functions– ISO 19136: Geographic information - Geography markup language (GML) (Shared OGC standard)– ISO 19139: Geographic information - Metadata - XML schema implementation – ISO 8601 Time
13
Relevant Standards (cont’d)
• Open Geospatial Consortium– Geography Markup Language (GML), a toolkit for building data descriptions– SensorML– Catalog Service (ebXML profile)– Web Feature Service (WFS)– Web Coverage Service (WCS)– Web Map Service (WMS)– Web Processing Service (WPS)
• DOD– NCES catalog service– JMBL– DDMS metadata standard
• Based on Dublin Core• Incorporates IC/ISM security markup
• FAA/Eurocontrol– Aeronautical Information Exchange Model (AIXM)– Weather Exchange Model (WXXM)– Models and notional data dissemination mechanisms based on ISO/OGC
standards
14
Existing Access Services & Compliance
• DOD – JMBL– W3C, OASIS compliant
• XML 1.0, SOAP, WSDL, Schema, WS-I
• NOAA –DWML– W3C compliant
• XML 1.0, SOAP, WSDL, Schema
• NOAA – CAP– W3C, OASIS compliant
• XML 1.0, CAP1.0
• FAA/NWS – ADDS– W3C compliant
• XML 1.0, REST
15
Sample Capabilities• JMBL
– Gridded atmospheric (areas, points, lines, slant angle) and ocean forecasts, observations (raw, decoded, selected parameter), alphanumeric messages (METARS, TAFS, PIREPS, AIRMETS, SIGMETS, …), station info
• DWML– Forecasts (point, selected parameters/character of the day), city forecasts &
observations, national highs/lows
• CAP– Text warnings, watches, advisories, outlooks, alerts
• ADDS– METARS, TAFS, PIREPS, AIRMETS, SIGMETS, station info– WCS in development for icing, turbulence, winds, temperatures, ceiling, visibility,
echo top, and VIL
16
What type of services?
• Data exchange/use cases– Grids/obs/warnings
• JAG will collect and define use cases to identify current and near-term capabilities– Sample template follows– Level of granularity:
• One system level slide• Two or three user task level slides• Data retrieval by end-user
– Optionally include an OV-1 graphic for system level use case– Target delivery date is October 1, 2007
• Email use cases to Ken Barnett and cc to Tim Hopkins• Focus on aviation use cases
• Can we recommend alignment of current systems to achieve some goal… how do we leverage?
17
Simple (Trivial) Text-Only Use Case Example(From W3C Schema versioning use cases)
Web services content" scenario C: original instance with new schema
• Brief summary: An instance of the original schema will be accepted by the new schema.
• Basic course of events:– 1. An instance of the original schema is processed.– 2. The processor identifies it as a version of the new schema.– 3. The system processes the instance.
• Desired outcome:– The processor determines that it is a version of the new schema. – Valid against the new schema. – PSVI returns all information items expected from the new schema (even
though it is an old instance). • This scenario may occur when an old client invokes a new release
of the service.
18
General Form for Use CasesGoal:
Paragraph (or two) describing background and goal of use case at high level
Level:
Level of the use case. The exact terminology used is somewhat flexible. A use case summarizing a behaviour at a high level uses ‘Summary’, while a more detailed user-level behaviour specifies ‘user’. See reference for list of possible and their meanings.
Preconditions:
Set of conditions that are assumed to be true at the start of the use case. This allows the detailed steps that follow to focus on only those steps that are important to the particular use case.
Main Success Scenario:
1. First step in scenario. Steps are usually one or two sentences in length
2. Second step….
3. Third step…. Generally the number of steps is between 3 (for fine-grained use cases) and 20, but this is not fixed.
4. Etc….
Extensions:
3a. Alternate path for step 3. Alternate paths can describe handling of a particular error condition, or any key variation of the use case that is deemed relevant with respect to the requirements it imposes on a design
19
Example: Access of Satellite ImageryGoal:
Access satellite imagery data for a given region of interest at a specific time. Returned imagery is in coordinate reference system chosen by user
Level:
User
Preconditions:
Satellite data of interest exists in database on the ground and is accessible via a LAN with sufficient bandwidth to download the image. The user has the necessary security clearance to access the data.
Main Success Scenario:
1. User brings up geospatial data analysis tool.
2. Tool dynamically discovers available data sources (catalogs)
2. User makes request for metadata describing all satellite imagery from yesterday within 500 miles of Borneo
3. Tool presents user with list of available data sets, their accompanying metadata, and the capabilities of the data access services available to access the data (available formats, coordinate reference systems, etc…)
4. User makes request for data of highest interest, specifying the desired spatial, temporal, and coordinate system properties.
5. System accesses the raw data image, produces the desired image, and returns it (or a reference to it) to the user
6. Image is displayed in geospatial data analysis tool.
Extensions:
4a. No imagery currently exists for target area. A request for imagery is submitted to the entity responsible for scheduling satellite imagery data collections.
4b. The status (accepted/pending, rejected) of the request is returned to the user
20
Systems Agency JAG XML Use CasesExist ToDo
Due
AWIPS NOAA/NWS x AWIPS-2 Canonical(ie tbd) 1 System/6 User 8/21/2007
NDFD NOAA/NWS x None 2 System/4 User
Web Services NOAA/NWS x CAP KML DWML WFS Yes(CAP) 1 System/4 User 8/21/2007
NWSTG NOAA/NWS x NONE
ADDS FAA/NCAR/GSD x Non-gridded Text Data Yes 1 System/8 User 8/21/2007
WDAC AFWA x JMBL Yes (UML) 1 System/5 User 8/21/2007
JET AFWA x JMBL, ESRI Yes 1 System/5 User
WPMDS/CDC AFWA x Planned Yes(CDC) 1 System/4 User 8/21/2007
ITWS FAA/LL x None Yes 1 System/4 User
CIWS FAA/LL x Experimental HDF-5 SOAP Yes 1 System/4 User 8/21/2007
NOP/NMDSF CNMOC x JMBL 1 System/4 User
CAGIPS CNMOC x JMBL 1 System/4 User
NDGD NOAA/NWS x TBD
Preliminary System List JAG – XML & Web Services
21
Follow On
• Next meeting– Virtual meeting with Web Ex or Go To Meeting– Target is October 15, 2007– Ken to set up a test meeting ahead to identify
participation issues– Agenda
• Discuss use cases to identify baseline of current and near-term capability
• Outline a business case for resources to establish– Standardized, XML-based schema exchange format– Consistent, national net-centric weather services interfaces
• Pick a date for follow on face to face meeting
22
Roadmap Objective
• Shape JMBL into JMBL++ to become a national and international standard for machine-to-machine environmental data exchange
• Evaluate other technologies for adoption as special purpose services within JMBL++– Leading candidates include DWML, CAP, ADDS, etc.– E.g., propose CAP to for alert messaging for the meteorological
community
• The overall approach is driven by use cases
23
Roadmap• Brief OGC meeting at NCAR (Sep 07)
• Use Case Gathering (Oct 07)– Derive use cases from selected existing systems– Augment with near-term use cases– ROM cost to this point is roughly 440 hours (senior
analyst/engineer)
• Obtain resources to work remainder of roadmap (Oct 07)– ROM cost will depend on outcome of use case analysis– Additional team representation will be needed
• Use Case Abstraction (Nov 07)– Identify commonality– Consolidate
24
Roadmap• Service Design (Feb 08)
– Assess how well existing services (JMBL, DWML, CAP, ADDS, …) • Meet needs of use cases• Comply with standards such as
– ISO (19xxx), OGC (GML), HDF5, NetCDF – Some of these are data model 'building blocks' and require application profiles
– Apply use cases to shape JMBL into JMBL++ to become a national and international standard for environmental data exchange
• Seek close EUROCONTROL collaboration on conceptual data model, …
– Identify functionality gaps in existing services, missing services
– Augment existing services to fill functionality gaps• Work within standards bodies working groups• Create service 'profiles' for weather COI
– Propose new services as needed• 'Flight path service', etc.• Welcome use cases from EUROCONTROL for further collaboration
25
Roadmap (cont’d)
• Beyond Feb 08– Address registry, discovery, metadata, catalog, …– File format issues (netCDF-4/binary xml)– Examine REST as a complement to SOAP– Common business rules for server behavior
26
Building Application-Level 'Profiles'
Existing Format/Building Block COI-developed Profiles
Data Model/Building Block
XML Schema
GML
ISO 19xxxStandards
XML-based Formats
JMBL++ Data Model
WXXM (like) Application Schema
CAP
27
Data Format Layers
HDF-5
HDF5-EOS
HDF-5
NetCDF-4
NetCDF-4-CF
NetCDF-4-CF-WX?
Binary Formats …
HDF5-EOS-WX?
Existing Format/Building Block COI-developed Profiles
Data Model/Building Block
29
POCs
TECHNOLOGY POC
AWIPS Tim Hopkins and Brian Gockel
NDFD Tim Kempisty & Co.
Web Services Bob Bunge & Co.
TG Bob Bunge
ADDS Aaron Braeckel
WDAC Eric Wise
JET Eric Wise
WPMDS/CDC Eric Wise
ITWS Oliver Newell
CIWS Oliver Newell
NOP/NMDSF Roy Ladner
CAGIPS Roy Ladner
NDGD Tim Kempisty
30
Areas for national collaboration.(* = Areas we expect will come up at the Aug 07 meeting with EUROCONTROL.)
• Use Service Oriented Architecture (SOA) approach (tri-agency agreement)
• *Setting an action plan to resolve issues• Catalog• Discovery – taxonomy• Subscription• CAP- warning advisory service• Shared security model• *Version control• *Conceptual Data Model (UML, ER)• Exchange data formats
– NetCDF4 (CF), HDF-5 (EOS)• *Data dictionary
31
• *XML schemas• *Access services
– *OGC standards– *W3C standards– *General access by major data types– *Specific access for major customers– Formats supported in delivery– Common business rules– Communication challenged users (bandwidth and protocol)
• *Standards followed– Must be non-proprietary
• Preserve independence from physical data base
Areas for national collaboration.(* = Areas we expect will come up at the Aug 07 meeting with EUROCONTROL.)
32
Deliverable approach• Develop pieces to build business plan• What are the resources needed?
– Staff– Level of technical effort– Time lines for projects/demos
• Areas to focus on – JMBL, OGC standards, etc• * WMO/ICAO angle• * Technology args
– Net-centric– What standards: W3C/OASIS/ISO/OGC – What level interop/validation?
• Standardize business rule profile?– Must be non-proprietary
• * Short term – EUROCONTROL support
33
NNEW SWIM JMB NOAA
NCARNOAA/GSD MIT/LL DOC MITRE Boeing AFWA CNMOC NESDIS NCDC &
CTRS NWSNCEP(9)
• OGC Services• GML-based data format• netCDF4, HDF5 grids• OWL ontologies• architectural documents• Use cases
• architectural documents• Use cases • Websphere (& related technologies)
• AWIPS• NDFD• Web services• NWSTG• GML-based data format• netCDF4, HDF5 grids• OWL ontologies