VOYAGER Data Explorer: Architecture and TechnologiesSee also Design and Applications

Built and used Used by a Virtual Community on Aerosols

Layered Map

Time Chart

Providers Users

Vector

GIS Data

XDim DataSQL Tables

WebImages

Voyager Web Services

Publish, Find, Bind

Data & Tool Catalog

Uniform Access

Scatter Chart

S u p p o r tC o o r d i n a t i o n T e c h n o l o g i e s

Web Services Components and Actions

• Service providers publish services to a service broker.

• Service users find the needed service and get access key from a service broker• With the access key, users bind to the service provider• The result is a dynamic binding mechanism between the service users and providers

Service Broker

Service Provider

PublishFind

BindServiceUser

Components: Provider – User – Broker

Actions: Publish – Find - Bind

Web Application: Chained Web Services

• A Web Service Provider may also be a User of other services • Multiple web services can be chained into an interactive workflow system • The result is an agile application that can be created ‘just in time’ by the user for a specific need

Service Broker

Service Provider/User

PublishFind

BindServiceUser

Chain

Service Provider

Bind

Chain

Web PublishHTTP, FTP

Data Access though a Web Service Adapter

Service Broker

Publish

Service Consumer

Find

Access

Ordinary web content can be delivered as a Web Service through a Proxy Server.

• The Adapter Service converts HTTP/FTP service to XML Web service

• The Adopter Service publishes the web service to the Broker

• The User finds the data from the broker and accesses the Adapter to get distributed data

Service Adapter

WebServer

ServiceUser Chain

Interoperability through Web Services

• Interoperability: Independent of platform and language, based on XML Web Services

• Service Transparency: Published capabilities of components; implementation opaque.

• Just-in-time integration: Discovery, access to and ad-hoc chaining of services.

• Legacy Support: Encapsulating existing components and exposing them as services.

4 D Geo-Environmental Data Cube (X, Y, Z, T)

Environmental data represent measurements in the physical world which has space (X, Y, Z) and time (T) as its dimensions.

The specific inherent dimensions for geo-environmental data are: Longitude X, Latitude Y, Elevation Z and DateTime T.

The needs for finding, sharing and integration of geo-environmental data requires that data are ‘coded’ in multidimensional data space

4D Geo-Environmental Data Illustration

• Animated 3D Maps

Hierarchy of Data Objects:

DataGranule, Data Series, DataCube

Measure A measure (in OLAP terminology) represent numerical

values for a specific entity to be analyzed (e.g. temperature, wind speed, pollutant).

A collection of measures form a special dimension ‘ Measures’ (??Can Measures be Dimensions??)

Data GranulesA data granules– discrete, atomic data entities that cannot be further broken down.

A data series is an ordered collection of data granules DataSeries is a collection of DataGranules having common attributes

All data points in a measure represent the same measured parameter e.g. temperature. Hence, they share the same units and dimensionality.

The data points of a measure are enclosed in a conceptual multidimensional data cube; each data point occupies a volume (slice or point) in the data cube.

Data points in a measure share the same dimensions; Conversely, each data point has to have the dimensional coordinates in the data cube of the measure that it belongs to.

Dimension Y

DataSeriesDimension Z

Dimension X

DataCube

DataGranule

Catalog of Multidimensional Datasets

• Designed to Publish, Find(Select), Bind(Access) and Render distributed datasets• Publishing is though an open web interface for user/broker registration of datasets• Finding a dataset is aided by a metadata on Provider and Dataset• Binding (data access) information is contained in the Dimensional Tables• Rendering parameters are also contained in the Dimensional Tables

Multi-dimensional Data Access

• In array notation, the granule ‘value’ is accessed as – MyGranule = My1DArray(i)

– MyGranule = My2DArray(i,j)

– MyGranule = MynDArray(i,j,…..n)

• In order to select a data granule, a controller is assigned to each data dimension

1D Dataset

e.g. Time selector

i

j

k

j

i

Data Granule

Selection

i2D Dataset

e.g. Param & Time selector

3D Dataset

e.g. Param, Time & Location

DVoy Data Space

XminZmin

Zmin

XmaxYmin

Ymax

Data Views

Z

X

Y

Data Space for a Measure

Ymax

Xmin XmaxYmin

DataZmin

Zmax

•aergqegqet

View Data Space

Data Granule

Measure Data Space

Geo-referenced Map Image: Time Indexed

Regular Time Series; Fixed Georect; Image Trim

<measure code="TOMS01" name="TOMS - Aerosol Index - Glob" descr="Earth probe TOMS aerosol index">

<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"/>

<data_facts>< fact_attributes >

<data name ="TOMS" type="map image" descript="/>

<granul_query handler_assembly= (access handler)…./>

<image width=“640" height=“480“ margin_left="69" margin_top="46"/>

< georect lat_ min="-65" lat_ max="65“ lon_min="-180“ lon_max="180“ unit ="DecDeg“ />

</ fact_attributes >

</ data_facts>

<data_dimensions>

<time_dimension

handler_assembly="CAPITA.data.index.impl"

handler_class="CAPITA.data.index.impl.datetime_interval"

start_time="1996-07-25"

end_time="yesterday">

sample_periodicity _unit = "day"

sample_periodicity_mult = “1"

</time_dimension>

</data_dimensions>

</measure>

TOMSFact

TimeIndex Int, Calc from date

DailyImage Img, Parsed from date

LatMin Fixed, -65

LatMax Fixed, 65

LonMin Fixed, - 180

LonMax Fixed, 180

TOMSTime

TimeIndex

DateTime

Virtual Tables (No physical StarTables) XML Example: TOMS

ftp://jwocky.gsfc.nasa.gov/pub/eptoms/images/aerosol/y2002/ea020510.gif

Geo-referenced Map Image: Image IndexedVariable Time , Var GeoRect, Var Sensor

SatGeoImg

CapImgID Universal ID

ProviderURL URL

SourceURL URL

ImageURL URL

ProviderAbbrev NASA GSFC

ImageDescription Text

Sensor Enumerated

DateTime DateTime

LatMin -65

LatMax 65

LonMin - 180

LonMax 180

ImageWidth For size verification

ImageHeight

MarginLeft For trimming

MarginTop

MarginRight

MarginBottom

SQL Table

<measure code=“SeaWiFS01" name=“SeaWiFS – Reflectance Images - Glob" descr=" SeaWiFS – Reflectance Images - Global">

<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"/>

<data_facts>< fact_attributes >

<data name ="SeaWiFS" type="map image" descript=“stuff stuff “/>

<granul_query handler_assembly= (access handler)…./>

<fact table, SQL same as CIRA >

</ fact_attributes > … see table left

</ data_facts>

</measure>

NO DIMENSION TABLES

XML Example: SeaWiFS

Astronaut Photos: Image IndexedVariable Time, Image Center, Platform SatImg

NASAImageID Universal ID

ProviderURL http://eol.jsc.nasa.gov/

SourceURL http://eol.jsc.nasa.gov/scripts/sseop/photo.pl?mission=ISS004&roll=E&frame=11079

ImageURL http://eol.jsc.nasa.gov/sseop/images/ESC/small/ISS004/ISS004-E-11079.JPG

ProviderAbbrev NASA JSC

ImageDescription Text

Platform ISS004

GeoRegion GUATEMALA

Features SMOKE, GULF OF HONDURAS

DateTime 20020501 212700

Lat 17.6

Lat -90.2

SQL Table

<measure code=“AstroPhoto01" name=“Astronaut Photographs " descr=" Astronaut Photographs ">

<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"

/>

<data_facts>< fact_attributes >

<data name ="SeaWiFS" type="map image" descript=“stuff stuff “/>

<granul_query handler_assembly= (access handler)…./>

<fact table, SQL same as CIRA >

</ fact_attributes > … see table left

</ data_facts>

</measure>

XML Example: ISS004

Atmospheric Photos: Image IndexedVariable Time, Image Center, PlatformAtmoImg

ImageID Universal ID

ProviderURL http://capita.wustl.edu/capita/people/RHusar/rhusar.html

SourceURL http://capita.wustl.edu/capita/people/RHusar/Pic/9812SanFranciscoAGU/9812SanFranciscoAGU.htm

ImageURL http://capita.wustl.edu/capita/people/RHusar/Pic/9812SanFranciscoAGU/Dcp01000Enh.jpg

Provider Abbrev R Husar

ImageDescription Text

Platform Airplane

GeoRegion NEVADA

Features DUST PLUME

DateTime 19981208 1800

Lat 36.75

Lat -115.15

SQL Table

<measure code=“AtmosPhoto01" name=“Atmospheric Photographs " descr=" Atmospheric Photographs ">

<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"

/>

<data_facts>< fact_attributes >

<data name ="SeaWiFS" type="map image" descript=“stuff stuff “/>

<granul_query handler_assembly= (access handler)…./>

<fact table, SQL same as CIRA >

</ fact_attributes > … see table left

</ data_facts>

</measure>

XML Example: Dust Plume Photo

Interoperability through a Layered Protocol Stack

• Web Services are implemented on a layered stack of technologies and standards

• The lower layers enable binding and exchange of messages; higher levels enable interoperability

• Applications are formed dynamically from distributed components through publish-find-bind mechanisms

TCP/IP, HTTP, FTP

ASCII, XML, etc.

HTML, XML OGC -GML

OGC Coverage, CoordTransfom, WMS

HTTP, SOAP

WSDL

UDDI OGC Catalog

WSFL, XLANG

StandardsInteroperability

Comm. Protocols

Data Encoding

Data Schema

Data Binding

Web Service

Service Integr.

Service Discovery

Service Descript.

Connectivity

Web Services Standards

• Discovery and Integration: UDDI (Universal Description, Discovery and Integration)

• Service Description: WSDL (Web Services Description Language)

• Content Envelope: SOAP (Simple Object Access Protocol)

• Data Encoding: XML (Extensible Modeling Language)

• Service Chaining: WSFL (Web Services Flow Language, IBM) or XLANG (Message flow language, Microsoft)

ChainWSFL/XLANG

Service Broker

Service Provider/User

PublishUDDI, WSDL

FindUDDI, WSDL

BindSOAP, XML

ServiceUser

Chain

Service Provider

Bind

Publish, Find, Bind

Voyager Data Services Architecture

Scatter Chart

Text, Table

Data View & ProcessLayered Map

Cursor

Homogeneous data used by Viewing and Processing

services in ‘agile’ applications

Heterogeneous data: different types, coding and

access protocols

Connects data providers with data users

Transforms hetero to virtual homogeneous data

Time Chart

Providers Users

XML WebServices

Satellite

Vector

GIS Data

XDim Data

OLAP Cube

SQL Table

HTTP, FTP

Web

Text

OpenGISServices

Images

Access Services

Data & Tool Catalog

Uniform Access/Retrieval

Select, Overlay, Explore; Multidimensional data

Maintain Distributed Data; Heterogeneous coding , access

Connect providers to users; Homogenize data access

Voyager Spatio-Temporal Data Explorer Built and Used by a Virtual Community

Layered Map

Time Chart

Providers Users

Vector

GIS Data

XDim Data

SQL Table

Web

Images

Voyager Web Services

Publish, Find, Bind

Data & Tool Catalog

Uniform Access/Retrieval Scatter Chart

Navigation Service

DVoy: Components and Data Flow

Legacy Data

Publish

(DataSet)

DataSet Records

• Provider Descript.

• Service Descript.

• Measure Access

Find (Measure)

DataSet Recs

Selected Measure Record

• Dvoy

Data Wrapping

WebServiceBind (Measure, FocusCube)

Data Delivery

WebServiceFocusCube, GlobCursor,

Measure, Granule

Time Chart

Layered Map

DataToView

DataForCursorAndView

Catalog Service Data Services

Data Delivery

WebService

Presentation Services

Data Focus Range

Rendering

Cursor

Viewer Layers

Dim1: LonDim2: Lat

Data provided by each dimension of a View:

Dim1.Type, Dim1.Min, Dim1.Max

Dim2.Type, Dim2.Min, Dim2.Max

….

Current Dim.Types:

Latitude, Longitude, DateTime, Elevation

Federated Data Services Architecture

XML WebServices

Satellite

Vector

GIS Data

XDim Data

OLAPCube

SQLTable

HTTPServices

Text Data

WebPage

TextData

Scatter Chart

Text, Table

Data View & Process TierLayered Map

Cursor

Data Warehouse Tier

Data View

Manager

Connection

Manager

Data Access

Manager

Cursor-Query

Manager

OpenGISServices

Data are rendered by linked Data Views (map, time, text)

Distributed data of multiple types (spatial, temporal text)

The Broker handles the views, connections, data access, cursor

Time Chart

Major Service Categories (OGC)

Service Category Description

Human Interaction Managing user interfaces, graphics, presentation.

 

Info. Management Managing and storage of metadata, schemas, datasets.

 

Workflow Services that support specific tasks or work-related activities.

 

Processing Data processing, computations; no data storage or transfer  

Communication Services that encode and transfer data across networks.

 

Sys. Management Managing system components, applications, networks (access).

 

0000 OGC web service [ROOT]1000 Human interaction1100 Portrayal1110 Geospatial viewer1111 Animation1112 Mosaicing1113 Perspective1114 Imagery1120 Geospatial symbol editor1130 Feature generalization editor1200 Service interaction editor1300 Registry browser2000 Information Management2100 Feature access2200 Coverage access2210 Real-time sensor2300 Map access2400 Gazetteer2500 Registry2600 Sensor access2700 Order handling3000 Workflow3100 Chain definition3200 Enactment3300 Subscription4000 Processing4100 Spatial4110 Coordinate conversion4120 Coordinate transformation4130 Representation conversion4140 Orthorectification4150 Subsetting4160 Sampling4170 Feature manipulation4180 Feature generalization4190 Route determination41A0 Positioning4200 Thematic4210 Geoparameter calculation4220 Thematic classification4221 Unsupervised4222 Supervised4230 Change detection4240 Radiometric correction4250 Geospatial analysis4260 Image processing4261 Reduced resolution generation4262 Image manipulation4263 Image synthesis4270 Geoparsing4280 Geocoding4300 Temporal4310 Reference system transformation4320 Subsetting4330 Sampling4340 Proximity analysis4400 Metadata4410 Statistical analysis4420 Annotation5000 Communication5100 Encoding5200 Format conversion5300 Messaging6000 System Management

OGC code Service class

Semi-Static Views (slices) through 4D Data Space

• Possible Cross-sections through the 4 D Data space

- data point..Temperature (xi, yi, zi, ti)

- image Temperature (xrange , yrange, zrange, trange)

XY MAP: Z,T fixed

Vertical Profile:XYT fixed

Time Chart: X,Y,Z fixed

Vertical Cross sect: YT fixed Vertical Cross sect: XT fixed

Vertical Profile Trend:

X,Y fixed

Data Catalog

• All the data in the system are to be distributed on the Web and maintained by their custodians

• The purpose of the catalog is to help finding and and accessing the data

• Catalog would be limited to data that can be accessed/merged in DVOY

DVoy Queries as Web Services

Purpose: Locating relevant data measures fir specific location and time

Abstract Query: Find available measures in MyDataCube

Web Service: input: MyDataCube; output: List of measures, MeasureDataCube

Design: Measure table with bounding MeasureDataCube cubes

Implementation: SQL measures table with MeasureDataCubeSELECT Measures, MeasureDataCube WHERE MeasureDataCube in MyDataCube

Distinct Locations, Times, Heights

D-DADS Architecture (OOOOLD)

ARC/INFO

VirtualDataCube

ArcSDETranslator

OLAPService

Provider

DataCube

LegacyDatabase

CustomOLAP

Translator

DataCube

SQLDatabase

OLAP ServiceProvider

GISTable

OLAP

StandardizedDescription &

Format

Database(SQL,

Oracle,etc.)

ArcSDETranslator

GISMap

DataProviders

Data Access andManipulation Tools

UserInteraction

DataCube

ArcIMS

Dvoy Proto

DVoy Project

• Goal: Web based data finding/access/delivery system

• Support: – EPA WebVis (Stefan): Aug 2001- Apr 2003

– NSF Collaboration: Aug 2001- Aug 2003

• CAPITA $$ is secured until Apr 2003, after that only very limited

• Need to sell project NOW to continue after Apr 2003

• Partners: Bret, Stefan, Poirot, Sci peers

DVoy: Data Access, Display, Browser, Content

• Data Access: Indexed data access framework. (Kari, ‘done’)

• Data Display: Spatial, temporal other data rendering (Kari & Jer, not done)

• Browser: A specific ‘application’ - linked multidim views (Kari & Jer, not done)

• Content: Encoding and registering data sets (Bob & Sean, in progress)

Project Schedule: May-July 2002

Goal for May-July 2002: Data Viewer, Dvoy BrowserMay 16: Data access framework ‘ready’

Viewer, some data

May 24: Viewer Proto Demo/selling to partners: Bret, Stefan, Poirot, Add content CAPITA

Jun 7: Spatial/Temporal Viewer ReadyDemo/selling to ‘outsiders’: NASA, NSF Report, NOAA, EPA, Sci. Peers

Add content CAPITA, partners

Jul 17: Functional browserDevelop other ‘applications’ – data finding/renderings service web pages

Add content CAPITA, partners

SQL Server

Data Access though a Web Service Portal

Service Broker

PublishUDDI, WSDL

Service Consumer

FindUDDI, WSDL

Access

SOAP, XML

Ordinary web content can be delivered as a Web Service through a Proxy Server.

• The Portal Server supplies a web server-to-web-service ‘adapter’

• The Portal Server publishes the web service to the Broker

• The User accesses the Portal to get the distributed Web Server data

DataSets

TimeViewLocView

Star Schema

DimTablesFactTable

ServiceUser Chain

Service Portal