Sharing Hydrologic Data with the CUAHSI

Hydrologic Information System

SupportEAR 0622374

CUAHSI

HISSharing hydrologic data

http://his.cuahsi.org/

David Tarboton, David Maidment, Ilya Zaslavsky, Dan Ames, Jon Goodall, Jeffery Horsburgh, Tim Whiteaker, Michael Piasecki, David Valentine,

Tom Whitenack, Kim Schreuders

Consortium of Universities for the Advancement of Hydrologic Science, Inc.

An organization representing more than one hundred United States universities, receives support from the National Science Foundation to develop infrastructure and services for the advancement of hydrologic science and education in the U.S. http://www.cuahsi.org/

• 110 US University members

• 6 affiliate members• 12 International

affiliate members (as of March 2009)

GoalA system that enhances access to hydrologic data for education and research to better understand hydrologic processes. • How can we better organize hydrologic data to enhance the

analysis it can support?• How can we better provide access to the data sources, tools

and models that enable the synthesis, visualization and evaluation of the behavior of hydrologic systems?

PrecipitationWater quantity MeteorologyWater quality

CUAHSI HISThe CUAHSI Hydrologic Information System (HIS) is an internet based system to support the sharing of hydrologic data. It is comprised of hydrologic databases and servers connected through web services as well as software for data publication, discovery and access.

Data Discovery and Integration platform

Data Publication platform

Data Synthesis and Research platform

Data Services

Metadata Services Metadata Search

HIS Central

HydroDesktopHydroServer

Service registration

Catalog harvesting

Service and data theme metadata

Data carts

Water Data ServicesSpatial Data Services

Like search portals Google, Yahoo, Bing

Like browsersLike web servers

Like HTML

Base StationComputer(s)

Telemetry Network

Sensors

Query, Visualize, and Edit data using ODM Tools

Excel, text

ODMDatabase

ODM Data

Loader

Streaming Data

Loader GetSitesGetSiteInfoGetVariableInfoGetValues

WaterOneFlowWeb Service

WaterML

Platform independent

AnalysisGIS

MatlabSplus

RIDL

JavaC++VB

Discovery and Access

Hydro Desktop

HydroExcel

Water Metadata Catalog

Harvester

Service Registry Hydrotagger

HIS Central

CUAHSI Water Data Services System

USGS NWIS

EPA STORET NCDC Others

• Set of query functions • Returns data in WaterML

Slide from David Valentine

WaterML and WaterOneFlowWaterML is an XML language for communicating water dataWaterOneFlow is a set of web services based on WaterML

ISO Time valuequalifier

GetVariables

GetValues

What are the basic attributes to be associated with hydrologic data values and how can these best be

organized?

Streamflow

Flux towerdata

Precipitation& Climate

Groundwaterlevels

Water Quality

Soil moisture data

Observations Data Model (ODM) – a relational model at the single observation level

• Stores observation data made at points• Metadata for unambiguous interpretation• Traceable heritage from raw measurements to usable information• Standard format for data sharing• Cross dimension retrieval and analysis

Space, S

Time, T

Variables, V

s

t

Vi

vi (s,t)“Where”

“What”

“When”

A data value

Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), A Relational Model for Environmental and Water Resources Data, Water Resour. Res., 44: W05406, doi:10.1029/2007WR006392.

CUAHSI Observations Data Model http://his.cuahsi.org/odmdatabases.html

Stage and Streamflow Example

Loading data into ODM

• Interactive ODM Data Loader– Loads data from spreadsheets and

comma separated tables in simple format

• Streaming Data Loader (SDL)– Loads data from datalogger files on

a prescribed schedule– Interactive configuration

• SQL Server Integration Services (SSIS)– Microsoft application accompanying

SQL Server useful for programming complex loading or data management functions

ODM Data Loader

SDL

SSIS

Dynamic controlled vocabulary moderation system

Local ODMDatabase

Master ODM Controlled Vocabulary

ODM Website

ODM ControlledVocabulary Moderator

ODM Data Manager

ODMControlled Vocabulary

Web Services

ODM Tools

Local Server

XML

http://his.cuahsi.org/mastercvreg.html From Jeff Horsburgh

• Publishers– Register a data service

• Users– Find a data service

• Supported by– Metadata Catalog

HIS Central

http://hiscentral.cuahsi.org

Ontology – to support concept based data discovery

15

HydroTagger

Each Variable in your data is connected to a corresponding Concept

ODM ODM ODM

WaterOneFlow WaterOneFlow WaterOneFlow

HydroServerDatabase

ODM Databases and WaterOneFlow Web Services

ArcGIS Server Spatial Data Services

SpatialServices

WaterOneFlowServices

Map Server Time Series AnalystHydroServer Website HydroServer

Capabilities Web Service

HydroServer Database

Configuration Tool

17

HydroDesktopHarvesting and analyzing data from web services

• Add shapefiles to map• Change symbology and

labels• Print and export map• GIS toolbox

GIS

• Search for data• Download data• Display time series• Export data

Hydrology

http://hydrodesktop.codeplex.com

HydroDesktopObservations

Models

Climate

GIS

Remote Sensing

• Platform for data discovery, access, integration and analysis• Uses MapWindow 6 open source GIS software development toolkit• Plug-in architecture (extendibility that enables customization)• Data abstraction layer• http://hydrodesktop.codeplex.com open source software development portal• Windows and Mac compatible (through Mono)

CUAHSI Water Data Services

43 services15,000 variables1.8 million sites9 million series4.3 billion data

NCDC Integrated Station Hourly Data

Hourly weather data up to 36 hours ago

13,628 sites across globe

34 variables

Published by National Climate Data Center and populated with weather observations from national weather services

http://water.sdsc.edu/wateroneflow/NCDC/ISH_1_0.asmx?WSDL

USGS Instantaneous Data

Real time, instantaneous data over the last 60 days

11188 sites, nationally for the US

80 variables

Published by USGS National Water Information System

Reynolds Creek Experimental Watershed

1 data service84 sites65 variables372 series17.8 million data

http://idahowaters.uidaho.edu/RCEW_ODWS/cuahsi_1_0.asmx?WSDL

Published by USDA-ARS as part of an Idaho Waters project

Dry Creek Experimental Watershed (DCEW) (28 km2 semi-arid steep topography, Boise Front)

68 Sites20 Variables5924511 values

Published by Jim McNamara, Boise State University

http://icewater.boisestate.edu/dcew2dataservices/cuahsi_1_0.asmx?WSDL

Databases: Structured data sets to facilitate data integrity and effective sharing and analysis.- Standards- Metadata- Unambiguous interpretation

Analysis: Tools to provide windows into the database to support visualization, queries, analysis, and data driven discovery.

Models: Numerical implementations of hydrologic theory to integrate process understanding, test hypotheses and provide hydrologic forecasts.

Advancement of water science is critically dependent on integration of water information

Databases Analysis

Models

ODM

Web Services

Conclusions• Data Storage in an Observations Data Model (ODM)• Data Access through internet-based Water Data

Services using a consistent data language, called WaterML

• Data Indexing through a National Water Metadata Catalog

• Data Discovery through federated map and thematic keyword search system

The combination of these capabilities creates a common window on water observations data for the United States unlike any that has existed before.