CUAHSI Hydrologic Information Systems David R. Maidment and Ernest To Center for Research in Water Resources, University of Texas at Austin Hydrosystems

CUAHSI Hydrologic Information Systems

David R. Maidment and Ernest ToCenter for Research in Water Resources,

University of Texas at Austin

Hydrosystems LaboratoryUniversity of Illinois at Urbana-Champaign,

18 August 2006


• Introduction

• HIS Server

• CUAHSI web services • Demo by of Corpus Christi Bay by Ernest To

• Data models and some longer range thinking


• Introduction

• HIS Server

• CUAHSI web services • Demo of Corpus Christi Bay by Ernest To


Ocean Sciences

CUAHSI-Hydrologic Information Systems

• CUAHSI – Consortium of Universities for the Advancement of Hydrologic Science, Inc

• Formed in 2001 as a legal entity

• Program office in Washington (5 staff)

• Supported by the National Science Foundation

Earth Sciences

AtmosphericSciences

UCAR

CUAHSI

Unidata

HISNational Science Foundation

Geosciences Directorate

CUAHSI Member Institutions

115 Universities as of August 2006

Common Vision: WATERS Network

Sensors and Measurement Facility

CA

dx)x(r)x(Q

Synthesis

Informatics

Server

Workstation

domainDNS

Disk array

Observatories/ Environmental Field Facilities

A combined CLEANER-CUAHSI effort


• Introduction

• HIS Server



Digital WatershedHow can hydrologists integrate observed and

modeled data from various sources into a single description of the environment?

Digital WatershedHydrologic Observation

Data

GeospatialData

Weather and ClimateData

Remote SensingData

(NetCDF)

(GIS)(Relational database)

(EOS-HDF)

Digital Watershed

A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather

and climate data into a connected database for a hydrologic region

HIS ServersHydrologic

ObservationsServer

GIS Data Server

Weather and ClimateServer

Remote SensingServer

Digital Watershed

HIS Servers provide hydrologic observations, weather and climate, GIS and remote sensing data. For HIS version 1.0, the focus is a hydrologic observations server for data from gages and monitoring sites at point locations.

CUAHSI Hydrologic Information System Levels

National HIS – San Diego Supercomputer Center

Workgroup HIS – research center or academic department

Personal HIS – an individual hydrologic scientist

HIS Server

HIS Analyst

Map interface, observations catalogs and web services for national data sources

Map interface, observations catalogs and web services for regional data sources; observations databases and web services for individual investigator data

Application templates and HydroObjects for direct ingestion of data into analysis environments: Excel, ArcGIS, Matlab, programming languages; MyDB for storage of analysis data

HIS Server

• Supports data discovery, delivery and publication– Data discovery – how do I

find the data I want?• Map interface and

observations catalogs

– Data delivery – how do I acquire the data I want?

• Use web services or retrieve from local database

– Data Publication – how do I publish my observation data?

• Use Observations Data Model

Observations CatalogSpecifies what variables are measured at each site, over what time interval,

and how many observations of each variable are available

HIS Server Architecture

• Map front end – ArcGIS Server 9.2 (being programmed by ESRI Water Resources for CUAHSI)

• Relational database – SQL/Server 2005 or Express

• Web services library – VB.Net programs accessed as a Web Service Description Language (WSDL)

National and Workgroup HIS

National HIS has a polygon in it marking the region of coverage of a workgroup HISserver

Workgroup HIS has local observations catalogs for coverage of national data sources in its region. These local catalogs are partitioned from the national observations catalogs.

For HIS 1.0 the National and Workgroup HIS servers will not be dynamically connected.

National HIS Workgroup HIS

Point Observations Information Model

Data Source

Network

Sites

Variables

Values

{Value, Time, Qualifier}

USGS

Streamflow gages

Neuse River near Clayton, NC

Discharge, stage (Daily or instantaneous)

206 cfs, 13 August 2006

• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• A qualifier is a symbol that provides additional information about the value

Data Discovery and Delivery

Data Source

Network

Sites

Variables

Values

Observations metadata

Observations data

HIS ServerObservations

Catalog

Web services

Data Discovery

Data Delivery

• HIS facilitates data discovery by building and maintaining observations catalogs• Data delivery occurs through web services from remote data archives or local observations databases. Water resource agencies support data delivery services.


• Introduction

• HIS Server



Web Services with HIS Server

• Publication services for local observations databases

• Ingestion Services for remote data archives

Send data out from theserver

Enable users to access data in remote archives

Rainfall & SnowWater quantity

and quality

Remote sensing

Water Data

Modeling Meteorology

Soil water

Water Data Web Sites

NWISWeb site output# agency_cd Agency Code# site_no USGS station number# dv_dt date of daily mean streamflow# dv_va daily mean streamflow value, in cubic-feet per-second# dv_cd daily mean streamflow value qualification code## Sites in this file include:# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC#agency_cd site_no dv_dt dv_va dv_cdUSGS 02087500 2003-09-01 1190USGS 02087500 2003-09-02 649USGS 02087500 2003-09-03 525USGS 02087500 2003-09-04 486USGS 02087500 2003-09-05 733USGS 02087500 2003-09-06 585USGS 02087500 2003-09-07 485USGS 02087500 2003-09-08 463USGS 02087500 2003-09-09 673USGS 02087500 2003-09-10 517USGS 02087500 2003-09-11 454

Time series of streamflow at a gaging station

USGS has committedto supporting CUAHSI’sGetValues function

Observation Stations

Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System

USGS National Water Information System NOAA Climate Reference Network

Map for the US

Water Quality Measurement Sites in EPA Storet

Substantial variation in data availability from states

Data from Bora Beran, Drexel University

Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

Geographic Integration of Storet and TCEQ Data in HIS

CUAHSI Hydrologic Data Access System

A common data window for accessing, viewing and downloading hydrologic information

USGSUSGS

NASANASANCDCNCDCEPAEPA NWSNWS

Observatory DataObservatory Data

http://river.sdsc.edu/HDAS

Arc Hydro Server will be a customization of ArcGIS Server 9.2 for serving

water observational

data

NWISNWIS

ArcGISArcGIS

ExcelExcel

NCARNCAR

UnidataUnidata

NASANASAStoretStoret

NCDCNCDC

AmerifluxAmeriflux

MatlabMatlab

AccessAccess JavaJava

FortranFortran

Visual BasicVisual Basic

C/C++C/C++

Some operational services

CUAHSI Web ServicesCUAHSI Web Services

Data SourcesData Sources

ApplicationsApplications

Extract

Transform

Load

http://www.cuahsi.org/his/

http://www.cuahsi.org/his



CUAHSI Web Services

Web ServicesLibrary

Web Application: Data Portal

Your application• Excel, ArcGIS, Matlab• Fortran, C/C++, Visual Basic• Hydrologic model• …………….

Your operating system• Windows, Unix, Linux, Mac

Internet Simple Object Access Protocol


• Introduction

• HIS Server



Example: Corpus Christi Bay Environmental Info System

• Workgroup HIS implementation

• Uses ODM to store hydrology and environmental data from state agencies and academic investigators.

• Contains web-services to regional data repositories (e.g. TCOON).

Water quality data sites in Corpus Christi Bay(maps by Tyler Jantzen)

Demo: TXHIS ODM webservice

ODM(Observations Data Model)

= Observations Catalog + Values Table +Metadata Tables

Excel CUAHSI Web service

How Excel connects to ODM

• Obtains inputs for CUAHSI web methods from relevant cells.

• Available Web methods are GetSiteInfo, GetVariableInfo GetValues methods.

converts standardized request to SQLquery.

imports VB object into Excel and graphs it

converts response to a standardized XML.

Observations Data

Model

SQL query

Response

HydroObjects

converts XML to VB object

parses user inputs into a standardized CUAHSI web method request.


• Introduction

• HIS Server



Series and FieldsFeatures

Point, line, area, volumeDiscrete space representation

Series – ordered sequence of numbersTime series – indexed by time

Frequency series – indexed by frequency

Surfaces Fields – multidimensional arrays

Scalar fields – single value at each locationVector fields – magnitude and direction Random fields – probability distribution

Continuous space representation

mm / 3 hours

Precipitation Evaporation

North American Regional Reanalysis of Climate

Variation during the day, July 2003

NetCDF format

Space, L

Time, T

Variable, V

D

Data Cube – What, Where, When

“What”

“Where”

“When”

A data value

Continuous Space-Time Data Model -- NetCDF

Space, L

Time, T

Variables, V

D

Coordinate dimensions

{X}

Variable dimensions{Y}

Space, FeatureID

Time, TSDateTime

Variables, TSTypeID

TSValue

Discrete Space-Time Data Model

Geostatistics

Time Series Analysis

Multivariate analysis

Hydrologic Statistics

How do we understand space-time correlation fields of many variables?

Water OneFlow• Like Geospatial OneStop, we need a “Water

OneFlow” – a common window for water data and models

• Advancement of water science is critically dependent on integration of water information

Federal

AcademicLocal

State

Conclusions

• This is a complex and important problem that will not be solved soon

• Web services architecture will work and is valuable

• Major water agencies are buying into our web services design, in particular the USGS

• We need to think more deeply and abstractly about the way data is used to represent water and the water environment

Documents

CUAHSI Hydrologic Information Systems David R. Maidment and Ernest To Center for Research in Water Resources, University of Texas at Austin Hydrosystems