Remote Instrumentation and Sensor Networks on the Great Barrier Reef CIMA based sensor networks… Ian Atkinson [email protected]

Remote Instrumentation and Sensor Networks on the Great Barrier Reef

CIMA based sensor networks…

Ian Atkinson

[email protected]™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

New Technologies…

• Sensor Networks: will enable us to monitor our environment more closely than ever before.

• Grid computing: will enable the distribution of data collected to researchers around the globe in real time, as well as storing the massive amounts of historical data in online repositories.

• The challenge: Streamlining the transfer of data from sensors to researchers and managers.

Major Projects

1. ReefGrid

2. Remote Instrument Monitoring

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

http://roadnet.ucsd.eduhttp://roadnet.ucsd.edu

RoadNet: SIO, SDSC, UCSDRoadNet: SIO, SDSC, UCSD



Frank Vernon, SIO







http://marine.rutgers.edu/cool

Rutgers Cool Room: Costal Observatories

QuickTime™ and aTIFF (Uncompressed) decompressor


ReefGrid• ReefGrid is a pilot project to develop the technologies to

allow real time sensor network monitoring of the Great Barrier Reef (GBR).

• NCRIS-Integrated Marine Observation System GBR monitoring program

• Encompasses the entire e-Research data flow

SensorsData

Aggregation

DataTranspo

rt DataQA/QC

Data Ingestion

andStorage

Data Dissemination

• Foundation for QOOS: Queensland Oceans Observing System

ReefGrid @ Davies

• Davies Reef is the location for the initial deployment of a sensor network on the GBR

• Davies Reef is ~100km East of Townsville. Data has to be sent 80km to AIMS.

GBR Network?

Local Communications Networks

ReefGrid @ Davies

• AIMS’ Weather Station at Davies Reeg is being used as a base for the sensor network.

• A hybrid power system with wind and solar energy and battery back up powers the system.

ReefGrid @ DaviesThe Sensors

• A range of sensor technologies is being used. The most important two are 1-Wire Sensors and Ambient Systems’ Wireless uNodes

• These technologies provide low cost sensors that can be deployed in large numbers covering a wide area.

Data Aggregati

on

DataTranspo

rtDataQA/QC

Data Ingestion

andStorage

Data Disseminatio

n

Sensors

Costal Radars (CODAR)• HF ocean surface radar to monitor surface currents,

wind directions and wave heights• Capricorn and Bunker Group of reefs and islands in

the southern part of the GBR• 100 x100 km, 3km pixel• April 2006 deployment







Mal Heron, JCU

ReefGrid @ DaviesData Aggregation

Data Aggregati

on

DataTranspor

tDataQA/QC

Data Ingestion

andStorage

Data Disseminatio

n

Sensors

• Software tools that allow us to organise many different types of sensing devices for simple reference, access & probing.

• System is scalable so that many sensors can be added.

Individual Sensors

Data Aggregation

Data Sinks

Sensor Gateway: Use Case

• To add / remove / re-configure a sensor– Make the change at the hardware level– Update the configuration database

• Sink• Source• Root• Sensor

– Push the new configuration file to the appropriate sink

• See http://www.reefgrid.org

ReefGrid @ DaviesData Transport

SensorsData

Aggregation

DataTranspo

rtDataQA/QC

Data Ingestion

andStorage

Data Disseminatio

n

Two aspects of Data Transport:

1. the physical connection to sensor networks many 10’s of km off the coast

2. the protocols for interacting with the sensor network (retrieving data and interrogating the system).


SensorsData

Aggregation

DataTranspor

tDataQA/QC

Data Ingestion

andStorage

Data Disseminatio

n

• microwave link to provide data communications between the sensor network at Davies Reef and the mainland.

• "evaporation duct" just above the ocean, allows microwaves to propagate around the curved surface of the Earth.

• removes the need for a line-of-site connection between the two ends of the microwave link.

JAINIS


SensorsData

Aggregation

DataTranspo

rtDataQA/QC

Data Ingestion

andStorage

Data Disseminatio

n

• The protocols for communications with the Grid are tied to an overall dataflow that includes QA/QC and ingestion of data into storage facilities.

• JAINIS (JCU And INdiana University Instrument Services). • CIMA provides a consistent and reusable framework for

including shared instrument resources distributed Grids.• Several enhancements have been made to the CIMA

implementation

JAINIS

ReefGrid @ DaviesJAINIS

SensorsData

Aggregation

DataTranspo

rtDataQA/QC

Data Ingestion

andStorage

Data Disseminatio

n

• JAINIS has a structure that has an Instrument Representative that allows the instrument (or data sink in ReefGrid) to be represented as a web service.

• This web service can be accessed by a Data Manager to control the flow of data through QA/QC and preprocessing to a data repository.

• The Web Services can also be used to stream live data and to provide interactive control over the sensor network.

ReefGrid @ DaviesJAINIS

CIMAInstrument

Representative

Data Manager

CIM

A S

OA

PIn

terf

ace

GridSphere Portal

SRB Data Repository (data and metadata)

Kepler mediatedExperimental

logic

SRB MCAT

PGL

Applications/Processing

Data Sinks/Sensors

CIMA core code from IUJCU/DART reengineered CIMA codeSDSC Components

Workflow & computational Pipelines, Automatic metatdata generation

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Live Data Feeds,Instrument Control

ReefGrid @ DaviesData Manager

Data Aggregati

on

DataTranspo

rtDataQA/QC

Data Ingestion

and Storage

Data Disseminatio

n

Sensors

• JAINIS uses Kepler as its Data Manager.

• Kepler provides a graphical user interface allowing the user to define workflows.

• Metadata can be automatically generated in Kepler based on input data from the Instrument Representative

ReefGrid @ DaviesData Manager

Data Aggregati

on

DataTranspo

rtDataQA/QC

Data Ingestion

and Storage

Data Disseminatio

n

Sensors

• QA/QC routines are performed in Kepler.

• Preprocessing can also be performed in Kepler (e.g. converting units of a measurement)

• Kepler can controls the location where data is stored (creating file names and directories, metadata)

ReefGrid @ DaviesData Ingestion and Storage

Data Aggregati

on

DataTranspor

tDataQA/QC

Data Ingestion

andStorage

Data Disseminatio

n

Sensors

• In the JAINIS implementation, the data is stored using SRB (Storage Resource Broker).

• Heterogenous storage devices act as single logical storage • Metadata Catalogue Service (MCAT)

– Stores metadata associated with objects

– Metadata is vital for finding data in large data stores, as well as describing how the data was collected, what QA/QC and what pre-processing has been done

Personal Grid Library: PGL• Implementation of a web-based digital library

(Gridsphere portlets) • Provides an interface to templated metadata

manipulation – Define object classes (Images, reports, datasets...)– Define appropriate metadata tags – Define metadata display template

• Uses SRB to manage the digital objects and the metadata

• Exists by placing a metadata template file in a directory - used by interface to render library

http://plone.jcu.edu.au/hpc/staff/projects/hpc-software/personal-grid-library/documentation/pgl-project-information

PGL: Interface and metadata/annotations

http://ngportal.hpc.jcu.edu/gridsphere/gridsphere



ReefGrid @ DaviesData Ingestion and Storage

Data Aggregati

on

DataTranspor

tDataQA/QC

Data Ingestion

andStorage

Data Disseminatio

n

Sensors

• Integrating SRB with content management systems (plone).

JCU e-Research and Sensor Network Collaborations

• ARC Research Network on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP).

• ARC Research Network for Materials and Molecular Sciences (MMSN).

• Key partner of Queensland Cyber-Infrastructure Foundation (QCIF).

• Australian Partnership for Advanced Computing (APAC).• Ambient Systems and University of Twenté (Netherlands)• Indiana University (USA).• San Diego Supercomputing Centre. • EM Solutions (Brisbane).


Sensor Data Flows SummarySensor Data Flows Summary

––Wireless data links

Sensor Networks• Temp, salinity, • Light, turbidity, etc.

Data ‘Sinks’ Internet

Microwave links - via humidity ducting

SRB MCAT

Data Repository

Disk/Tape Storage(multiple locations)

Data Sinks enable•Simple sensor configuration Interface•Scalability / ease of maintenance•Model for ‘plug and play’ sensor configuration•Security•Automatically update backend data structures•Developed by DART for Davies Reef SN Platform•Network & sensor monitoring

AODC-JF& other portals

Data Manager

CIM

A

SO

AP

Inte

rfac

e

Kepler mediatedExperimental

logic

Workflow & computational pipelines, Automatic metatdata generation

Local and remoteQuickTime™ and a

TIFF (LZW) decompressorare needed to see this picture.


are needed to see this picture. QuickTime™ and aTIFF (Uncompressed) decompressor






Grid computing



Computeresources

Custom portals / Research codes

Sensor Devices• Inexpensive, commodity

inspired• Ad hoc network• Above and below water

wireless network & wired• Many partners involved




Great Barrier Reef -Ocean Observing System

Registry and directory server

Data Acquisition

Code(s)

Register

NotifyWeb Services interface

Request/Response

Events/Streaming

Locate

Analysis Codes

Storage Storage

InstrumentSemantics,Metadata





















QuickTime™ and aTIFF (LZW) decompressorare needed to see this picture.Microwave linkMicrowave link

Quic

kTim

e™

and

aTIF

F (

Un

com

pre

ssed

) de

com

pre

sso

rare

ne

ed

ed

to s

ee

this

pic

ture

.

DIMSIM: Distributed Integrated Multi-Sensor and Instrument Middleware

Documents

Remote Instrumentation and Sensor Networks on the Great Barrier Reef CIMA based sensor networks… Ian Atkinson [email protected]