FCRPS BiOp RME Data Management Pilot Project: Protocol Manager

FCRPS BiOp RME Data Management Pilot Project: Protocol ManagerMike Beaty, USBR PN-Region GIS & Remote Sensing Manager

Northwest Environmental Data Network (NED) Executive Meeting

Portland, OR

December 7, 2005

I. Introduction

A. What is Protocol Manager?

1. Relational Database Application

2. Dynamic Data Dictionary

3. Data Management Tool

I. Introduction

B. What does Protocol Manager do?

1. Supports Biological Monitoring Programs and Projects

2. Fully Documents Data Elements and their Properties

3. Stores Protocol and Method Reference Documents

I. Introduction

C. Why use Protocol Manager?

1. Foster Coordination and Collaboration

2. Improve Data Quality & Consistency

3. Enforce Data Standards

4. Provide Data Transparency

5. Facilitate Data Exchange and Usage

6. Compare, Improve, and Develop Protocols

I. Introduction

D. What can NED do for Protocol Manager?

1. Support Pacific Northwest Aquatics Monitoring Partnership (PNAMP) Workgroup Functions

a. Data Analysis

b. Content Management

c. Protocol Standards

I. Introduction

D. What can NED do for Protocol Manager?

2.Partner with USGS National Biological Information Infrastructure (NBII) Protocol Library and Monitoring Locator National Project

a. Form Regional Consortium Node

b. Leverage NBII Information Technology (IT) Resources

c. Utilize National and International Scientific Expertise

II. Pilot Overview

A. Brief History

1. Research, Monitoring & Evaluation (RME) Data Management Work Plan, May 2003

2. John Day Pilot, October 2003

a. Data Analyst

b. Data Dictionary

II. Pilot Overview

A. Brief History

3. Coordinate with Wenatchee Regional Technical Team (RTT) Pilot

a. Upper Columbia Monitoring Strategy, Hillman, February 2004

4. Publish John Day (& U.Columbia) Data Dictionary, June 2004

II. Pilot OverviewA. Brief History

5. Host Database Design Process, June 2004

a. ArcHydro Geospatial Data Model concepts and relationships applied to monitoring data

b. NPS FEAT Protocol Manager DB application replaces static data dictionary

II. Pilot Overview

A. Brief History

6. NOAAF Status Trend Monitoring (STM) Database initiated, September 2004

a. Wenatchee 2004 physical and biological data loading completed, August 2005

b. EMAP (ODEQ & Wenatchee) & PIBO 2004 loading in progress at NWFSC


7. National Biological Monitoring Protocol Library and Monitoring Locator Project initiated, November 2005.

a. USGS NBII, National Park Service (NPS), & International Association of Fish and Wildlife Agencies (IAFWA)

b. USGS NBII: $500K & Technical Resources


8. Current Status

a. Version 0.7 (.Net)

i. January 2006 release

ii. NPS NIFC cost share through contractor

iii. Documentation

b. Desktop Installations

i. Multiple versions require coordination

c. U.S. Government owns software code (public)


9. Next Steps

a. Grow Protocol Manager Project Partnership

i. NED/PNAMP

ii. USGS NBII

b. Develop Version 2.0

i. Fully Functional Suite of Features

c. Plan Web Enabled Application

i. Centralized Relational Database Management System Host & Services Oriented Architecture

II. Pilot Overview

B. Resources (USBR PN-Region GIS)

1. October 2003 – December 2005

2. 2+ Staff Years utilized

a. GIS/RS support: 33 %

b. Data analysis: 67%

i. Protocol Manager application dev: 5%

II. Pilot Overview

B. USBR Resources to Date

3. Contracting: $133K

a. Data Analysis & Dictionary: $90K

b. Protocol Manager Development: $43K

4. Total Cost: $325K

a. Data Analysis: $250K

b. Protocol Manager Development: $75K

II. Pilot Overview

C. Key Findings

1. Protocols are integral to monitoring data

a. Protocols = Metadata

b. Metadata = Data

c. Protocols = Data

II. Pilot Overview

C. Key Findings

2. Protocols are defined by their data management functions

a. Prescribe data elements

b. Describe methods and procedures

i. Collect field data

ii. Process summary values

iii. Validate values

II. Pilot Overview

C. Key Findings

3. “Data management systems must be designed from an end-to-end perspective (from data collection through data entry to database management, report generation and data sharing).” – Data Management for Field Collection, June 2004, Welches, OR Data Management Workshop

III. Monitoring Data Processes

Pose Scientific Questions

Design Experiments and monitoring protocols

Data needs

Display results

Collect and Analyze data

Data Dictionary(monitoring indicators and

protocols)

Empirical Method Road Map

GDB

GISAFG

Needs Analysis

A. Analytical Framework Empirical Method

III. Monitoring Data ProcessesB. Field Data Processing Flow

Prep for field

Collect data

Record data

Capture data

Updaterepository

AnalyzeindicatorsTransfer data

SubjectMatterExpert

FieldWorker

DataAnalyst

Coordinator

DataSteward

Researcher

III. Monitoring Data ProcessesC. Interaction between Tabular Data and ArcHydro Geodatabase

RawData

GeneratedData

RepositoryDatabase

Field Data Tables

Data CollectionMethods

Field formsField formsOutput

files

Data ValidationMethods

EvaluateProtocols/Methods

FieldForms

Data SummaryMethods

ProtocolMetadataRecords

ProtocolMetadataRecords

MetadataRecords

(Protocols)

DocumentProtocols/Methods

DataAnalysis

ReviseProtocols/Methods

Protocol Manager

Protocol

Status Trend Monitoring Database

Monitoring Program

IV. Protocol Manager Contents

A. Monitoring Programs & Protocols

B. Terms

C. Relationships of Terms

D. Protocol Manager

E. Comparing Protocols & Methods

Monitoring Programs & ProtocolsMonitoring Program Protocols Specified

PNAMP (w/ OWEB for Bank Stability)

PIBO

John Day AFG

plus other protocols in development or undocumented

Upper ColumbiaMonitoring Strategy

EMAP EMAP (2001)

PIBO (2002)

EMAP (2001)

PIBO (2004)PIBO (2003)

EMAP (2001) PIBO (‘02-’04)

EMAP (2001) OPSW (1999) WFC (1998) WDFW (2000)

WFPB (1983) Zaroban (2000) Schuett-Hames (1999)

BURPTAC (1999) Hawkins et al. (1993)

Platts et al. (1983)

Reeves et al. (2001)

Overton et al. (2000)

Parmenter et al. (2003)

Dolloff et al. (1996) Reynolds et al. (1996)

Van Deventer and Platts (1989)

Anderson and Neumann (1996)

Strange (1996)

Borgerson et al. (1992)

Mosey and Murphy (1996)

and more…

Terms

Subject Area Classification Habitat / Physical Biological

Indicator Group Ecoregion

Physiography

Geology

Geomorphic Features

Valley Characteristics

Channel Characteristics

Water Quality

Habitat Access

Habitat Quality

Channel Condition

Riparian Condition

Flows & Hydrology

Watershed Condition

Adults

Redds

Parr / Juveniles

Smolts

Macro Invertebrates

Terms

Biological Adults

Redds

Parr / Juveniles

Smolts

Macro invertebrates

Escapement/numberAge structureSizeSex ratioOrigin (hatchery or wild)GeneticsFecundity

TransportComposition

Number smoltsSize smoltsGenetics smolts

Abundance/distributionSize parr

Subject Area Indicator

AttributeNumberDistribution

Indicator Group

Protocol

Method

Protocol

Collection Method

Attribute Name

Collection Method

Attribute Name

Attribute Name

Attribute Name

Indicator

Summary Value

Indicator

Indicator

Summary Method

IndicatorGroup

Relationships of Terms

Relationship of Terms

EMAP, 2001

Dominant Substrate (EMAP, 2001)

Distance Left Bank

Large Wood Debris

Bankfull Depth

Size Class Code

DIST_LB

BF_DEPTH

SIZE_CLS

Embeddedness EMBED

Substrate Flag FLAG_SUB

Transect TRANSECT

Transect Position TRANSDIR

Habitat Event ID HAB_EVENT

CollectionAttributes

Dominate Substrate

Comments COMMENTS

Links to Site info:Site ID, Date, Stream ID, Visit No., Team ID, Location, etc.

Fields from data collections

Bank Measurements

ProcedureAttribute

Attribute

Attribute

Attribute

Procedure

Procedure

Procedure

Short Name: EMAP, 2003

Large Woody Debris

ProcedureAttribute

Attribute

Attribute

Attribute

Procedure

Procedure

Procedure

Name: EMAP West Surface Water Wadeable Streams

Torrent Evidence

ProcedureAttribute

Attribute

Attribute

Attribute

Procedure

Procedure

Procedure

Stream Flow - NBO

ProcedureAttribute

Attribute

Attribute

Attribute

Procedure

Procedure

Procedure

Contact: Phil KaufmanDoc Ref: Peck, et al., 2003, EMAP West Surface Water Wadeable Streams, EPA, Corvallis,OR

- Library Protocols

- EMAP Surface Waters – Wadeable Streams, 2003

+

+

+

Bank Measurements

Large Woody Debris

Torrent Evidence

Protocol Manager

ProtocolProtocol

MethodsMethods

Merritt, 2004

Bankfull Diameter 0.1-0.3m Length 1.5-5m WETSDSLBankfull Diameter 0.3-0.6cm Length 1.5-5m WETMDSLBankfull Diameter 0.6-0.8m Length 1.5-5m WETLDSLBankfull Diameter >0.8m Length 1.5-5m WETXDSL

Bankfull Diameter 10-15cm Length 1-3m 10_SHTBankfull Diameter 10-15cm Length >3-6m 10_MEDBankfull Diameter 10-15cm Length >6m 10_LONBankfull Diameter >15-30cm Length 1-3m 15_SHTBankfull Diameter >15-30cm Length >3-6m 15_MEDBankfull Diameter >15-30cm Length >6m 15_LONBankfull Diameter >30cm Length 1-3m 30_SHTBankfull Diameter >30cm Length >3-6m 30_MEDBankfull Diameter >30cm Length >6m 30_LON

Large Woody Debris

Attribute Name Field Name


Bankfull Diameter >10cm Length >1m LOD


Bankfull Diameter 0.1-0.3m Length 5-15m WETSDMLBankfull Diameter 0.3-0.6cm Length 5-15m WETMDMLBankfull Diameter 0.6-0.8m Length 5-15m WETLDMLBankfull Diameter >0.8m Length 5-15m WETXDMLBankfull Diameter 0.1-0.3m Length >15m WETSDLLBankfull Diameter 0.3-0.6cm Length >15m WETMDLLBankfull Diameter 0.6-0.8m Length >15m WETLDLLBankfull Diameter >0.8m Length >15m WETXDLLAbove Bankfull Diameter 0.1-0.3m Length 1.5-5m DRYSDSLAbove Bankfull Diameter 0.3-0.6cm Length 1.5-5m DRYMDSLAbove Bankfull Diameter 0.6-0.8m Length 1.5-5m DRYLDSLAbove Bankfull Diameter >0.8m Length 1.5-5m DRYXDSLAbove Bankfull Diameter 0.1-0.3m Length 5-15m DRYSDMLAbove Bankfull Diameter 0.3-0.6cm Length 5-15m DRYMDMLAbove Bankfull Diameter 0.6-0.8m Length 5-15m DRYLDMLAbove Bankfull Diameter >0.8m Length 5-15m DRYXDMLAbove Bankfull Diameter 0.1-0.3m Length >15m DRYSDLLAbove Bankfull Diameter 0.3-0.6cm Length >15m DRYMDLLAbove Bankfull Diameter 0.6-0.8m Length >15m DRYLDLLAbove Bankfull Diameter >0.8m Length >15m DRYXDLL

Comparing Protocols & Methods

EMAP, 2001

BURPTAC, 1999

Large Woody Debris (Merritt, 2004)

Large Woody Debris (EMAP, 2001)

Large Woody Debris (BURPTAC, 1999)

IV. Protocol Manager Screens