Upload
lester-alexander
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
DOI Climate Change Task Force
CC Impacts Data Collection, Integration, and Management
Workgroup
Recommendations of the Data Management/Integration
Subcommittee
Steve FancyNational Park ServiceInventory and Monitoring Program Leader
Chaco Culture Natl. Historical Park
The climate change crisis is pushing agencies to work together in ways that we’ve never been able to do before.
“A crisis is a terrible thing to waste”
“Research funding agencies need to recognize that preservation of and access to digital data are central to their mission, and need to be supported accordingly” Nature: 10 Sep 2009
Climate Change Response
• Data and information sharing and integration are the key to collaboration among DOI bureaus and their many partners in addressing the significant climate change challenges.
• We need a significant DOI response to develop effective and efficient means of finding, retrieving, using, and sharing the best available data and information. This will require both a top-down and bottom-up approach to addressing Department-wide data management issues.
• The key to data sharing and integration is modernization of data systems using reusable and sharable building blocks (“web services”) based on DOI and industry standards to allow data exchange across multiple data systems.
Key Points from DOI Data Management/Integration Subcommittee:
Service-oriented Architecture
Service-oriented architecture allows agencies to maintain their internal systems, yet make portions of their data available for sharing
by using a common set of standards and processes.
NPS
FWS
USGS
NASA
NOAA
States
Internal data
Shared dataEnd users
NPSNPS
ResearchersResearchers
StatesStates
NOAA
General PublicGeneral Public
FWS
Natural Resource
Portal
NPS
FWS
USGS
NASA
NOAA
Air France
Lufthansa
DeltaFrontier USAir
United Alamo
Hertz
NationalBudget Enterprise
Avis Best Western
Days Inn
HiltonMarriott Motel 6
Holiday Inn
Expedia.comOrbitz.com
Common Service Standards Common Service Standards Common Service Standards
Hotels.com
Using a set of common data service exchange standards makes disparate data available for sharing
Recovery PlansSpecies ReportsSpecies StatusMigratory BirdsRefuge LocationsCritical HabitatWetlandsEtc.
ReferencesTaxonomySpecies OccurrencesPark LocationsMonitoring DataWater QualityVegetationEtc.
PublicationsPhenologyRemote SensingStream GaugesClimate StationsExotic SpeciesITIS, PLANTSMaps, etc.
Data.gov
DOI
Common Service Standards Common Service Standards Common Service Standards
Using a set of common data service exchange standards makes disparate data available for sharing
FWS Data
NPS Data
Natural Resource Portal
USGS, Other Agencies
Researchers
• Integration of Resource Management Applications
• Service-oriented Architecture (SOA): build once, use many times services that can be re-used by many applications
• Using DOI and industry standards to allow data exchange and integration among data systems
• DOI award for “Best Agency SOA Application”
Why Service-oriented Architecture?• Modular: Each agency/organization can build their own system
incrementally and iteratively, when they’re able. Agencies don’t all have to get together and “figure it all out” in advance.
• SOA can leverage existing investments; more efficient– Capitalize on existing resources, including developers, software
languages, hardware platforms, databases, and applications– “Reusable building blocks” developed by others can be shared
and built upon, rather than everyone starting from scratch.
• Adaptive Management for IT: SOA positions us to respond rapidly to a changing environment once initial building blocks are in place.
• SOA is DOI, industry, and Federal Government Best Practice– Private sector investment in development of methods.
Service-oriented ArchitectureSOA: silos are deconstructed into reusable, multipurpose “services”
user identitypark codereferencesdata sets
access control data entry
data downloadsreports
observationstaxonomyvouchers
user identitypark codereferencesdata sets
access control data entry
data downloadsreports
observationstaxonomyvouchers
user identitypark codereferencesdata sets
access controldata entry
data downloadsdata uploads
reports
user identitypark codereferencesdata sets
access controldata entry
data downloadsdata uploads
reports
Species Data
References
user identitypark codereferencesdata sets
access controldata entry
data downloadsdata uploads
reports
user identitypark codereferencesdata sets
access controldata entry
data downloadsdata uploads
reports
Data Store
Silos: stand-alone systems that duplicate data and don’t interact well
User identityservice
User identityservice
Unit ServiceUnit Service
Reference Service
Reference Service
Taxonomy Service
Taxonomy Service
SpeciesService
SpeciesService
NotificationService
NotificationService
ATBI All-TaxaInventories
ResearchPermit
And Reporting
Researchers
States
NOAA
University Cooperators
Legacy Systems New Integrated System
APCAM InvasiveSpecies
External users such as other agencies, universities, or organizations can access services we want to share
Internally, services can be shared and re-used
USFWS
Examples of common, reusable services
References(Metadata)
Taxonomy
Identity Management
Notification
Feedback
Fluency in and crosswalks among key taxonomic systems. Allows analysis and synthesis of information across multiple classification systems. Can accommodate biological classification as well as other systems (e.g., geology, vegetation)
Metadata for documents, data sets, articles, images; links to full-text documents and files; allows searching all content including PDFs. Retrieve data, documents, other file types.
Eliminates user logins and passwords. System determines who you are and your specific permissions for data sets and subsets (e.g., who can view certain sensitive data, edit or delete particular records, upload data)
Request and receive customized notifications (via email or other means) when specific events occur in an information system (e.g., a subject-specific record added or changed)
Submit suggestions or bugs directly to information system managers; progress of resolution visible. Engages users directly in improving information system and helps ensure it meets their needs.
UnitsInformation on designated geographic or administrative areas such as parks, wilderness, or regions. Includes ability to link and cross-link units (e.g., one unit may comprise many units)
Organization - StructureTeams
• Functional• Architecture• User Interface
Design• Development• Quality Assurance• Infrastructure
• … and Project Management
Project Management
Configuration Management
Functional AnalysisChange Control Board
Quality Assurance Development
Architecture
Iterative Development Process
3-4
we
ek
itera
tion
s
User
User
Person 1
User
User Community
Organization - Methodology
Rational Unified Process• Project plan is divided into iterations
• Project plan tackles high risk in first few iterations
• An iteration is divided into multiple tasks (or work items)
• Each iteration results in an executable application or service
• Each iteration consistently delivers demo-able software
Quality Assurance Development
Architecture
Iterative Development Process
3-4
wee
k ite
ratio
ns
Building the IRMA Infrastructure
Testing Role-based Teams
User Interface Consistency
User Requirements
Iterative Development
Reporting Portal Help AuditingError
Handling
Foundation
Processes
Frameworks
Supporting ServicesIdentity Feedback Notification Park Units File
Resource DataSpecies References Observations Vouchers Taxonomy
Hardware Software Staff Organization
NPS Experience with SOA – Key Points
• Required changes in how people worked together and our development approach;– SOA is something you do, not something you buy– Organized people into functional teams, rather than one
developer per application
• Structural change: Decoupled the “what” from the “how”– “What”: Analysts focus on user requirements– “How”: Developers focus on technology
• End Result:– Improved applications & services; happier users– Lets you quickly build custom business processes from existing
services (e.g. service orchestration) The organization can survive loss of key players
– Is a sophisticated undertaking, but well worth it
Natural Resource Database Template
• NRDT standardizes data elements that are common to most I&M data sets (e.g., locations, events)
• Allows for higher-level integration of data sets across networks or regions (rolling up the data)
• Allows for linkage to GIS• More efficient application development• Users across the country can share code, share
approaches or solutions to specific problems, and use a common database vocabulary.
A standardized yet flexible database core for developing natural resource-related applications
Metadata
CoreTables
Protocol-specific Tables
Shapefile
Optional GISFeatures
Coverage
Geodatabase
Core Tables
NRDT Front-end Application Builder (FAB)
• A Microsoft Access file that provides a user-interface and administrative tools for developing NRDT applications
– is linked to NRDT back-end database file– automates table-linking, compaction, backups, and
lookup table management– provides main menu, data gateway form for selecting
records, and data entry forms for entering new records– works with NRDT back-end files with any of the primary
key options
Natural Resource Program CenterInventory & Monitoring Program
NRDT Front-end Application Builder
•User-interface template for NRDT•Includes many built-in features and frequently-used utilities•Gives applications a standardized look-and-feel
San Francisco Area Network Sonoran Desert Network
• Currently available to NPS users; read-only, non-sensitive data• Additional functionality and data added every few months• Legacy systems no longer needed by summer 2010• Non-sensitive data expected to be publically available in 2010
Simple search tool results in list of references and attached files if available
250,000+ records now available.
Natural Resource Documents and Data (References)
Simple search tool results in certified species list and related data. Users can view, print, download results.
Park Species Information
Taxonomy service available to search and crosswalk with multiple taxonomic systems.
Taxonomic Information
NPS Natural Resource & GIS ProgramsInventory and Monitoring Program
FWS Meeting - Sept 2009
IRMA Integration
NPS Natural Resource & GIS ProgramsInventory and Monitoring Program
FWS Meeting - Sept 2009
NPScape Measures
• Habitat availability• Fragmentation and isolation• Watershed health• Anthropogenic threats to park resources
Land CoverEnhanced NLCD 1992NLCD 1992NLCD 2001NLCD Change ProductNLCD Impervious SurfaceNLCD Tree CanopyHistoric Natural Fire RegimeLandFire - all productsLand cover diversity (Simpson's)NatureServe Ecological SystemsGAP/ReGAPLand cover characteristicsForest fragmentationMorphological pattern metricsForest cover typesConverted and natural landcover
LandformDEM - 10, 30, and 120 meter
Slope and aspect
Depth to bedrock
Sand, silt, clay fractions
Crop capability
Geology
Climate Precipitation Temperature (min/max, variability) Growing season daysNDVISea surface temperature
TransportationRoads (Multiple data sources)Railroads (U.S. and Canada)National Waterways
PopulationNightime lights (1992/93, 2000)Populated placesU.S. citiesU.S. urban areas1990, 2000 Census, block groupPopulation projections by countySEDAC census gridsHousing density (1940-2040)Inventoried roadless areaAgriculture census by countyWater use by countyConservation risk indexWildland Urban Interface
BoundariesOmernick Ecoregions (and CEC)Bailey Ecoregions Physiographic ProvincesUNEP Large Marine Ecosystems StatesCounties NPS Units (with various buffers)NPS Vital Sign NetworksProtected Areas BoundariesFederal LandsNational Wilderness Preservation SystemContinental DivideNCDC Climate DivisionsNEON Domains
HydrologyHydrologic Units (4, 6, 8, 12-digit)NHD (med and high resolution)ImpoundmentsAquifersGround water climate response networkSea ice (North America)
Data Sets that we have available:
Office of Inventory, Monitoring, and EvaluationNatural Resource Program Center
NPS Natural Resource & GIS ProgramsInventory and Monitoring Program
FWS Meeting – Sept 2009
Integration Approaches for Monitoring Data
• NRDT upload and map display functions– Upload and centralize data from desktop databases– Upload monitoring locations and attributes:
• Visualize and query– Access MDB SS2008 ArcSDE/ArcGIS Server– Eventual integration within IRMA frameworks
NPS Natural Resource & GIS ProgramsInventory and Monitoring Program
FWS Meeting – Sept 2009
Upload from NRDT
Run demo
Inventory & Monitoring Program“In just a few short years, the I&M networks have become known as a key source and supplier of reliable, organized, and retrievable information about parks that was formerly unavailable, misplaced, or lost to managers and others who needed the information for sound decisions or sound science.”
FY 2010: Accelerate development Packaging & Delivery of data for modeling Close partnership with USFWS