Upload
cody-phillips
View
215
Download
0
Embed Size (px)
Citation preview
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Current Air Quality Information ‘Ecosystem’ (Draft for Feedback)
AQ information includes emissions, ambient & satellite data and model outputs
The distributed data are produced and provided by agencies, mostly through portals
Providers have different access protocols, formats, and information usage conditions
This lack of interoperability causes the under-utilization of the rich data resources
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Future Integrated AQ information System (Draft for Feedback)
DataMart
VIEWS
NEISGEI
AIRNow
AQMod
DAACs
ASOS
NEI
Emission
IDEA
GASP
Missions
WeaMod
Forecast
GloMod
FireInv
Data Federation Distributed, Virtual, Uniform
AQ Forecasting
AQ Compliance
Status and Trends
Network Assess.
Data Processing Filtering, Aggregation, Fusion
Info Products Reports, Websites
Data are maintained by custodians and exposed through ‘portals’ Mediators uniformly ‘wrap’ data and provide processing servicesAnalysts program the services to create application-specific productsResponsibility is shared among data providers and mediator/ integratorsESIPFed can provide the infrastructure and tools for the AQ info system
Mediators
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Air Quality Information Providers
• AQ information includes emissions, ambient (surface) and satellite data and model outputs • The information is provided by multiple Agencies, have different form and is• AQ data usage requires considerable processing and integrating
EmissionAmbientSatelliteModel
Form | ContentNOAA
GASP
NASA
DAACs
NASA
IDEA
NASA
Missions
EPA
NEI
EPA
NEISGEINOAA
FireInv
State/Local
Emission
NOAA
ASOSRPO
VIEWS
EPA
AIRNow
EPA-AQS
DataMart
NOAA
WeaMod
EPA
AQModel
NASA
GloModel
NOAA
Forecast
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
DataFed WS Output Data Types
• UrlGranuleType• TimePointType• TimeDimensionType• MapVectorType• MapTrajectoryType• MapTimePointType• MapPointType• MapLocationTableType• MapImageLatLonType• MapGridType• ImageType• HtmlType• DotNetTableType• DataSetType
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Interagency Working Group for Earth Obs. (IWGEO) Global Earth Observing System of Systems (GEOSS)
Decision Support Tools
Assessments
Decision Support Systems
Decision Support Tools
Assessments
Decision Support Systems
Decision Support
Assessments
Decision Support Systems
ManagementDecisions
PolicyDecisions
Societal BenefitsHigh PerformanceComputing,Communication,& Visualization
Standards &Interoperability
Predictions
Observations
Monitoring & Measurements
remotely-sensedin situ
Earth Science Models• Oceans• Ice• Land• Atmosphere• Solid Earth• Biosphere
Monitoring & Measurements
remotely-sensedin situ
Earth Observation Systems
• Remotely-sensed• In situ
Earth Science Models• Oceans• Ice• Land• Atmosphere• Solid Earth• Biosphere
Earth System Models• Oceans• Ice• Land• Atmosphere• Solid Earth• Biosphere
DATA
On-going feedback to optimize value and reduce gaps
T. Karl, NOAA, NCDC
Integrated Observing Systems
OBSERVING SYSTEM TIMELINE
21st CenturyAtmospheric Observations
Data Systems
Technology Development
Innovations
Breakthrough
Efficiencies Cost
Mass Productions
Space Observations
Ocean Observations
Innovations
BreakthroughEfficiencies
Cost
Mass Productions
66
T. Karl, NOAA, NCDC
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Tools for Users
• Pare down large file sizes of high resolution data and products.
• (re-) Group different data sets to create needed products – such as initialization files for model development, analysis, and intercomparison.
• Subset the data: – in parameter space
– in physical space
– in temporal space
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Collaborations: How do we get there?Collaborations: How do we get there?
• Data transport is being actively pursued: OPeNDAP, SOAP, ...
• Earth System Partners need to be able to find and use various data sets, wherever they may be, whatever format...
• THREDDS can provide dynamic access and generate catalogs
• GCMD is a major resource for metadata management for the entire GeoSciences community- this activity must evolve!
• Ontology projects such as SWEET in conjunction with THREDDS and GCMD can provide individual data sources, data variables and metadata management for the community.
G. Rutledge: Emerging Tools for Distributed Data Access and Collaborations
•Data systems based on the integration of independently developed system elements offer many more opportunities than more traditional centrally developed ones.•P. Cornillon
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Tools & Data
Information Systems
ProfessionalProductivity
InformationManagement
SharedServices
. . . Becoming More Intelligent And Distributed. . . Becoming More Intelligent And Distributed
Web Services Networks
GIS is Evolving to a Web Services EnvironmentGIS is Evolving to a Web Services Environment
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
GIS Networks Will Allow Us to Connect andIntegrate Distributed GIS Resources
. . . Making Virtual Collaborations Possible. . . Making Virtual Collaborations Possible
MapsMapsModelsModels
GeoDataGeoDataSetsSets
Peer-to-Peer GISPeer-to-Peer GIS
MetadataMetadata
Data ModelsData Models
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
PervasivePervasiveComputingComputing
Terabyte/Second CommunicationsTerabyte/Second Communications
• Faster HardwareFaster Hardware• Distributed ComputingDistributed Computing• Mobile/WirelessMobile/Wireless• Services Oriented Services Oriented
ArchitectureArchitecture• Large Data RepositoriesLarge Data Repositories• GIS SoftwareGIS Software
Capacity In 10 YearsCapacity In 10 Years• 100x Computing100x Computing• 1000x Storage1000x Storage• 5000x Networks5000x Networks
Enabling Technology
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
InternetWeb Services
GIS Portals Support Data Dissemination
Select Select Format Format
Data Data ConversionConversion
Zoom to Zoom to ExtentExtent
. . . Clip/Zip/Ship. . . Clip/Zip/Ship
TIGER
DXF
VPF
S57
GML
XMC
MIF
Geomedia
SDTS
DLGDWG
DGNCAD
. . . Supporting Interoperability. . . Supporting Interoperability
IMS ServerIMS Server
Support Many Support Many FormatsFormats
Many Standard Formats And Many Standard Formats And
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Interoperability Is Important
ConversionConversion
Direct Read Direct Read (API)(API)
DBMSDBMS IntegrationIntegration
. . . Focus Is On Simple and. . . Focus Is On Simple and Practical Approaches That WorkPractical Approaches That Work
Web Web ServicesServices
GISGISServerServer
There Are Many Standards . . .There Are Many Standards . . .
XML/SOAPXML/SOAP
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Direct Read & UseDirect Read & UseDynamic Dynamic Read/Conversion/UseRead/Conversion/Use
Custom Format Custom Format ConvertersConverters
MIF GML
M.S.
MIF
Standards And Direct Proprietary InterfacesStandards And Direct Proprietary Interfaces
Interoperability Technology Is A Fundamental Part Of GIS Products
. . . Supporting Complex Data Transformation. . . Supporting Complex Data Transformation
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
For GIS Networks to Work Either Everyone For GIS Networks to Work Either Everyone Uses Same Software, Data Formats, and Data Models . . . Uses Same Software, Data Formats, and Data Models . . .
They Use Interoperability ProceduresThey Use Interoperability Procedures
. . . Geoprocessing Models Can Transform. . . Geoprocessing Models Can TransformData AutomaticallyData Automatically
. . . OR. . . OR
• Format Conversion• Schema Reorganization (ETL)• Scale Projection Changes• Generalization• Merge
GeoprocessingGeoprocessingModelsModels
Interoperability Is Important
. . . Enhancing Collaboration. . . Enhancing Collaboration
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Geoprocessing On Servers
GISGIS
. . . Distributed Workflow & Process . . . Distributed Workflow & Process ModelsModels
Distributing Spatial Analysis And ModelingDistributing Spatial Analysis And Modeling
NowNow FutureFuture
GISGIS
BrowserBrowserDesktopDesktop
Data SetsData Sets
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Managing Multi Dimensional Geographic Data Sets And Simulation Modeling
• Data ModelingData Modeling• Tools for ManipulationTools for Manipulation
– QueryQuery– Change AnalysisChange Analysis– Iterative ProcessingIterative Processing– VisualizationVisualization
–AnimationAnimation–ChartingCharting
With Particular Focus on TimeWith Particular Focus on Time
FutureFuture
T1T1
. . . Iterative/Recursive Modeling. . . Iterative/Recursive Modeling
Simulation / Time LoopingSimulation / Time Looping
New Folder\ELNINO_Final.aviaa
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
GIS Will Maintain Distributed Geographic Knowledge
Relationships Will be via “Messaging”(Sending/Receiving Web Services Messages)
Geodatabases Will be Distributed and FederatedGeodatabases Will be Distributed and Federated
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Serving Globes Over the Web
. . . Serving 3D Virtual Geography. . . Serving 3D Virtual Geography
Globe WebGlobe WebServerServer
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Personal GIS Server
. . . Users Will . . . Users Will ShareShare And Serve Their KnowledgeAnd Serve Their Knowledge
Supporting• Map Services• Metadata Catalog (Searching
& Harvesting)• Download
– Data – Models– Data Models
• Easy to Use• Simple to Install
GeodatabaseGeodatabase
Web ServiceWeb ServiceGIS GIS
DesktopDesktop PersonalPersonalServerServer
MetadataMetadata
ModelsModels
MapsMaps
GeodataGeodataSetsSets
DataDataModelsModels
Will Allow Peer to Peer CollaborationWill Allow Peer to Peer Collaboration
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
TT TT
TT
TTTT
UpdateUpdateMessagesMessages
NationalNational
StateState
LocalLocal
• ReplicatedReplicated• Periodically UpdatedPeriodically Updated• History/Archiving History/Archiving
Geodatabases Will Support Geodatabases Will Support Distributed Data ManagementDistributed Data Management
TT = Transactions= Transactions
22
Infusion Confusion Solutions:Putting Technology to Work
Earth Science Data System Working Groupon Technology Infusion
Karen Moe, NASA/ESTORob Raskin, NASA/JPL
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
• One of four groups established by the REASoN CAN– Standards & Interfaces– Metrics Planning &
Reporting– Reuse Frameworks– Technology Infusion
• Outgrowth of SEEDS– Strategic Evolution of ESE
Data Systems– Explored ways to support
NASA ES strategy• More PI production
processing• Measurement-oriented
systemsREASoN = Research, Applications, and Education Solutions NetworkCAN = Cooperative Agreement NoticeESDSWG = Earth Science Data System Working Groups
What is the Technology Infusion Working Group?
SEEDS
REASoN CAN
ESE Strategic
Plan
ProjectsProjects
Projects
• • • Data Life Cycle
ESDSWG• Standards• Metrics• Reuse• Infusion
New in 2005
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Effective Technology
Infusion
Enterprise Context• Constrained budgets• Broad data service provider community
Pragmatic Infusion Approaches
• Information sharing• Demonstration
testbeds
Emerging Technologies
• Technology investments• Web and grid computing
• Linux clusters
Organizational Goals• Lower system costs• Increase community
participation• Increase flexibility &
responsivenessInternal
Opportunities
Drivers
External
Why is Technology Infusion Important?Drivers and Opportunities
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Why is Technology Infusion Important?
Meeting ESE Goals Requires Tech Infusion
• Science and application needs– Faster & better models– Near-real-time data – Easier data fusion
• Science data system needs– Enable open distributed
architecture for PI processing
– Fill capability gaps in current systems
– Support evolution
New Research
New Applications
New System Capabilities
System Capability
Vision
Technology Infusion
Technology Identificatio
n / Developmen
t
Science & App Needs
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Capability Needs
Technology Projections
Technology Roadmaps
Technology DevelopmentTechnology
Infusion
Operational Systems Identified
Gaps
Solicitation Formulation
Peer Review & Competitive Selection
Capability Vision
Technology Infusion is Part of a Larger System Evolution Process
• Think globally, act locally– How can we improve technology infusion across the
community?– How can you successfully infuse technology in your own
projects?
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
What Capabilities are Needed?
Capability BenefitAssisted information discovery
Identify needed data quickly and easily
Seamless information access Enable access to any data from anywhere
Assisted knowledge building Provide research and operations assistance
Interactive analysis environments
Reduce research algorithm implementation from months to hours
Super-scalable analysis portals
Provide computing power and data storage on demand
Interoperable information services
Increase synergy within the ESE community through service chaining
Integrated modeling frameworks
Enable linked and ensemble models for improved predictive capability
Responsive information logistics
Ensure research priorities are met and enable new uses of ESE data
Verifiable information quality Provide confidence in products and enable community data providers
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
THREEDS - Topics
• Traditional Unidata Approach– Mainly meteorological data– Subscription system pushes data to user sites– Unidata Program Center provides data analysis tools for
use on data at user sites
• THREDDS Enhancements– Broader menu of Earth system data– Local client access from remote servers– Less arcane, more general and accessible tools– Integration of data and analysis tools into educational
modules and digital libraries
THREEDS
The THREDDS (Thematic Realtime Environmental Distributed Data Services) project is developing middleware to bridge the gap between data providers and data users. The goal is to simplify the discovery and use of scientific data and to allow scientific publications and educational materials to reference scientific data.
The mission of THREDDS is for students, educators and researchers to publish, contribute, find, and interact with data relating to the Earth system in a convenient, effective, and integrated fashion. Just as the World Wide Web and digital-library technologies have simplified the process of publishing and accessing multimedia documents, THREDDS is building infrastructure needed for publishing and accessing scientific data in a similarly convenient fashion.
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
THREDDS THematic Real-time Environmental Distributed Data Services
Connecting people, documents and data
PeoplePeople
DocumentsDocuments DataData
DRAFT April 28, 2005 ESIP AQ Cluster, [email protected]
Summary
• Universities have used Unidata tools to acquire, analyze, and display real-time atmospheric data for nearly 20 years
• THREDDS – along with related client/server access and display technologies-- makes an even broader menu of Earth system data to a more diverse community of users
• THREDDS technologies enable the creation of compound educational modules and scientific publications with embedded pointers to datasets and tools.