InSAR Working Group Meeting, Oxnard, Oct 2004
1
IT discussion
Day 1 afternoon breakout session
http://solidearth.jpl.nasa.gov/insar
InSAR Working Group Meeting, Oxnard, Oct 2004
2
GDS Data Flow
UpdateCatalog
Data FTP
Stanford University
Caltech MIT NSIDC Howard University
SDSC Long Term Archive5 years near-line
EDC Permanent Archive
University of Miami
Alaska SARFacility (ASF)
Mission Control Center (MCC)
DLR-GSOCGround StationWeilheim (WHM)
Short Term Archive 1 year on-line
CatalogScience Team
Archive Manager
Collection requests, acquisition requests, data requests
Acquisition plan, status, holdings, software, ancillary data
Catalog redirects user to appropriate FTP address for L0 data retrieval
L1 Data
Internet - 2
User
Internet - 2
Internet and Internet-2
Network Transfer Subsystem (NTS)Stanford University & NSIDC•Archive Manager
Caltech
UserCatalog
Archive Manager
User
NSIDC
User
NSIDCStanford University
SDSC Long Term Archive5 years near-line
Archive Manager
Alaska SAR Facility (ASF)
JPL Mission Ops
300 Mb/sec downlink
Populate STArchive
Catalog
Archive Manager
UserRedirect
PopulateLT Archive
InBackground
User
NSIDC
Catalog
MIT EDC Permanent Archive
SIO Science Planning Bd.
User Acq.Request
Catalog
SIO SciencePlanning Bd.
JPL Mission Ops
Mission ControlCenter (MCC)
DLR-GSOC Ground StationWeilheim (WHM)
Mission ControlCenter (MCC)
JPL Mission Ops
Status toUser
Catalog
Network Transfer Subsystem (NTS)Stanford University & NSIDC•Archive Manager
User Data.Request
Transfer toNTS
PopulatePermanentArchive in
Background
InSAR Working Group Meeting, Oxnard, Oct 2004
3
Solid Earth Research Virtual Observatory (SERVO)Solid Earth Research Virtual Observatory (SERVO)
Tier2 Center
Archive
SERVO
…Goddard JPL Ames
InstituteInstituteInstituteInstitute
Fully functional problem solving environment
•Plug and play composing of parallel programs from algorithmic modules
•On-demand downloads of 100 GB in 5 minutes•106 volume elements rendering in real-time
•Program-to-program communication in milliseconds
•Approximately 100 model codesData cache
~TBytes/day
Tier2 CenterTier2 CenterTier2 Center
Tier 0 +1
Tier 1
Tier 3
Tier 4
Tier2 Center
•1 PB per year data rate in 2010
•Distributed Heterogeneous Real-Time Datasets
Observations
Archive
Downlink
Archive Downlink
Downlink
…
……
…… …
…
100 TeraFLOPs sustained
Tier 2
Workstations, other portals
100 - 1000 Mbits/sec
InSAR Working Group Meeting, Oxnard, Oct 2004
4
National Lambda Rail (NLR)
15808 Terminal, Regen or OADM siteFiber route
NLRComing Up NowInitially 4 10G
WavelengthsFull Footprint Ops
by 3Q or 4Q04 Internet2 HOPI Initiative (w/HEP) To 40 10G Waves in Future
Transition beginning now to optical, multi-wavelength Community owned or leased fiber networks for R&E
PITPIT
PORPOR
FREFRE
RALRAL
WALWAL
NASNASPHOPHO
OLGOLGATLATL
CHICHI
CLECLE
KANKAN
OGDOGD
SACSAC BOSBOSNYCNYC
WDCWDC
STRSTR
DALDAL
DENDEN
LAXLAX
SVLSVL
SEASEA
SDGSDG
JACJAC Regional Dark Fiber Initiatives
in 18 U.S. States
InSAR Working Group Meeting, Oxnard, Oct 2004
5
UCSD
StarLightChicago
UIC EVL
NU
Solution: Move to Internet Protocol Over Dedicated Optical Lightpaths
NASA JPL
NASA Ames
PNWGPSeattle
Pacific Wave
NLR
NLR
NASA Goddard2Level 3
Level 3
4
CalREN-XD
4
CENICSan DiegoGigaPOP
2
UCI
ISI CENICLos Angeles
GigaPOP
2
U Amsterdam
NetherLightAmsterdam
CA*net4
1GE lambda
10GE lambda
NASA Goddard IT Pathfinder Working GroupEarth and Climate Scientists-
Creating a Virtual Collaboratory Between
Goddard and SIO (UCSD)
Coupling NASA Centers
to NSF OptIPuter
InSAR Working Group Meeting, Oxnard, Oct 2004
6
Database Database
Analysis and VisualizationPortal
RepositoriesFederated Databases
Data Filter
Services
Field Trip DataStreaming Data
Sensors
?DiscoveryServices
SERVOGrid
ResearchSimulations
Research Education
CustomizationServices
From Research
to Education
EducationGrid ComputerFarmGeoscience Research and Education Grids
GISGrid
Sensor GridDatabase Grid
Compute Grid
InSAR Working Group Meeting, Oxnard, Oct 2004
8
Schematic of a Typical Ontology Manager
DataResource
Grids
MetadataManagement
TopicMining
Authoring ToolOntology
Architecture
DatabaseArchitecture
Task Pool ManagerSemantic -basedQuery Generator
DB1 DB2Data
Source iDBm
DataSource1
Conn.Pool
É É
DBMS1 DBMSn
Conn.Pool
Conn.Pool
Conn.Pool
FederatedDatabase Services
Onto-logy
Onto-logy
Onto-logy
Onto-logy
Onto-logy
Data Integration API (Semantic-based Wrapper)
Meta-ontologies(metadata of ontologies
assocaited with local databases)
InSAR Working Group Meeting, Oxnard, Oct 2004
9
What will be needed as technology for:
• Access to data through data bases and the web
• Software for data mining and data exploration
• Software for simulating, interpreting, visualizing data
InSAR Working Group Meeting, Oxnard, Oct 2004
10
Who are the End Users outside the InSAR Research Community?
• Engineering• Education• Government/Policy decision makers• Operational/Hazards• Commercial
InSAR Working Group Meeting, Oxnard, Oct 2004
11
Issues Associated with Access to data through databases
• Standards for domestic and international missions
• Security• Quality and Validation• Bandwidth• Interoperability – XML & GML• Content addressable storage• Near real time or real time
– Rapid response tasking of satellite
InSAR Working Group Meeting, Oxnard, Oct 2004
12
S/w and H/w
• Data exploration and analysis– Higher level functionality is desirable– Portal architecture– Peer to peer– Collaborative– Error quantification– Graduated authentication
• S/w development– Curation– Reliability and robustness– Fusion
InSAR Working Group Meeting, Oxnard, Oct 2004
13
Simulating, interpreting, and visualizing data
• Large scale simulation & modeling capability is needed & should be part of mission plan
• Centralized and Grid computing resources
• Code sharing (open source)• Code validation and curation• Reliability and robustness