View
953
Download
3
Category
Tags:
Preview:
DESCRIPTION
Citation preview
NPPNational Polar-orbiting Operational Environmental
Satellite System Preparatory Project
Interface Data Processing Segment (IDPS)
Joe Mulligan, IPO Ground Systems Division Deputy Chief
Marge Ripley, IPO IDPS IPT Lead
NPPNational Polar-orbiting Operational Environmental
Satellite System Preparatory Project
Interface Data Processing Segment (IDPS)
Joe Mulligan, IPO Ground Systems Division Deputy Chief
Marge Ripley, IPO IDPS IPT Lead
Timely, Accurate, Reliable Data from Sensors to Users
C3SDHN &
FEP(@ IDPS)
IDP@
Centrals
C3S
IDPS
Centrals SMD
Field Terminals (LRD, HRD)
FieldTerminalSoftware
CONUS Gateways
(4x)SMD
HRD
GroundReceptor
GroundStation
Mission Management Center
Stored Mission Data flow forCentrals and Science Data Users
C3SDHN &
FEP(@ IDPS)
IDP@
Centrals
IDP@
Centrals
C3S
IDPS
Science Users
SMD
Field Terminals (HRD)
FieldTerminalSoftware
FieldTerminalSoftware
HRD Data flow for Field Users
Field Users
CONUS Gateways
(2x)
Svalbard Ground
Mission Management Center
Deliver DataProducts
CLASS
Deliver DataProducts
Sensors Bus
Space Vehicle 1
Bus
NPP Spacecraft
Sensors
NPP Mission Data Flow
SDS
A/D Conversion
Packetization
Compression
Anc.
Sensor
Data
CCSDS (mux, code, frame)
Comm
Xmitter
SEN
SOR
S
OT
HE
R
SU
BS
YS
TE
MS
Cal.Source
\ Data Store
ENVIRONMENTALSOURCE
COMPONENTS
SPA
CE
SE
GM
EN
T
RDR
Production
EDR
Production
SDR
Production
EDR Level
SDR Level
RDR Level IDP
S
Comm
Receiver
Comm
Processing
CCSDS Level
C3S
NPP Data Product Definitions
NASA’s Science Data Segment will independently assess the quality of the science and environmental data records from the NPP mission for their
ability to support climate research.
All data products will be provided to NOAA’s Comprehensive Large-Array data Stewardship System (CLASS) for distribution to the broader scientific
user community.
The plan for the follow on mission is to add two more Centrals: FNMOC and NAVO.
NPP will provide data for processing at each of the two centrals: AFWA and NESDIS. Each Central will then disseminate the data to their user
communities.
The data products generated on IDPS are delivered to the Centrals, CLASS and SDS based on their data subscriptions.
NPOESS Preparatory Project (NPP) Science Data
Segment (SDS)
Air Force Weather Agency (AFWA)
NOAA Satellite and Information Service
(NESDIS)
Fleet Numerical Meteorology and
Oceanography Center (FNMOC)
Naval Oceanographic Office (NAVO)
Comprehensive Large Array-data Stewardship
System (CLASS)
Interfaces
Infrastructure
Ingest
Data Management
Data Delivery
Processing
Data QualityMonitor
DQM performs quality monitoring of the data products. It allows automated and ad-hoc processing in support of data quality notifications from the PRO subsystem. It also provides the interface and tools for the data quality engineers to analyze the data and generate reports.
This subsystem is implemented only at the NESDIS.
The Data Delivery Subsystem is the primary mechanism for data delivery. Users have the capability to submit data requests via a GUI or use of an API. HDF5 formatting, packaging and
aggregation of NPOESS Data products is performed.
The Data Management Subsystem manages the data within IDPS. Its primary function is sharing data between processes and the other subsystems. It provides on-line repository of
data, inventory and metadata.
The Processing (PRO) subsystem performs the data processing for product generation. PRO takes RDRs, ancillary (ANC) and auxiliary (AUX) data as inputs to generate Sensor Data
Records (SDRs), Temperature Data Records (TDRs), geolocation information, Intermediate Products (IPs), and Environmental Data Records (EDRs). PRO assesses the data quality of the SDR, TDR, and EDRs and sends real-time quality notifications to the Data Quality Monitoring
(DQM) subsystem to alert the operators of possible quality degradation.
The Ingest subsystem is the primary input path for data into IDPS. Stored Mission Data and Mission Support Data are ingested and used to create Raw Data Records (RDRs).
The Infrastructure subsystem manages IDPS by initiating tasks, monitoring processes, monitoring product generation, and maintaining log files of Status, Error, and Informational
messages. It also provides common utilities, tools and a scheduler. It interfaces with NPOESS Enterprise Management (EM) as well as providing the main GUI for IDP Operator management,
control and status.
The Interface Data Processing Segment (IDPS) consists of the data processing elements (DPEs) located at each of the four Centrals to generate and deliver NPOESS and NPP products and
associated metadata to the Centrals for additional application specific processing and dissemination. Each IDPS consists of six subsystems: Infrastructure, Ingest, Processing, Data
Management, Data Delivery and Data Quality Monitoring.
IDPS Functional Overview
•The current PRO design is based on an I-P-O model– Input (I) retrieves all needed input data –Processing (P) creates EDRs and IPs from the input data–Output (O) sends products and metadata to the DMS for
storage, formatting & delivery • The use of modular software isolates impacts of changes to the
basic architecture and provides for rapid, low impact recovery from hardware and software failures
• Additionally, granule based processing allows the generation of high quality products within latency timelines
• This construct allows the decoupling of algorithms from I/O
Input OutputProcessing
I-P-O Construct
NPP EDRs
NPP System LatencyNPP System Latency for 47th Orbit
4-copy Robustness Test executed 6/18 - 6/24/2009
0:00
10:00
20:00
30:00
40:00
50:00
60:00
70:00
80:00
90:00
100:00
110:00
120:00
130:00
140:00
150:00
Granule ID (increasing)
Sys
tem
Lat
ency
(m
m:s
s)
CrIMSS EDR
VIIRS EDR
ATMS SDR
CrIS SDR
OMPS SDR
VIIRS SDR
ATMS TDR
ATMS RDR
CrIS RDR
OMPS RDR
VIIRS RDR
Age @ Arrival
Max Allowable Latency for most EDRs
- NHF, Surface Albedo, & Sea Ice Characterization get 10 more minutes
- Final product for each chain shown for clarity - CERES is not delivered to the ACI.
IDPS is optimized to reach data latency requirements
Nominal Robustness TestLatency Orbit (Orbit 47)
0:00
10:00
20:00
30:00
40:00
50:00
60:00
70:00
80:00
90:00
100:00
110:00
120:00
CrIMSS-EDR
VIIRS-Aeros-EDR
VIIRS-AF-EDR
VIIRS-CBH-EDR
VIIRS-CCL-EDR
VIIRS-CEPS-EDR
VIIRS-COT-EDR
VIIRS-CTH-EDR
VIIRS-CTP-EDR
VIIRS-CTT-EDR
VIIRS-I1-IMG-EDR
VIIRS-I2-IMG-EDR
VIIRS-I3-IMG-EDR
VIIRS-I4-IMG-EDR
VIIRS-I5-IMG-EDR
VIIRS-IST-EDR
VIIRS-LST-EDR
VIIRS-M1ST-EDR
VIIRS-M2ND-EDR
VIIRS-M3RD-EDR
VIIRS-M4TH-EDR
VIIRS-M5TH-EDR
VIIRS-M6TH-EDR
VIIRS-NCC-EDR
VIIRS-NHF-EDR
VIIRS-OCC-EDR
VIIRS-SA-EDR
VIIRS-SCD-BINARY-SNOW-FRAC-EDR
VIIRS-SCD-BINARY-SNOW-MAP-EDR
VIIRS-SIC-EDR
VIIRS-SST-EDR
VIIRS-ST-EDR
VIIRS-SusMat-EDR
VIIRS-VI-EDR
CrIMSS-Latency-EDR
VIIRS-Latency-EDR
IDPS Data Latency
• Each Central will have an IDPS − High performance IBM computing hardware− POWER architecture
• Consists of two hardware suites− Operational string− Integration and Test string
IDPS IBM Architecture
•The HW is scalable and expandable− Can add new capabilities without impacting the existing system
• Combining software and hardware flexibility and robustness− Allows stringent performance requirements to be met
• Fault tolerant hardware and software − Ensures high quality data products− Available to users within operational latency timelines
IDPS Flexibility
Summary
• IDPS generates the data products for end users.•Central Users will get data directly from IDPS•Science Users will get data from CLASS•SDS will get data directly from IDPS
•25 EDRs will be produced by IDPS
•IDPS is meeting Latency requirements
• IDPS is installed at AFWA and NESDIS and is currently undergoing system level testing in preparation for launch.
Questions ?
Questions ?
Recommended