Page 1GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Resulting Technical Specification

Page 1 GlobColour CDR Meeting – July 10-11, 2006, ESRIN

All rights reserved © 2006, ACRI-ST

Resulting Technical Specification



Technical approach and discussionDevelopment of the processing system

System design

Targets:

Maximum re-use and integration of existing processing / data handling package.

Flexibility to activate one merging technique or the other




System overview

Acceptance of any data sources

Stand alone processing system (PC based HW / under GNU/linux oper. syst.)

Intermediate individual sensors level 3 products

Dissemination of the DDS, PPS, FPS

Link with BEAM




GLOBCOLOURprocessing system

Main modules:

Pre-processing Spatial binning Temporal binning Merging

MERISL2

SeaWIFSOcean color L2

AquaOcean color L2

ParasolOcean Color L2

Preprocessing Preprocessing Preprocessing Preprocessing

(1) Spatial binning

(2) Temporal binningin daily mode

(3) Merging

(4) Temporal binningin periodic mode

MERISL3 track

SeaWIFSL3 track

MODIS/AquaL3 track

ParasolL3 track

MERISL3 daily

SeaWIFSL3 daily

MODIS/AquaL3 daily

POLDEROcean Color L3d

Daily merged4.63km/0.25°/1°/QL

8-days4.63km/0.25°/1°/QL

Monthly4.63km/0.25°/1°/QL

DDS L2Sensor (4.63km)

DDS L3Merged (4.63km)

DDS L2Sensor (1km)Diagnostic

sites table

Level 3 data archive


DDS data archive

Daily level 3 data cache




L3 products at track level daily L3 products per instrument (temporal binning +data-day) merged daily L3 products (merging) 8-days and monthly L3 products (temporal binning)

Technical approach and discussionUpdate of the methodology

GLOBCOLOURprocessing system

MERISL2

SeaWIFSOcean color L2

AquaOcean color L2

ParasolOcean Color L2

Preprocessing Preprocessing Preprocessing Preprocessing

(1) Spatial binning

(2) Temporal binningin daily mode

(3) Merging

(4) Temporal binningin periodic mode

MERISL3 track

SeaWIFSL3 track

MODIS/AquaL3 track

ParasolL3 track

MERISL3 daily

SeaWIFSL3 daily

MODIS/AquaL3 daily

POLDEROcean Color L3d

Daily merged4.63km/0.25°/1°/QL

8-days4.63km/0.25°/1°/QL

Monthly4.63km/0.25°/1°/QL


DDS L3Merged (4.63km)

DDS L2Sensor (1km)Diagnostic

sites table



DDS data archive

Daily level 3 data cache




Globcolour processor development




GLOBCOLOUR processing system

Pre-processing components:

Application of a cross calibration LUT Computations of Fully Normalised Water Leaving Radiance Computation of a common Kd(490)





Spatial binning

Application of aflux-conservativeresampling algorithm (using the Sutherland-Hodgeman areaClipping technique)

L2 grid

L3 grid





Spatial binningMain software component developed: Product reader for all types of Level-2 input products; MERIS L2 (Envisat PDS format), SeaWiFS and MODIS /Aqua Ocean Colour Level 2 in (HDF-EOS), POLDER Ocean Colour Level 2 (HDF format) Resampling Component DDS Level-2/Level-3 Extraction Component DDS Level-2/Level 3 granule netCDF writer Product writer for the spatially binned, intermediate level-3 netCDF products





MergingSeveral software components for merging are currently under IDL breadboard implementation: One or more implementations for the merging method to be used, e.g.

a. Weighted Averagingb. Optimal Interpolationc. Blended Analysisd. GMS

An ocean colour retrieval processor starting from merged sensor spectra





Temporal binningThe core software components have been developed: A component performing the weighted averaging DDS Level-3 extraction component DDS Level-3 granule netCDF writer Level-3 data product writer using netCDF format

Components not yet consolidated:

A component used for computing the error statistics Data-day algorithm for all instruments (breadboard under validation)




Input Products Overview

Uncompressed data volume




Output Products Overview




Output parameters (1/3)












ISIN grid main characteristics




Output products variables - definition (1/2)




Output products variables – storage (2/2)

+ 15 attributes: scaling factors, names, unit, etc…




Format modification since TS 1.1 (globCOLOUR processor v1.0)

Variables are no more scaled due to hot points occurrence

hot points

The hot points are due to a high coding quantisation error at very small weight bins

Impact on final product size has been assessed and takeninto account in the re-estimation of the required resources




DDS/PPS/FPS products – overall list (1/2)




DDS/PPS/FPS products – content (2/2)



Technical approach and discussionProduction of the Full Product Set

Processing requirements

one month of data from the 4 sensors processed in 5 days ten years of data processed in 6 months NRT L3 processed in less than 24 hours

Preliminary data flow requirements and system sizing has been re-assessed using the globCOLOUR processor v1.0

installed on the APADIC system at ACRI-ST




APADIC systemin ACRI-ST




Resources requirements analysis



























Technical approach and discussionDiagnostic Data Set

DDS objectives

The processing system creates DDS granules for a minimum of 20 predefined diagnostic sites. DDS granule are extracted from Level-2 input products and Level-3 output products

A DDS granule covers approximately 100km x 100km and comprises all layers of information of the input or output products.

The granules are provided in the netCDF format following the globCOLOUR DDS format (following CF 1.0 rules)..




DDS important characteristics

DDS are defined by center lon-lat and temporal period. Close DDS lead to almost identical DDS L2 products.

DDS bins are computed using a nearest interpolation approach. One pixel of the L2 track is projected on a single DDS bin: holes exist in the built DDS map (see next slides). Identified by flags.

DDS from L2 input products: PC 1 kmDDS from L3 output products: ISIN 4.63 km (automatic reprojection on PC map performed by DDS tools)




DDS holes MERIS MODIS




DDS holes - flags




Diagnostic Sites

Two purposesSensors characterisation against in situ dataCross-characterisation for specific biogeographical areas

The DDS location and spatial extent are selected from the existing validation sites, e.g. IOCCG/SIMBIOS DDS, ESA validation sites, using the same protocol of selection as the one used in SIMBIOS.

.




Diagnostic SitesFirst pre-selection(1/2)

.

N° SITE LOCATION Lat (°) Long (°) PI

1 MOBY Hawaii 20.8 N 157.2 W D.Clark(Nasa)

2 BOUSSOLE Ligurian Sea 43.3 N 7.1 E D. Antoine

3 Venice Tower Adriatic Sea 45 N 12.5 E G. Zibordi

4 BATS Sargasso S. 32 N 64.5 W N. Nelson

5 CARIACO Carib.Sea 11 N 65 W F. Muller-Karger

6 CALCOFI California 35 N 125 W G. Mitchell

7 Gulf of Maine USA-Canada 43 N 69 W B. Balch

8 LEO-15 New Jersey 39 N 74 W O. Schofield (?)

9 Benguela South Africa 32.5 S 17.4 E S. Bernard

10 Helgoland North Sea 54 N 7.5 E R. Doerffer

11 Channel English Ch. 50 N 3 W CASIX (?)

12 Sopot Baltic Sea 55.2 N 19 E Wozniak (?)

13 Palmer Antarctic 65 S 65 W B. Prezelin (?)

14 Rapa Nui S.Pacific 23 S 118 W Nobody

15 Concepcion Chile 36.5 S 73 W O. Ulloa

16 Taiwan-Str China 22.5 N 118 E SL Shang

17 Yellow Sea China 35 N 122 E JW Tang



-150° -100° -50° 0° 50° 100° 150°

-50°

0°

50°


Diagnostic SitesFirst pre-selection(2/2)

GlobCOLOUR Diagnostic Data Set locations




Already generated DDS L2 products

MERIS MODIS SeaWiFS Total BOUSSOLE 244 210 311 765 NILU 32 41 47 120 NOMAD 151 107 244 502 Total 427 358 602 1387



Globcolour processor status

Version 1.0: used at ACRI and at UoP to generate DDS

regression tests

Version 2.0: to be released 1st of September

Documents

Page 1GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Resulting Technical Specification