Update on Grid_based activities at ESA

Update on Grid_based activities at ESA

[email protected] [email protected]

(presented by Ivan Petiteville, Guoqing Li)

CEOS WGISS, Boulder, Colorado - September 2008

Summary

G-POD: Grid Processing on Demand @ESA - recent new services

DEGREE: roadmap for Earth Science Grid

GENESI-DR: –Building the federation of European Earth Science Digital Repositories

–Contribution to GEO Work Plan 2009-2011

G-POD is…

a generic GRID-based operational environment where specific data handling applications can be seamlessly plugged into system.

it provides the necessary flexibility for building an application virtual environment with quick accessibility to data, computing resources and results.

From the G-POD web portal the user can easily manage all its tasks.

Recent G-POD results…

Arctic ice on the verge of another all-time low– Following last summer's record minimum ice cover in the Arctic, current observations

from ESA's Envisat satellite suggest that the extent of polar sea-ice may again shrink to a level very close to that of last year.Envisat observations from mid-August depict that a new record of low sea-ice coverage could be reached in a matter of weeks.

– Each mosaic contains approximately 200 Envisat images. – See http://www.esa.int/esaCP/SEMCKX0SAKF_index_0.html

FAIRE service: Webportal

User can easily browse for the required products specifying:

– the geographical area of interest

– the acquisition time, – if required, limiting the search

for a given mode or pass.Data:

– 100% ASAR Medium Resolution products (WSM, IMM)

– Samples of ASAR Full Resolution products

ASAR Med Res is useful for flood monitoring (75m, high ENL, NRT)

NRT crisis data and NRT archive data

Output:– Calibrated, Coregistered,

orthorectied product– Change detection map

FAIRE service: achievements and future developments

Main 2007 Achievements– Geometric validation of FAIRE products completed– System now in the hands of application experts: DLR ZKI,

EC/JRC, SERTIT, UNOSAT– The benefit of the system has been assessed against User

criteria: availability, timeliness, reliability, fitness for purpose and affordability

– Operational where FAIRE has been used: +/- 10 activations 2008: Develop Faire V2

– Make use of updated SAR Toolboxes (e.g. NEST)– Ensure long-term maintenance– Improved service based on user feedback and lessons learnt

Longer term: R&D to enhance SAR data access for crisis mapping

– R&D action on systematic flood monitoring– Methodological issues still open, for instance: what

reference data for change detection? – Expansion to other types of hazard– Expansion to other NRT applications of SAR (e.g. illicit crop

monitoring)

FAIRE Results: Charter Call 203, Cyclone in Myanmar

Faire output Crisis map provided by SERTIT

FAIRE Results: Charter Call 218, Floods in Nepal

Date series: 22 June 2008, 27 July 2008, 31 August 2008

Pre-Flood waters on 22 June 2008 Flood waters on 27 July 2008

Flood waters on 31 August 2008

“The flood map under reference (31st August 2008) is nice and the extend it has covered is

impressive” ICIMOD

FAIRE Results: Hurricane Ike, 14 September 2008

GPOD: new services underdevelopment Interferometric

application for subsidence monitoring (cooperation with IREA CNR, Naples, Italy)

Systematic monitoring of predefined areas of the Antarctica ice shelf

DEGREE IST 2006- 034619

Dissemination and Exploitation of GRids in Earth sciencE

Earth Earth Science Science

CommunitCommunityy

RequirementsRequirementsRequirementsRequirements

TTechnologiesechnologiesTTechnologiesechnologies

FeedbackFeedbackFeedbackFeedback

ApplicationsApplicationsApplicationsApplications

DisseminationDisseminationDisseminationDissemination

ES ES VOsVOs

EGEE & EGEE & various various GRIDSGRIDS

Build a bridge linking the ES

and Grid communities

The DEGREE team:IISAS, Slovakia (Coordinator)CNRS, France KNMI, The Netherlands UNINE, SwitzerlandCRS4, ItalySCAI, Germany GCRAS, RussiaESA-ESRIN, ItalyCGG Veritas, FranceDutch Space, The Netherlands

Coordinator: Ladislav Hluchy (IISAS)

DEGREE IST 2005-034619

Objectives / Challenges / Vision

• Strategic Objectives– Disseminate, promote uptake of Grid in wider ES community,

integrate newcomers– Reduce the gap between ES Users and Grid Technology– Explain and convince ES users of Grid benefits and capability to

tackle new and complex problems• Challenges

– Promote integration of ES tools and applications in EGEE and other Grid environments

– Evaluating complementary technologies that are under development

– Collect and analyze the requirements of ES complex applications– Exploit GRID as infrastructure for international e-collaboration

development (sharing of data, tools, models…)• Vision

– A community wide Earth Science Knowledge Infrastructure


A roadmap for a dedicated ES Grid based platform

Sho

rt termLon

g term

Starting point: • successful deployment of ES applications and demonstration of the

viability and benefits of using the Grid • middleware gaps identified (requirements analysed with Grid developers)• need for increased awareness of Grid benefits in the ES community• increased specialised participation in Grid developments

Considering the vision and plans of three major ES community programmes/initiatives:

GEOSS SEIS INSPIRE

today



Long term

Short term• To build the ES-Grid community and to significantly reduce the gaps

towards a basic Grid infrastructure suitable for the needs of ES community

• Porting of applications from different ES disciplines throughout a wide community, initially for dissemination and moving gradually to more applications that produce useful results

1-3 years

Sho

rt term



Long term

Medium term• Solve outstanding blocking issues for ES applications deployment on

Grid, including secure and easy access to data and quality of service.• Porting of applications from different ES disciplines throughout a wide

community, publication of new scientific results which could not have been achieved without ES Grid infrastructures.

• Facilitate access for everyday ES users to Grid.

2-5 years

Sho

rt term



Long term

Long termestablish a standard ES Grid platform fully integrating the major emerging ES standards (OGC, ISO 19115, OAIS, etc.). This ES Grid Platform shall be able to provide ES users with easy and seamless access to distributed and diverse data and to support them in enlarging collaboration, creativity and productivity.

4-8 years

Sho

rt term



Policy makers ES Scientists

INFRA-2007-1.2.1 • Scientific Digital RepositoriesDuration • January 1, 2008 – December 31, 2009Proposal no. • 212073 Total EC funding • 4.4

M€

INFRA-2007-1.2.1 • Scientific Digital RepositoriesDuration • January 1, 2008 – December 31, 2009Proposal no. • 212073 Total EC funding • 4.4

M€

For scientists to access and share seamlessly

observations of the Earth System and derived

information and knowledge

Digital Repositories

Ground European Network

for Earth Science Interoperations

Repository

Repository

On Demand product

On Demand product

GENESI-DR collaboration platform

GENESI-DR collaboration platform

GENESI-DR projects goals

• To provide a base for (establishing) a world-wide e-infrastructure for Earth Science repositories

• To provide reliable, easy, effective, and operational access to a variety of data sources (space and ground)

• To harmonise operations at key Earth Science data repositories

• To demonstrate effective curation and prepare the frame for long term preservation

• To validate capabilities to access distributed repositories for involving new communities, including education…

• To integrate new scientific and technological derived paradigms

GENESI-DR challenges

• Have single access to space, airborne and in situ sensors data

• Combine data with user tools and models

• Overcome DR specific data policies • Establish a world-wide leadership• Capitalise on high-speed connectivity

for user data access / exchange

Complex web of sensors

Fram

e

Information System

interdisciplinary processes

Atm

osph

ere

Oce

an

Biosphere

CryosphereAtm

osph

ere

Oce

an

Biosphere

CryosphereAtm

osph

ere

Oce

an

Biosphere

Cryosphere

Coupled and

With GENESI-DR Earth Scientists could:

GENESI-DR e-infrastructure to be integrated by 2009

GENESI-DR e-infrastructure to be integrated by 2009

Architecture Highlights

• Exploit and strengthen best practice in distributed data archiving, discovery, access and processing

• A set of services for indexing, searching, sharing and storing very large spatial data sets.

• Publication and of metadata and syndication of data between peers automatically

• Discovery of data relevant to an application, establishment of data usage / access rights

• Processing on user demand in open Grid environment

– Pre-processing (avoid bulk data transfer)– Move algorithm to data

Standard interfaces for data access

•Simple interfaces to complex standards– Import / export support for ISO (ISO 19115, 19139) and

OGC standards for geospatial metadata and processing (WPS)

– Conformance with basis of emerging INSPIRE Implementing Rules for Metadata and Discovery

– Atom+GeoRSS+Dublin Core for wider use and OpenSearch protocol with Geo extensions

– CSW interfaces via GeoNetwork (FAO)

Relation with external bodies (1)

• Relations with EC: – GENESI-DR attendance of “e-infrastructure concertation meeting”

(Jun 08) • standardisation across e-infrastructures need to be focused on

communities – GENESI-DR can play coordination role in Earth Science (for data access /exchange)

• , Relations with EC funded projects– Contact under formalisation: EGEE (European Grid infrastructure),

SeaDataNet (Marine ground data infrastructure), METAFOR (Climate Digital Repository), CYCLOPS (disaster management)

– Interest in EC (GEO related projects) in accessing GRID infrastructures

– New contacts established: EuAsiaGrid (includes disaster management grid), SEEGrid (includes seismic grid), SIB-ESS-C (siberial ES cluster) and Irkutsk Regional Information System for Environmental Protection

• Possible access to other repositories and applications

Relation with external bodies (2)

• Open Grid Forum and Open Geospatial Consortium– Meetings in Frascati (Jan 08), Vienna (EGU Apr 08), Barcelona

(OGF#23 Jun 08) on possible cooperation OGF-OGC (Standardisation of WPS - Web Processing Service)

– Contribute to workshop on Digital Repositories at OGF#23

• Relation with other ESA activities (GMES…)– GISHEO (funded via PECS in Romania, 3y, ~700KEuro):

development of EO education Grid – consider GENESI-DR environment in line with EDUSPACE

– GLOBCOVER, GLOBCOLOUR, GLOBCARBON, GLOBMODEL (ESA funded projects) with multi-year global environmental products generated …

– Preservation (and Data Policy) activities to be synchronised with ESA-GSCB established LTDP working group

GENESI-DR and GEOSS

• GENESI-DR is involved in DA-09-02a GEO Task that aims at establishing “alliances among existing data centers for effective management of large volumes and diverse types of Earth observation data”

• Participation to other tasks of the GEO Work Plan 2009-2011 is being discussed in the GENESI-DR partnership:

– AR-09-01a Enabling Deployment of a GEOSS Architecture – DA-09-01c - Data, metadata and Products Harmonisation, and in the following SBA tasks– DI-06-09 - Use of Satellites for Risk Management (GPOD

FAIRE)– AG-06-02 - Data Utilisation in Fishery and Aquaculture.

Formal relation between GENESI-DR and GEOSS:

Thank you !

Documents

Update on Grid_based activities at ESA