Www.eu-eela.eu E-science grid facility for Europe and Latin America EELA-2 Achievements and Impact so far Towards a Latin American Grid Initiative (LGI)

www.eu-eela.eu

E-science grid facility forEurope and Latin America

EELA-2 Achievements and Impact so farTowards a Latin American Grid Initiative (LGI)

Philippe Gavillet (Deputy Project Coordinator) on behalf of EELA-2

Contributions from B.Marechal (PC), R.Barbera (TC), F.Brasileiro (JRA1 AM)

Barcelona (Spain) – EGEE’09 – 21-25.09.2009 2www.eu-eela.eu

Outline

• EELA-2 in a nutshell

• State of the Art – A few highlights

• The EELA-2 model for long-term sustainability

• Outlook


EELA-2 objectives

• Provide an empowered Grid Facility with versatile services fulfilling application requirements

• Ensure production quality services• Expand the current EELA e-Infrastructure• Look for new communities• Ensure the long term sustainability of the e-

Infrastructure beyond the term of the project


Project in numbers

EELA-2 (CP-CSA under FP7)

• E-science grid facility for Europe and Latin America• EC support: 2.1 M€• CETA-CIEMAT extra funds: 0.3 M€• Currently 16 Countries (11 in LA)• 2 new countries (Panama & Uruguay)• 1 International Organisation (CLARA)• Currently 78 Members (62 in LA)• 32 Institutions joining (31 in LA)• Currently 14 JRUs (9 in LA)• 4 new JRUs (3 in LA)


Countries and Resources Centres


NA2: Dissemination & Training (1/2)

Much more training effort delivered than pledged


NA2: Dissemination & Training (2/2)Status vs Roadmap

1


NA3: Application Support (1/3)

The EELA-2 “gridification universe”

Barcelona (Spain) – EGEE’09 – 21-25.09.2009 9www.eu-eela.eu 9

2

NA3: Application Support (2/3)

27 apps using the EELA-2 Grid facility

19 papers published


NA3: Application Support (3/3) Some examples…

www.eu-eela.euwww.eu-eela.eu

Water control

NMR tomography Fire growth simulation Heart simulator


• “Expanding the current e-Infrastructure to consist of 41 Resource Centres (13 in Europe and 28 in Latin America) mobilising about 3000 computing cores and more than 700 Terabytes of storage space, at start of the project and to further grow by 20 % in computing and 15 % in storage over the duration of the project.”

• “Providing the full set of Infrastructure and Application-oriented Grid Services needed by all types of applications in their scientific environment.”

The current infrastructure has 22 Resource The current infrastructure has 22 Resource Centres (11 in Europe and 11 in Latin Centres (11 in Europe and 11 in Latin America) mobilising about 5800 computing America) mobilising about 5800 computing cores and more than 192 Terabytes of cores and more than 192 Terabytes of storage.storage.

The current infrastructure has 22 Resource The current infrastructure has 22 Resource Centres (11 in Europe and 11 in Latin Centres (11 in Europe and 11 in Latin America) mobilising about 5800 computing America) mobilising about 5800 computing cores and more than 192 Terabytes of cores and more than 192 Terabytes of storage.storage.

Two Core Service Centres deployed (EU and LA), Two Core Service Centres deployed (EU and LA), providing a full set of Grid Services (information providing a full set of Grid Services (information systems, data and metadata catalogues, etc.) and systems, data and metadata catalogues, etc.) and the services developed by JRA1.the services developed by JRA1.

Two Core Service Centres deployed (EU and LA), Two Core Service Centres deployed (EU and LA), providing a full set of Grid Services (information providing a full set of Grid Services (information systems, data and metadata catalogues, etc.) and systems, data and metadata catalogues, etc.) and the services developed by JRA1.the services developed by JRA1.

SA1: Grid Infrastructure Service (1/3)


• “Building the support of the e-Infrastructure to provide high reliability services to partnerships spanning the Atlantic Ocean by:”

– “Providing a complete set of Global Services from central EELA-2 Operation and Networking Support Centres (EOC/ENSC).”

– “Targeting the creation of Regional Operation Centres (ROC) in Latin America, covering one or more countries.”

– “Pursuing the aggressive EELA Training and Dissemination programme.”

– “Creating a comprehensive knowledge repository on Grid operation federated with the EGEE ones.”

A full set of ancillary tools was deployed on the EOC, which A full set of ancillary tools was deployed on the EOC, which acts as a central co-ordination point for the infrastructure.acts as a central co-ordination point for the infrastructure.

http://eoc.eu-eela.euhttp://eoc.eu-eela.eu

A full set of ancillary tools was deployed on the EOC, which A full set of ancillary tools was deployed on the EOC, which acts as a central co-ordination point for the infrastructure.acts as a central co-ordination point for the infrastructure.

http://eoc.eu-eela.euhttp://eoc.eu-eela.eu

Four OOD teams were engaged in Operations; they prefigure Four OOD teams were engaged in Operations; they prefigure the support teams of future Latin American ROCs.the support teams of future Latin American ROCs.Four OOD teams were engaged in Operations; they prefigure Four OOD teams were engaged in Operations; they prefigure the support teams of future Latin American ROCs.the support teams of future Latin American ROCs.SA1 contributed trainers and infrastructure support to NA2 SA1 contributed trainers and infrastructure support to NA2 and NA3.and NA3.SA1 contributed trainers and infrastructure support to NA2 SA1 contributed trainers and infrastructure support to NA2 and NA3.and NA3.

As first step, SA1 and EGEE-SA1 agreed on common protocols As first step, SA1 and EGEE-SA1 agreed on common protocols and procedures.and procedures.As first step, SA1 and EGEE-SA1 agreed on common protocols As first step, SA1 and EGEE-SA1 agreed on common protocols and procedures.and procedures.

OOD Teams

SA1: Grid Infrastructure Service (2/3)


SA2: Network Resource Provision (1/2)Integration of Network Support in EOC


SA2: Network Resource Provision (2/2) EELA-2 Network Monitoring


JRA1: Application Services (1/3)

• EELA-2 Software Repository• Data Management

– Digital archives– Secure storage– Cooperative annotation of data

• Job management– VO compliant Virtual Machine environments– Workflow for OurGrid jobs– WatchDog (monitoring and control of job execution on the gLite Worker Node)

• Catalogue and file management– lcg-rec toolkit (recursive version of the lgc-* suite of commands)


JRA1: Infrastructure Services (2/3)

• X.509-enabled OurGrid middleware available– Enables an OurGrid system to use the same PKI and Grid Security

Infrastructure technologies used by gLite systems

• Interoperability between OurGrid and gLite planned for M16 is advancing very well– MoU signed with other projects working on similar goals (EDGeS and SEE-

GRID-SCI)

• Port of gLite to Microsoft Windows platforms– User Interface:

Workload Mngt, File transfer, Data Mngt, Info System

• Support for grid operation– SAGE (Storage Accounting for Grid Environments)– Multiple middleware Resource Centres

gLite and Globus Toolkit gLite and OurGrid


JRA1: Use of new Services (3/3)

A number of applications are either using or being upgraded to use the new Services

Service Applications

GSAF Aîuri – UFRJ (Brazil)

OurGrid DistBlast - UnB (Brazil), BRAMS - UFCG (Brazil) and Cinefilia - INFN (Italy)

watchdog CROSS-Fire – U. of Minho (Portugal)

lcg-rec toolkit

CROSS-Fire – U. of Minho (Portugal)

Service Applications

GSAF gRREEMM - Cubaenergia (Cuba)

Secure Storage

CardioGrid Portal - UBA (Argentina)

watchdog G-InterproScan and G-HMMER - UNIANDES (Colombia)

lcg-rec toolkit

META-Dock - UNAM (Mexico)

Using Being upgraded


Next to EELA-2….


The “global” Grid coverage


The Latin American Grid Initiative (LGI)(http://documents.eu-eela.org/record/1119/files/)

GridLGI is composed of three layers, each of them

including Grid and Network components:• At the lower Institute level:

– The Resource Centre (RC) is the actual site of the Grid computing resources;

– The backbone LAN or MAN network connects the Computing and Storage Elements (CE & SE) at the RC site;

• At the intermediate national level:– The National Grid Initiative (NGI) integrates all Grid

Institutions in the country and federates under its Grid Operation Centre (GOC) all their RCs in a National Grid (GridNGI);

– The underlying National Research & Education Network (NREN) links country-wide the Research and Education Institutes and optimizes / monitors their network activity via its associated Network Operation Centre (NOC);

• At the upper international layer:– The Latin American Grid Initiative (LGI) which brings

together all NGIs and supervises the various GridNGIs operation from its Grid Support Centre (GSC);

– CLARA: it is the WAN network which integrates the Latin American NRENs and supervises the overall RedCLARA network activity by means of its Network Support Centre (NSC).


RC tasks


GOC tasks


GSC tasks


NSC tasks


Training & dissemination tasks


User support tasks


NGI / LGI Management-Governance

A tight liaison with the NREN is strongly suggested/recommended.A tight liaison with the NREN is strongly suggested/recommended.Local conditions permitting, NGI should be incorporated into the NRENLocal conditions permitting, NGI should be incorporated into the NREN


Preparing the “After EELA-2” (1/4)

M06: Definition of the Strategy – WNA1.1: “Strategy towards Long Term Sustainability”

http://documents.eu-eela.org/record/1248/files/

Four lines of actions carried on in parallel

• Establish the conditions of a sustainable Latin

American Grid Infrastructure

– M06: DSA1.3: “The Long-Term Latin American Grid Initiative:

Model & Operation Cost Estimate”– M16: WNA1.2: “Time Profiles for the Staffing of the Latin

American Grid Initiative (LGI)”

http://documents.eu-eela.org/record/1307/files/


Preparing the “After EELA-2” (2/4) Collaboration with Grid-related projects

– Pro-active participation in all Concertation meetings with other EGEE-

related projects at EGEE’07, EGEE’08, ICT’08 (BELIEF Workshop)

– Active contribution to the CHAIN preparation phase Unfortunately we could not fit well our mandatory Services: SA1, SA2, JRA1

activities in the proposal (CSA-type)

Decision to go for a CP-CSA proposal to achieve our main objective: Garantee the

sustainability of the LA part of the e_Infrastructure

CIEMAT in CHAIN leadering WP3: ”Present and emerging needs of trans-

continental scientific communities” addressing VRC support

Wish to collaborate on VRC support with CHAIN, EGI,..

• Common Application database with inputs from regions

• User guide, self-training material, Help desk,..


Preparing the “After EELA-2” (3/4) Interaction with / Support from User Communities

– Over Year-1: New collaborators joining EELA-2

2 Partners (Panama, Uruguay) 36 New Members (> 15

Applications)

– JRU / NGI status In place in: Argentina, Brazil,

Chile, Colombia, Ecuador, France, Ireland, Italy, Mexico, Panama, Peru, Portugal, Spain, Venezuela

Missing: Cuba

– Initiative with eResearch 2020 and Belief of an “e-Infrastructure Survey”


Preparing the “After EELA-2” (4/4) Interaction with / Support from Decision Makers

• Actions– Decision Makers interactions already made:

Panama, Oct. 08 Lima, Nov. 08 Monterrey (Mexico), Oct. 08 Quito (Ecuador), Dec. 08 Bogotá (Colombia), Feb. 09

– In depth discussions (EELA-2 presentations, e-Science status in the country, e-Science Virtual days, prospective,..)

– Production of dissemination materials (Posters, Flyers,..)


Summary & Conclusions

• EELA-2 is well on track to achieve its objectives;• EELA-2 has devised a comprehensive model and

architecture of the Latin American Grid Initiative;• The collaboration with CLARA has been settled and is

becoming effective;• EELA-2 has developed the necessary synergies with

the other Grid projects, User Communities, Decision Makers;

• However, the roof will be solid only if the pillars will be strong so Latin American countries should proceed with the creation of NGIs in close relationships with NRENs.


Thank you very much! …Any Questions ?

Documents

Www.eu-eela.eu E-science grid facility for Europe and Latin America EELA-2 Achievements and Impact so far Towards a Latin American Grid Initiative (LGI)