Towards a National Grid Infrastructure -
The European View
Dieter Kranzlmüller(GUP, Joh. Kepler University Linz)
Chile Structuring Workshop
www.eu-egi.eu 2
Stakeholders in Europe
NationalFunding Agency 1
NationalFunding Agency 2
NationalFunding Agency n
…
International Scientific and Research Collaboration
European Commission
onGrids
Chile Structuring Workshop
www.eu-egi.eu 3
Stakeholders in Europe
European Commission
NationalFunding Agency 1
NationalFunding Agency 2
NationalFunding Agency n
…
NationalGrid
Project
RegionalGrid
Institution
…LocalGrid
Community
European Grid Projects
International Scientific and Research Collaboration
Chile Structuring Workshop
www.eu-egi.eu 4
Stakeholders in Europe
European Commission
NationalFunding Agency 1
NationalFunding Agency 2
NationalFunding Agency n
…
NationalGrid
Project
RegionalGrid
Institution
…LocalGrid
Community
European Grid Projects
International Scientific and Research Collaboration
Chile Structuring Workshop
www.eu-egi.eu 5
Univ. Linz - March 2008 5
250 Sites48 Countries50,000 CPUs13 PBytes>5000 Users>200 VOs>140,000 Jobs/Day
ArcheologyAstronomyAstrophysicsCivil ProtectionComp. ChemistryEarth SciencesFinanceFusionGeophysicsHigh Energy PhysicsLife SciencesMultimediaMaterial Sciences…
32%
www.eu-egi.eu 6
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
The EGEE Series of Projects
• EGEE– 01 April 2004 – 31 March 2006– 71 Partners, 27 Countries– EC Contribution: 32 Mio €
• EGEE-II– 01 April 2006 – 30 April 2008– 91 Partners (11 JRU – 48 partners), 32 Countries– EC Contribution: 36 Mio €
• EGEE-III– 01 May 2008 – 30 April 2010– 42 Partners (mostly JRUs)– EC Contribution: 32 Mio €
INFRA-2007-1.2.3: e-Science Grid Infrastructures
Indicative budget: 50 Mio €
EGEE-III > 60 %
Chile Structuring Workshop
www.eu-egi.eu 7
The Need for a Permanent, Common Grid Infrastructure
Production Grids are reality today, but …
• Dependency: Some application domains depend on grids already today by using them for production runs
• Protection of Investment: Investment in grids, both from funding organizations and from users, need to be protected
• Perspective: Today’s grid users are grid enthusiasts, tomorrows grid users ask for a longer term perspective
Chile Structuring Workshop
www.eu-egi.eu 8
Stakeholders in Europe
European Commission
NationalFunding Agency 1
NationalFunding Agency 2
NationalFunding Agency n
…
NationalGrid
Project
RegionalGrid
Institution
…LocalGrid
Community
European Grid Projects
International Scientific and Research Collaboration
e
-Inf
rast
ruct
ure
Ref
lect
ion
Gro
up
(e-I
RG
)
Chile Structuring Workshop
www.eu-egi.eu 9
e-IRG Delegates Meeting, London, 12/2005
Recommendation:
“The e-IRG recognises that the current project-based financing model of grids (e.g., EGEE, DEISA) presents continuity and interoperability problems, and that new financing and governance models need to be explored – taking into account the role of national grid initiatives as recommended in the Luxembourg e-IRG meeting.”
Chile Structuring Workshop
www.eu-egi.eu 10
Grids in Europewww.eu-egi.eu
Chile Structuring Workshop
www.eu-egi.eu 11
Characteristics of NGIs
Each NGI • … should be a recognized national body
with a single point-of-contact • … should mobilize national funding and resources• … should ensure the operation of a national e-
Infrastructure• … should support user communities (application
independent, and open to new user communities and resource providers)
• … should contribute and adhere to international standards and policies
Available soon: Guidelines for NGIs – Prepared by EGI_DS
Chile Structuring Workshop
www.eu-egi.eu 12
e-IRG Recommendations on Sustainable e-Infrastructures
I: governments and the Commission should develop policies and mechanisms to encourage increased investment in a more coherent and interoperable way across Europe
II: the existing e-Infrastructure projects must be superseded by integrated sustainable services at national and European levels
III: e-Infrastructures must be application-neutral and open to all user communities and resource providers. National funding agencies should be encouraged to fund multi-disciplinary and inclusive infrastructures rather than disciplinary-specific alternatives
Chile Structuring Workshop
www.eu-egi.eu 13
e-IRG Recommendations on Sustainable e-Infrastructures
IV: e-Infrastructures must inter-operate and adopt international standard services and protocols in order to qualify for funding
V: the Commission should, within the seventh Framework Programme, develop a pan-European e-Infrastructure which explicitly encourages the further integration of national e-Infrastructure initiatives
e-IRG Task Force on Sustainable e-Infrastructures (SeI)http://www.e-irg.org/publ/2006-Report_e-IRG_TF-SEI.pdf
Chile Structuring Workshop
www.eu-egi.eu 14
European Commission
“…for Grids we would like to see the move towards
long-term sustainable initiatives less dependent upon
EU-funded project cycles”
Viviane Reding, Commissioner, European Commission, at the EGEE’06 Conference, September 25, 2006
Chile Structuring Workshop
www.eu-egi.eu 15
Slide courtesy of Kyriakos Baxevanidis, EC
Broad scale test-beds
Production quality
facilities
Sustainable
e-Infrastructures
(utility model)
Sustainable grid/data-based e-
Infrastructures (utility model)
Towards sustainable grid-empowered e-Infrastructures
Chile Structuring Workshop
www.eu-egi.eu 16
Slide courtesy of Kyriakos Baxevanidis, EC
The first e-Infrastructure Call in FP7
Year 2009Year 2009Year 2008Year 2008Year 2007Year 2007
1. e-Science Grid Infrastructures
2. Scientific Digital Repositories
3. Deployment of e-Infrastructures for new Scientific Communities
4. New Research Infrastructures – Design studies
5. New Research Infrastructures – Preparatory phase
6. Support measures (studies, policy initiatives,international co-operation,…)
Publication: early 2007Closure: spring 2007
•Support conceptual design studies for new RI (or major upgrades of existing ones) of clear European dimension and interest; such studies will help to assess technical and financial feasibility of proposed new RI
•Action should also foster emergence of new organisational models to consolidate a sustainable approach to e-Infrastructures, in particular in the domain of grids and data repositories
•New service provisioning schemes to be more neutral and open to all user communities and resource providers
Chile Structuring Workshop
www.eu-egi.eu 17
Evolution
Testbeds Utility ServiceRoutine Usage
National
Global
European e-Infrastructure
Chile Structuring Workshop
www.eu-egi.eu 18
Stakeholders in Europe
European Commission
NationalFunding Agency 1
NationalFunding Agency 2
NationalFunding Agency n
…
NationalGrid
Project
RegionalGrid
Institution
…LocalGrid
Community
European Grid Projects
International Scientific and Research Collaboration
e
-Inf
rast
ruct
ure
Ref
lect
ion
Gro
up
(e-I
RG
) NationalGrid
Initiative 1
NationalGrid
Initiative 2
…National
GridInitiative n
Chile Structuring Workshop
www.eu-egi.eu 19
European Grid Initiative
Goal:• Ensure the long-term sustainability of grid infrastructures
in Europe by establishing a new federated model bringing together NGIs to build the EGI Organization
Objectives:• Ensure the long-term sustainability of the European e-
infrastructure• Coordinate the integration and interaction between
National Grid Infrastructures• Operate the European level of the production Grid
infrastructure for a wide range of scientific disciplines to link National Grid Infrastructures
Chile Structuring Workshop
www.eu-egi.eu 20
• Ensure the long-term sustainability of the European e-infrastructure• Coordinate the integration and interaction between National Grid
Infrastructures• Operate the European level of the production Grid infrastructure for a wide
range of scientific disciplines to link National Grid Infrastructures• Provide global services and support that complement and/or coordinate
national services (Authentication, VO-support, security, etc);• Coordinate middleware development and standardization to enhance the
infrastructure by soliciting targeted developments from leading EU and National Grid middleware development projects;
• Advise National and European Funding Agencies in establishing their programmes for future software developments based on agreed user needs and development standards;
• Integrate, test, validate and package software from leading Grid middleware development projects and make it widely available;
• Provide documentation and training material for the middleware and operations. (NGIs may wish to make the material available in turn in their local language);
• Take into account developments made by national e-science projects which were aimed at supporting diverse communities.
• Link the European infrastructure with similar infrastructures elsewhere;• Promote Grid interface standards based on practical experience gained from
Grid operations and middleware integration activities, in consultation with relevant standards organizations;
• Collaborate closely with industry as technology and service providers, as well as Grid users, to promote the rapid and successful uptake of Grid technology by European industry.
EGI Vision Paper
http://www.eu-egi.eu/vision.pdf
Chile Structuring Workshop
www.eu-egi.eu 21
38 European NGIs
+ Asia, US, Latin America
+ OGF-Europe
+ PRACE
+ …
Chile Structuring Workshop
www.eu-egi.eu 22
EGI Design Study
Project funded by the European Commission within FP7-INFRASTRUCTURES-2007-1, 1.2.1 Design Studies
Goal:• Conceptual setup and operation of a new organizational
model of a sustainable pan-European grid infrastructure
EGI Organization:• Coordination and operation of a common multi-national,
multi-disciplinary Grid infrastructure– To enable and support international Grid-based collaboration– To provide support and added value to NGIs– To liaise with corresponding infrastructures outside Europe
Chile Structuring Workshop
www.eu-egi.eu 23
EGI Webpage www.eu-egi.eu
Chile Structuring Workshop
www.eu-egi.eu 24
EGI Design Study
Project funded by the European Commission within FP7-INFRASTRUCTURES-2007-1, 1.2.1 Design Studies
Goal:• Conceptual setup and operation of a new organizational
model of a sustainable pan-European grid infrastructure• Consortium: 9 Partners EGI Preparation Team
Chile Structuring Workshop
www.eu-egi.eu 25
EGI Preparation Team
Members:• Johannes Kepler Universität Linz (GUP)• Greek Research and Technology Network S.A. (GRNET)• Istituto Nazionale di Fisica Nucleare (INFN)• CSC - Scientific Computing Ltd. (CSC)• CESNET, z.s.p.o. (CESNET)• European Organization for Nuclear Research (CERN)• Verein zur Förderung eines Deutschen
Forschungsnetzes - DFN-Verein (DFN)• Science & Technology Facilities Council (STFC)• Centre National de la Recherche Scientifique(CNRS)
Chile Structuring Workshop
www.eu-egi.eu 26
EGI Design Study
Project funded by the European Commission within FP7-INFRASTRUCTURES-2007-1, 1.2.1 Design Studies
Goal:• Conceptual setup and operation of a new organizational
model of a sustainable pan-European grid infrastructure• Consortium: 9 Partners EGI Preparation Team• NGI Representatives EGI Advisory Board
Chile Structuring Workshop
www.eu-egi.eu 27
EGI Advisory Board
Chile Structuring Workshop
www.eu-egi.eu 28
EGI Design Study
Project funded by the European Commission within FP7-INFRASTRUCTURES-2007-1, 1.2.1 Design Studies
Goal:• Conceptual setup and operation of a new organizational
model of a sustainable pan-European grid infrastructure• Consortium: 9 Partners EGI Preparation Team• NGI Representatives EGI Advisory Board
• Person months: ~300• Duration: 1 Sept 2007 – 30 Nov 2009 (27 Months)
Chile Structuring Workshop
www.eu-egi.eu 29
Work Distribution
• WP2: EGI Requirements Consolidation(Fotis Karayannis, GRNET)
• WP3: EGI functionality definition(Laura Perini, INFN)
• WP4: Study of EGI legal and organisational options(Beatrice Merlin, CNRS)
• WP5: Establishment of EGI(Jürgen Knobloch, CERN)
• WP6: EGI Promotion and Links with Other Initiatives(Per Öster, CSC)
Chile Structuring Workshop
www.eu-egi.eu 30
EGI Knowledge Base - Main
NGI Representatives to provide their
input and update their local information
http://knowledge.eu-egi.eu
Chile Structuring Workshop
www.eu-egi.eu 31
EGI Knowledge Base Austria
Chile Structuring Workshop
www.eu-egi.eu 32
EGI Knowledge Base
Chile Structuring Workshop
www.eu-egi.eu 33
EGI Functionality Overview
• Management, Outreach & Dissemination, Representation of European Grid Efforts, Industry take-up
• Operations & Resource Provisioning & Security
• Middleware Coordination: Build & Test, Component Selection & Validation & Deployment, Standardization & Policies
• Application Support, User Support & Training
Chile Structuring Workshop
www.eu-egi.eu 34
Industrial Involvement
• Roles in EGI:– Customer using the EGI Grid for daily work– Provider offering services on the EGI Grid
• Actions for Industry:– Define requirements for industry to use EGI– Evaluate and support the definition of the EGI
functionality– Collaborate on the development of business models
for Grids– Develop future services to be provided on EGI, e.g. in
the domain of security
Chile Structuring Workshop
www.eu-egi.eu 35
e-IRG Recommendations on Industrial Uptake
• Recommendation 1: Coordination between EC funded projects with business related objectives has to be improved.
• Recommendation 2: To aid entrepreneurship, public organisations employing researchers should be encouraged to define employment conditions that permit individuals to continue their research while developing business opportunities in parallel.
• Recommendation 3: Pre-commercial procurement can be geared to contribute directly to the development of important new lead markets for innovative products and services and thereby to the creation of growth and jobs in Europe.
Executive Summary of e-IRG Open Workshop,Heidelberg, Germany, 19-20 April 2007
http://e-irg.eu/workshopGermany/summary.pdf
e-IRG Workshop April 20 200736 High Performance Computing Center Stuttgart
Michael M. Resch
Science or Industry
• Goal– Breakthrough or best solution
• Purpose of HPC– Object of investigation or tool
• Technical Issues– Interesting or irrelevant
• Legal issues– Irrelevant or basis of business
• Economic issues– Relevant or key to success
• Mode of collaboration– Cooperation or competition
from e-IRG Workshop, Heidelberg, 2007
e-IRG Workshop April 20 200737 High Performance Computing Center Stuttgart
Michael M. Resch
Recomm. for HPC – Grids - Industry
1. It is necessary to identify mutual interests of HPC@science and HPC@industry
2. It is necessary to identify a working business model by addressing the legal issues and economic issues of a mixed usage model
3. It is necessary to identify the required technical solutions to deliver the requested high level service and not CPU cycles
from e-IRG Workshop, Heidelberg, 2007
Chile Structuring Workshop
www.eu-egi.eu 38
EGI – European Grid Initiative
• EGI Grid Infrastructure should – be a large-scale, production Grid infrastructure – built on national grids that interoperate seamlessly at
many levels, – offering reliable and predictable services to a wide
range of applications
• The Future EGI Organization represents the “Glue” between various grid communities in Europe and beyond
• EGI_DS defines required mechanisms and functionalities of the EGI Organization