Embed Size (px)
Citation preview
eScience in France
Vincent Breton September 30th 2014
Plan-‐e mee:ng
09:14:54 1
Credits: R. David, F. Desprez, S. Gervois, L. Gouarin, D. Margery, J. Pansanel, G. Romier
There is no na:onal e-‐science centre in France
• HPC – E-‐infrastructure founding and interna:onal representa:on in PRACE : GENCI
– Engineering resources at na:onal TIer1 centers: (IDRIS, CINES, TGCC) and Maison de la Simula:on
• Grid and cloud compu:ng – E-‐infrastructure founding in research communi:es
– E-‐science siloed in research communi:es
09:14:55 2
French landscape: RENATER educa:on and research network
09:14:55 3
Bridging e-‐infrastructure to science
10:29:42 4
Grid and cloud produc:on infrastructure (integrated in EGI)
Grid and cloud infrastructure for research in computer sciences
HPC pyramid (Tier-‐0 in PRACE)
France Grilles Grid5000 Groupe Calcul
e-‐science
e-‐Infrastructure layer
GENCI
Training in scientific computing
The French working group Groupe Calcul has been funded by the CNRS and deals with organizing conferences, meetings, seminars on various aspects of scientific computing.
Its primary purpose is to facilitate improved interactions between people working in scientific or high-performance computing.
September 29, 2014
Training in scientific computing
This is one of the major players in training in the field of scientific computing in France: more than 20 schools or seminars organized in 5 years.
The Groupe Calcul also organizes an annual meeting of the Regional Computing Centres in France. It will be the seventh time in October.
September 29, 2014
For more information
Main website (in French) http://calcul.math.cnrs.fr
Mailing-list (1400 subscribers) https://listes.mathrice.fr/math.cnrs.fr/info/calcul
September 29, 2014
Together with France Grilles
" Two scientific conferences organized in common with France Grilles
http://mesogrilles2012.sciencesconf.org/ http://succes2013.sciencesconf.org/
Next to come in 2015 !
September 29, 2014
GRID’5000 • Testbed for research on distributed systems
• Born from the observation that we need a better and larger testbed • High Performance Computing, Grids, Peer-to-peer systems, Cloud computing • A complete access to the nodes’ hardware in an exclusive mode (from one node to the whole infrastructure): Hardware as a service
• RIaaS : Real Infrastructure as a Service ! ? • History, a community effort
• 2003: Project started (ACI GRID) • 2005: Opened to users
• Funding • INRIA, CNRS, and many local entities (regions, universities)
• One rule: only for research on distributed systems • → no production usage • Free nodes during daytime to prepare experiments • Large-scale experiments during nights and week-ends (no long jobs)
30/09/14 - 9
Current Status (Sept. 2014 data) • 10 sites (1 outside France) • Dedicated 10 Gbps backbone provided by Renater (french NREN) • 24 clusters • 1006 nodes • 8014 cores • Diverse technologies
• Intel (65%), AMD (35%) • CPUs from one to 12 cores • Ethernet 1G, 10G, • Infiniband {S, D, Q}DR • Two GPU clusters • 2 Xeon Phi • 2 data clusters (3-5 disks/node)
• More than 500 users per year • Hardware renewed regularly
30/09/14 - 10
Grid’5000 Mission
Support high quality, reproducible experiments on a distributed system testbed
Two areas of work
30/09/14 - 11
• Improve trustworthiness • Testbed description • Experiment description • Control of experimental conditions • Automate experiments • Monitoring & measurement
• Improve scope and scale • Handle large number of nodes • Automate experiments • Handle failures • Monitoring and measurements
Both goals raise similar challenges
GRID’5000 Software Stack
• Resource management: OAR
• System reconfiguration: Kadeploy
• Network isolation: KaVLAN
• Monitoring: Ganglia, Kaspied, Kwapi
• Putting all together GRID’5000 API
30/09/14 - 12
An experiment over Grid’5000 • Description and verification of the environment • Reconfiguring the testbed to meet experimental needs • Monitoring experiments, extracting and analyzing data • Improving control and description of experiments
Some recent experiments over Grid’5000 • Energy monitoring and management
• Evaluation of Green Strategies for Energy-Aware Framework in Large Scale Distributed Systems
• Evaluation of different wattmeters • Estimation of energy consumption with or without application expertise
• Cloud Computing and virtualization • Sky computing between US and France using Hadoop • Virtual machines deployment and migration (up to 10240 VMs) • Experiments using major CloudKits (Nimbus, OpenStack) and VM stacks (Xen, kvm)
• High Performance Computing • Replay of Curie traces for resource management systems over an emulated environment • Comparison of component based approaches and MPI/threads applications
• Big data management • Optimization of MapReduce frameworks with high performance data management systems • High performance data movements over multicore machines validated with a climate
simulation application
30/09/14 - 13
Conclusion and Open Challenges • Computer-Science is also an experimental science • There are different and complementary approaches for doing experiments in
computer-science • Computer-science is not at the same level than other sciences • But things are improving …
• GRiD’5000: a test-bed for experimentation on distributed systems with a unique combination of features • Hardware-as-a-Service cloud
• redeployment of operating system on the bare hardware by users
• Access to various technologies (CPUs, high performance networks, etc.) • Networking: dedicated backbone, monitoring, isolation • Programmable through an API • Energy consumption monitoring
• Useful platform • More than 750 publications with Grid’5000 in their tag (HAL) • Between 500 and 600 users per year since 2006
30/09/14 - 14
• Is a Scien:fic Interest Group… – Created in 2010 by 8 partners: CEA, CNRS,CPU, INRA, INRIA, INSERM, MESR, RENATER…
– To steer up and coordinate the na:onal strategy in the fields of grids and clouds
• Vision: – Build and operate a na:onal distributed compu:ng infrastructure open to all sciences and to developing countries
• France Grilles does not own the resources – Resources owned by user communi:es
• France Grilles provides a framework and a por]olio of services – To share resources, exper:se and know how – To promote innova:on and ini:a:ves – To foster collabora:on at na:onal and interna:onal levels
– To reach out to the long tail of users
France-‐Grilles backbone: LCG-‐France
France-‐Grilles spine: CC-‐IN2P3
Resources integrated in EGI
5 1 1
218 54
9 1 5 9 11 15 13 11
755 99 50
9 23
1
10
100
1000
Chim
ie
Mathé
ma:
ques
Sciences de l'H
omme
et Société
Inform
a:qu
e Calcul
parallèle ou distribu
é et
partagé
Inform
a:qu
e autre
Planète et Univers
Astroph
ysique
Planète et Univers
Océan, A
tmosph
ère
Planète et Univers
Sciences de la Terre
Sciences de
l'enviro
nnem
ent
Sciences de l'ingén
ieur
Sciences du Vivant Bio-‐
Inform
a:qu
e, Biologie
Systém
ique
Sciences du Vivant
Ingénierie bioméd
icale
Sciences du Vivant
autre
Physique
des Hautes
Energies -‐ Expé
rien
ce
Physique
Nucléaire
Expé
rimen
tale et
théo
riqu
e
Physique
Physique
Astroph
ysique
Physique
autre
Over 1500 scien0fic publica0ons june 2010 – April 2014
Conclusion
• eScience has a complex paeern in France – Many actors
– Ac:ve collabora:on • Future: towards a global data infrastructure
10:00:16 20
