EMI INFSO-RI-261611 JRA1.5-Infra-Cloud Computing Task Force Shahbaz Memon (JUELICH)

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

JRA1.5-Infra-Cloud Computing Task Force

Shahbaz Memon (JUELICH)

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

2

• Cloud developments at NIIFI• WnoDeS and OCCI• Cloud Projects StratusLab and Venus-C• UNICORE Cloud Status• ARGUS-EE• Hydra Cloud Integration (Cancelled)• eScience Applications and Next Plans

EMI Cloud Computing Task Force, 22-24 Nov 2010, Prague

Session Plan

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• „The StratusLab Toolkit will integrate cloud and virtualization technologies and services within grid sites and enrich existing computing infrastructures with “Infrastructure as a Service” (IaaS) provisioning paradigms. “

• Current Activities– Operation of a Grid site in a commercial public cloud– Execution of Grid sites on private/hybrid clouds– Provision of infrastructure cloud interfaces to grid

sites

3

StratusLab


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Grid Services on the Cloud• Customized Environments for Virtual

Organizations• Customized Environments for Users• Sharing of Dataset and Machine Images• Provision of Common Appliances• VO/User Services on the Cloud• Deployment of a Group of Machines• Hybrid Infrastructures

4

StratusLab: Usecases


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

5

Architecture

Grid Resource Center

StratusLabDistribution

PrivateCloud

Cloud API

Grid Services

PublicClouds

users

[C.Loomis. StratusLab]


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI Cloud Computing Task Force, 22-24 Nov 2010, Prague 6

• Combining Grid and Cloud Technologies– Easier deployment, maintenance of sites for

administrators– More flexible, capable, and dynamic

infrastructure for users• Challenge for Cloud Community

– Meet users’ and admins’ expectations– Building more trust between actors

StratusLab - Conclusion

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

7

Venus-C

VENUS-C will develop and deploy a Cloud Computing service for research and industry communities in Europe by offering an industrial-quality service-oriented platform based on virtualisation technologies.

VENUS-C aims to facilitate and empower these communities through the easy deployment of end-user services, in order to make e-Infrastructures more widely valuable across a spectrum of research fields without the complexity of existing grids and high up-front costs.

Cloud computing eScienceScientific cloud

Key concepts

Virtual multidisciplinary EnviroNments USing Cloud infrastructures

[Ake Edlund. Venus-C]


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

8

Venus-C: Goals

• Bring early adopters to the cloud – from start• Attract new users during the project – through an open

call• Give users multiple choices - Azure, OpenNebula and

Eucalyptus• Contribute actively to DCI Collaborative Roadmap

– EGI-InSPIRE, EMI, IGE, StratusLab, EDGI, VENUS-C• Contribute through ECEE to “interoperability-today”

effort


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI Cloud Computing Task Force, 22-24 Nov 2010, Prague 9[Ake Edlund. Venus-C]

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

10

Supporting Multi-disciplinary Research

Biomedicine (3): Integrating widely used tools for Bioinformatics, System Biology and Drug Discovery into the VENUS-C infrastructure.

Civil Protection & Emergencies (1): Early fire risk detection, through an application that will run models on the VENUS-C infrastructure, based on multiple data sources.

Civil Engineering (2): Support complex computing tasks on Building Information Management for green constructions and dynamic building structure analysis.

Data for Science (1): Integrating computing through VENUS-C on data repositories. In particular focus will be on Marine Biodiversity through Aquamaps.

[Ake Edlund. Venus-C]


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

11

Venus-C: Conclusions

• A cloud-based e-Infrastructure for eScience, currently missing from Europe’s service portfolio

• Ensuring a leap forward for the European Research Area (ERA)

• Integrating flexible easy-to-use utility services, complementing current computing services like grids & supercomputers

Europe as a hub of excellence in e-Science

• Provisioning, operation and user-testing of an industrial quality, virtualised infrastructure, open to the research & scientific community

• Individual partners benefit, all considering a sustainable model once the project finishes

Sustainable and continuous services of

production quality 24/7

• A scalable solution which would enable SMEs and small research labs to compete with larger organizations

• Know-how produced in the project and then exploitable beyond science

Innovation by exploiting know-how beyond

science (public services, large scale

experimentation, ….)[Ake Edlund. Venus-C]


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

12

• StratusLab– Development at the IaaS tier– Release 1.0 StratusLab toolkit

• OpenNebula, WebMonitor, CLI, Quattor installation• Reference Release

http://www.stratuslab.org/doku.php?id=referenceservices

• Venus-C– Reference Architecture released in the mid of

December-2010

Collaborations




EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Open Source Cloud Platforms

13EMI Cloud Computing Task Force, 22-24 Nov 2010, Prague

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

14

Anatomy of Open Source Cloud: Single Unit

[Tim Jones Anatomy of an open source cloud, IBM Developer Works]


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

15

Anatomy of Open Source Cloud: Infrastructure

[Tim Jones Anatomy of an open source cloud, IBM Developer Works]


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Platforms– Eucalyptus – OpenNebula– Nimbus– Xen Cloud Platform– OpenQRM

16

Open source IaaS Platforms


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

17

• No solutions available yet..• UNICORE server to support scaling

capability– UNICORE as an interface to cloud

middleware• OCCI implementations, may be OpenNebula

– Introduce UNICORE Elastic TSIs to serve resource exhaustion and/or implicit virtualized instance over an infrastructure

UNICORE Cloud Integration


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

18

UNICORE Architecture


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• A-REX, CREAM, U.TSI, U.XNJS, U.EMIXS, EES, Hydra...

• Evaluate WnoDeS• Evaluate integration scenarios with off the

shelf cloud computing components– OpenNebula (StratusLab and Venus-C),

LibVirt, StratusLab toolkit IaaS, Venus-C PaaS distribution

19

EMI and Clouds


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• To support existing Job, Data, and Information management usecases within a cloud infrastructure

• Middleware integration with public / private / hybrid cloud infrastructures

• Security Challenges– Authn / Authz (Every cloud provider

consumes different type of credentials)

20

Scenarios


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Report on Reference Architecture? Timeline defined

• Merge with Virtualization? Better to segregate them

• Need a separate mailing list? Not required • Do we integrate EMI components with

mulitiple open source providers or decide on any one? Depends on Evaluation and Requirements

• Regular telecons? Bi-weekly from January21

Open Questions and To Dos


EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Thank you

22EMI Cloud Computing Task Force, 22-24 Nov 2010, Prague

EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611

Documents

EMI INFSO-RI-261611 JRA1.5-Infra-Cloud Computing Task Force Shahbaz Memon (JUELICH)