Upload
truongkhanh
View
223
Download
1
Embed Size (px)
Citation preview
Open Source based C-DAC Cloud Infrastructure &
Services
19 Mar 2014NISG DeitY Industrial Cloud Workshop
New Delhi
Centre for Development of Advanced Computing(C-DAC) Bangalore
19 Mar 2014 1
Dr. Prahlada Rao B.BJoint Director
C-DAC Cloud Services 2 EGov
C-DAC
• CDAC was launched as India’s initiative to Design & Develop HighPerformance Computers Indigenously in 1988
• Premier R&D organization under the Department of Electronics andInformation Technology (DeitY), Govt. of India
• C-DAC is engaged in R&D and Development of PARAM Series of High Performance Machines and also Specialized in :
– Cloud/Grid Computing Technologies– E-Governance Applications– Multilingual , Ubiquitous Computing
– Health Informatics– Embedded, Mobile computing– GIS…– Educational Research: ACTS
• Centre for Development of Advanced Computing : http://www.cdac.in
19 Mar 2014 2C-DAC Cloud Services 2 EGov
Presentation Plan
• About C-DAC• C-DAC Cloud Computing Activities
o C-DAC Cloud Infrastructureo Scientific Cloud Computingo SuMegha-Cloud Lab Kit
• Service Delivery Model• Cloud Disseminations• Conclusions
19 Mar 2014 C-DAC Cloud Services 2 EGov3
Supercomputing Evolution @ C-DAC
• Param 8000
• Param 9000
• Param 10000
• ParamPadma
• GARUDA Grid Comp
• Param Yuva II
• Cloud Comp
• PetaFlop Comp
1 Gbps NKN connectivity
2012
419 Mar 2014 C-DAC Cloud Services 2 EGov
70 + Partners 10000 CPUs –550TF
EGI, CHAIN-REDS
220TB Storage
1500 +Certificates
NKN
19 Mar 2014 5C-DAC Cloud Services 2 EGov
Focus Areas
• PAN C-DAC Cloud Infrastructure• C-DAC Cloud Stack • Enablement of multi-centric applications on C-DAC Cloud
• Cloud Interoperability
19 Mar 2014 6C-DAC Cloud Services 2 EGov
Highlights
• Infrastructure as a service • Cloud Stacks :
– Meghdooth– SuMegha (Scientific Cloud)
• Storage as a Service • Enablement of Multi centric Applications
19 Mar 2014 7C-DAC Cloud Services 2 EGov
HPC as a Service:
MPI/Hadoop Virtual Cluster(VC)Problem Solving Environments
Bangalore:HPC Applications
PAN C-DAC Cloud Initiatives: Overview
Chennai: Meghdooth Cloud Stack
Bangalore: SuMegha: Scientific Cloud Stack
Noida:HMS Deployment
Mohali:Courses
Kolkatta:NE Courses
TVM:HealthcareApplications
Mumbai:DisasterRecovery
Hyderabad:E-learning e-shikshak
Pune:Machine Translation MANTRA
Bangalore: PAN C-DAC Cloud Infrastructure
PSE NGS
PSE SFM
SaaSIaaS StaaS
19 Mar 2014 C-DAC Cloud Services 2 EGov 8
Pan C-DAC Cloud Computing
C-DAC local afforts to setup cloud with limited infrastructure. Cloud Stacks implemented
Based on open source. Eucalyptus, Nimbus, OpenStack as the core engine. Many extensions to meet production requirement.
Monitoring, metering and billing supported. Elasticity based on runtime parameters. Scientific Cloud Implemented (IaaS, SaaS, PaaS). Scientific Cloud Stack-SuMegha Released. Cloud Storage as a Service (CloudVault) Developed SuMegha_Cloud LabKit (to Build Privite Clouds) Developed MeghaSikshak Cloud based eLearning Tool Developed Currently Applications or Enabled on local infrastructure.
19 Mar 20149
C-DAC Cloud Services 2 EGov
Pan C-DAC Cloud Infrastructure• Physical Cloud infrastructure Set up • Host Cloud Middleware Stacks: Meghdoot, Sumegha-Scientific Cloud • Availability of the Cloud Infrastructure for applications.• Based on Meghdoot/SuMegha Cloud Stacks.• Physical Security, Power, Air Conditioning Available.• C-DAC Centres able to access resources for building and experimenting
with cloud applications.• Repository: for Application Snapshots, Tools, and common tools.• Provide tools for repositories, cloud management. • Disaster Recovery, backup for projects.
1019 Mar 2014 C-DAC Cloud Services 2 EGov
PAN C-DAC Cloud: Applications
Pan C-DAC Cloud Computing Supports: Pan C-DAC Cloud infrastructure Operating with: Meghdoot /
Sumegha Stacks. Cloud Enable Applications of Pan C-DAC Cloud Project
o Hospital Information System(HIS) C-DAC, Noidao HIS- Health Care Applications C-DAC, Thiruvananthapuramo Learning Management System (e-Sikshak) C-DAC, Hyderabado MANTRA (MAchiNe assisted TRAnslation tool) C-DAC Puneo Disaster Recovery as a Service (DRaaS) C-DAC Mumbai o eSanjeevani (A web based health care solution) C-DAC Mohali
1119 Mar 2014C-DAC Cloud Services 2 EGov
Highlights
• Infrastructure as a service • Cloud Stacks :
– Meghdooth– SuMegha (Scientific Cloud)
• Storage as a Service • Enablement of Multi centric Applications
19 Mar 2014 13C-DAC Cloud Services 2 EGov
C-DAC Scientific CloudScientific Cloud is a model for on-demand access to a shared pool of HPC resources (e.g.,networks, servers, storage, applications) that can be easily provisioned as and when needed by the researchers and scientists.Benefits of Scientific Cloud• Easy access to large infrastructures• On demand access to HPC resources• Virtual ownership of cloud resources• Ease of deployment• Scientific Cloud provide an alternative for HPC outreach
19 Mar 2014 C-DAC Cloud Services 2 EGov 15
SuMegha: Scientific Cloudwww.sumegha.in
Provides on demand access to Compute, Storage resources and enables parallel environments.Services Supported : Infrastructure as a service(IaaS) Storage as a service(StaaS) Software as a service(SaaS)
Stakeholders Faculty and Students from engineering collegesResearchers
19 Mar 2014 16C-DAC Cloud Services 2 EGov
SuMegha: Scientific Cloudwww.sumegha.in
• Cluster as an Infrastructure Service (IaaS)• Provisioning of on demand HPC clusters• Provisioning of Map reduce clusters to process large data sets • Support for MPI as a tightly-coupled applications category. • Design methodology for providing/supporting installation of MPI
libraries in multiple machines simultaneously• Storage as a Service (StaaS): CloudVault• SaaS:
– Job Submission Portal (C_JSP) , Cloud Access Portals– Monitoring tools
• PaaS: Program development and analysis Tools• Applications as Service (AaaS) PSEs
– PSE for Climate Modelling– PSE’s for Bio Informatics
• Types of applications supported– Sequential & Concurrent Applications– Data Parallel Applications - Map Reduce– Parallel Programs: OpenMP ,Pthreads, MPI
19 Mar 2014C-DAC Cloud Services 2
EGov17
SuMegha StackManagemen
tSCIENTIIFC APPLICATIONS
(Bioinformatics, Climate modeling)
Cloud Middleware (NIMBUS, Openstack Swift)
SLA & Policy Mana-gement
Monitoring
(NAGIOS+ Ganglia)
Accounting, Metering & Billing
Operating System (RHEL)Hypervisors (XEN)
AU
T
H
&
S
E
C
U
R
I
T
Y
)
SaaS(Job submission portal)
PaaS(HADOOP, MPI with High PerfIO)
IaaS (VM, Virtual Clusters , storage)
Interconnect ( Ethernet & infiniband)
Cluster Virtualization (Contextualization)
Scientific Cloud portal and Problem Solving Environments
Computational and Storage resources
18
StaaS(Storage as a Service)
19 Mar 2014 C-DAC Cloud Services 2 EGov
SciCloud Service Offerings
19 Mar 2014
C-DAC Cloud Services 2 EGov
GUIIaaSPaaSSaaS
Cloud Portal( Selects the Resource)
Virtual Machine
(OS, Time, Type)
Virtual Cluster
(No. of nodes, type,Time)
HadoopCluster
(No. of nodes, Time)
MPI Cluster(No. of nodes, Time)
Cloud Vault(StaaS)
Web interface
Small
Medium
Large
PSEs forBioinformatics
Climate Modelling
Job Submission Portal
19
Infrastructure as a Service(IaaS)
• Accessible using Self service portalhttp://scientificcloud.cdacb.in/• Infrastructure as a Service :
Provisioning of virtual machines(VM) & virtual clusters (VC)– VM Specs
• Small, medium, large images• Access to cloud storage(cloud vault)• Access to additional elastic storage, through CIFS
gateway– VC Specs
• Upto 8 nodes in a cluster• Each node: Small, medium, Large• Type of cluster : MPI, HADOOP• Access to cloud storage(cloud vault)• Access to additional elastic storage, through CIFS
gateway
• Secure Access :Secure ssh based access to the VMs & VCs through internet using secure key
VM Size Details Size Detail
Size No. Of Virtual CPU RAM (GB)
Small 1 1
Medium 2 2
Large 4 4
19 Mar 2014 20C-DAC Cloud Services 2 EGov
Job Submission Portal for Virtual Clusters
• Web based interface to provide an easy & user friendly mechanism for job submission to MPI based virtual clusters.
• Enables users to run Sequential & Parallel jobs
Features•Job submission through Torque job scheduler.•Job monitoring to know the Status of submitted jobs•Output/Error files can be Viewed/Downloaded from the Portal•User Management on Virtual Cluster
19 Mar 2014 21C-DAC Cloud Services 2 EGov
Cloud Vault
End-User
Desktop ApplicationWeb Application
Mobile Application
Swift
Storage and Data Center
Physical Storage
OpenStack Object Storage
Cloud Vault is your personal hard drive in thecloud. Store any kind of file like Documents,Music, Videos, Photos, and Virtual MachineImages on CDAC Cloud Vault secure servers.
19 Mar 2014 C-DAC Cloud Services 2 EGov 22
Cloud Vault provides access to storage space for storing and accessing data from anywhere, anytime
http://cloudvault.cdacb.in:8084
Features
– Easy access to Storage : Supports easy accessmechanisms such as Web interface, Linux clients
– File and directory operations : list, upload, download,delete
– Data Sharing: Enables users to share their data withthe other registered users of cloud vault in acontrolled way
– Interfacing with the Virtual Machines & VirtualClusters
– Reliability & Availability: Provides 99.5% up-timeguarantee and 3-way redundancy for data.
– Huge file support : Allows users to upload files fromrange of MBs, GBs (upto 40GB)
– Data isolation and security: Data of one user isisolated & secured from other
Storage as a Service : Cloud Vault
19 Mar 2014 23C-DAC Cloud Services 2 EGov
PSE for SFM• Web based problem
solving environment torun seasonal forecastmodel on MPI basedvirtual cluster provisionedby SuMegha
• Enables on linevisualization of the results
• Facility is provided to runGrADS commands fromthe browser, whichvisualize the necessaryforecast parameters.
19 Mar 2014 24C-DAC Cloud Services 2 EGov
PSE for NGS• Problem Solving Environment for Next Generation
Sequencing (PSE for NGS) • Salient Features:
– Executes the complete workflow Preprocessing(Quality of input data),Indexing, Alignment , post processing, Visualization
– Run-Time Logs for debugging and Directory Trees to navigate easily among the projects or different output files/directories.
– Huge Datasets uploading via Internet (More than 10GB supported).
– Common View window for visualisations and other textual outputs.
– Notification Centre for the user to know which step is going on at any given time and other information like which project is active and which is pending.
19 Mar 2014 28C-DAC Cloud Services 2 EGov
SuMegha Cloud Kit • Packaged software kit to enable the automated deployment of
private scientific clouds on the hardware.• Suitable for academic institutes & research organizations who
want to setup cloud labs.Benefits Automated deployment of cloud setup for Infrastructure as a service Provides web based tools that supports easy and quick access to the
Virtual machines and Virtual HPC clusters Simple and quick installation of all the cloud components. Provides Pre-built OS images with complex parallel environments like MPI
and Map Reduce to cater the HPC needs. Provides Golden images with preloaded HPC applications like Seasonal
Forecast Model ,Cloud Next generation pipeline with the required visualization tools and job submission portal for easy parallel job submission
Enables the users to setup cloud lab without having expertise of system administration.
19 Mar 2014 30C-DAC Cloud Services 2 EGov
SuMegha Software Stack & Supported ImagesComponents : Hypervisor
Xen Cloud Middleware
Nimbus, Openstack Swift Storage
Glusterfs Portal
SuMegha portal Job Submission Portal Cloud Vault Portal
PSE PSE for SFM PSE for NGS Pipeline
Supported Images: Virtual Machines
CentOS (small, large, medium)
Job Submission Portal
Virtual Clusters MPICH enabled with job
submission portal HADOOP enabled
PSE enabled images SFM NGS Pipeline
19 Mar 2014 31C-DAC Cloud Services 2 EGov
Enabling Applications on Meghrajusing SuMegha Cloud Lab Kit
19 Mar 2014 32C-DAC Cloud Services 2 EGov
SuMegha Cloud Lab kitSophisticated Packaged software Kit to enable the automateddeployment of private clouds on the hardware.
Highlights– Open source components: Xen Hypervisor , Nimbus(cloud middleware) and GlusterFS– Cloud tools & Dashboard: Cloud Portal, Job Submission Portal and Problem Solving
Environment (PSE), Cloud monitoring.– Automated Installer: Installation and configuration of private cloud on the
recommended hardware to setup Cloud Lab without needing expert knowledge ofSystem Administration.
– Secured Access to VMs through SSH using private keys– Golden Images: VM’s of different sizes - small, medium, large with CentOS operating
system– Parallel Programming environments with MPI libraries and HADOOP framework.
Advantages• Provides a hand holding mechanism for moving existing applications & building new
applications on cloud• Lower cost to application developers :Application developer need not pay for the
cloud resources at the application development & testing phase• Enhanced cloud adoption in several domains
19 Mar 2014 33C-DAC Cloud Services 2 EGov
Phases of Cloud Application Enablement using SuMegha Lab kit
• Enabling Existing Application– Setup cloud testing environment using SuMegha lab kit auto
installer– Test application on local cloud setup using VMs– Deploy application on Meghraj Cloud
• Developing New Application– Developing Application on SuMegha Development Environment
(SDE) using Cloud APIs(openstack, eucalyptus,etc)– Deploy & Test Application on local setup– Deploy Application on Meghraj cloud
19 Mar 2014 34C-DAC Cloud Services 2 EGov
Elastic Framework for Cloud Applications
Elasticity Framework allows cloud resources (VMs, Storage) to dynamically scale-up or down as required by the application at run-time
Building Blocks• User defined application configurations.• Cloud monitoring.• Cloud Scalar.
19 Mar 2014 35C-DAC Cloud Services 2 EGov
TENANTAPP
STORE
SuMegha Elasticity
Framework
Monitoring Configuration
Deployment Configuration
and Scripts
Cloud API Production
Cloud A
Production Cloud B
Developer Private Cloud (Development Environment) SuMegha Lab
kit1
2
3
1
2
3
4
4
19 Mar 2014 36C-DAC Cloud Services 2 EGov
• Developer Of cloud Application :-1. Create his application 2. Provide Deployment configuration and scripts to the Elasticity
frame work3. Test on private cloud Created Using SuMegha Cloud Labkit4. List that in AppStore
• User of Cloud Application 1. Choose the cloud application and provide the monitoring
threshold listed in app store2. The deployment configuration will be taken from Elasticity
Framework.3. Setting the threshold in the motoring component of Elasticity
Framework.4. Deployed on the cloud chosen by user.
19 Mar 2014 37C-DAC Cloud Services 2 EGov
Steps to Cloud Application Enablement using SuMegha Lab kit
Points to Ponder
• Licensing for commercial cloud platforms supported by SuMegha
• Billing & Pricing based on cloud platform chosen by the users
19 Mar 2014 38C-DAC Cloud Services 2 EGov
Cloud Publications/Patents• Provisional patent on “A CLOUD STORAGE SERVICE FOR CLOUD COMPUTING”, Indian
Patent Office, Provisional Patent Application No. 1392/CHE/2013, Patent Filed on: 28 March 2013.
• Shivay Veer Sharma, Anil Bhargava, Payal Saluja, Prahlada Rao B. B., Accessing E-Infrastructures using CDAC Scientific Cloud (CSC) Services , 2nd IEEE CCEM 2013, Bangalore (Poster)
• Deepanshu S, Aman A, Ramesh Naidu Laveti, Arunachalam B, Vineeth Simon Arackal, Prahlada Rao B B; Problem Solving Environment for Seasonal Forecast Model on CDAC Scientific Cloud; ACC 2013, Bangalore
• AmanArora, Pavan Kumar, Vineeth Simon Arackal, Janaki Chintalapati, ArunachalamB,Prahlada Rao B B, Cloud NGS Tool: An Online Pipeline for Next Generation Sequencing Data; ACC 2013, Bangalore
• Vineeth Simon Arackal, AmanArora, Deepanshu Saxena, Arunachalam B, Prahlada Rao B B; SciInterface: A Web-Based Job Submission Mechanism for Scientific Cloud Computing; IEEE CCEM 2013, Bangalore
• Payal Saluja, Prahlada Rao BB, Ankit Mittal, Rameez Ahmad, C-DAC Scientific Cloud: On demand provisioning of HPC resources for Scientific Applications, paper published & presented at 18th International conference of parallel and distributed processing techniques and applications(PDPTA 2012) 6th-19th July 2012 at Las Vegas, Nevada, USA
• Prahlada Rao B. B, Payal Saluja, Neetu Sharma, Ankit Mittal, Shivay Veer Sharma, Cloud Computing for Internet of Things & Sensing Based Applications poster paper published and presented at 6th International conference on Sensing Technology (ICST 2012), 18-21st Dec 2012, Kolkata, India
19 Mar 2014 C-DAC Cloud Services 2 EGov 39
Dissemination: Workshops/Trainings
Workshops Conducted: (3)o Indo_Taiwan International Workshop on Cloud Computing & Its Applications, 7-8th Nov 2013, Bangaloreo Prahlada Rao B. B. , National Workshop on Cloud Computing, conducted at CDAC Knowledge Park,
Bangalore, 21st May 2011 .(80+ participants from Industry, Academia)o National Work Shop on Virtualization, Feb 2011, C-DAC Bangalore ((70+ participants from Industry,
Academia)Tutorialso Prahlada Rao B.B., Payal Saluja, Vineeth Simon Arackle, Half-Day Tutorial “Open Source Platforms for
Scientific Clouds” CSI’s Annual Conference on Advances in Cloud Computing (ACC2012), 26th July 2012, Bangalore.India (60+ participants from Industry, Academia)
o Prahlada Rao, Vineeth Simon Arackal, Payal Saluja Tutorial on “Cloud Computing for HPC” C-DAC’s National Conference on Parallel Computing Technologies (PARCOMPTECH 2013) 21-23 Feb , Bangalore, India
o Prahlada Rao B.B , Tutorial on “Scientific Clouds” International Conference on Advances in Computing, Communications and Informatics (ICACCI-2013) , Mysore
o Prahlada Rao B.B , Invited Talk on “Trends in Cloud Computing” International conference on Advances in Cloud Computing (ACC 2013), Bangalore, 19-20 Oct 2013, India
o Cloud Computing Talks: 15+ Invited /KeyNote Talks in Intl Conferences/ Workshops, and TEQiPprograms
19 Mar 2014 C-DAC Cloud Services 2 EGov 40
Dissemination: Technical Talks Given• Prahlada Rao B.B., “Resent Trends in Cloud Computing” Technology day Talk C-DAC Bangalore
10th May 2013.• Prahlada Rao B.B., “CDAC Cloud Initiatives” Workshop on National Cloud Innovation Council, C-
DAC Bangalore 29th April 2013.• Prahlada Rao B.B., “HPC for BigData” International Workshop on Data Analytics and Applications
(IWDAA) 2013, BITS, Goa Campus, 1st Mar2013.• Prahlada Rao B.B, Payal Saluja, Neetu Sharma, Ankit Mittal, Shivay Veer Sharma, “Cloud Computing
for IOT & Sensing Based Applicationss”, in IEEE International Conference on Sensing Technologies, ICST 2012, Kollkata, India, Dec 2012.
• Prahlada Rao B.B., Tutorial On “Cloud Computing”, International Conference on Trends in Knowledge and Information Dynamics, ICTK 2012, organized by ISI, NIMHANS Conv Centre, Bangalore, India, 10 July 2012.(150 participants from Industry, Academia)
• Payal Saluja, Prahlada Rao, Ankit Mittal, Rameez Ahmad, “CDAC Scientific Cloud: On Demand Provisioning of Resources for Scientific Applications”, in Proceedings of 18th International Conference PDPTA 2012, Las vegas, Nevada, USA, 16-19 July 2012.
• Dr. Prahlada Rao BB, Cloud Computing for Telemedicine, National Workshop on Telemedicine Today & Tomorrow, C-DAC Mohali, An Invited Talk, 6th July 2012.
• Vineeth Simon Arackal, Arunachalam B, Payal Saluja, Prahlada Rao B B, Co-existence of Cloud and Grid: A Case Study in Service Oriented Grid GARUDA. The International Conference on Information Technology- New Generations (ITNG 2011), Las Vegas, USA. 11-13April 2011.
• Vineeth Simon Arackel, “ Coexistence of Grid and Cloud” at "Grid To Cloud (G2C)- Challenges andApplications" Workshop at VIT, Vellore, 2nd April 2011.
4119 Mar 2014 C-DAC Cloud Services 2 EGov
Conclusions
19 Mar 2014 C-DAC Cloud Services 2 EGov 42
• Middleware Development for Cloud, Grid & HPC:C-DAC Strength• C-DAC Cloud Activities
o Open Source based C-DAC Cloud Testbedo Cloud Applications of National Importanceo Scientific Cloud Computing Researcho SuMegha- Cloud LabKit for Privite Cloud Buildingo Cloud Interoperability Researcho Elastic Cloud Applications Development
• C-DAC Founding member of IntercloudTestbed-IEEE • Active member CCICI (Cloud Computing Innovation Council for India)