Crash Course in Open Source Cloud Computing

Mark R. Hinkle VP of Community Cloud.com Twitter: @mrhinkle Email: mrhinkle@gmail.com

Crash Course in Open Source Cloud Computing

2

%whoami •  Responsible for Driving Adoption of

CloudStack Open Source Cloud Computing Software

•  Former manager of Zenoss Open Source project 100,000 users, 1.5 million downloads

•  Former Linux Desktop Advocate (Zealot?) •  Former LinuxWorld Magazine Editor-in-Chief •  Open Management Consortium Conspirator •  Open Desktop Consortium Instigator •  Author - “Windows to Linux Business Desktop

Migration” - Thomson •  NetDirector Project - Open Source

Con!guration Management Project •  Sometimes Author and Blogger at

SocializedSoftware.com/NetworkWorld •  Start-up junkie, Glutton for punishment

3

Agenda

•  Cutting Through the Cloud Computing Hype

•  Open Source Building Blocks for Cloud Computing

•  Open Source Tools for Cloud Management

•  Questions (and Hopefully Answers)

Cloud Computing: Cutting through the Hype

5

Cloud Computing Adoption

•  365,000 web sites are running on Amazon EC2 - Netcraft May 2010

•  £76 billion spent on Cloud Computing in the U.K. in 2010 - IT Candor

•  Cloud Computing will be a $126 billion market by 2012 - IBM

•  20% of Businesses won’t have IT Assets by 2012 - Gartner

•  December 2010,Amazon Web Services housed 262 billion objects, more than doubling in size from 102 billion objects at the close of 2009 - Amazon

6

Open Source Virtual Adoption

•  41% of users prefer to deploy servers virtually, 29.3% use virtualization whenever possible

•  Xen is 2nd most popular virtualization technology after VMware followed by Linux Kernel-Based Virtual Machines (KVM) with 21.3% of the market

•  49.2% of users planed to deploy hosted Linux servers in 2010 while 32.6% indicated that they would be deploying hosted Microsoft Windows.

•  50.8% indicated they used no specific management tools for cloud computing, 33.3% indicated using tools provided by their hosting provider

Source: Zenoss Cloud Computing State of the Union - http://www.zenoss.com/in/virtualization_survey.html

7

Five Characteristics of Clouds

1. On-Demand Self-Service

2. Broad Network Access

3. Resource Pooling

4. Rapid Elasticity

5. Measured Service

8

Cloud Computing Service Models USER CLOUD a.k.a. SOFTWARE AS A SERVICE Single application, multi-tenancy, network-based, one-to-many delivery of applications, all users have same access to features. Examples: Salesforce.com, Google Docs, Red Hat Network/RHEL DEVELOPMENT CLOUD a.k.a. PLATFORM-AS-A-SERVICE Application developer model, Application deployed to an elastic service that autoscales, low administrative overhead. No concept of virtual machines or operating system. Code it and deploy it. Examples: Google AppEngine, Windows Azure, Rackspace Site, Red Hat Makara SYSTEMS CLOUD a.k.a INFRASTRUCTURE-AS-A-SERVICE Servers and storage are made available in a scalable way over a network. Examples: EC2,Rackspace CloudFiles, OpenStack, CloudStack, Eucalyptus, Ubuntu Enterprise Cloud, OpenNebula

9 Deployment Models Public,Private & Hybrid Clouds

10

Benefits of Cloud Computing

Public Cloud •  Pay for What You Need •  Numerous Geographical

Options •  Virtually unlimited

capacity •  Op-Ex versus Cap-Ex •  Automation Capabilities

Private Cloud •  Full control of

infrastructure •  Security (if you are good) •  Automation Capabilities •  Flexibility, Flexibility,

Flexibility

Building Compute Clouds with Open Source Software

12

Open Source Hypervisors

Open Source •  Xen Cloud Platform (XCP) •  KVM – Kernel-based Virtualization •  VirtualBox* - Oracle supported Virtualization Solutions •  OpenVZ* - Container-based, Similar to Solaris Containers or BSD Zones •  LXC – User Space chrooted installs

Proprietary •  VMware •  Citrix Xenserver •  Microsoft Hyper-V

13

Open Source Compute Clouds Year Started License Hypervisors

Supported

2010 (Development Since 2008)

GPL Xenserver, VMware, KVM, Hyper-V

2008 GPL Xen, KVM, VMware

2010 (Developed by NASA by Anso Labs peviously)

Apache Xen, KVM, Hyper-V

2009 (Karmic Koala) GPL Xen,KVM

2009 (Development 2006)

LGPL v3 VMware ESX and ESXi, Microsoft Hyper-V, Xen, KVM and Virtual Box

Ubuntu Enterprise Cloud

14 Open Source Cloud Computing Storage •  GlusterFS – Scale Out NAS system agregating storage

over Ethernet or Infiniband •  CEPH – Distributed file storage system •  OpenStack Storage (Swift) – Long-term storage object

storage system •  Sheepdog – Distributed storage for KVM hypervisors •  NFS – Old standby, tried and true, not designed for

cloud scale or performance

15 Cloud APIs Aren’t Created Equal Open Source Abstractions •  jclouds •  libcloud •  deltacloud

16

Private Cloud Architecture

Managing Clouds with Open Source Tools

18

Cloudy Tools

•  Network Capable •  Cloud “Aware” •  Easy-to-Integrate •  Adhere to Open Standards •  Lend Themselves to Automation

19

The Myth of the Nines Availability % Downtime per

Year Downtime per Month

Downtime per Week

99.9% (three nines) 8.76 hours 43.2 minutes 10.1 minutes

99.95% 4.38 hours 21.56 minutes 5.04 minutes

99.99% (four nines) 52.6 minutes 4.32 minutes 1.01 minutes

99.999% (!ve nines) 5.26 minutes 25.9 minutes 6.05 minutes

99.9999% (six nines) 31.5 seconds 2.59 seconds .0605 seconds

Average polling interval for monitoring? 5 minutes? Even superhuman operations people can’t be alerted and take action in under 5 minutes.

One outage per year could drop service level to three nines or worse.

20 Automation Unlocks the Potential of the Cloud

•  MeatCloud, Can’t Keep up with Cloud Computing

•  Devops & Agile IT Philosophy

•  Script Repetitive Tasks •  Automate, Automate,

Automate

21

4 Types of Management Tools

Provisioning Installation of operating systems and other software

Configuration Management Sets the parameters for servers, can specify installation parameters

Orchestration/Automation Automate tasks across systems

Monitoring Records errors and health of IT infrastructure

22

Management Toolchains

Con!guration

Patching and Provisioning

Monitoring

Toolchain (n): A set of tools where the output of one tool becomes the input of another tool

23 Open Source Provisioning Tools

Year Started Language License Installation Targets

Cobbler 2007 Python GPL Red Hat, OpenSUSE Fedora, Debian, Ubuntu

Fully Automatic Installation (FAI)

2000 Perl GPL Debian

Kickstart ? Python GPL Most .dep and RPM based Linux distros

Spacewalk 2008 Perl, Python, Java GPL Fedora, Centos

Viper 2008 Perl GPL Debian

24 Open Source Configuration Management Tools

Year Started Language License Client/Server

Bcfg2 2003 Python BSD Yes

Cfengine 1993 C Apache Yes

Chef 2009 Ruby Apache Chef Solo – No Chef Server - Yes

Puppet 2004 Ruby GPL yes

25 Open Source Monitoring Tools

Year Started License Language Type of Monitoring

Collection Methods

Cacti 2001 GPL PHP Performance SNMP, syslog

Nagios 1999 GPL C/PHP Availability SNMP,TCP, ICMP, IPMI, syslog

OpenNMS 2000 GPL Java Availability/Performance

SNMP,

Zabbix 2001 GPL C/PHP Availability/ Performance and more

SNMP, TCP/ICMP, IPMI, Synthetic Transactions

Zenoss 2005 GPL Python Availability, Performance, Event Management

SNMP, ICMP, SSH, syslog, WMI

26 Open Source Automation/Orchestration Tools

Year Started

Language License Client/Server

Support Organization

AutomateIT 2009 Ruby GPL No None

Capistrano 2006 Ruby MIT Yes None

RunDeck 2010 Java Apache Yes DTO Solutions

Func 2007 Python GPL Yes Fedora Project

MCollective 2009 Ruby Apache Yes PuppetLabs

27

Automated Toolchain

Cloud Image Launch

OS Install

System Con!guration

Application Service Orchestration

Capistrano RunDeck Fabric Func

BCFG2 Cfengine Chef Puppet

Cloud: Eucalyptus OpenStack CloudStack Abiquo

OS Install: Kickstart Cobbler Spacewalk

Command and Control

Con!guration

Bootstrapping Prov

isio

ning

Act

ivity

Questions?

29

Contact Me

Professional: mrhinkle@cloud.com Personal: mrhinkle@gmail.com

Professional: mrhinkle@cloud.com Personal: mrhinkle@gmail.com

Professional: http://open.cloud.com Personal: http://socializedsoftware.com

Twitter: @mrhinkle

Mark R. Hinkle VP of Community Open Source Fanboy

Crash Course in Cloud Computing by Mark R. Hinkle is licensed under a

Creative Commons Attribution-ShareAlike 3.0 United States License.