Introduction to Apache CloudStack by David Nalley

Introduction to Apache CloudStackDavid Nalley

@ke4qqq

ke4qqq@apache.org

#whoami

Recovering sysadmin

Apache CloudStack and Apache jclouds

Work for Citrix Open Source Business Office

VP Infra, Apache Software Foundation

Intro to Apache CloudStack

Questions

Intro to Apache CloudStack

A brief history of CloudStack

2008-2010 – Stealth mode startup (VMOps)

2009 – First production in public cloud

2010 – Released as open source (GPLv3)

2011 – Cloud.com acquired by Citrix

2012 – Transferred to the ASF (ASLv2)

2013 – Graduated as TLP

Intro to Apache CloudStack

Cloud

Infrastructure as a Service

• Compute, Network, Storage

Cloud requires:

• On-demand

• Self-service

• Scalable

• Measurable

Intro to Apache CloudStack

What is CloudStack

Orchestration platform for Infrastructure as a

Service

Focused heavily on compute

Somewhat monolithic – by design

Written in Java

Intro to Apache CloudStack

Where do we dive in?

Intro to Apache CloudStack

IaaS is Orchestration

IaaS doesn’t really provide new services, it

orchestrates existing services

•Hypervisors

•Storage

•Network

•Isolation

•Accounting

Intro to Apache CloudStack

CloudStack Hypervisor Support

KVM

XenServer

VMware vSphere

Hyper-V

Oracle VM (reappearing in 4.5)

LXC

Baremetal

Intro to Apache CloudStack

CloudStack Storage - Primary

Local

NFS

iSCSI

Distributed (GlusterFS, Ceph, etc)

$anything_hypervisor_supports

Intro to Apache CloudStack

CloudStack Storage - Secondary

NFS

Object Storage (S3 or Swift API)

Intro to Apache CloudStack

Networking

The bane of any cloud operators existence

Intro to Apache CloudStack

Networking - Challenges

Network guys are luddites (generally)

We have scale issues

Intro to Apache CloudStack

Scale

VLANs – don’t scale

Firewalls – don’t scale

Loadbalancers – don’t scale

Intro to Apache CloudStack

How we scale the network

Amazon’s Security Groups

In CloudStack we reuse the Security Groups

terminology.

SDN

The interesting pieces aren’t in Layer 2-3,

but 4-7, and still being developed.

Intro to Apache CloudStack

Resources, hierarchy, failure

domains

Regions

Zones (DCs)

Pods

Clusters

Machines

Intro to Apache CloudStack

Actual use

UI is beautiful

…..but…..

API is where real people do work.

Intro to Apache CloudStack

API

Native CloudStack API

http://cloudstack.apache.org/docs/api/

EC2, GCE APIs

https://github.com/imduffy15/ec2stack

https://github.com/NOPping/gstack

Intro to Apache CloudStack

API

Use an abstraction library (jclouds, fog,

libcloud)

Use tools that use abstraction libraries

(knife-cloudstack,

Intro to Apache CloudStack

Stop thinking of VMs

{

"name": "hadoop_cluster_a",

"description": "A small hadoop cluster with hbase",

"version": "1.0",

"environment": "production",

"servers": [

{

"name": "zookeeper-a, zookeeper-b, zookeeper-c",

"description": "Zookeeper nodes",

"template": "rhel-5.6-base",

"service": "small",

"port_rules": "2181",

"run_list": "role[cluster_a], role[zookeeper_server]",

"actions": [

{ "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }

]

},

{

"name": "hadoop-master",

"description": "Hadoop master node",

"template": "rhel-5.6-base",

"service": "large",

"networks": "app-net, storage-net",

"port_rules": "50070, 50030, 60010",

"run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"

},

{

"name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",

"description": "Hadoop worker nodes",

"template": "rhel-5.6-base",

"service": "medium",

"port_rules": "50075, 50060, 60030",

"run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",

"actions": [

{ "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },

{ "http_request": "http://${hadoop-master}:50070/index.jsp" }

]

}

]

}

{

"name": "hadoop_cluster_a",

"description": "A small hadoop cluster with hbase",

"version": "1.0",

"environment": "production",

"servers": [

{

"name": "zookeeper-a, zookeeper-b, zookeeper-c",

"description": "Zookeeper nodes",

"template": "rhel-5.6-base",

"service": "small",

"port_rules": "2181",

"run_list": "role[cluster_a], role[zookeeper_server]",

"actions": [

{ "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }

]

},

{

"name": "hadoop-master",

"description": "Hadoop master node",

"template": "rhel-5.6-base",

"service": "large",

"networks": "app-net, storage-net",

"port_rules": "50070, 50030, 60010",

"run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"

},

{

"name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",

"description": "Hadoop worker nodes",

"template": "rhel-5.6-base",

"service": "medium",

"port_rules": "50075, 50060, 60030",

"run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",

"actions": [

{ "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },

{ "http_request": "http://${hadoop-master}:50070/index.jsp" }

]

knife cs stack create hadoop_cluster_a

Prereqs

Configuration management

Automated provisioning

Monitoring

Intro to Apache CloudStack

Jevon’s Paradox

Intro to Apache CloudStack

Rants

Portability is a red herring – Config

Management is a prereq

Portability is pointless – moving data is the

problem

Scale is hard – expect to iterate, a lot.

Failure is even harder – fail proactively

Intro to Apache CloudStack

Next gen predictions

Lots of people need IaaS

Many people need workload orchestration

(Kubernetes, Apache Mesos, etc)

Intro to Apache CloudStack

Questions

Intro to Apache CloudStack

Get Involved

Web: http://cloudstack.apache.org/

Mailing Lists: cloudstack.apache.org/mailing-lists.html

IRC: irc.freenode.net: 6667 #cloudstack

Twitter: @cloudstack

LinkedIn: www.linkedin.com/groups/CloudStack-Users-Group-3144859

If it didn’t happen on the mailing list, it didn’t happen.

Intro to Apache CloudStack