OpenStack 101 - All Things Open 2015

© 2014 VMware Inc. All rights reserved.

OpenStack 101Or: “Take the red pill”

Mark T. Voelker, OpenStack Architect

Oct. 20, 2015All Things Open 2015

CONFIDENTIAL 2

Remember the first time you were asked to stand up a

distributed system?

Or maybe a series of cooperating ones?

CONFIDENTIAL 3

Turns out, distributed systems are both very powerful

…and sometimes hard to learn.

CONFIDENTIAL 4

But at least the fine manuals are all pretty clear for these things.

…right?

?

CONFIDENTIAL 5

A little help making that first leap goes a long way.

So where do you start?

6

“A computer nerd….is somebody who uses a computer in order to use a computer.”–Douglas Adams

Mark T. Voelker (@marktvoelker)• OpenStack Architect @ VMware, OpenStack ATC, Former Puppet-OpenStack core dev, Triangle

OpenStack Meetup founder, OS Foundation Member #54, DefCore Committee Member

• Fact: can be bribed with doughnuts

• OpenStack community member since 2011, VMware since 11 months ago.

• In copious (hah!) spare time: data center OS’s, Big Data, Massively Scalable Data Centers, DevOps, outdoorsey stuff, making sawdust

Let’s begin.

“OpenStack software controls large pools of compute, storage, and networking resources throughout a datacenter, managed through a dashboard or via the OpenStack API. OpenStack works with popular enterprise and open source

technologies making it ideal for heterogeneous infrastructure…The software is built by a thriving community of developers, in collaboration with users, and is designed in the open…”

What if I told you….

…ok, let me rephrase that.

Basically: it’s software to run cloud services (compute, storage, network, etc) and the community behind that software.

History• Founded July 2010 by Rackspace, NASA, & friends

– NASA contributed a compute controller (Nova)

– Rackspace contributed an object storage controller (Swift)

• 12th release (Liberty) just went live a few days ago

– The project now follows a 6-month release cycle

• Hundreds of companies and thousands of people contribute

– 1,933 contributors from 164 organizations in the last release

– It’s probably easier to list IT/cloud companies that aren’t involved somehow than it is to list the ones that are:

Structure

Structure• The OpenStack Foundation

– Membership free for individuals

– Platinum, Gold, Corporate memberships paid for by member companies

– Board of Directors comprised of Platinum, some Gold, and generally elected members

• Provides strategic & financial oversight of Foundation resources & staff

– Controls the OpenStack brand & logo, coordinates events, etc

• The Technical Committee

– Provides technical leadership for OpenStack as a whole

– Enforces OpenStack ideals (Openness, Transparency, Commonality, Integration, Quality)

– Elected by active technical contributors (ATC’s) to the project

• Project Team Leaders

– Elected to lead individual projects by contributors to those projects

• User Committee

– Represents users with the Technical Committee and Board of Directors

All to give you this: power.

The power to build a cloud out of the stuff you want, or use one Someone else has built for you to consume.

How to Participate• IRC Channels and Mailing Lists

• User/Meetup Groups

– Local to RTP? http://meetup.com/Triangle-OpenStack-Meetup/

• Code is in git, reviewed in Gerrit, mirrored on GitHub, Bugs in Launchpad

• Two annual Design Summit/Conferences

– I leave for Tokyo this Saturday

– Spring 2016: Austin

– Fall 2016: Barcelona

• There’s a welcome guide here.

• You can try out OpenStack as a user here or get a development instance running on your laptop to start hacking on here.

What are people using it for?

Much more info fromannual User Survey data

With hundreds of projects available, it’s a very flexible platform.

You get to choose which projects you deploy.

But a few projects are where the majority of the focus (and users) are.

Horizon (GUI)

Nova (Compute) Neutron (Networking)Swift (Object Storage)

Cinder (Block storage)

Glance

(VM Image Service)

Keystone

(Identity Service)

OpenStack: The Software

AWS Management Console

EC2 VPC/ELBS3

EBS

Ceilometer

(Telemetry Service)

Trove (Database as a Service)

Heat (Orchestration)

Sahara (Data Processing)

EMR

RDS

AWS CloudFormation

and AWS comparables

DefCore: the new interoperability standard for OpenStack Powered™ products

• The DefCore Committee creates Guidelines to which products must adhere if they use the OpenStack name or OpenStack Powered logo.

• Guidelines contain a list of Capabilities that products must expose and tests they must pass in order to prove it.

• Guidelines also contain Designated Sections of code products must use to provide those Capabilities

Keystone: the Identity Service

• Provides a central service for authentication and authorization as well as a service catalog (e.g. a list of where the API endpoints of other services are)

• Abstracts various backend auth services

– SQL databases

– LDAP/AD

• Uses a bearer token model

– Clients are assigned a token which they present to other services in headers

• Multiple token types

– UUID

– PKI/PKIz

– Fernet

• Can federate

– E.g. use another keystone as a service provider via SAML assertion

Primitives:• Tokens• Services• Endpoints• Domains

• Projects• Groups• Credentials• Roles• Policies

Glance:The Image Service

• Houses virtual machine images that can later be launched as instances

• Abstracts various image containers and disk formats

– Note: this does not mean that we magically make all image formats work on all hypervisors although there are some ways to do conversion

– Bare, OVF, AKI, ARI, AMI

– Qcow2, raw, VHD, AKI/ARI/AMI, ISO, VDI, VMDK

• Multiple storage backends

– File, Swift, Ceph, etc

Primitives:• Images• Metadata• Tags• Tasks

Nova: The Compute Controller

• Basically, it’s what takes care of launching VM instances (think Amazon EC2) and plugging things into them.

• Nova abstracts hypervisors and pools of computer hardware.

• Most operations can be involved with a REST API call, a CLI client, or a few clicks in the Horizon web GUI.

• A few high-level features:

– Supports most hypervisors

– Distributed, mostly asynchronous architecture

– Public REST API, SQL backend DB, AMQP for RPC

– Supports security groups

– Several means of providing resource segregation

• Host aggregates

• Availability zones

• Regions

• Cells

Some primitives: • Flavors• Servers• Keypairs• Quotas• Aggregates

Neutron: the network controller

• Provides tenants with the ability to create isolated or shared L2 and L3 virtual networks, route between them, and connect compute instances to them

• Abstracts various networking backends

– SDN controllers

– Physical switches

– Dozens of backend plugins, both open source and proprietary

• Supports IPv4 and IPv6 (depending on plugin)

• Organized as a “stadium” project to house many networking subprojects with their own lieutenants

• Can also provide L4-L7 services

– Load Balancing as a Service

– VPN as a Service

– Firewall as a Service

Primitives:• Networks• Subnets• Ports• Quotas• Security groups• Routers

• VIPs• Health Monitors• Pools• Members

Swift: The object storage service

• Provides highly available, distributed, eventually consistent object storage.

• Can (and often is) run completely independently of OpenStack Compute.

• Optimized around durability and availability

• One of the most long-lived API’s in OpenStack (still v1)

• Similar in some respects to Hadoop HDFS and Amazon S3

– Replicates objects over multiple machines (usually at least 3)

– Replicas lost due to hardware failures can be re-replicated

– Clusters can be rebalanced

Primitives:• Accounts• Containers• Objects

Cinder: the block storage service

• Provides persistent block storage volumes to compute instances

• Abstracts underlying storage systems

• Originally part of Nova itself, but split out into it’s own project since the Folsom release

• Dozens of drivers

– EMC, NetApp, LVM, VMware, Gluster, Nexenta, NFS, Ceph, SolidFire, etc etc etc

• Volumes appear to instances as block devices

– E.g. a virtual hard drive

• There’s a separate service for shared file systems called Manila

Primitives:• Volumes• Backups• Snapshots• Quota sets• QoS Specs

And much more!

• Documentation• CI & Infrastructure• Client libraries• Oslo (common libs)• DevStack• Tempest (integration tests)• Rally (benchmarking/scale

test)• Modules for deploying with

Ansible, Puppet, Chef, Salt

• Metering service• DNS as a Service• Data Processing Service• Bare metal service• Container service• Orchestration service• Key management service• Queue service• Database as a service

Questions?

Thank You

@marktvoelker