Introduction To OpenStack IaaS - John...

Introduction To

OpenStack

Haim AteyaGroup Manager, EMC

Session Goals

Get the audience familiar with OpenStack IaaS platform

Go over the project, releases and community around it

Review and explain OpenStack terminology

Deep dive into OpenStack projects and architecture

Review installation, deployment and testing around OpenStack

Agenda

Quick introduction to OpenStack project

Explain the OpenStack architecture and how its built

Get you familiar with the different terminology and concepts

Get you familiar with OpenStack services (components)

Go over installation methods and tools

Review risks

What Is OpenStack ?As Described by Wikipedia:

“OpenStack is a cloud computing

project aimed at providing an

Infrastructure as a service (IaaS).”

What Is OpenStack ?As Described by OpenStack foundation:

“Aims to produce the ubiquitous Open

Source Cloud Computing platform that

will meet the needs of public and

private clouds regardless of size, by

being simple to implement and

massively scalable.”

OpenStack Capabilities Virtual machines (VMs) on demand

o provisioning

o snapshotting

Networks

Storage for VMs and arbitrary files

Multi-tenancy

o quotas for different projects, users

o user can be associated with multiple projects

Metering

History Begun in 2010 as a joint project of Rackspace hosting and NASA to build

Cloud based operating system

Free and open source software platform under the apache license

Actively driven by a strong open-source community with thousands of developers and more than 500 companies that actively contributing to the project: IBM, Red Hat, HP, Cisco, Intel, Google, Oracle, Dell, EMC, VMware.

11 OpenStack releases so far

OpenStack In A Nutshell Is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.

OpenStack In A Nutshell - Cont Controls large pools of storage, network and compute resources throughout

a data-center.

Believes in open source, open design, open development, all in an open community that encourages participation by anyone

Consists of a series of interrelated projects delivering various components for a cloud infrastructure solution

Statistics One of the fastest growing open-source communities in the world with more

than 15,0000 contributors Code submission Code reviews Testing Documentation

OpenStack Integrated Programs

Each OpenStack Program

Is also a “top-level” OpenStack component

Has an elected “Project Technical Lead” (PTL)

Has separate developers and design teams

Has a well defined public API

projects have a Restful (JSON/HTTP) API

OpenStack DistributionsDistributions

ReleaseName

Release Date Code Name Of Included Components

Austin October 21 2010 Nova, Swift

Bexar February 3, 2011 Nova, Swift, Glance

Cactus April 15, 2011 Nova, Swift, Glance

Diablo September 22, 2011 Nova, Swift, Glance

Essex April 5, 2012 Nova, Swift, Glance, Horizon, Keystone

Folsom September 27, 2012 Nova, Swift, Glance, Horizon, Keystone, Quantum, Cinder

Grizzly April 4, 2013 Nova, Swift, Glance, Horizon, Keystone, Quantum, Cinder

Havana October 17, 2013 Nova, Swift, Glance, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer

Icehouse April 17, 2014 Nova, Swift, Glance, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer, Trove

Juno October 16, 2014 Nova, Swift, Glance, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer, Trove, Sahara

Kilo April 30, 2015 Nova, Swift, Glance, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer, Trove, Sahara, Ironic

Liberty October 16, 2015 Nova, Swift, Glance, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer, Trove, Sahara, Ironic, Zaqar, Manila, Designate, Barbican

Project Timeline

OpenStack Architecture (beginning)

OpenStack Architecture (Now)

OpenStack TerminologyPTL – Project Team Lead for one of OpenStack projects (cinder, glance, swift, etc.)

Blueprint – a detailed suggestion for new code enhancement \ project in upstream (Open Source community)

Incubation – the process in which new project is maintained until it becomes one of the core projects

Instance – virtual machine running in Nova node

Image – an attribute of a virtual machine which includes disk and OVF

Tenant – logical entity that creates a shared pool of resources (network, storage, compute, security) between users & groups.

OpenStack TerminologyFlavor - virtual hardware templates which defines the hardware characteristics of a virtual machine (tiny, medium, large, small)

Volume – virtual disk attached to a VM instance (mostly refers to block device)

Ephemeral storage – disk or storage device is not persistent and will be destroyed upon VM instance termination

OpenStackServices

Common Attributes

Every OpenStack service

share a common set of

attributes

Common Attributes API – each service has 3 sets of API’s

CLI

Rest

GUI (via Horizon)

Database – all services store their data on a rational data base (MariaDB)

Scaling – each service support scale up \ out of the different internal

components

Security & Access – keystone controls all access between the different

services

Communication – all services are communicating using AMQP

OpenStack Deployments

OpenStack can be configured and deployed in many ways

The configuration should be set according to customer \ site needs

Main deployments are:

All-in-one (POC’s)

Standalone \ multi-node

Distributed \ partly distributed

Highly available

Semi Distributed Deployment

Cinder – Block Storage OpenStack Block Storage service provides persistent block storage

management for virtual hard drives

Cinder supports creation, deletion and management of block devices (volumes and snapshots)

Cinder provides an API that allow various vendors to write their own driver and support their designated storage array:

EMC – VMXA, XtremIO, VNX, ScaleIO

HP – HPE 3PAR

Pure – Pure Storage

Redhat (Linux) – GlusterFS, NFS, LVM, RHS

Nimble Storage

Cinder – Basic Features

Create/delete volumes

Specify custom "types/extra-specs”

Clone

Copy image to volume and volume

to image

Point in time copy (snapshots of

volumes)

Create volume from snapshot

Backup volume (to object store,

SWIFT and CEPH)

Transfer volume ownership

Per tenant quota

Customized scheduling filters

Cinder – Internal Services

openstack-cinder-api

openstack-cinder-backup

openstack-cinder-scheduler

openstack-cinder-volume

cinder

Swift – Object Storage Provides an HTTP accessible storage for large amounts of data including

static entities such as videos, images, email messages, files, or VM images.

Objects are stored as a binaries on the underlying storage along with metadata attributes such as files.

Support horizontal scaling and failover \ redundancy with software based data-replication

File has no meaning – stored as blobs of data (useful for big data, NoSQL)

No file hierarchy

Swift – Basic Features

Distributed

Scalable to several Peta-bytes

Highly available (by design)

Robust

S3 API support

Can work on simple (commodity

hardware)

Support several storage types

Quotas and access control

Swift – Internal Services

openstack-swift-account

openstack-swift-container

openstack-swift-object

openstack-swift-proxy-swift

Glance – Image Repository OpenStack Image service acts as a registry for virtual disk images

Users can add new images or take a snapshot of an existing server for immediate storage

Registered images can be stored in the Object Storage service or in other locations, such as simple file systems or external Web servers

Glance – Supported Images aki/ami/ari (Amazon kernel,

ramdisk, or machine image)

iso (archive format for optical discs, such as CDs)

qcow2 (Qemu/KVM, supports Copy on Write)

raw (unstructured format)

vhd - Hyper-V, VMware, Xen, Microsoft, and VirtualBox

vdi (Qemu/VirtualBox)

vmdk (VMware)

Glance – Internal Servicesopenstack-glance-apiopenstack-glance-registry glance

Nova - Compute OpenStack Compute serves as the core of the OpenStack cloud by providing

virtual machines on demand

Compute schedules virtual machines to run on a set of nodes by defining drivers that interact with underlying virtualization mechanisms

Compute interacts with the Identity service to authenticate instance and database access

Nova – Compute (cont.)

Compute supports the libvirt driver libvirtd that uses KVM as the hypervisor. The hypervisor creates Instances \ servers Flavors \ instance type Drivers (for virt drivers) Support bare metal provision Support several API (for virtualization vendors)

Nova – Internal Services

openstack-nova-api

openstack-nova-cert

openstack-nova-compute

openstack-nova-conductor

openstack-nova-consoleauth

openstack-nova-network

openstack-nova-novncproxy

openstack-nova-scheduler

nova

Neutron – Networking (SDN)

Handles management of virtual network infrastructure in OpenStack cloud

Including networks, subnets, routers, switches, firewalls and VPN.

Users can create network, control traffic and connect servers and devices to

one or more networks

Flexible networking models

IP addresses can dedicated or floating

Support FWaaS, LBaaS and VPNaaS

Support GRE and VXLAN tunneling

Neutron – Internal Services

network agent

neutron-dhcp-agent

neutron-ml2

neutron-server

neutron

ovs-agent

open Vswitch

Horizon - Dashboard

Self service UI (allows you to see and manage your cloud)

Python WSGI application (written in python)

Stateless client of OpenStack API's

Support Horizontal scaling

Session state must be kept (using Django)

Interact with all other services (nova, cinder, glance, swift, neutron)

Horizon – Dashboard (View)

Keystone – Identity Management OpenStack Identity provides user authentication and authorization to all

OpenStack components

Supports multiple authentication mechanisms, including user name and password credentials, token-based systems, and AWS-style log-ins

User account management, including associated information such as a name and password

Keystone – Internal Services

openstack-keystone

keystone

Heat – Orchestration

Automation and orchestration framework for OpenStack cloud

Consumes all other Openstack API's

Allows the user to define all his configuration (VM's, storage, network,

packages, bootstrapping information) and kick it into a template

You can version your deployment and repeatable one

Includes monitoring and auto-scaling

Additional Projects Trove - OpenStack Database-as-a-Service allows users to select, provision, and

operate a variety of relational and non-relational database

Sahara – OpenStack Hadoop-as-a-Service allows rapidly configure, reliably auto-deploy and scale Hadoop clusters on OpenStack

Ironic – Bare metal provisioning that enables the user to provision physical, or bare metal machines, for a variety of hardware vendors with hardware-specific drivers

TripleO – TripleO is a community developed approach and set of tools for deploying, and managing an OpenStack cloud.

Ceilometer - Telemetry provides user-level usage datafor OpenStack-based clouds

Installation, DebuggingAnd Testing

Installation Methods PackStack – OpenStack installer (deprecated – very good for POC’s and small

deployments

DevStack – Development environment that allows developers to install basic OpenStack server and debug their code

TryStack – an easy way to try OpenStack on remote servers (http://trystack.org/)

TripleO - “OpenStack on OpenStack”. It is an official OpenStack project with the goal of allowing you to deploy and manage a production cloud onto bare metal hardware using a subset of existing OpenStack components

“In God we trust, The rest we test”

Tempest

OpenStack integration testing suite

Includes set of integration tests to be run against a live OpenStack cluster

Both tests and infrastructure are written in Python

Based on Unittest2 and nose runner

Very good for API testing

Every developer \ code contributor must test his code in OpenStack CI

environment before submission and code review

Troubleshooting

Reasons

Many stability issues

Not enterprise \ production ready

Involves complex technologies

o Linux (kernel)

o Storage

o Network

o Database

o Code (python)

Lack of support

Deployment Risks

Fairly new project

Open Source

No official support

Complex troubleshooting

Requires unique knowledge in many fields

OpenStack - Summary OpenStack – open source software for building IaaS

o OpenStack release cycle is every 6 months

o OpenStack is an umbrella over multiple independent

programs (components)

o All OpenStack components talk Restful API

o Most OpenStack components have dedicated DB

(SQL) and MQ (QP), some talk to 3rd party

Components using their native APIs

Q & A

Q & A

The End (part 3)

Time to move to the next part …

Dive Into OpenStack