94
OpenStack Introduction Presenter: Jason, Tsung-Cheng, HOU Advisor: Wanjiun Liao June 7 th , 2012 1

OpenStack Framework Introduction

Embed Size (px)

DESCRIPTION

Presentation of OpenStack survey to Internet Research Lab at National Taiwan University, Taiwan. OpenStack framework and architecture overview. (ppt slide for download.) Materials collected from various resources, not originally produced by the author. Briefly explained Nova, Swift, Glance, Keystone, and Quantum.

Citation preview

Page 1: OpenStack Framework Introduction

OpenStack Introduction

Presenter: Jason, Tsung-Cheng, HOUAdvisor: Wanjiun Liao

June 7th, 2012 1

Page 2: OpenStack Framework Introduction

Motivation

• What does a cloud OS look like?• How are they building IaaS Platform?• What are current industry trend?• How will the cloud system press the network?• OpenStack

– Founded by NASA and Rackspace in 2010– Currently 178 companies and 3386 people– Growing fast now, latest release Essex, Apr. 5th

2

Page 3: OpenStack Framework Introduction

OpenStack Status

• OpenStack– Founded by NASA and Rackspace in 2010– Currently 178 companies and 3386 people– Was only 125 and 1500 in fall, 2011.– Growing fast now, latest release Essex, Apr. 5th

• Aligned release cycle with Ubuntu, Apr. / Oct.• Aim to be the “Linux” in cloud computing sys.• Open-source v.s. Amazon and vmware• Start-ups are happening around OpenStack• Still lacks big use cases and implementation

3

Page 4: OpenStack Framework Introduction

4

Page 5: OpenStack Framework Introduction

Agenda

• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking

5

Page 6: OpenStack Framework Introduction

Enterprises are building clouds to...

Where is your enterprise on the path to cloud?

Meet departmental resource needs & timelinesControl & monitor the entire environment

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Virtualization

Page 7: OpenStack Framework Introduction

Datacenters are being virtualized, Servers are firstHypervisors provide abstraction between SW and HW (Servers)

HOST 1 HOST 2 HOST 3 HOST 4, ETC.

VMs

Hypervisor:Turns 1 server into many “virtual machines” (instances or VMs)(VMWare ESX, Citrix XEN Server, KVM, Etc.)

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Virtualization

Automation & Efficiency

Better resource utilization for each server

Hardware abstraction for each server

Compute Pool Network Pool Storage PoolVirtualized Servers Virtualized Networks Virtualized Storage

Next: Storage, Network…the building blocks

Resource pools for apps starting to form...

Flexibility, Efficiency are key drivers

Page 8: OpenStack Framework Introduction

But questions arise as the environment grows...“VM sprawl” can make things unmanageable very quickly

Where should you provision new VMs? How do you keep track of it all?

+

How do you empower employees to self-service?

USERS ADMINS

How do you make your apps cloud aware?

APPS

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Virtualization

Automation & Efficiency

A Cloud Management Layer Is Missing

Page 9: OpenStack Framework Introduction

APPS

Solution: OpenStack, The Cloud Operating SystemA new management layer that adds automation and control

Creates Pools of Resources Automates The Network

USERS ADMINS

CLOUD OPERATING SYSTEM

Connects to apps via APIsSelf-service Portals for users

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Server Virtualization

Automation & Efficiency

Page 10: OpenStack Framework Introduction

Enterprise Private Clouds run cloud operations systems…

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Server Virtualization

Automation & Efficiency

What’s next?

Public Clouds run cloud operating systems…

But you can’t interoperate ifpublic clouds are built on proprietary software

Page 11: OpenStack Framework Introduction

Common PlatformImagine having a across clouds

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Virtualization

Automation & Efficiency

Seamlessly transporting workloads

Page 12: OpenStack Framework Introduction

2. Cloud Data Center 3. Cloud FederationServer Virtualization1. Virtualization

Automation & Efficiency

A common platform is here.OpenStack is open source software powering public and private clouds.

Public Cloud:OpenStack powers someof the worlds largest publiccloud deployments.

Private Cloud:Run OpenStack software

in your own corporatedata centers

Washington

EuropeCaliforniaTexasPrivate Cloud Private Cloud

Public Cloud

Public Cloud

Common software platform making

Federation possible

OpenStack enables cloud federationConnecting clouds to create global resource pools

Page 13: OpenStack Framework Introduction

In Summary, the Cloud Operating System enables enterprises to:

1. Control and automate pools of resources2. Efficiently allocate resources3. Empower admins & users via self-service portals4. Empower developers to make apps cloud-aware

via APIsTop 3 Benefits of a Common Platform1. Easy to migrate data and applications to public

clouds when conditions are right- based on security policies, economics, and other key business criteria

2. No longer locked in with major investment in one provider

3. Disaster recovery

Page 14: OpenStack Framework Introduction

Core Components in Essex• Release Apr. 5th, 2012

14

• Dashboard: Access and control portal for admin and users, also web-based

• Identity: Unified authentication across whole system

• Object Storage: Large-scale redundant storage of static objects, not a file system

• Image Service: Store, retrieve, discover, register, and deliver VM images

• Compute: Large-scale deployment of automatically provisions VMs and related SWs

Page 15: OpenStack Framework Introduction

15

Page 16: OpenStack Framework Introduction

OpenStack Compute Key Features

2. Horizontally and massively scalable

1. REST-based API

3. Hardware agnostic: supports a variety of standard hardware

4. Hypervisor agnostic: support for Xen ,Citrix XenServer, Microsoft Hyper-V, KVM, UML, LXC and ESX

Page 17: OpenStack Framework Introduction

OpenStack Storage Key Features

4. Scalable to multiple petabytes, billions of objects

1. REST-based API

6. Account/Container/Object structure (not file system, no nesting) plus Replication (N copies of accounts, containers, objects)

5. No central database required

2. Data distributed evenly throughout system

3. Runs on standard hardware

Page 18: OpenStack Framework Introduction

OpenStack Image Service Key Features

2. REST-based API1. Store & retrieve VM images

3. Compatible with all common image formats

4. Storage agnostic: Store images locally, or use OpenStack Object Storage, HTTP, or S3

Page 19: OpenStack Framework Introduction
Page 20: OpenStack Framework Introduction
Page 21: OpenStack Framework Introduction
Page 22: OpenStack Framework Introduction
Page 23: OpenStack Framework Introduction
Page 24: OpenStack Framework Introduction

Agenda

• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking

24

Page 25: OpenStack Framework Introduction

Chief Reference

• CIS 607: Seminar in Cloud Computing, Spring 2012, by Dr. Allen D. Malony

• "Applied Computational Instrument for Scientific Synthesis" (ACISS)

• University of Oregon (UO), groups may configure their own VM images and carry out research work as if had dedicated clusters.

• This course delivers hands-on operations on ACISS and cloud computing knowledge to students.

25

Page 26: OpenStack Framework Introduction

VM and Hypervisor

• Virtual Machine: A software package, sometimes using hardware acceleration, that allows an isolated guest operating system to run within a host operating system.

• Stateless: Once shut down, all HW states disappear.

• Hypervisor: A software platform that is responsible for creating, running, and destroying multiple virtual machines.

• OpenStack is hypervisor agnostic.• Type I and Type II hypervisor 26

Page 27: OpenStack Framework Introduction

27

Page 28: OpenStack Framework Introduction

Bridged Networking• One network card acts as

many devices.• Host does not need an IP

address.• Hypervisor sets virtual

MAC address for guest machine.

• ACISS uses bridges, along with Virtual Local Area Networks (VLANs) to segment traffic and assign network addresses.

28

Page 29: OpenStack Framework Introduction

Network Block Storage

• Network Block Storage: Make data persistent by mounting a network block storage device.

• NFS Mounts: Many machines may access simultaneously. Limited permissions.

• iSCSI Mounts. Only one machine may access at any given time. Unlimited permissions.

29

Page 30: OpenStack Framework Introduction

Object Storage

• Persistent storage of objects on a network.• Generally “write once, read many.”• Durable storage with redundant copies• Access Control Lists determine visibility for

owner and authorized users.• Amazon’s S3 is an example of this.• ACISS uses OpenStack Swift.• Swift uses same API as S3.

30

Page 31: OpenStack Framework Introduction

Virtual Machine Images

• Disk images that can be booted on a virtual machine by a hypervisor.

• Can be a single image that contains boot loader, kernel and operating system.

• Boot loader and kernel can be separated.• Allows for custom kernels and resizable

images.

31

Page 32: OpenStack Framework Introduction

Image Service

• Stores and catalogs virtual machine images.– Keep track of VMs, trace and recover.

• Provides for discovery, registration, and delivery of images to hypervisors.

• Allows for many image formats and for linking of loaders and kernels to images.– There may be different types of virtualization

technologies, different kernels, etc.

• Usually built on object storage systems.• Glance on Swift.

32

Page 33: OpenStack Framework Introduction

Cloud Computing

• The course defines in the following way:– The orchestration of hypervisors, networking,

block storage, and image, and identity services to provide on demand virtual machines.

• Hence, meeting required characteristics of cloud computing.– On-demand self-service– Resource pooling– Rapid elasticity– …

33

Page 34: OpenStack Framework Introduction

Agenda

• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking

34

Page 35: OpenStack Framework Introduction

Keystone Main Functions

• Provides 4 primary services:– Identity: User information authentication– Token: After logged in, replace account-password– Service catalog: Service units registered– Policies: Enforces different user levels

• Can be backed by different databases.– LDAP– SQL– Key Value Stores (KVS)

35

Page 36: OpenStack Framework Introduction

Keystone: Identity

• User information:– username/password– Metadata (e-mail, etc.)– Tenant - organizes users into projects or group.– Role - define a user’s role and permissions in a

project.

• A user must belong to at least one tenant, and may belong to many tenants

• Roles are assigned to user/tenant pairs– Common roles: Member, Admin

36

Page 37: OpenStack Framework Introduction

Keystone: Token

• Once a user’s identity has been verified with a acc/pswd pair, a short-lived (24 hr) token is issued.

• Tokens are a stand-in for the acc/pswd.• OpenStack services hold on to tokens and

use them to query keystone during operations.

• For example, once Nova can use a token to determine if an authenticated user has authorization to delete an instance.

37

Page 38: OpenStack Framework Introduction

Keystone: Catalog

• OpenStack service endpoints are registered with Keystone to create a service catalog.

• A client for a service connects to Keystone, and determines an endpoint to call based on the returned catalog.

• Behind the scenes, services can be moved to different endpoints. A client can find online services by querying Keystone endpoint.

• Also allows for service load distribution with multiple endpoints to a single service.

38

Page 39: OpenStack Framework Introduction

Keystone: Catalog

• Every catalog entry has five elements:– region: the name given to a collection of cloud

services– service id: the service the endpoint is associated

with (Glance, Nova, Swift, Keystone)– public url: the public facing endpoint for the

service– internal url: the internal facing endpoint. Usually

the same as the public url– admin url: the endpoint for service administration

39

Page 40: OpenStack Framework Introduction

40

Page 41: OpenStack Framework Introduction

41

Page 42: OpenStack Framework Introduction

Agenda

• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking

42

Page 43: OpenStack Framework Introduction

Swift

• Object storage, objects “live” on an endpoint.– An endpoint could be any storage device

• Every object belongs to a user/account pair.– keystone tenant : swift account– keystone user : swift user– keystone role : swift group

• Proxy, Ring, and Workers• Account, Container, Object

43

Page 44: OpenStack Framework Introduction

Swift: Proxy Server

• Handles incoming requests via the OpenStack Object API or raw HTTP.

• Accepts files to upload, modifications to metadata or container creation.

• Serve files or container listing to web browsers.

• Several types of Ring files• May utilize an optional cache to improve

performance.

44

Page 45: OpenStack Framework Introduction

Swift: Workers

• Keep a distributed database of replicated objects.

• Workers are divided into reliability zones. • Copies of data are distributed across multiple

zones.• There are many types of workers:

– Account server, container server, object server– Housekeeping: Replication, updater, auditor

45

Page 46: OpenStack Framework Introduction
Page 47: OpenStack Framework Introduction

Swift: Ring

• Maps names to entities and locations– Stores data based on zones, devices, partitions, and replicas

• There are three types of items:– Account, container, object

• The locations are determined by a ring file• Worker IP addresses are loaded into a ring

builder.• Storage ids and locations are computed using

a hashing algorithm to evenly distribute items across the workers.

47

Page 48: OpenStack Framework Introduction

Swift: Ring• Account and container storage id has a

database, storing object metadata.• Proxy makes distributed searches across the

databases for item requests.• The ring builder can add / remove nodes, and

rebalance distribution of files across servers.

48

Page 49: OpenStack Framework Introduction
Page 50: OpenStack Framework Introduction

←Stores object metadata

↑Stores container / object metadata

↓Physical arrangement

↑ Logical view

← Stores real objects

Duplicated storage, load balancing

Page 51: OpenStack Framework Introduction

Workers can be a account server, a container server, or an object server

Page 52: OpenStack Framework Introduction
Page 53: OpenStack Framework Introduction

# of account < # of container < # of object servers

Different zones ↑

Page 54: OpenStack Framework Introduction
Page 55: OpenStack Framework Introduction

55

Page 56: OpenStack Framework Introduction

Agenda

• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking

56

Page 57: OpenStack Framework Introduction

Glance

• Image storage and indexing.• Keeps a database of metadata associated

with an image, discover, register, and retrieve.

• Built on top of Swift, images store in Swift• Two servers:

– Glance-api: public interface for uploading and managing images.

– Glance-registry: private interface to metadata database

• Support multiple image formats 57

Page 58: OpenStack Framework Introduction

58

Page 59: OpenStack Framework Introduction

59

Page 60: OpenStack Framework Introduction
Page 61: OpenStack Framework Introduction

61

Page 62: OpenStack Framework Introduction

Agenda

• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking

62

Page 63: OpenStack Framework Introduction

Nova

• Major components:– API: public facing interface– Message Queue: Broker to handle interactions

between services, currently based on RabbitMQ– Scheduler: coordinates all services, determines

placement of new resources requested– Compute Worker: hosts VMs, controls hypervisor

and VMs when receives cmds on Msg Queue– Volume: manages permanent storage

63

Page 64: OpenStack Framework Introduction

Nova

• Major components:– Network: manages networking

• Was originally a component in Nova• Default gateway, network controller• DHCP server, address mgmt• The network part in Nova will be enhanced by the

project named “Quantum”, to be released.• Will introduce Quantum later.

64

Page 65: OpenStack Framework Introduction

Nova Messaging and Data

• Messaging is managed through RabbitMQ– Server that allows messages to be posted to channels.– Subscribers to channels receive messages.– Services regularly announce availability.– Scheduler regularly reads for availability.– Scheduler makes requests to services.

• Persistent data stored in a database.– VM metadata, network topology, volume metadata, known

services

65

Page 66: OpenStack Framework Introduction

Messaging (RabbitMQ)• Get data from point A to point B• Decouple publishers and consumers• Queueing for later delivery• Load balancing and scalability• RabbitMQ is an AMQP messaging broker• Advanced Message Queueing Protocol• Network wire-level protocol• Internet protocol - like HTTP, TCP - but

ASYNCHRONOUS

66

Page 67: OpenStack Framework Introduction

Messaging (RabbitMQ)

67

Page 68: OpenStack Framework Introduction

Messaging (RabbitMQ)

68

Page 69: OpenStack Framework Introduction

69

Page 70: OpenStack Framework Introduction

70

Page 71: OpenStack Framework Introduction
Page 72: OpenStack Framework Introduction
Page 73: OpenStack Framework Introduction

73

Page 74: OpenStack Framework Introduction

74

Page 75: OpenStack Framework Introduction

Agenda

• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking

75

Page 76: OpenStack Framework Introduction

Without Quantum• Originally, Nova handles all networking by:

– Linux bridge networking– Virtual interfaces connecting network through the

physical interface– Assigns VM IP address– Fixed IP: Returns when VM shuts down– Floating IP: Can be reassigned online

• Network Manager provides VN to enable compute servers to interact with each other and the public network

• A Blog states currently 90% Nova bugs are network related76

Page 77: OpenStack Framework Introduction

Original Network Manager

• Each VM network owned by one network host – Simply a Linux running Nova-network daemon

• Nova Network node is the only gateway• Flat Network Manager:

– Linux networking bridge forms a subnet– All instances attached same bridge– Manually Configure server, controller, and IP

• Flat DHCP Network Manager:– Add DHCP server along same bridge

• Later: VLAN Network Manager77

Page 78: OpenStack Framework Introduction

Bridged Networking• One network card acts as

many devices.• Host does not need an IP

address.• Hypervisor sets virtual

MAC address for guest machine.

• ACISS uses bridges, along with Virtual Local Area Networks (VLANs) to segment traffic and assign network addresses.

78

Page 79: OpenStack Framework Introduction

79

The only gatewayWith security measures

Linux running Nova-network daemon

Network host will act as the gateway for all the NICs bridged into that network.VMs bridged in to a raw Ethernet device

Page 80: OpenStack Framework Introduction

80

• DHCP server also tracks IP leases and releases

• Re-uses and assigns IP addresses dynamically

• Sets up a routing table for outside forwarding

• Compute optionally have public IP

• Network host is a single point of failure and bottleneck

• Backup network host• A new proposed model → →• Multi-NIC→Multiple networks

Page 81: OpenStack Framework Introduction

81

Page 82: OpenStack Framework Introduction

VLAN Network Manager

• Current default mode for OpenStack• Nova creates a VLAN and bridge for each project.

– Requires switches with VLAN tagging (IEEE 802.1Q).– A range of private IPs, only accessible inside VLAN.

• A special VPN instance (code named cloudpipe) needs to be created. Generates (certificate, key) for users to access VPN automatically.

• Provides a private network segment for each project, accessed via dedicated VPN connection from the Internet. Each project with own VLAN, Linux networking bridge, and DHCP server.

82

Page 83: OpenStack Framework Introduction
Page 84: OpenStack Framework Introduction
Page 85: OpenStack Framework Introduction
Page 86: OpenStack Framework Introduction
Page 87: OpenStack Framework Introduction
Page 88: OpenStack Framework Introduction

Plugin

• The component where the ‘virtual networking’ magic happens. Fulfills API contract by implementing the ‘Plugin Interface’

• Tenants expect same behavior from Quantum API regardless of the particular plugin employed

• Available Quantum Plugins:– Open vSwitch: Builds isolated networks with OVS and L2-in-L3 tunnels.

– Cisco UCS: Isolation based on VLAN and net-profiles applied to Cisco UCS

– converged network adapters

– Linux Bridge: Build isolated networks with VLAN interfaces and linux bridge

– NTT-Data Ryu: Acts as a proxy for the NTT Ryu platform

– Nicira NVP: Acts as a proxy for the Nicira NVP platform

88

Page 89: OpenStack Framework Introduction

The Quantum Manager

• Nova’s network manager for Quantum. Forwards network related requests.

• Also, provides other network services such as IP address management, DHCP, NAT, Floating IPs…

• Virtual Networking: A label nowadays applied to too many solutions and products.– Securely partitioning the network– Defining virtual network topologies– Automating network provisioning

89

Page 90: OpenStack Framework Introduction
Page 91: OpenStack Framework Introduction
Page 92: OpenStack Framework Introduction
Page 93: OpenStack Framework Introduction

The Near Future

• Folsom release, Fall 2012– Become a core OpenStack project– Merge with IP Address Management service– Improve API quality and documentation– Improve GUI, i.e. Quantum Horizon plugin– Possible more plugins, Build more network

services on top of the basic building block• Each service with its own tenant-facing API • IP routing, Distributed Firewall, LB, NAT, VPN,

bridging…• Quantum is NOT SDN, but in theory can transform anything into SDN.

93

Page 94: OpenStack Framework Introduction

Reference• OpenStack Documentation

http://docs.openstack.org/

• Dr. Allen D. Malony, CIS 607: Seminar in Cloud Computing, Spring 2012, U. Oregonhttp://prodigal.nic.uoregon.edu/~hoge/cis607/

• Bret Piatt, OpenStack Overview, OpenStack Tutorialhttp://salsahpc.indiana.edu/CloudCom2010/slides/PDF/tutorials/OpenStackTutorialIEEECloudCom.pdfhttp://www.omg.org/news/meetings/tc/ca-10/special-events/pdf/5-3_Piatt.pdf

• Vishvananda Ishaya, Networking in Novahttp://unchainyourbrain.com/openstack/13-networking-in-nova

• Sandy Walsh, OpenStack 101 Technical Overviewhttp://www.slideshare.net/openstackcommgr/openstack-101-technical-overview

• Jaesuk Ahn, OpenStack, XenSummit Asiahttp://www.slideshare.net/ckpeter/openstack-at-xen-summit-asiahttp://www.slideshare.net/xen_com_mgr/2-xs-asia11kahnopenstack

• Salvatore Orlando, Quantum: Virtual Networks for Openstackhttp://qconlondon.com/dl/qcon-london-2012/slides/SalvatoreOrlando_QuantumVirtualNetworksForOpenStackClouds.pdf

• Dan Wendlandt, Openstack Quantum: Virtual Networks for OpenStackhttp://www.ovirt.org/wp-content/uploads/2011/11/Quantum_Ovirt_discussion.pdf

• Daneyon Hansen, OpenStack @ CISCOhttp://www.cisco.com/web/strategy/docs/gov/openstack_presentation.pdf

• Rick Clark, Cisco and OpenStackhttp://www.ogf.org/OGF32/materials/2310/ogf32-isod-Cisco-OpenStack-July2011.pdf

94