Architect Openstack

7/21/2019 Architect Openstack

http://slidepdf.com/reader/full/architect-openstack 1/17

Architecting OpenStack for enterpriserealityBy Paul Miller

April 7, 2014

This report was underwritten by Canonical.



Architecting OpenStack for Enterprise Reality 2

TABLE OF CONTENTS

Executive summary ................................................................................................................................... 3

Adding cloud to the enterprise IT mix ........................................................................................................ 4

From virtualization to the cloud .............................................................................................................. 5

Public, Private, Hybrid ........................................................................................................................... 5

The role of VMware ............................................................................................................................... 6

The road to the cloud ............................................................................................................................ 6

OpenStack ................................................................................................................................................ 7

Key components ................................................................................................................................... 8

Adoption to date .................................................................................................................................. 10

Building a bridge ..................................................................................................................................... 11

SDN – hype or value? ............................................................................................................................. 13

Managing change .................................................................................................................................... 14

Key takeaways ........................................................................................................................................ 16

About Paul Miller ..................................................................................................................................... 17

About Gigaom Research ......................................................................................................................... 17




Executive summaryEnterprise IT managers are watching the open-source cloud infrastructure project OpenStack with

interest, hoping it might offer an easy way to begin exploiting the cloud alongside their existing IT estates.

In this report, we briefly introduce each of OpenStack’s core components before exploring the ways

OpenStack might realistically add value alongside existing investment in widely deployed on-premise

solutions such as those dependent on VMware’s product family.

Today’s enterprise data center is typically already heavily virtualized. Pools of servers are available for use

across the organization, in a manner that appears increasingly cloud-like. With VMware still dominating

this market for on-premise virtualization, we could argue that customers who have embraced VMware’s

model of virtualization have no real need to take the additional steps required to deploy either public or

private cloud solutions.

In this report, we explore some of the ways in which VMware virtualization and OpenStack-powered

clouds complement each other, and we discuss the efforts of OpenStack Foundation member VMware

and other project participants to simplify the process by which existing enterprise IT investments might

be enriched with the addition of OpenStack.

http://openstack.org/







Adding cloud to the enterprise IT mixThe enterprise IT landscape has changed dramatically in recent years, and many of those changes

continue to affect IT planning and procurement decisions today. The full implications of cloud

computing’s rise are not yet fully understood, but an earlier wave of transformation — the widespread

adoption of virtualization — is increasingly seen as a logical step on the way toward enterprise cloud

adoption.

With well over 50 percent of the world’s x86-based servers likely now devoted to hosting virtualized

workloads and virtualization often exceeding 75 percent of the server estate in larger enterprises,

virtualization is clearly an established technique in the IT toolkit. These virtualized pools of computing

capacity change the way in which IT is provisioned and managed, and they set adopters on a path that

typically leads them toward the even greater flexibility offered by a cloud solution.

Virtualization offers a number of benefits over hardware-based provisioning of IT, including:

Centralized management of IT capacity, offering economies of scale in purchasing, more-efficient

resource utilization, and so on

Cost, power, cooling, and space savings, as a smaller number of servers can be operated at higher

levels of utilization (virtualized servers typically operate at 80 percent to 90 percent of capacity,

compared with 50 percent to 60 percent or less for non-virtualized servers)

Reduction of vendor lock-in, as the virtualization process creates a layer of abstraction between

the applications and the physical hardware on which they happen to be running today

Faster provisioning, as new virtual machines can be created from a pool of available capacity far

faster (minutes) than a new physical server can be specified, approved, procured, delivered,

installed, and made available (weeks or even months)

Improved reliability, as virtual machines and their applications can often be moved from one

physical server to another without significant impact on users




Virtualization is not the answer to every IT challenge:

The hypervisor that controls the virtualization process introduces a slight performance overhead,

perhaps making it more efficient to leave servers devoted to a single application un-virtualized.

Some applications require dedicated access to specific hardware (such as a GPU for intensive

processing), and these will usually perform better without virtualization.

In certain circumstances, ensuring that a mission-critical application is able to draw on all of a

server’s resources may be more cost-effective, even if those resources may be underutilized much

of the time when that application is idle.

A number of applications still ship with licenses that do not permit virtualization.

Some older applications may not perform reliably in a virtualized environment.

From virtualization to the cloud

Once an organization recognizes and embraces virtualization’s core proposition of a device-independent

pool of computing capacity, seeing the additional value offered by cloud computing is relatively simple.

Self-service provisioning of virtual machines, elastic scaling up and down of compute capacity, the ability

to access additional computing power outside the data center when required, fine-grained metering, and

billing on the basis of consumption all offer clear and achievable benefits. For an organization that is

already virtualizing a lot of its workloads, the additional step to a cloud solution is often — in theory, at

least — not a large one.

Public, private, hybrid

Early cloud solutions, such as those offered by Amazon Web Services (AWS), tended to be in the public

cloud. They were compelling to startups without existing IT infrastructure, and even for larger

organizations they made a lot of sense for short bursts of activity such as the New York Times’ batch

conversion of unwieldy image formats for use online back in 2008. But for organizations with existing ITinfrastructure, established workflows, and compliance frameworks, moving mainstream workloads to the

public cloud was more complicated. Whether justified or not, the perception that the cloud might be less

secure, less reliable, or simply too different to existing systems created hurdles that slowed adoption.

Typically, those hurdles were only tackled when a pressing business requirement made change less

painful than preserving the status quo.

http://open.blogs.nytimes.com/2008/05/21/the-new-york-times-archives-amazon-web-services-timesmachine/












Private cloud solutions and, more recently, hybrid cloud solutions have emerged to tackle these perceived

shortcomings in the public cloud, lowering the barriers to adoption and simplifying the process of

realizing at least some of cloud computing’s benefits.

Activities such as the Eucalyptus project from the University of California, Santa Barbara (UCSB) quickly

offered software that allowed customers to run Amazon-compatible private clouds in their own data

centers. More recently other open-source initiatives like the CloudStack and OpenStack projects gained

traction and grew to become widely supported by a significant proportion of vendors operating in the

market. OpenStack, for example, powers public cloud offerings from Rackspace, Hewlett-Packard, and

others, and it can be downloaded to create private clouds that run inside customer data centers. In

principle, at least, public and private OpenStack clouds can be combined to create a hybrid cloud, and the

OpenStack code distributions from the likes of Rackspace and Canonical are explicitly marketed on this

promise.

The role of VMware

VMware dominates the server-virtualization market today, with IDC cited as suggesting a market share of

50 to 55 percent. That dominance is being squeezed — mostly by competitors Microsoft and Citrix — but

VMware remains a significant player in the space. The company’s vSphere hypervisor is likely to be

deployed at many organizations considering a move to the cloud, and some of the company’s other

products may also be used to manage some or all the existing on-premise infrastructure.

VMware is increasingly pushing cloud-like solutions such as the private vCloud Suite and its more

recently launched hybrid equivalent. Both of these are most likely to appeal to customers with an ongoing

and near-exclusive commitment to VMware’s family of products. More-cautious customers may be wary

of the growing risk of lock-in and will therefore look elsewhere.

The road to the cloud

The perception that VMware’s cloud products are the only way to move from a VMware virtualized data

center to a cloud-based model is not necessarily true. The perception that companies with heavily virtualized IT infrastructure (using VMware or one of its competitors) need to throw a lot of that

investment away and begin again as they adopt a different cloud solution such as OpenStack’s is also not

the case, as we shall see below.

http://www.techweekeurope.co.uk/comment/vmworld-vmware-cloud-mumford-and-sons-129505



http://www.idc.com/tracker/showproductinfo.jsp?prod_id=39



https://www.vmware.com/products/vcloud-suite/



http://www.vmware.com/products/vcloud-hybrid-service










OpenStackLaunched in 2010 by Rackspace and NASA and supported by a broad and growing set of technology

companies, the OpenStack project today dominates the discussion of private and hybrid clouds.

Significant backers such as HP and Rackspace also offer public clouds to compete with AWS, powered by

OpenStack. Other open-source cloud projects such as CloudStack have loyal followings of their own, and

they are frequently described as easier to deploy than OpenStack. But OpenStack’s broad industry

backing, plus the speed with which projects form to tackle perceived weaknesses in the code, make it the

open-source AWS alternative to beat.

Google Trends data, tracking interest in competing open-source cloud projects

Source: Google

OpenStack continues to evolve rapidly, with new versions of the code released roughly every six months.

The current version, OpenStack Havana, was released in October 2013. Core capabilities around compute

and storage are relatively mature, but other aspects of the project are not so complete. Across the project,

more emphasis tends to be paid to core functionality than to ease of use, sometimes leading newcomers

to consider OpenStack modules complex or difficult to deploy. A wide range of companies, including

Canonical, Mirantis, and Rackspace, offer professional-services engagements designed to mask some of




this complexity behind delivery of an installation tailored to meet their clients’ requirements. These

companies and others also offer their own distributions of the OpenStack code, often adding richer

installation tools or tighter integration with other open-source projects (such as Ubuntu, in Canonical’s

case) or their own products.

Key components

Core components of the OpenStack cloud

Source: OpenStack

OpenStack originally launched with a focus on two core modules, an object-storage module (Swift),

contributed by founding partner Rackspace, and a compute module (Nova), contributed by founding

partner NASA. Development on each of these has continued, with a growing number of contributions

from others too.

The OpenStack project now offers nine core modules, composed of:

1. Nova (compute). One of the original OpenStack modules and still the most widely deployed,

Nova is broadly equivalent to Amazon’s Elastic Compute Cloud (EC2). Nova is central to any

OpenStack deployment, providing the APIs that developers use to start, manage, and stop virtual

machines within an OpenStack cloud. Nova is designed to be horizontally scalable and to operate

effectively on commodity hardware. Nova does not include a hypervisor of its own, but it is




designed to manage the deployment of most major hypervisors, including KVM, Xen, and

VMware’s ESX (via an API call to vCenter). As well as the x86 architectures typically found in

today’s data centers, Nova can also run on alternative infrastructures such as those using low -

power Atom chips designed by ARM.

2. Swift (object storage). The second of OpenStack’s original modules, Swift is loosely similar to

Amazon’s Simple Storage Service (S3). Swift provides OpenStack users with a scalable and

redundant object-storage solution, and it should not be confused with the block-storage module

Cinder. Contributors such as SwiftStack have also commercialized Swift for use in OpenStack and

non-OpenStack environments.

3.

Cinder (block storage). Cinder is OpenStack’s block-storage module, designed to manage a

wide range of commercial storage arrays in delivering persistent block-level storage to high-

performance applications such as databases. A further project, Ceph, has been growing in

popularity as a replacement for (or adjunct to) both Swift and Cinder. Ceph is offered as a

supported option within the OpenStack distributions of companies such as Canonical.

4. Neutron (networking). Neutron (previously known as Quantum) is OpenStack’s networking

module, designed to manage communication among OpenStack instances across a wide range of

physical and virtual network architectures. Neutron supports OpenFlow, one of the principal

specifications for the emerging area of software-defined networking (SDN).

5. Horizon (dashboard). Horizon is OpenStack’s web-based dashboard, augmenting the APIs

offered by each OpenStack module with a single graphical management console.

6. Keystone (identity service). Keystone is OpenStack’s central directory service, which manages

registration, authorization, and authentication of users. Keystone can integrate with existing

authentication services such as LDAP to reuse user credentials created elsewhere.

7.

Glance (image service). Glance is OpenStack’s repository of disk and server images, which can

be used to store and quickly deploy predefined virtual machines (for example, an Ubuntu web

server or database server or a CentOS development machine). Images may be stored locally within

a single OpenStack cloud or shared across a number of clouds with querying via a standard REST

interface.

https://ceph.com/

https://www.opennetworking.org/

https://www.opennetworking.org/

https://ceph.com/




8. Ceilometer (telemetry). Ceilometer offers a single repository for storing usage data from

across an OpenStack cloud. This usage data is intended to support billing systems and audit

processes, and it also aids in the general monitoring of a cloud’s performance under load.

9.

Heat (orchestration). Heat is OpenStack’s orchestration service, designed to support human

and machine-driven management of a cloud, its infrastructure, and its applications. Heat’s

primary focus is the management of infrastructure, but it is designed to work with widely used

software-configuration tools such as Puppet (see disclosure) and Chef in order to offer an

integrated view across the whole.

( Disclosure: Puppet Labs is backed by True, a venture capital firm that is an investor in the parentcompany of Gigaom.)

Adoption to date

According to October 2013 results from the OpenStack Foundation’s ongoing survey of its users,

OpenStack adoption broadly mirrors trends observed in other cloud activities. The majority of reported

deployments are small, with 45 percent constituting less than 100 virtual machine instances and only 6

percent with more than 10,000 instances. Similarly, 67 percent of deployments are across fewer than 50

physical servers, and only 8 percent require more than 1,000. OpenStack use is still dominated by proofs

of concept, with 32 percent of survey respondents reporting running some form of production workload.

Open-source technologies dominate the environments in which OpenStack was deployed at the time of

the survey, with Linux distributions such as Ubuntu (55 percent overall) and CentOS (24 percent overall)

clearly the default choice for host operating systems at all scales of deployment. The KVM hypervisor

used by many Linux distributions is also dominant in 62 percent of responses, but Microsoft’s HyperV

and VMware’s ESX also make the list of chosen hypervisors (3 percent and 8 percent, respectively). The

appearance of enterprise-grade networking from Cisco (10 percent) and VMware’s Nicira (6 percent) as

well as storage solutions from the likes of NetApp (8 percent) and EMC (3 percent) combine to suggest

that some, at least, are trying to integrate OpenStack with solutions less frequently associated withadopters of open-source projects. Effective deployments that include these companies’ mainstream

solutions will, of course, be key to more-widespread adoption of OpenStack in the future.

http://www.openstack.org/blog/2013/11/openstack-user-survey-october-2013/



https://www.openstack.org/user-survey/








Building a bridge VMware as a company is keen to remain relevant as its biggest customers move from a largely virtualized

IT infrastructure (that VMware dominates) toward a model in which public and/or private clouds play an

increasingly significant role. Equally, those advocating the greater adoption of cloud infrastructure

benefit if prospective customers see that their new cloud projects will be able to leverage existing

investment in the virtualization of their data centers. For the moment, at least, it is in the interests of both

VMware and the cloud’s champions to be seen to be working together, even as each works to extend the

reach and capability of its own emergent alternative solutions (VMware’s private cloud and hybrid cloud

vCloud offerings, for example).

Integration between OpenStack and VMware

Source: VMware




Officially sanctioned and supported free drivers already exist to support interoperation between

OpenStack’s Nova nodes and vSphere’s compute cluster capabilities and to direct OpenStack Cinder

requests to vSphere’s storage services. There are also drivers in Canonical’s OpenStack distribution to

exploit the software-defined networking (SDN) capabilities of Nicira NVP (acquired by VMware and now

marketed as VMware NSX) within OpenStack’s Neutron.

http://ir.vmware.com/releasedetail.cfm?ReleaseID=756729








SDN: hype or value?The virtualization of compute and storage is relatively well-understood and accepted within the broader

IT industry. As discussed above, the majority of enterprise-compute workloads are now virtualized, and

the creation of virtual pools of storage is also well-advanced. The virtualization of networking — or

software-defined networking (SDN) — is at an earlier stage of adoption. Many organizations have

significant investments in perfectly serviceable physical network devices from established incumbents

such as OpenStack Foundation member Cisco and are at an early stage in evaluating the additional

benefits of virtualizing their network. VMware’s 2012 acquisition of SDN startup Nicira was one

validation of the trend, and even stalwarts of the physical-networking paradigm today offer SDN products.

OpenStack’s Neutron module is designed to integrate with existing SDN projects such as OpenFlow, as

well as connecting relatively easily to commercial SDN products from VMware and others.

Software-defined networking is at an early point in adoption, but most indicators suggest that the SDN

market is heading toward significant growth. The foundations laid in Neutron should enable those

deploying OpenStack clouds to benefit from a wide range of SDN solutions as these emerge in the market.

http://gigaom.com/2012/07/23/vmware-to-buy-nicira-for-1-26b-in-a-strategic-leap-of-faith/



http://www.cisco.com/web/solutions/trends/open_network_environment/index.html








Managing changeOrganizations with an existing investment in server virtualization from VMware or one of its competitors

would not likely consider throwing that investment away in order to move wholesale to a completely new

cloud. However, even organizations with fully virtualized IT infrastructure will benefit from the elastic

and self-service nature of a well-architected cloud solution. Adding the ability to draw on additional

compute capacity from outside the data center when required simply makes the proposition more

compelling.

VMware’s own cloud products offer one means of achieving these ends, but it is also increasingly feasible

to implement more-open cloud environments (such as OpenStack) without giving up any of the benefits

seen in the already virtualized data center.

Use of the same hypervisor (e.g., KVM) and operating system (e.g., Ubuntu) both on- and off-premise

certainly simplifies that process of extending a cloud, but cooperation among the technology companies

in this space means it is often possible to move workloads across architectures. PayPal, for example,

integrates its existing VMware investment with an OpenStack cloud. That cloud combines virtual

machines using both OpenStack’s dominant KVM hypervisor and VMware’s ESX under a single

management layer.

As OpenStack matures, the code distributions from various partners are becoming increasingly robust

and more tailored to deployment in the sort of mixed environments likely to be found in many production

settings. Both Canonical and Mirantis, for example, offer their own OpenStack distributions, and both

have signed agreements and undertaken development work with VMware to simplify real-world

deployments like PayPal’s.

Production environments are rarely as neat and single-source as the clusters used for pilot deployments

or devtest activities. There are no convincing indications that IT buyers are likely to restrict their options

by buying more from a smaller set of vendors, which would suggest that the IT landscape will continue to

be diverse and complex. Indeed, as the number of choices on the market continues to expand, the

complexly diverse nature of most IT deployments will only grow. As such, efforts to improve

interoperability among different pieces of the whole should be welcomed, and activity to improve

interoperability among VMware solutions and open-source clouds powered by OpenStack is one recent

example of this.

https://www.openstack.org/summit/openstack-summit-hong-kong-2013/session-videos/presentation/marriage-of-openstack-with-kvm-and-esx-at-paypal








We have come a long way since early hype-filled discussions in which OpenStack was often unrealistically

pitched as a direct replacement for much of an enterprise's existing IT estate. There is now far less

interest in simply replacing existing systems and processes and far more in discovering the most cost-

effective and advantageous ways to blend the best of old and new.

OpenStack has clearly reached a level of maturity at which it is feasible to deploy for key workloads inside

the enterprise data center. The project's rich partner ecosystem includes both the technical

underpinnings to integrate established infrastructure and systems (such as VMware-based virtualization)

and the consultancy and services expertise to support these deployments in production environments.

For those who are ready to embrace a hybridized solution and who wish to reduce the perceived risk of

becoming too dependent on a single technology partner, it's time to seriously explore the opportunity

offered by the OpenStack ecosystem.




Key takeaways The virtualization of servers is increasingly common, especially in larger enterprise data centers,

and VMware continues to dominate this market today.

OpenStack attracts much of the attention in the open-source cloud space. Adoption still lags far

behind industry leader Amazon, but a growing number of organizations publicly support

OpenStack. These include public and private cloud operators such as Rackspace, Hewlett-Packard,

IBM, and others, as well as smaller companies like Canonical and Mirantis, which can help with

local OpenStack deployments.

Virtualization is a step on the path toward cloud deployment, and it introduces many of the

concepts and procedures needed for an effective cloud.

Organizations do not need to adopt a VMware cloud solution to benefit from existing investment

in VMware virtualization.

Equally, there is no need to throw away existing investment in virtualization in order to build an

OpenStack cloud.

VMware is an active member of the OpenStack Foundation, and there are supported drivers that

simplify the process of managing VMware virtual machines within an OpenStack cloud.

OpenStack continues to evolve, with new code released every six months. There may be value in

working with a partner if you are deploying an OpenStack cloud for production workloads.



About Paul MillerPaul Miller is an analyst and consultant, based in the East Yorkshire (U.K.) market town of Beverley and

working with clients worldwide. He helps clients understand the opportunities and pitfalls around cloud

computing, big data, and open data, as well as presents, podcasts, and writes for a number of industry

channels. His background includes public policy and standards roles, several years in senior management

at a U.K. software company, and a Ph.D. in Archaeology.

Paul was the curator for GigaOM Research’s infrastructure and cloud computing channel during 2011,

routinely acts as a moderator for Gigaom Research webinars, and has authored a number of underwritten

research papers such as this one.

About Gigaom ResearchGigaom Research gives you insider access to expert industry insights on emerging markets. Focused on

delivering highly relevant and timely research to the people who need it most, our analysis, reports, and

original research come from the most respected voices in the industry. Whether you’re beginning to learn

about a new market or are an industry insider, Gigaom Research addresses the need for relevant,

illuminating insights into the industry’s most dynamic markets.

Visit us at: research.gigaom.com.

© 2014 Giga Omni Media, Inc. All Rights Reserved.

This publication may be used only as expressly permitted by license from Gigaom and may not be accessed, used, copied,distributed, published, sold, publicly displayed, or otherwise exploited without the express prior written permission of Gigaom. Forlicensing information, please contact us.

http://pro.gigaom.com/


mailto:[email protected]





Documents

Architect Openstack