Pivotal CF on Vblock Systems

vce.com

© 2014 VCE Company, LLC. All rights reserved.

PIVOTAL CF ON VBLOCK™ SYSTEMS – SOLUTION AND INFRASTRUCTURE

SCALING OVERVIEW

Solution Validation

January 2014

VCE TECHNOLOGY ALLIANCE PARTNER

© 2014 VCE Company, LLC. All rights reserved. 2

Terminology ......................................................................................................... 3 Business Case ..................................................................................................... 4 Executive Summary ........................................................................................................................ 4 Scope .............................................................................................................................................. 5 Audience ......................................................................................................................................... 5 Why Pivotal CF on Vblock Systems? .............................................................................................. 5 New Approach ................................................................................................................................ 6 Vblock™ System Advantage ............................................................................... 7 Vblock System 300 ......................................................................................................................... 7 Vblock System 700 ......................................................................................................................... 7 Vblock™ Data Protection and Mobility ............................................................................................ 7 VCE Vision™ Intelligent Operations ............................................................................................... 7 Solution Architecture .......................................................................................... 8 Pivotal CF Architecture Overview ................................................................................................... 9

Pivotal CF Elastic Runtime Service ........................................................................................................... 10 Pivotal CF Operations Manager ................................................................................................................ 11 Automated Health Management ................................................................................................................ 13 Pivotal CF Services ................................................................................................................................... 13

Infrastructure Layer Monitoring and Provisioning ......................................... 14 Monitoring ..................................................................................................................................... 14 Provisioning .................................................................................................................................. 16 Infrastructure Validation and Performance ..................................................... 16 Testing Scope ............................................................................................................................... 16 Test setup ..................................................................................................................................... 17 Compute........................................................................................................................................ 18

StressTest ................................................................................................................................................. 18 Normalization ............................................................................................................................................. 18 Vertical Scaling .......................................................................................................................................... 20 Horizontal Scaling ...................................................................................................................................... 22

Storage.......................................................................................................................................... 24 Network ......................................................................................................................................... 24 Findings ......................................................................................................................................... 25 Conclusion ......................................................................................................... 25 Next Steps..................................................................................................................................... 25


Terminology Table 1 shows the terminology and abbreviations used in this document.

Abbreviation Description

vCPU Virtual or Logical CPU within VMware and Pivotal CF

Threading A code task spawned as a discrete block of code executed by a vCPU

IO latency Read and write latency in a virtual environment

Transaction Any relational SQL or other database operation that must succeed or fail or roll back

Virtual memory Memory presented to an application executing in Pivotal CF supported platform

Droplet “A droplet is the result of the application staging process, it is an uploaded application to which a buildpack has been applied. It is the original application (eg., a .WAR file for Java), that has been compiled with all the required libraries called out in the buildpack. The DEA has a wrapper around it that accepts one input—the port where it should listen for HTTP requests—and has two methods, a start and a stop.”1

DEA Droplet Execution Agent stages and runs applications in Linux-virtualized containers, called Warden containers. Warden is an enterprise grade container technology developed by Pivotal, and is similar to LXC or Docker. The DEA subscribes to messaging from the Cloud Controller and responds to requests by creating Wardens as required. The DEA receives droplets and then starts or stops them as required. Information about the droplet instances is periodically broadcasted using NATS.

Warden “Warden manages isolated, ephemeral, and resource-controlled environments in a virtual machine. The primary goal of Warden is to provide a simple API for managing isolated environments. These isolated environments, or containers, can be limited in terms of CPU usage, memory usage, disk usage, and network access. As of writing, the only supported OS is Linux.”2

Cloud Controller “The Cloud Controller is written in Ruby and provides REST API endpoints for clients to access the system. The Cloud Controller maintains a database with tables for orgs, spaces, apps, services, service instances, user roles, and more.”3

(GO) Router “The Router4 routes traffic coming into Cloud Foundry to the appropriate component— usually Cloud Controller or a running application on a DEA node. The router is implemented in GoLang. Implementing a custom router in GoLang gives full control over every connection to the router, which makes it easier to support WebSockets and other types of traffic (e.g. via HTTP CONNECT) in the architecture5. All routing logic is contained in a single process, removing unnecessary latency.”

1 http://docs.cloudfoundryPivotal CF.com/docs/using/terms.html 2 http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/warden.html 3 http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/cloud-controller.html 4 http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/router.html 5 http://docs.cloudfoundry.com/docs/running/architecture/


Abbreviation Description

Health Manager “The Health Manager is essential to ensuring that apps running on Cloud Foundry remain available. It is needed to restart applications whenever the DEA running an app shuts down for any reason; Warden kills the app because it violated a quota; or the application process exits with a non-zero exit code.”6

NATS “NATS is a lightweight publish, subscribe and distributed queuing messaging system written in Ruby. The various Cloud Foundry components use NATS to communicate with each other.”7

UAA “The UAA is the identity management service for Cloud Foundry. Its primary role is as an OAuth2 provider, issuing tokens for client applications to use when they act on behalf of Cloud Foundry users. It can also authenticate users with their Cloud Foundry credentials, and can act as an SSO service using those credentials (or others). It has endpoints for managing user accounts and for registering OAuth2 clients, as well as various other management functions.”8

BOSH “BOSH is an open source tool chain for release engineering, deployment, and lifecycle management of large-scale distributed services.

BOSH was developed to deploy Cloud Foundry, however it has a more general purpose —it can be used to deploy other distributed services”9

Instance A single application droplet running in a Warden container constitutes one instance.

Business Case

Executive Summary Enterprises are constantly looking for new ways to gain competitive advantage. In today’s rapidly- evolving global landscape, they are increasingly more aware that technology is at the heart of what is driving business success, and fine-tuning the IT investment strategy is imperative. One of the key technology-enabled initiatives is to turn a collection of application infrastructure, development and related services; such as application platform, integration, business process management, and database services; into a software model with a platform-as-a-service (PaaS) approach. With PaaS organizations can drive faster time-to-results, gain better business outcomes, and enable new business opportunities with productivity and scale of “software.”

However, to become a software-driven enterprise, organizations are often impeded by the complexity of deploying, upgrading, and scaling a PaaS integrated with Infrastructure-as-a-Service (IaaS) on their private cloud. Pivotal CF, the leading enterprise PaaS solution, powered by Cloud Foundry, is built to solve this complexity and to deliver a turnkey experience for scaling and updating PaaS on the private cloud. It enables developers to provision and bind leading application and data services on a unified platform with virtually no down time. It also empowers businesses to deliver applications and update them with new features at a velocity and scale previously only available to Internet giants. Another key aspect of gaining faster time-to-results with PaaS is to deploy it on a converged system acting as a standardized infrastructure with adaptability and investment protection. It delivers a pre-engineered, pretested, and fully-documented approach, allowing for a tremendous improvement in the time and expense to bring an IaaS offering to market. Vblock™ Systems from VCE, the

6 http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/health-manager.html 7 http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/messaging-nats.html 8 http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/uaa.html 9 http://docs.cloudfoundryPivotal CF.com/docs/running/bosh/


world’s most advanced converged infrastructures, provide predictable performance at scale and are integrated with industry-leading solutions for virtualization, collaboration, and data protection. Vblock Systems also support a range of applications and mixed workload environments, helping IT meet aggressive timelines for new initiatives and respond quickly to changing business needs. This is why a growing number of organizations are pursuing the deployment of Pivotal CF on Vblock Systems.

This document will look at the combined synergies that can be gained by deploying Pivotal CF on a Vblock System. In this document, we will:

! Introduce the solution benefits of Pivotal CF and Vblock Systems

! Provide an overview of Pivotal CF architecture and Vblock Systems

! Explain how infrastructure monitoring and provision can be automated

! Validate and measure how infrastructure components of Vblock Systems function with Pivotal CF

The combination of Pivotal CF from Pivotal and Vblock Systems from VCE, serves as an ideal foundation for a converged stack of IaaS and PaaS, enabling organizations to deploy unique cloud applications to outperform in the marketplace, control costs and risks, and drive innovations.

Scope This paper describes high-level solution architecture, key functions for scaling Pivotal CF, infrastructure monitoring and provisioning, and performance and validation when deploying Pivotal CF on a Vblock System. It assumes that a reader has a high-level understanding of IaaS and PaaS for cloud deployment. The lower level of functional discussion is out of scope for this document.

Audience This document is intended for large enterprises and service providers that are evaluating or planning to deploy Pivotal CF on Vblock Systems.

Why Pivotal CF on Vblock Systems? In today’s dynamic environment, agility is vital to turning cloud-based services into strategic advantage. Many organizations across the industries are looking to PaaS offerings to develop new applications quicker and at great scale. At the same time, adoption of converged systems has accelerated for an enterprise to take advantage of the faster time-to-value, higher availability, and lifetime assurance unique to the pre-integrated, tested, and validated solutions.

Pivotal One™ is a comprehensive, multi-cloud Enterprise PaaS comprised of a set of application and data services that run on top of Pivotal CF™, the leading enterprise distribution of the Cloud Foundry® platform. This new solution will allow enterprise companies to bring new innovation to market faster than ever before, enabling agile development teams to rapidly update and scale applications across public or private clouds, and instantly expand and upgrade with no downtime.

Vblock Systems from VCE also represents the next evolution of IT—one that unleashes simplicity by delivering the extraordinary efficiency and business agility of virtualization and cloud computing. Seamlessly integrating best-in-class compute, network, and storage technologies from industry leaders, such as Cisco, EMC, and VMware, Vblock Systems provide dynamic pools of resources that can be intelligently provisioned and managed to address changing demands and business opportunities.

By deploying Pivotal CF on Vblock Systems, organizations are better positioned to achieve the speed, agility, and predictability needed to be successful in deploying business-critical cloud applications. This solution white paper describes the VCE solution for implementing Pivotal CF on Vblock System 300 and Vblock System 700 that are suited to enterprises and service-provider environments.


New Approach IT has been maturing its use of the cloud to drive a better business outcome for differentiation and innovation. Traditionally, cloud-based IT implementations are challenged with gaps in operations, applications, and infrastructure from people, process, and technology standpoints.

Typical challenges include:

! Inefficiency and delay in managing the application development process in disparate, heterogeneous environments

! Limited adaptability in managing disparate infrastructures to handle multiple application developments across the lifecycle

! Lack of predictability and scaling for the future, as transitioning from individual development environments to QA and production

In order to solve these challenges, we need a solution that can

! Transform individual silos into a single platform

! Address a hybrid cloud approach that incorporates private, public, and existing environments

! Evolve legacy applications into a new platform

By taking a single platform for a hybrid, multi-cloud environment that embraces legacy applications and systems, an organization can achieve the following benefits:

! Speed – Time-to-Value

o Fast—increase velocity of IT service contribution to business from application development to infrastructure deployment and monitoring

o Improved—deploy high value-generating applications quickly instead of managing scripting

! Agility – Operational Efficiency

o Adaptable—scale environments elastically based on business demand

o Extensible—embrace and transform legacy environments to a platform approach

! Predictability – Reduced Risk and Cost

o Standardized—use scale as an advantage by replicating a proven, highly available deployment across data centers

o Manageable—increase portability across private/public infrastructure while reducing footprint and power needs

Table 2 shows the business benefits of using Pivotal CF on a Vblock™ System.

Benefit Pivotal CF Vblock Systems

Speed – Time-to-Value ! Speed application and data service deployment from months to days

! Scale hundreds of application instances in seconds

! Get up-and-running with 40+-day lead time

! Meet 5X faster time for new services

Agility – Operational Efficiency

! Ensure faster updates and upgrades with minimal downtime

! Reduce infrastructure wiring, and eliminate deployment bottlenecks

! Scale storage/network compute tuned to demand

! Rapidly deploy mixed workloads on a single platform


Benefit Pivotal CF Vblock Systems

Predictability – Reduced Risk and Cost

! Improve application performance and availability

! De-risk and separate concerns of application development and operations

! Achieve 96% reduction in downtime

! Reduce annual data center costs by up to 50%

Source: VCE Systems metrics based on DC White Paper: Convergence with Vblock Systems: A Value Measurement, September 2013

Vblock™ System Advantage The expanded Vblock Systems portfolio is built on a single, highly-secure, standardized, market-leading infrastructure. Pre-integrated, tested, and validated, the systems can be quickly deployed, easily distributed, and counted on to increase the predictability and timeliness of IT projects for solution providers and end users alike. In this validation and performance paper, we simulate a hybrid Vblock System 300 and Vblock System 700 environment that addresses large enterprise and service-provider deployment needs.

Vblock System 300 The Vblock System 300 family is ideally suited to achieve the scale needed for large virtualization and cloud computing implementations. Engineered and validated to address a broad spectrum of mixed-workloads, the Vblock System 300 family offers growing enterprise businesses a range of choices to minimize business risk and Total Cost of Ownership (TCO). Additional information on Vblock System 300 can be found at http://www.vce.com/products/vblock/system-300

Vblock System 700 The Vblock System 700 family is comprised of enterprise- and service provider-class systems designed to help organizations benefit from virtualized and cloud computing—faster and easier. The systems are designed for mission-critical application environments, offering the highest level of data and application availability. Additional information on Vblock System 700 can be found at http://www.vce.com/products/vblock/system-700

Vblock™ Data Protection and Mobility Vblock™ Data Protection and Mobility Solution contributes to true peace of mind by providing your choice of safeguards, optimized for virtualized or cloud environments. Count on highly reliable backup and recovery, data replication, business continuity, and workload mobility that ensure critical protection for applications and data running on Vblock Systems. Additional information on Vblock Data Protection can be found at http://www.vce.com/products/integrated/data-protection.

VCE Vision™ Intelligent Operations VCE Vision™ Intelligent Operations software enables and simplifies converged operations by dynamically providing a high level of intelligence to your existing management toolset. This next generation of the software will enable IT to easily update non-compliant software or firmware and quickly scan for and report on configuration security issues in need of remediation. VCE offers unprecedented choice in the management platforms used with Vblock Systems. Plus, using VMware vCenter Operations Manager for Vblock System analytics will be significantly simplified—for maximum utilization and performance. More information is available at http://www.vce.com/products/vblock/vce-vision.


Solution Architecture This paper will highlight the combined benefits that can be achieved by the deployment of the leading platform-as-a-service (PaaS) offering, Pivotal CF based on Cloud Foundry, with the leading converged infrastructure-as-a-service (IaaS) offering engineered by VCE as shown in Figure 1.

Vblock Systems from VCE is an optimized information technology system that accelerates the adoption of converged infrastructure and cloud-based computing models that dramatically reduce the cost of IT while improving time-to-market for customers. Consisting of industry leading compute, network, and storage solutions, VCE products enable IT organizations to

! Standardize processes and applications

! Improve utilization and lower operational costs

! Accelerate technology adoption and innovation

! Maximize adaptability and investment protection

Pivotal CF is the leading enterprise PaaS, developed by Pivotal on the popular open-source Cloud Foundry platform. It delivers an always-available, turnkey experience for scaling and updating PaaS on the private cloud. Just as the IaaS layer provides an abstraction from the infrastructure components, the PaaS layer offers the abstraction of key components of the data-driven software platform. This includes an Elastic Runtime Service for deploying, scaling, and managing applications; a flexible multi-language buildpack architecture that allows for the detection, download, and configuration of the appropriate runtimes, containers, and libraries; an extensible-services architecture that allows for the dynamic binding of internal and external services to the Pivotal CF platform; and an Operations Manager (CF BOSH) for deploying, managing, scaling, and updating a PCF installation.

Figure 1 illustrates how Pivotal and VCE high-level approach that address PaaS and IaaS stack together


Pivotal CF Architecture Overview

Figure 2 illustrates the solution components within Pivotal CF that interface with Infrastructure through VMware

Cloud Foundry is not a monolithic software stack. Rather, it is a scalable set of idempotent distributed services that operate in concert to provide a highly reliable and scalable PaaS environment. As shown in Figure 2, this is the Cloud Foundry inner shell. Components are dynamically-discoverable and loosely-coupled, exposing health through HTTP endpoints so agents can collect state information (app state and system state) and act on that information.

Pivotal CF offers a comprehensive set of the essential elements of an enterprise platform-as-a-service (PaaS) offering that allows customers to change the way they manage software in order to compete in this new industrial era. These essential elements allow enterprises to become more agile in their software delivery by allowing enterprises to concentrate on the components of their software that will provide real business value and not spend time worrying about the infrastructure and middleware configurations that tie up countless months of critical-resources time.

In the following section, we will examine three key components of Pivotal CF architecture that are crucial for performance and scalability considerations.


Pivotal CF Elastic Runtime Service

Figure 3 depicts the Pivotal CF Elastic Runtime Service

Pivotal CF provides the ideal self-service platform for development teams of any size to start small and rapidly evolve new ideas into complex applications, increasing business agility. Developers upload their application source files using the CF CLI or via plugins for popular IDE and build tools like Eclipse, Maven, Gradle, Jenkins, Bamboo, and more. The simplified semantics of the CF CLI and REST API allow users to manage and scale applications, as well as create and bind zero-touch data services. Pivotal CF’s role-based control and lifecycle staging isolation enable different teams to be responsible for early development, testing, staging, and ultimately production. A web console allows developers and application owners to monitor, manage, and administer users and services.

Developers simply push applications to Pivotal CF for an “it just works” experience. Pivotal CF uses a flexible approach called buildpacks to dynamically assemble and configure a complete runtime environment for executing a particular class of applications. Rather than specifying how to run applications, developers can rely on buildpacks to detect, download, and configure the appropriate runtimes, containers, and libraries.

Figure 3 above shows how the various Pivotal CF components work together to allow a developer to deploy their applications. A developer will communicate what application they want to push to the platform and which service to bind to the application. The developer does not have to be concerned with the underlying plumbing, or how these components are glued together. The Pivotal CF components handle the service provisioning, application staging, and service bindings. The CF CLI also provides the ability for the developer to scale out the number of instances of an application that are deployed by using a simple CF scale command and entering the number of instances to deploy.

The Pivotal CF platform handles the deployment of the application into the droplet execution agent (DEA). The DEA stages and runs applications in Warden containers that are responsible for managing isolated and resource-controlled environments. Once deployed, the application is available for end-user access. The platform also ensures that the appropriate routing configuration is handled within the platform through the Router component. An end-point URL is provided to access the application, and the application is available in the elastic runtime of the pivotal CF platform. Figure 4 below shows how the Pivotal CF components work together to provide application services to the end user based on the deployed application.


Figure 4 depicts the Pivotal CF workflow to provide applications to the end users

Pivotal CF Operations Manager The Pivotal CF Operations Manager is an enterprise PaaS management component that manages all aspects of the platform, from the one-click installation through the intuitive interface for horizontal and vertical scaling of the platform, applications, and services, with zero downtime. Pivotal CF is the only PaaS that supports direct IaaS API integration for turnkey deployment and full life-cycle management. Pivotal CF Operations Manager installs from a single virtual appliance (OVA). From there, Operations Manager fully automates the deployment of Pivotal CF, using IaaS APIs, onto a resource pool of Linux VMs with Linux containers.

Scaling of Pivotal CF components is done using the same Operations Manager console as shown in Figure 5. PaaS administrators make changes to the number of instances in the settings tab of the runtime services and perform an install. Pivotal CF does a comparison of the currently-deployed counts and deploys additional components for each component size that has increased. This allows the PaaS administrator to dynamically scale each of the components independently based on the scope of the applications and end-user traffic.


Figure 5 shows the Pivotal CF Operations Manager resources

The Pivotal CF Operations Manager also allows for the same installation and management capabilities for services hosted in the Pivotal CF runtime such as Pivotal HD or Rabbit MQ. The interface provides the ability to specify the same type of resource-sizing parameters that includes the instance counts, CPU, RAM, and disk space. These capabilities provide for a consistent operations capability across the entire Pivotal CF platform.

Figure 6 shows the Pivotal CF Operations Manager for Pivotal HD


Automated Health Management

Figure 6 shows the Pivotal CF Application Health Management

The Pivotal CF Health Manager component is a third-generation application health manager that continuously monitors all applications running on a Pivotal CF instance and intervenes to make the actual state match the intended state as shown in Figure 6 above.

This is an important element of an enterprise PaaS that will be critical for every enterprise. The Health Manager and the DEA work in concert to ensure that the expected applications, along with the number of instances, are maintained. In the event of an application failure, the Health Manager will communicate to the Cloud Controller any anomalies detected in the expected applications, and the Cloud Controller will automatically issue commands to restart the expected application instances.

Another part of health monitoring is ensuring that each application is given its fair share of the available PaaS resources, and that no individual application can adversely impact the overall health of the PaaS environment. The DEA runs applications within Warden containers that are responsible for managing isolated- and resource-controlled environments. Quotas for resources such as memory and CPU can be set for each Warden container in the Pivotal CF environment. The DEA monitors each Warden for adherence to the assigned quotas and will kill a process that is in violation of these quotas to ensure a healthy platform for all applications hosted in the PaaS. The normal health management process as described previously will kick in again, and new instances of any killed applications can be restarted.

Pivotal CF Services Enterprise applications rely on services to provide capabilities such as messaging and data persistence. Pivotal CF provides a flexible architecture to incorporate these services with applications hosted in the Pivotal CF runtime. Pivotal CF provides two ways in which services can be used by applications: managed services and user-provided services. Managed Services have been integrated via APIs and provides for the provisioning of new service instances and credentials on demand. User-provided Service Instances are a mechanism to deliver credentials to applications for service instances that have been pre-provisioned outside of the Pivotal CF platform.

The managed services approach is desirable when enterprises want to provide a set of services that can be dynamically provisioned for new application instances along with being managed within the Pivotal CF runtime. Pivotal provides a series of managed services including the following;

! Pivotal HD Service – this service provides the ability to build, manage, and scale Hadoop as a natively-integrated Pivotal CF Service. Via the Service Broker, applications can bind to this service, automatically


assigning capacity in Hadoop Distributed File System (HDFS), a database in HAWQ, and a resource queue in YARN. This reduces development cycle time by eliminating common Hadoop development complexities around deployment, security, networking, and resource management.

! Pivotal AX Service – this service offers a self-service analytics environment to the teams responsible for the creation, collection, storage, querying, and visualization of data and lowers the barriers of entry to analyze Pivotal CF-deployed applications and relevant supporting data to discover and communicate meaningful patterns that affect profitability and future product direction. Pivotal AX service is on-premises analytics software that is purpose-built on Pivotal HD and deploys and scales as a Cloud Foundry Service.

! Pivotal RabbitMQ Service – this service provides increased application speed, scalability, and reliability by delivering asynchronous messaging to applications. A message broker for applications running on Pivotal CF, RabbitMQ Service applications can integrate with other Pivotal CF applications and with applications outside Pivotal CF using the service broker.

In addition, Pivotal provides a fully-documented Service Broker API to allow developers or service providers to incorporate their own set of services into the Pivotal CF runtime.

In scenarios where an enterprise already has existing, pre-provisioned services that live outside of the Pivotal CF platform, a user-provided service instance can be utilized to bind an application hosted in the Pivotal CF runtime to this external service. This flexibility accommodates all types of application and service configurations.

Infrastructure Layer Monitoring and Provisioning

Monitoring One important consideration in a PaaS strategy is that, while the PaaS layer does automate the component placement and installation on top of the IaaS layer, it does not take over the initial and ongoing operations and management of the underlying IaaS layer. Existing IaaS administrators will continue with their operations management using their tools of choice to manage, monitor, and maintain the operational efficiencies of the IaaS Platform.

A tool such as vCenter Operations Manager from VMware can greatly simplify and automate operations management of the IaaS layer. vCenter Operations Manager provides real-time analytics along with capacity management capabilities that are critical functions for the management and monitoring of the IaaS layer. The real-time analytics of vCenter Operations Manager works by learning the operational patterns in the infrastructure layer, and once the normal patterns are recognized, the system detects anomalies and alerts the IaaS administrator of potential issues with the infrastructure shown in Figure 7 below.


Figure 7 shows the vCenter Operations Manager

The vCenter Operations Intelligent Operations provides the follow capabilities:

! Proactively avoid performance issues and gain deep insights into the health, risk, and efficiency of your virtual and physical infrastructure, operating systems, and applications.

! Performance data is abstracted to health, risk, and efficiency measures that allow IT to efficiently identify building performance problems with less effort.

! Customizable operations dashboards visualize key performance indicators, provide role-based access, and enable better collaboration across infrastructure, operations, and applications teams.

! Integrated capacity management and cost metering capabilities track the usage, cost, and value of IT resources being consumed.

! Intelligent operations groups provide operational visibility into logical groups of resources and applications.

! Flexible group policies prioritize alert types and notifications for business-critical applications, production workloads, or business units.

Planning for capacity involves analyzing, optimizing, and forecasting data to determine how much capacity is available and whether you are making efficient use of the infrastructure. vCenter Operations Manager provides a series of badges and reports that allows IaaS administrators to understand remaining capacity in the infrastructure based on current sizing and utilization information from the analytics shown in Figure 8 below.


Figure 8 shows the vCenter Operations Manager for capacity scenarios

Forecasting data for capacity risk in vCenter Operations Manager involves creating capacity scenarios to examine the demand and supply of resources in the virtual infrastructure. A what-if scenario is a supposition about how capacity and load change if certain conditions are changed, without making actual changes to your virtual infrastructure. If you implement the scenario, you know in advance what your capacity requirements are and what additional capacity might be needed.

Provisioning Organizations can also automate infrastructure deployment and provisioning, performing non-intrusive elastic provisioning to add or release compute resources within a given infrastructure service.

Enabled by VCE Vision Intelligent Operations, EMC Ionix Unified Infrastructure Manager (UIM), and VMware vCenter Operations Manager; infrastructure, application, and development teams can collaborate in configuring and provisioning network access, storage, and compute profiles. Traditional methods to stand up a new virtualization cluster that aggregates key data center functions such as compute, networking, storage, and memory has been shown to take upwards of 160 days.10 Starting with a Vblock System reduces this time due to pre-integrated components that are factory tested and pre-integrated. Additional tools from VCE, EMC, and VMware help to further automate the additional tasks of virtualizing the infrastructure and the associated management tasks. This includes capabilities such as automating the provisioning, compliance checking, firmware upgrades, and scanning and reporting on security configuration issues. All these capabilities enable enterprises to automate the provisioning and management of their IaaS layer and provide for greater agility.

Infrastructure Validation and Performance

Testing Scope Stress testing on a hybrid Vblock System (see Test Setup below for details on the configuration) was performed to identify how Pivotal CF would perform under stress on the underlying infrastructure components. This testing was

10 IDC Whitepaper – Convergence with Vblock Systems: A Value Measurement http://www.vce.com/asset/documents/noindex/idc-vblock-value-whitepaper.pdf


performed to help the infrastructure administrators identify how Pivotal CF utilizes the various infrastructure components, including CPU, memory, networking, and storage.

A vSphere cluster was configured based on the requirements specified in the Pivotal CF Getting Started guide. This guide recommends a minimal capacity based on the starting configuration of the Pivotal CF virtual machines that are deployed during the installation process. Table 3 below summarizes the virtual machines with their associated resource sizes after a Pivotal CF installation.

Table 3 summarizes the virtual machines with resource sizes

INSTANCES VM RAM (GB) DISK (GB) CPU

1 NATS 1 10 1

1 Health Manager 3 36 1

1 NFS Server 1 100 1

1 Cloud Controller Database 1 4 1

1 Cloud Controller 1 2 1

1 Router 1 2 1

1 Syslog 1 10 1

1 Collector 1 2 1

1 UAA Database 1 10 1

1 UAA 1 3 1

1 SAML Login 1 3 1

1 Console Database 1 3 1

1 DEA 16 11 2

1 Loggregator Server 1 2 1

1 Loggregator Router 1 2 1

6 Compilation 1 4 2

21 TOTALS 33 204 18

Test setup For this validation and performance testing, we have chosen a hybrid structure where Vblock System 700 and Vblock System 300, based on EMC Symmetrix VMAX and VNX, respectively.

Vblock System Environment

! EMC VMAX10K and VNX 5300

! 2 x UCS B200-M3 Blades

! 2 x Intel E5 2620 Processors

! 6 x 2.00 GHz cores per socket

! 2 Hyper-threads per core

! 192 GB RAM


! 96GB Licensed

! VMWare vCenter Server 5.1.0

! VMWare vSphere Hypervisor ESXi 5.1.0

Compute

StressTest The stress application performs a repeated series of CPU intensive tasks within its container with varying memory as part of the Compute testing. The application uses standard Cloud Foundry libraries and tools, and can be deployed into a standalone instance, or multiple, distributed instances. The StressTest application is not responsible for, nor does not task, storage or memory I/O.

Initial testing involved a single 8 vCPU DEA. A single 128MB instance of StressTest was uploaded into the DEA and invoked via HTTP request.

Normalization The size of the deployed instances was scaled to 256MB, 512MB, 1GB, and 2GB to confirm the StressTest CPU stress isolation as shown in Figure 9 below.

C:\node_stress_test\node_stress_test>cf instances stress_master

DL is deprecated, please use Fiddle

Using manifest file manifest.yml

Getting instances for stress_master... OK

instance #0: running

started: 2013-11-08 02:53:36 PM

Output of cf instances command illustrating number of running instances

Figure 9 shows averaged results of StressTest at incremental instance sizes

StressTest was also demonstrated to consume 100% of the deployed Warden. It was observed that a Warden is isolated to a single logical vCPU, instantiated within a DEA. The single StressTest instance deployed within the normalization phase of testing was found to completely consume 1 of the 4 vCPUs deployed to the host DEA. It was also found that the DEA successfully contained the StressTest Warden instance, by restricting access to additional resources, proving that application isolation worked as expected in Figures 9, 10, and 11.

8000

13000

18000

128MB 256MB 512MB 1G 2G

Stress 2/2000 Single Instance Computational Time (ms) at Instances 128MB -‐ 2GB


Figure 10 shows Single StressTest (node.js) consuming 1/8 DEA resources

Figure 11 shows vCenter view – DEA containment of warden resources


Vertical Scaling The next phase of testing involved spawning multiple instances of the StressTest within a single DEA with the intention of consuming up to 100% of DEA resources. For this test 8 StressTest instances were deployed into a single 8 vCPU DEA. Multiple calls to the StressTest application instance gateway were made: first 2, then 4, 6, and then 8; to demonstrate the effect of vertical scaling. As memory sizing was proved to have no impact to StressTest testing earlier, 512MB instances were deployed as shown in Figure 12.

C:\node_stress_test\node_stress_test>cf scale stress_master



Instances> 8

1: 128M

2: 256M

3: 512M

4: 1G

Memory Limit> 512M

Scaling stress_master... OK

Stopping stress_master... OK

Preparing to start stress_master... OK

Checking status of app 'stress_master'...

4 of 8 instances running (4 running, 4 starting)

Push successful! App 'stress_master' available at http://stresstest.cfapp.vce.superna.net

TIP: The system will continue to start all requested app instances. Try 'cf app' to monitor app status.

C:\node_stress_test\node_stress_test>cf instances stress_master



Getting instances for stress_master... OK


started: 2013-11-08 07:07:15 PM


started: 2013-11-08 07:07:15 PM



started: 2013-11-08 07:07:15 PM


started: 2013-11-08 07:07:15 PM


started: 2013-11-08 07:07:15 PM


started: 2013-11-08 07:07:15 PM


started: 2013-11-08 07:17:15 PM


started: 2013-11-08 07:07:15 PM

Figure 12 shows scaling up to 8 instances (0 to 7)

Table 4 illustrates compute resource consumption on a single DEA

Table 4 shows DEA OS CPU% utilization, reported by 'top'

Instances DEA CPU% Util. (TOP)

1 12.5

2 25

4 56.7

6 74.5

8 99.4

12.5 25

56.7

74.5

99.4

0

20

40

60

80

100

120

1 2 4 6 8

DEA OS CPU Utilization (TOP)


Figure 13 shows DEA VM resource consumption OS CPU utilization plot (TOP)

Horizontal Scaling While the previous section addressed vertical scaling of a single DEA on a single blade, this section addresses horizontal scaling across two blades. First an additional DEA must be deployed. This operation is performed by the PaaS administrator from the Pivotal CF Operations Manager interface. Pivotal CF Operations Manager is the graphical interface to BOSH as shown in Figure 14 below.

Figure 14 shows Pivotal CF Operations Manager resources allocation

The PasS administrator logs into Pivotal CF Operations Manager and increases the number of DEA instances. For this test, one additional DEA will be deployed to the second available blade. The configuration is then updated, and BOSH seamlessly and automatically deploys a new DEA for use as shown in Figures 15 and 16 below.


Figure 15 shows DEA VM resource consumption


Figure 16 shows Blade Resource Consumption

Storage As discussed earlier, Vblock System, as the IaaS component of this solution, can support elastic, non-disruptive, on-demand scaling of PaaS environment. EMC VNX and VMAX platforms, as part of the storage-tier of the infrastructure components, can be managed with EMC Unisphere with an easy-to-use wizard that provides an intuitive, context-based approach to configuring storage, creating replicas, monitoring the environment, and managing host connections.

Storage utilization within a Pivotal CF deployment will come from the initial platform storage requirements along with the storage requirements for the associated services utilized by the hosted applications. Typical storage-based services would be those involved with the persistence or sources of data the applications will utilize, such as a database service or a Hadoop platform, such as the Pivotal HD services described previously. The amount of storage these services will require will depend on the hosted application and the business scenario supported by the application. Application owners should be consulted in order to determine the specific storage requirements in these scenarios.

It is noted that Pivotal CF application design best practices recommends the avoidance of writing to the local file system. This means that application layer is state-less and that it does not share data.

This recommendation is due to two reasons: the local file system storage is short-lived, and instances of the same application do not share a local file system (see http://docs.cloudfoundry.com/docs/using/app-arch/ for more information on Application design consideration). Instead, it is recommended to use a database service, a cloud storage provider, or a cache service as alternative approaches.

Network This particular Vblock System configuration consists of the host blade (Cisco B200-M3), the UCS 6248, and the Nexus 5548. This allows for up to 80Gbits of Ethernet data to be forwarded to Cisco UCS B200 Blade on a Vblock System, when deployed with the Cisco 2208XP fabric extenders and VIC 1240/1280 combination.

For use with Pivotal CF, when the BOSH director creates a virtual machine through the BOSH IaaS interface, it passes along configurations for networking and storage, as well as the location and credentials for the message bus and the blobstore. A BOSH agent is also embedded in the template so that BOSH can take control of virtual machines.

VMware virtual interfaces provided by VMM have been tested and demonstrated as being effective in limiting the flow of Ethernet traffic to a configure rate, without impact to the VMM/Hypervisor, allowing normal operation to all


infrastructure systems, even when the virtual Ethernet ‘wire’ is saturated with traffic. Excess traffic is discarded, and the TCP layer (windowing) transparently throttles packet flow.

Findings We summarize the findings from this infrastructure scaling and performance testing on Pivotal CF on Vblock Systems below.

! Application Isolation: The Warden component of Pivotal CF manages the isolation of environments in terms of resource utilization. This capability will ensure that the applications are contained within the limits prescribed and will only utilize the CPU, memory, disk, and network quotas defined. This ensures that no single application will adversely affect the overall infrastructure platform or neighboring applications. When reviewing infrastructure-monitoring tools, infrastructure administrators seeing adverse stress to an individual application should work with their PaaS administrators to add additional instances of the application.

! Vertical scaling: the compute linearly scaled and increased utilization with added instances on one DEA on the first blade. The memory sizing was proved to have no impact to StressTest testing. DEA CPU utilization increased linearly with increasing StressTest Warden instances.

! Horizontal scaling: the compute also demonstrated that a new DEA on the second blade also shows the expected horizontal scaling. When one additional DEA was deployed to the second available blade, the configuration was updated, and BOSH seamlessly and automatically deployed a new DEA for use.

Conclusion Pivotal CF is a hypervisor agnostic platform that offers next generation lightweight application portability for rapid workload execution scale-up, regardless of provider or underlying operating system. Vblock Systems provide the world’s most advanced converged infrastructure to drive faster time-to-results, ensure lifetime assurance, and reduce cost and risk of operations. The combined capabilities of a Vblock System for converged infrastructure and the Pivotal CF platform for an enterprise PaaS empowers enterprises to drive agility into their application development and operation lifecycle.

In this paper, we

! Introduced the solution benefits of Pivotal CF and Vblock Systems

! Provided an overview of Pivotal CF architecture and Vblock Systems

! Explained how infrastructure monitoring and provision can be automated

! Validated and measured how infrastructure components of Vblock Systems function with Pivotal CF

Organizations seeking to gain competitive advantage by becoming a more agile enterprise can look toward deploying Pivotal CF on a Vblock System and take advantage of speed, agility, and predictability unique to the joint offerings with the approaches described in this document.

Next Steps To learn more about this and other solutions, contact a VCE representative or visit www.vce.com.

ABOUT VCE VCE, formed by Cisco and EMC with investments from VMware and Intel, accelerates the adoption of converged infrastructure and cloud-based computing models that dramatically reduce the cost of IT while improving time to market for our customers. VCE, through the Vblock Systems, delivers the industry's only fully integrated and fully virtualized cloud infrastructure system. VCE solutions are available through an extensive partner network, and cover horizontal applications, vertical industry offerings, and


application development environments, allowing customers to focus on business innovation instead of integrating, validating, and managing IT infrastructure. For more information, go to vce.com.

Copyright © 2014 VCE Company, LLC. All rights reserved. VCE, VCE Vision, Vblock, and the VCE logo are registered trademarks or trademarks of VCE Company LLC or its affiliates in the United States and/or other countries. All other trademarks used herein are the property of their respective owners.

!

Technology

Pivotal CF on Vblock Systems