Removing Barriers Between Dev and Ops

2© 2015 Pivotal Software, Inc. All rights reserved. 2© 2015 Pivotal Software, Inc. All rights reserved.

Removing Barriers Between Dev and Ops It Takes a Platform

Devops @ VMworld1 September 2015

Cornelia Davis, Sr. Director, Cloud Foundry Platform Engineering@cdavisafc

3© 2015 Pivotal Software, Inc. All rights reserved.

Software is Changing Industries

$6B valuationFinancial Services

$3.5B valuationTravel & Hospitality

$19B valuation Transportation

$3.2B Acquisition by GoogleHome Automation

$20B valuation Entertainment

$1.1B acquisition Monsanto--Agriculture

4© 2015 Pivotal Software, Inc. All rights reserved.

Increased Expectations and Increased EfficienciesChanging development and release disciplines

• Agile/Developer Productivity/TDD• Continuous Delivery • DevOps• Cloud-native App Design• Culture/Talent

Time

Expectations& Efficiency

Innovation Gap

IaaS

Automation

DevOps, CD, Agile

Traditional IT

Startups

Enterprise PaaS

5© 2015 Pivotal Software, Inc. All rights reserved.

“Operations is the secret sauce”

http://radar.oreilly.com/2007/10/operations-is-a-competitive-ad.html

6© 2015 Pivotal Software, Inc. All rights reserved.

the difference is platformsthe difference is third platform

7© 2015 Pivotal Software, Inc. All rights reserved.

8© 2015 Pivotal Software, Inc. All rights reserved.

Cloud Foundry is…

The world’s leading open source platform-as-a-service. Supported by dozens of major organizations Language and framework agnostic Manages both VMs and containers Orchestrates both applications and data services Abstracts Infrastructure

Founded and commercialized by Pivotal, Inc.

9© 2015 Pivotal Software, Inc. All rights reserved.

An (Overly) Simple View of the World

Applications Stateless Run in Containers Horizontally Scalable Disposable No permanent storage

Data Services Stateful Run in Virtual Machines Multi-tenant Diagonally scalable Durable storage

10© 2015 Pivotal Software, Inc. All rights reserved.

Cloud Foundry: Applications and Services

Services (virtual machines): managed by “BOSH”

Applications (containers): managed by “Runtime”

11© 2015 Pivotal Software, Inc. All rights reserved.

Platform

Services (stateful): Run in virtual machines

Applications (stateless): Run in containers

Cloud Foundry manages transformations between existing and desired system states, using immutable infrastructure.

Amend

(Re)deploy

(Re)configure

(Re)connect

Monitor

Current State Desired

State

12© 2015 Pivotal Software, Inc. All rights reserved.

The Cloud Foundry Foundation: An Open Industry StandardPLATINUM

GOLD

SILVER

13© 2015 Pivotal Software, Inc. All rights reserved. 13© 2015 Pivotal Software, Inc. All rights reserved.

DevopsEliminating Barriers

14© 2015 Pivotal Software, Inc. All rights reserved.

The Cloud Platform EvolutionTRADITIONAL IAAS

Virtualization Platform

Operating System

Dat

abas

e

Web

S

erve

r

Mes

sagi

ng

Your Application Code

Physical Servers

Dat

abas

e

Web

S

erve

r

Mes

sagi

ng

Your Application Code

IAAS

Your Application Code

PAAS

Virtualized Infrastructure

PAAS

15© 2015 Pivotal Software, Inc. All rights reserved.

Application Dial Tone

Emits application “dial tone”• Config• Runtime• Logs• Metrics• Health Management• Security• Operations

Your Application Code

PAAS

Virtualized Infrastructure

16© 2015 Pivotal Software, Inc. All rights reserved.

Enabling DevopsD

atab

ase

Web

S

erve

r

Mes

sagi

ng

Your Application Code

PAAS

Virtualized Infrastructure

PAAS

Platform Operations

Application Developers

Application Operations

• Deploys platform• Makes standard runtimes and services

available• Monitors platform• Scales platform

(ensuring sufficient capacity)• Upgrades platform with zero-downtime

• Creates deployable artifact

• Config Prod environment• Deploys application to Prod• Monitors application• Scales application (capacity)• Deploys new app version with

zero downtime.

17© 2015 Pivotal Software, Inc. All rights reserved.

Operations Activities and ToolsPlatform Application

Deploy: Operations Manager Deploy: Application Manager, CLI, STS Plugin, CI/CD Pipelines

Monitor: logs and metrics Monitor: Agentless APMAPM integrationsApplication logs – cli, Apps Man, ELK

Scale: Operations Manager Scale: Apps Manager, cli, autoscaling

Upgrade: Canary, Rolling UpgradesZero Downtime

Upgrade: Blue/green, route mappings, gradual cutoverZero Downtime

18© 2015 Pivotal Software, Inc. All rights reserved.

Deploy

> cf push \ -p spring-music.war \ corneliastunes

19© 2015 Pivotal Software, Inc. All rights reserved.

cmt

Platform Enables Process

cmt v201cmt cmt cmt cmt v202 cmt

Prod

…

Test

Dev

…

20© 2015 Pivotal Software, Inc. All rights reserved.

The Deployment Pipeline

CommitTests

IntegrationTests

Deployable Artifact

Deploy to CF Test Env

Deploy to Prod

• Developer crafted (before impl.)• Stubs/mocks external services• Developer executed (before commit)• CI executed (after commit)

(after commit)• Developer or QA crafted• CI executed• Runtime context (buildpack applied)• Binds to test services (DB, messaging, etc.)

• “cf push”• Runtime context (buildpack applied)• Binds to test services (DB, messaging, etc.)• Periodic smoke tests

• “cf push”• Runtime context (buildpack applied)• Binds to prod services (DB,

messaging, etc.)• Periodic smoke tests• Monitoring

? ?

21© 2015 Pivotal Software, Inc. All rights reserved.

Monitoring in Cloud Foundry

• Syslog Endpoint Configuration via OpsMan

• Log search and persistence such as ELK

• Collector Component with plugins for JMX and DataDog

• OpsMetrics Tile and Hyperic

• CLI: cf logs appname• Apps Manager• Stayup.io ELK tile

• Agentless APM• APM integrations

Platform Applications

Met

rics

Logs

22© 2015 Pivotal Software, Inc. All rights reserved.

Application Logs

23© 2015 Pivotal Software, Inc. All rights reserved.

Application Metrics

24© 2015 Pivotal Software, Inc. All rights reserved.

Platform Logs

25© 2015 Pivotal Software, Inc. All rights reserved.

Platform Metrics

26© 2015 Pivotal Software, Inc. All rights reserved.

Vmware + Pivotal for Operational Excellence

vRealize Operations

Manager Time Series DBPivotal

Ops MetricsPivotal Cloud

Foundry - ERS

27© 2015 Pivotal Software, Inc. All rights reserved.

Scale

> cf scale \ -i 5 \ corneliastunes

28© 2015 Pivotal Software, Inc. All rights reserved.

Upgrade

29© 2015 Pivotal Software, Inc. All rights reserved.

Every Commit is a Candidate for Deployment

cmt v201cmt cmt cmt cmt v202 cmt

Prod

…

Test

Dev

…

30© 2015 Pivotal Software, Inc. All rights reserved.

Every Commit is a Candidate for Deployment

cmt v201cmt cmt cmt cmt v202 cmt

Prod

…

Test

…

App

PlatformCVE

31© 2015 Pivotal Software, Inc. All rights reserved.

PCF – Each Layer Upgradable with Zero Downtime

* Devs may bring a custom buildpack

runtime layer*

OS image

application layer

PCF Buildpack Apps

linux host & kernel

App container

PCF Supported

Dev Supported

Ops Supported

• Platform Provided• CVEs addressed by

Pivotal• Canary-style, zero-

downtime upgrades

• Platform or Developer Provided

• CVEs addressed by Pivotal (for Platform Provided)

• Blue/Green deployments

• Developer/App team provided

• Blue/Green deployments

32© 2015 Pivotal Software, Inc. All rights reserved.

Upgrade

33© 2015 Pivotal Software, Inc. All rights reserved.

Enabling DevopsD

atab

ase

Web

S

erve

r

Mes

sagi

ng

Your Application Code

PAAS

Virtualized Infrastructure

PAAS

Platform Operations

Application Developers

Application Operations

• Deploys platform• Makes standard runtimes and services

available• Monitors platform• Scales platform

(ensuring sufficient capacity)• Upgrades platform with zero-downtime

• Creates deployable artifact

• Config Prod environment• Deploys application to Prod• Monitors application• Scales application (capacity)• Deploys new app version with

zero downtime.

34© 2015 Pivotal Software, Inc. All rights reserved. 34© 2015 Pivotal Software, Inc. All rights reserved.

And on top of all that…… something else we’ve found in

enterprises

35Pivotal Confidential–Internal Use Only

Virtualized Infrastructure

Infrastructure-centric Operations Model

Your Application Code

Dat

abas

e

Web

S

erve

r

Mes

sagi

ng

IaaS+ (+automation) IT Ops Team

App Team(Dev and Ops)

36Pivotal Confidential–Internal Use Only

Application-centric Operations

Virtualized Infrastructure

Your Application Code

Dat

abas

e

Web

S

erve

r

Mes

sagi

ng

IaaS+ (+automation)

IT Ops Team

App Team

Your Application Code

PAAS

Virtualized Infrastructure

PaaS Ops Team

App Team(Dev and Ops)

Emits application “dial tone”• Configuration• Runtime• Logs• Metrics• Health Management• Security

37© 2015 Pivotal Software, Inc. All rights reserved. 37© Copyright 2015 Pivotal. All rights reserved.

Thank You

38© 2015 Pivotal Software, Inc. All rights reserved. 38© Copyright 2015 Pivotal. All rights reserved.

Questions?

@cdavisafc