Removing Barriers Between Dev and Ops

2© 2015 Pivotal Software, Inc. All rights reserved. 2© 2015 Pivotal Software, Inc. All rights reserved.

Removing Barriers Between Dev and Ops It Takes a Platform

Devops @ VMworld1 September 2015

Cornelia Davis, Sr. Director, Cloud Foundry Platform Engineering@cdavisafc

3© 2015 Pivotal Software, Inc. All rights reserved.

Software is Changing Industries

$6B valuationFinancial Services

$3.5B valuationTravel & Hospitality

$19B valuation Transportation

$3.2B Acquisition by GoogleHome Automation

$20B valuation Entertainment

$1.1B acquisition Monsanto--Agriculture


Increased Expectations and Increased EfficienciesChanging development and release disciplines

• Agile/Developer Productivity/TDD• Continuous Delivery • DevOps• Cloud-native App Design• Culture/Talent

Time

Expectations& Efficiency

Innovation Gap

IaaS

Automation

DevOps, CD, Agile

Traditional IT

Startups

Enterprise PaaS


“Operations is the secret sauce”

http://radar.oreilly.com/2007/10/operations-is-a-competitive-ad.html


the difference is platformsthe difference is third platform



Cloud Foundry is…

The world’s leading open source platform-as-a-service. Supported by dozens of major organizations Language and framework agnostic Manages both VMs and containers Orchestrates both applications and data services Abstracts Infrastructure

Founded and commercialized by Pivotal, Inc.


An (Overly) Simple View of the World

Applications Stateless Run in Containers Horizontally Scalable Disposable No permanent storage

Data Services Stateful Run in Virtual Machines Multi-tenant Diagonally scalable Durable storage


Cloud Foundry: Applications and Services

Services (virtual machines): managed by “BOSH”

Applications (containers): managed by “Runtime”


Platform

Services (stateful): Run in virtual machines

Applications (stateless): Run in containers

Cloud Foundry manages transformations between existing and desired system states, using immutable infrastructure.

Amend

(Re)deploy

(Re)configure

(Re)connect

Monitor

Current State Desired

State


The Cloud Foundry Foundation: An Open Industry StandardPLATINUM

GOLD

SILVER


DevopsEliminating Barriers


The Cloud Platform EvolutionTRADITIONAL IAAS

Virtualization Platform

Operating System

Dat

abas

e

Web

S

erve

r

Mes

sagi

ng

Your Application Code

Physical Servers

Dat

abas

e

Web

S

erve

r

Mes

sagi

ng


IAAS


PAAS

Virtualized Infrastructure

PAAS


Application Dial Tone

Emits application “dial tone”• Config• Runtime• Logs• Metrics• Health Management• Security• Operations


PAAS



Enabling DevopsD

atab

ase

Web

S

erve

r

Mes

sagi

ng


PAAS


PAAS

Platform Operations

Application Developers

Application Operations

• Deploys platform• Makes standard runtimes and services

available• Monitors platform• Scales platform

(ensuring sufficient capacity)• Upgrades platform with zero-downtime

• Creates deployable artifact

• Config Prod environment• Deploys application to Prod• Monitors application• Scales application (capacity)• Deploys new app version with

zero downtime.


Operations Activities and ToolsPlatform Application

Deploy: Operations Manager Deploy: Application Manager, CLI, STS Plugin, CI/CD Pipelines

Monitor: logs and metrics Monitor: Agentless APMAPM integrationsApplication logs – cli, Apps Man, ELK

Scale: Operations Manager Scale: Apps Manager, cli, autoscaling

Upgrade: Canary, Rolling UpgradesZero Downtime

Upgrade: Blue/green, route mappings, gradual cutoverZero Downtime


Deploy

> cf push \ -p spring-music.war \ corneliastunes


cmt

Platform Enables Process

cmt v201cmt cmt cmt cmt v202 cmt

Prod

…

Test

Dev

…


The Deployment Pipeline

CommitTests

IntegrationTests

Deployable Artifact

Deploy to CF Test Env

Deploy to Prod

• Developer crafted (before impl.)• Stubs/mocks external services• Developer executed (before commit)• CI executed (after commit)

(after commit)• Developer or QA crafted• CI executed• Runtime context (buildpack applied)• Binds to test services (DB, messaging, etc.)

• “cf push”• Runtime context (buildpack applied)• Binds to test services (DB, messaging, etc.)• Periodic smoke tests

• “cf push”• Runtime context (buildpack applied)• Binds to prod services (DB,

messaging, etc.)• Periodic smoke tests• Monitoring

? ?


Monitoring in Cloud Foundry

• Syslog Endpoint Configuration via OpsMan

• Log search and persistence such as ELK

• Collector Component with plugins for JMX and DataDog

• OpsMetrics Tile and Hyperic

• CLI: cf logs appname• Apps Manager• Stayup.io ELK tile

• Agentless APM• APM integrations

Platform Applications

Met

rics

Logs


Application Logs


Application Metrics


Platform Logs


Platform Metrics


Vmware + Pivotal for Operational Excellence

vRealize Operations

Manager Time Series DBPivotal

Ops MetricsPivotal Cloud

Foundry - ERS


Scale

> cf scale \ -i 5 \ corneliastunes


Upgrade


Every Commit is a Candidate for Deployment


Prod

…

Test

Dev

…


Every Commit is a Candidate for Deployment


Prod

…

Test

…

App

PlatformCVE


PCF – Each Layer Upgradable with Zero Downtime

* Devs may bring a custom buildpack

runtime layer*

OS image

application layer

PCF Buildpack Apps

linux host & kernel

App container

PCF Supported

Dev Supported

Ops Supported

• Platform Provided• CVEs addressed by

Pivotal• Canary-style, zero-

downtime upgrades

• Platform or Developer Provided

• CVEs addressed by Pivotal (for Platform Provided)

• Blue/Green deployments

• Developer/App team provided

• Blue/Green deployments


Upgrade


Enabling DevopsD

atab

ase

Web

S

erve

r

Mes

sagi

ng


PAAS


PAAS

Platform Operations

Application Developers

Application Operations

• Deploys platform• Makes standard runtimes and services

available• Monitors platform• Scales platform

(ensuring sufficient capacity)• Upgrades platform with zero-downtime

• Creates deployable artifact

• Config Prod environment• Deploys application to Prod• Monitors application• Scales application (capacity)• Deploys new app version with

zero downtime.


And on top of all that…… something else we’ve found in

enterprises

35Pivotal Confidential–Internal Use Only


Infrastructure-centric Operations Model


Dat

abas

e

Web

S

erve

r

Mes

sagi

ng

IaaS+ (+automation) IT Ops Team

App Team(Dev and Ops)

36Pivotal Confidential–Internal Use Only

Application-centric Operations



Dat

abas

e

Web

S

erve

r

Mes

sagi

ng

IaaS+ (+automation)

IT Ops Team

App Team


PAAS


PaaS Ops Team

App Team(Dev and Ops)

Emits application “dial tone”• Configuration• Runtime• Logs• Metrics• Health Management• Security

37© 2015 Pivotal Software, Inc. All rights reserved. 37© Copyright 2015 Pivotal. All rights reserved.

Thank You

38© 2015 Pivotal Software, Inc. All rights reserved. 38© Copyright 2015 Pivotal. All rights reserved.

Questions?

@cdavisafc