CNA1612BU Deploying real-world workloads on Kubernetes … · · 2017-10-12Deploying real-world workloads on Kubernetes and Pivotal Cloud Foundry VMworld 2017 ... • App teams

Fred Melo, Director of Technology, PivotalMerlin Glynn, Sr. Technical Product Manager, VMware

CNA1612BU

#VMworld #CNA1612BU

Deploying real-world workloads on Kubernetes and Pivotal Cloud Foundry

VMworld 2017 Content: Not fo

r publication or distri

bution

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Disclaimer

#CNA1612BU CONFIDENTIAL 2



bution

3

There Are Many ways to Package & Run Workloads …

CONTAINERS EVENT-DRIVEN FUNCTIONS

DATA SERVICESMICROSERVICES

BATCHES

MONOLITHIC

APPLICATIONS



bution

4

Container

Orchestrator

(CaaS)

Application

Platform

(PaaS)

Serverless

Functions

(FaaS)

CONTAINERS EVENT-DRIVEN FUNCTIONS

DATA SERVICESMICROSERVICES

BATCHES

MONOLITHIC

APPLICATIONS

The Goal: Pick the Right Approach for the Workload

IaaS



bution

5

Container

Orchestrator

Container Scheduling

Primitives for Network,

Routing, Logs & Metrics

CONTAINERDeveloper

Provides

Tool

Provides

Application

Platform

APPLICATION

Container Orchestrator

Serverless

Functions

FUNCTION

Application Platform

IaaS

Container Image & build

L7 Network & Routing

Logs, Metrics, Monitoring

Services Marketplace

Team, Quotas & Usage

Function scheduling

Function exec services

Choose The Right Tool For The Job




bution

6

Container

Orchestrator

Container Scheduling

Primitives for Network,

Routing, Logs & Metrics

CONTAINERDeveloper

Provides

Tool

Provides

Application

Platform

APPLICATION


Serverless

Functions

FUNCTION

IaaS

Container Image & build

L7 Network & Routing

Logs, Metrics, Monitoring

Services Marketplace

Team, Quotas & Usage

Choose The Right Tool For The Job …


Function scheduling



Lower complexity, higher efficiency

Higher flexibility, lower standardizationVMworld 2017 Content: Not fo


bution

7

Serverless

Functions

FUNCTION

IaaS


Container

Orchestrator

Application

Platform

Pivotal Cloud FoundryElastic Runtime

BOSH

ABSTRACTION

Pivotal Container

Service

CONTAINER APPLICATION


Function scheduling





bution

8

IaaS


Container

Orchestrator

Application

PlatformMonitoring

Tracing

Logging

Pivotal Cloud Foundry

Services

ABSTRACTION

BOSH

Pivotal Container

Service

CONTAINER


APPLICATION



bution

9

IaaS


Container

Orchestrator

CONTAINER

Application

Platform

APPLICATION


BOSH

Pivotal Container

Service

Monitoring

Tracing

Logging


Services

Multi-cloud Provide a reliable and

smooth experience for any cloud.

Open APIs Allow platform

operations from different toolsets and

the creation of CD pipelines.

Consistency Provide a consistent

setup experience, across different

cloud environment configurations.

Patches Patching platform components

with thousands of apps running should

feel normal.

Scaling Seamlessly scale platform

components to accommodate changing

demand.

Upgrades. How do you roll out new

versions of the platform with the lights on?

Platform

Operator



bution

1

0

Built with open-source Kubernetes. Constant

compatibility with the current stable release of Kubernetes,

operated by BOSH. No proprietary extensions.

Production-ready. highly available from apps to

infrastructure, no single points of failure. Built-in health

checks, scaling, auto-healing and rolling upgrades.

Multicloud. BOSH provides a reliable and consistent

operational experience. For any cloud.

Network management and security out-of-the-box with

VMware NSX-T. Multi-cloud, multi-hypervisor.

GCP APIs access. The GCP Service Broker allows apps

to transparently access Google Cloud APIs, from

anywhere. Easily move workloads to/from Google

Container Engine (GKE).

Fully automated Ops. Fully automated deploy, scale,

patch, upgrade. No downtime. Use CD pipelines to deploy

your platform, too.

BOSH

GCPServiceBroker

Harbor

NSX

-T

Kubernetes

K8s ClusterK8s Cluster

K8s Cluster

GCP Azure Openstack AWS

PK

S C

on

tro

ller



bution

11

Physical Infrastructure

VMware PKS

Kubernetes on BOSH (Kubo)

BOSH

NSX

Analytics Automation

SecurityOperations

Monitoring

GCP

Service Brokermasteretcd worker

Logging

vSANvSphere

masteretcd workerContainer

Registry

(PKS)



bution

What to deploy to CF vs K8s?Please help me decide!!



bution

13

Hardware

IaaS


Application

Platform

Strategic goal: Push as many workloads as technically

feasible to the top of the platform hierarchy

Higher flexibility and

less enforcement of

standards

Lower development

complexity and higher

operational efficiency



bution

14

BOSH

DATA SERVICESMICROSERVICESMONOLITHIC

APPLICATIONSCONTAINERS

Platform

Operator

Developer



bution

15

“Run this app for me, I don’t care how”

Platform builds containers The platform takes care of container creation and management

from source code. Pivotal makes sure they’re always up-to-date.

Out-of-box app logging, metrics, tracing and monitoring. No additional configuration

required.

Create services on demand On-demand service brokers allow dynamic creation and binding

of service dependencies, offered through a marketplace.

Perfect for 12-Factor apps

Application

Platform



bution

16

war

PCF Elastic Runtime

Availability Zone 1 Availability Zone 2 Availability Zone 3

Staging

Root

FS

Build

Pack

war

`cf push`

Drop

let

A

I

A

Imyapp.foo.com

PCF Routing PCF Routing PCF Routing

“Here is my source code

Run it on the cloud for me

I do not care how”

Developer

URL Request:

myapp.foo.com



bution

17

Cloud Native Application Development

- 12-Factor apps

- Spring Boot

- Microsoft .NET

- Any technology supported by the PCF Buildpacks

- Platform Provides Service Mesh

Legacy Application Development

Persistent Data Volumes -

Native Docker Tooling-

Non Standard Port Apps -

No Buildpack -

App Requires declarative hardware-




bution

18

“Run this container for me. I’ll provide and manage the image,

configure port binds, routes and dependencies”

Developer builds containers. Developers are responsible for containers creation and

management

More Customization K8s offers more extension points and customization possibilities.

Higher Flexibility. K8s allows explicitly port bind definitions and container co-location (Pods).

Ideal for packaged apps, apps/services exposing multiple ports and

where finer grain control is needed

ContainerOrchestrator



bution

19

K8s Cluster

Worker

`kubectl apply –f myapp.yml`

Developer

Worker

kube-proxyMaster

etcd

kube-proxy

Service: nodeport | ingress

POD POD

Load Balancer

URL Request:

myapp.foo.com/k8siscool

Docker

Registry



bution

20

• Persistence

– Cloud Provider Volumes

– MongoDB, CouchDB, Couchbase, Elastic Search, …

• Services Managed as a cluster

– nodes need to communicate with one another

– often with the help of service meshes such as Istio or Linkerd

– Spark, Elastic Search (ELK)

• Misc things like multiple ports, etc.

• Needing new architectural primitives

New Cloud Native Development …



bution

Apps are “already” or need to be containerized …

21

• ISVs

– “I’ve had two large banks ask me ‘My vendors are giving me their software as containers. Where do I run

them?’” - Forrester Analyst

– Enterprise Redis, AppDynamics, etc.

• App teams are using containers as their single deployable artifact

– App team requires control of whole stack (app & exec)

– But IT (Platform team) have a pretty healthy understanding of the controls that are needed.

• (Optimistic) Best practices and scanning

• (Pessimistic) Standardized images and container creation pipelines (um, yeah, that looks a bit like

buildpacks)

– Some app teams are building the “platforms” to deploy and manage their containerized workloads - Mesos,

Kubernetes, ….



bution

22

Application

PlatformContainer ApplicationElastic Runtime (ERT) Pivotal Container Service (PKS)

GC

P S

erv

ice

Bro

ke

rKubernetes

ClusterClusterCluster

Platform Services

Logging Metrics Monitoring

Other

Service

Brokers

AppApp

AppApp

AppApp

BOSH

NS

X-T

Choose The Right Abstraction PCF + K8S…

Developer

CF API

Platform

Operator

KUBERNETES API

Manual or Build

Automation



bution

23

Application


GC

P S

erv

ice

Bro

ke

rKubernetes


Platform Services


Other

Service

Brokers

AppApp

AppApp

AppApp

BOSH

NS

X-T

KUBERNETES API

PK

S

Co

ntr

olle

r

CF API


Developer



bution

24

Application


GC

P S

erv

ice

Bro

ke

rKubernetes


Platform Services


Other

Service

Brokers

AppApp

AppApp

AppApp

BOSH

NS

X-T

PK

S

Co

ntr

olle

r

CF Routing K8S RoutingCF Routing


Consumer

Or Micro-Service



bution

DRI … Architect for Agility

Virtual Data Center

• Architect the right Abstractions

• Automate Everything

• Build for Failure

Control Agility

vSphere NSX vSAN


PCF

PKS

BOSH powered Kubernetes

BO

SH

Platform

Operator

Developer

Wavefront

Self Service

• Automation

• Day 2 Operations

• Control

• Application Services or Container Services

• Application Logging & Monitoring

vRLI (Dev)

vRops

vRLI (Ops)

vRNI



bution

26

VMworld US Key Focus Description

CNA1509BU DRI Developer-Ready Infrastructure from VMware & Pivotal

CNA1612BU PCF & PKSUse Cases: Deploying real-world workloads on Kubernetes and Pivotal Cloud

Foundry

CNA2006BU DRIDeep Dive: Architecting Container Services with VMware and Pivotal

Developer Ready Infrastructure

CNA2080BU PKS Deep Dive: How to Deploy and Operationalize Kubernetes

CNA3429BU PKSBasics of Kubernetes on BOSH: Run Production-grade Kubernetes on the

SDDC

CNA3430BU PCFYour Enterprise Cloud-Native App Platform: An Introduction to Pivotal Cloud

Foundry

MGT2871BUPCF & vRops,

vRLI

Bridging the Operations Gap Between the Software-Defined Data Center

and Pivotal CF for VMware Deployments

NET1523BU PCF & NSX Integrating NSX and Cloud Foundry

PAR4411PU DRIEmerging Technologies with VMware and Pivotal - presented jointly by

VMware, Pivotal and Special Guest Speakers from Cognizant and WWT

Developer Ready Infrastructure @ VMworld



bution

Demo



bution



bution

Any Questions …



bution



bution



bution

Documents

CNA1612BU Deploying real-world workloads on Kubernetes … · · 2017-10-12Deploying real-world workloads on Kubernetes and Pivotal Cloud Foundry VMworld 2017 ... • App teams