Infrastructure Design for Kubernetes

Fabio ChiodiniAdvisory Platform Architect - EMEA

@FabioChiodini

Credits to @cdavisafc and @pczarkowski for a lot of EPIC contents ;)

© Copyright 2019 Dell Inc.5

Kubernetes.. What?

© Copyright 2019 Dell Inc.6

Kubernetes 101 - Containers, Runtime and Orchestrator

Infrastructure

C1

C2

C3

Operating System

Container Runtime

Ku

be

let

Ku

be

-pro

xy

Physical Network

API serverSchedulerController manager

etcd

Node 1Master

Overlay Network (Flannel/Open vSwitch/Weave)

OS

Infra Infrastructure

C1

C2

C3

Operating System

Container Runtime

Ku

be

let

Ku

be

-pro

xy

Node 2

Infrastructure

C1

C2

C3

Operating System

Container Runtime

Ku

be

let

Ku

be

-pro

xy

Node 3

© Copyright 2019 Dell Inc.7

Kubernetes is a Container Orchestrator

> kubectl

Storage NetworkingCompute

Kubernetes Dashboard

Dev / Apps IT / Ops

App User

© Copyright 2019 Dell Inc.8

But we need a little bit more to RUN IT in production

Storage NetworkingCompute

Dev / Apps

App User

IT / Ops

> kubectl

Kubernetes Dashboard

Load Balancing / Routing

Container Image

Registry

App Monitoring

App Logging

OS Updates

OS Images

K8S Updates

K8S Images

Log & Monitor

Recover & Restart

Backup & Restore

ExternalData Services

Cluster Provisioning

Provision & Scale

Command Line / API

Management GUI

Monitoring GUI

© Copyright 2019 Dell Inc.9

What will WE cover today?

Storage NetworkingCompute

Dev / Apps

App User

IT / Ops

> kubectl

Kubernetes Dashboard

Load Balancing / Routing

Container ImageRegistry

App Monitoring

App Logging

OS Updates

OS Images

K8S Updates

K8S Images

Log & Monitor

Recover & Restart

Backup & Restore

ExternalData Services

Cluster Provisioning

Provision & Scale

Command Line / API

Management GUI

Monitoring GUI

StorageHost Networking

Install, Maintain and Upgrade

Cluster

Provisioning

Routing/Load Balancing/More

Cluster Scaling

© Copyright 2019 Dell Inc.10

#SpoilerAlert

© Copyright 2019 Dell Inc.11

With Pivotal Container Service we got You covered!!

Storage NetworkingCompute

Dev / Apps

App User

IT / Ops

> kubectl

Kubernetes Dashboard

Load Balancing / Routing

Container ImageRegistry

OS Updates

OS Images

K8S Updates

K8S Images

Log & Monitor

Recover & Restart

Backup & Restore

ExternalData Services

Cluster Provisioning

Provision & Scale

vRealize LogInsight

Service Broker

PKS Control Plane

> pks

Operations Manager

vRealize Operations*

*integration

© Copyright 2019 Dell Inc.12

Upgrades

© Copyright 2019 Dell Inc.13

Here’s (in 5 sec) why You need automatic upgrades

Kubernetes 1.9 Kubernetes 1.10 Kubernetes 1.11 Kubernetes 1.12

December 2017 March 2018 June 2018 September 2018

Kubernetes 1.13

December 2018

Kubernetes 1.14

March 2019

CVE-2018-1002105

Kubernetes API

CVE-2019-6486

Go

December 2018 February 2019

CVE-2019-5736

runC

February 2019

CVE-2019-1002100

API Server DOS

March 2019

Kubernetes

Releases

Kubernetes

CVEs

20 Linux CVE in 2019

(454 in 2017)

Operating System

© Copyright 2019 Dell Inc.14

Let’s do some math…. In a Year:

4 major K8s updates x (30 clusters x 7 VMs) = 840 upgrades

30 clusters x 7 VMs = 210 OS upgrades

how many linux release this year?

Operating System

And what about CVEs?

© Copyright 2019 Dell Inc.15

DEMO:Upgrading K8s (and OS) in a click

© Copyright 2019 Dell Inc.16

© Copyright 2019 Dell Inc.17

Host types

© Copyright 2019 Dell Inc.18

Yes you need VMs (most of the times)• Have you automated your

baremetal deployments today?

• How much time does it take to provision a cluster?

• How do you patch (OS/K8s Firmware etc)?

• How do you remediate Failures?

• Can you repair/reinstall a node in a few minutes?

• [How many people do you need to manage it?]

• Scale unit: scale a VM vs scaling a server

• Time required to scale up a cluster

© Copyright 2019 Dell Inc.19

DEMO:Self Healing Kubernetes

© Copyright 2019 Dell Inc.20

DEMO:Scaling Kubernetes

© Copyright 2019 Dell Inc.21

I’ll give you some homeworks for these demos ;)

https://www.youtube.com/playlist?list=PLAdzTan_eSPTJ8yIslkADNxtbxzwF1PAu

© Copyright 2019 Dell Inc.22

Storage

© Copyright 2019 Dell Inc.23

Storage for Kubernetes.. How?

Pod

• Is [effectively] a Directory, possibly with data in it,

available to all containers in a Pod

• Usually Shares lifecycle of a Pod (Created when Pod

is created, destroyed when Pod is destroyed)

• Persistent Volumes outlive Pods

• Can be mounted from local disk, or from a network

storage device such as a EBS volume, iscsi, NFS, etc

Infrastructure

C1

C2

C3

Operating System

Container Runtime

© Copyright 2019 Dell Inc.24

https://github.com/container-storage-interface

Container Storage Interface (CSI)

© Copyright 2019 Dell Inc.25

DEMO:Adding Storage to Kubernetes

© Copyright 2019 Dell Inc.26

© Copyright 2019 Dell Inc.27

Wrapping Up

© Copyright 2019 Dell Inc.28

What You should plan to add to Kubernetes

Advanced

Networking Flexible Host types

Self Healing Hosts Automated

Upgrades

AutomationStorage Integration

Automated

Cluster

Provisioning

PKS Enterprise value-added features

Built into Kubernetes

Multi-container pods

Stateful Sets of pods

Persistent disks

Single tenant ingress

Pod scaling and high availability

Rolling upgrades to pods

Cluster provisioning and scaling

Embedded, hardened Operating System

Monitoring and recovery of cluster VMs and processes

Rolling upgrades to cluster infrastructure

Secure multi-tenant ingress

Secure container registry

© Copyright 2019 Dell Inc.29