Containers All the Way Down

From Immutable Infrastructureto Immutable Metal

Agenda

DEMO!

1. Immutable infrastructure in Docker Compose 2. Digital Rebar Server Provisioning infrastructure3. Install HA Kubernetes

Interstitial:

1. Fidelity Gap - iterating devops faster from dev to test, pilot and production.2. Immutable Infrastructure - don’t fix, destroy!

Rob Hirschfeld & RackN

CEO & co-Founder

We specialize in portable infrastructure automation

I’m also on the OpenStack Foundation board.

Social:

not just cat pictures, on my twitter: @zehicle

I love to blog about cloud, tech and culture! http://robhirschfeld.com

Digital Rebar (aka Crowbar v3)

“API for Metal” > Automates physical infrastructure

“Start to Scale” > Works on any platform from desktop to datacenter

“Open Ops” > Makes DevOps portable between sites

Deploys container platforms using containers.

Latest evolution of the first OpenStack Installer known as Crowbar.

http://rebar.digital

Digital Rebar with Docker Compose

Determines:

● which containers● dependencies between● port mapping● variables injection● start/stop/scale

BUT, single system only

Doc

ker C

ompo

se (1

5 co

ntai

ners

)

Consul

Rebar API

Rebar Engine

Postgresql

NTP

DNS

DHCP

Provisioner

...

Port Map

Port Map

Port Map

Port Map

Port Map

Port MapDocker

Hub

Digital Rebar with Consul

Consul (registry & shared keystore)

● registers all services● shared secrets ● & more stuff we don’t use

Rebar API & Orchestration (yellow)

Services Managed by Rebar (blue)

Services Used by Rebar (green)Docker Containers

ConsulForwarder

Rebar API

Postgresql

Rebar Engine

DNS

Provision

NTP

DHCP

Chef Loggers

Scale faults from the “Fidelity Gap”

Testing for production on a desktop or cloud is not sufficient.

Automation is required at all levels.

We want to use the SAME deployment at every level to eliminate translation errors.

Production

Dev

Test

PoC

Scale

Effo

rt

Easy

Hard

5 10 20 100+

How Do Deployments Fail?One step at a time

FidelityGap

Major Differences in:

● Networking● Timing / Sequential Ops● Need for fault tolerance● Process Requirements● Ops / Environmentals● User motivation / priorities● Ownership

Why a Fidelity Gap? Different needs

Scale

Effo

rt

Easy

Hard

5 10 20 100+

Desktop Cloud

Lab

Datacenter

Addressing the “Fidelity Gap”

Faithful ops between environments

Portable DevOps automation

Fast cycle times for developers

Transparent execution

True multi-node even when small

Mix-and-match environments

Production

Dev

Test

PoC

Scale

Effo

rt

Easy

Hard

5 10 20 100+

Digital Rebar

Immutable Infrastructure: don’t fix, destroy!

Always destroy+create, never change

Create “pristine” source of truth

Single and repeatable image generation process (but not “gold images”)

Expects to manage rapid turnover (required as change agent)

It’s immutable until it has state (in OOP, we’d say “instantiated”)

And not just for containers...

Kubernetes Metadata

Determines:

● which containers● dependencies between● port mapping● variables injection● start/stop/scale● tenant networking

AND multi-system infrastructure

Kub

erne

tes

Keystore

Database

Web Front

Service 1

Service 2

Batch Item

Foo

Bar

...

Port Map

Port Map

Port Map

Port Map

Port Map

Port MapDocker

Hub

Running Kubernetes

Master + Minion: cluster via etcd

Builds networking tunnel for pods

Additional pluggable services (L3)

Manages container

● life-cycle● placement● dependencies

Docker Containers

Docker Containers

Docker Containers

SDN Agent

App

KubernetesMinion

KubernetesMinion

KubernetesMinion

SDN Agent SDN Agent

App

AppApp

AppApp

App

AppApp

AppApp

App App

AppApp

AppApp

App

KubernetesMaster

etcd (shared store)

Flannel is weak SDN (basically UDP)

Requires kernel modification (fast!)

When L2 and L3 support is needed

Multi-datacenter connections

Mix infrastructure (docker, VMs, metal)

Expect to have multiple SDN options

+ OpenContrail

Docker Containers

Docker Containers

Docker Containers

Contrail Agent

App

KubernetesMinion

KubernetesMinion

KubernetesMinion

Contrail Agent

Contrail Agent

App

AppApp

AppApp

App

AppApp

AppApp

App App

AppApp

AppApp

App

KubernetesMaster

etcd (shared store)

ContrailController

ThankN you

Project:

project social: @digitalrebar

project landing page: http://rebar.digital

Rob:

let’s talk cloud and ops! @zehicle

long form tweets: http://robhirschfeld.com