Managing containers with Docker

INTRODUCTION TODOCKER

JOHANNES 'FISH' ZIEMKEtwitter/github: @discordianfish

Building Docker's infrastructure for central services

hub.docker.comdocker.comregistry

Former SoundCloud System and Infrastructureengineer

Scaled infrastructure by 10x in 2 years

DOCKER?Docker 1.4.1Open Source, written in Godeveloped publicly; GitHub, Mailinglists, DGABBuilds, packs and ships applications as lightweightcontainersBuild once, run (almost) everywhereLinux 3.10, OS X and Windows via transparent VMwrapper

CONTAINER VS VM

WHO CARES?

SOME DODocker 0.1 release Spring 2013by dotCloud (now Docker Inc)Rewrite of code that powers dotCloud PaaSSince release>70000 Images on central registry>54000 Dockerfiles on GitHub>100M Image downloads

WELL, OKAY..but why should I care?

TEST THINGS?Start always with a clean slateSpawn up complete test infrastructures in secondsRun your tests against various versions of libraries andservices

MAKE THINGS EASIER, HELPOR TEACH PEOPLE?

Provide students with software environmentDistribute complex setups as self-contained containerfor

Bioinformatics, Information sciencesYour favorite blogdocker run -p 8080:8080 -eURL=http://example.com fish/ghost

Empower people to run their own services

BUILDINGINFRASTRUCTURES?

90S INFRASTRUCTURESProprietary, homogeneous and often vertically scaledstackWaterfall design processRelease cycles of monthsHuge companies...targeting rather small audience/vertical businessesHuge IT/Ops departments

TODAY'SINFRASTRUCTURES

Agile development methods to move fastInterconnected services scaled horizontal inheterogenious environmentLots of spinning wheelsSeveral deploys per daySmall but fast growing startups targeting Millions ofusers

CHALLENGESManaging such infrastructures is incredible hardMillions of knobs and switchesBillions of possible combinationsHigh complexity make it hard to reason aboutNobody can completely understand it

Go away or I will replace you with a very small shell script

CONFIG MANAGEMENTDescribe what you want and how to get thereChange state somewhere, encode what might affectedCFEngine, Puppet, Chef, Ansible, SaltManage everything!Doesn't solve anything

MANAGING COMPLEXITYSimilar problems:

Software complexitymodules, classes, plugins

Human communicationnamed concepts like cat, nerd or car

Shipping goodsintermodal containers

Solution: Abstraction!

WHERE CAN DOCKER HELP?Containers = abstracted application, includingdependencies and configurationOperations = Adding/removing of containersThe container is the same where ever it runsLightweight; can be deployed/rolled back fast andeasilyIsolation makes sure one container isn't affectingothersClear separation of concerns

DEVELOPER: WHY I LOVEDOCKER

Because I just have to care about my container:

my librariesmy package managermy code

I own the container and don't care where it's running.

OPS: WHY I LOVE DOCKERBecause I just have to care about running containers:

provide systems with the Dockerresources planning & monitoringorchestration, remote access

I own the platform and don't care what's inside thecontainers

WHAT IS DOCKER EXACTLYDOING?

RUNNING COMMANDS ON:immutable, shippable, layered imageswith copy-on-write storage on topin isolated environmentvia RESTish API

SHIPPABLE IMAGESimmutablemultiple layersdefined by Dockerfile, built by builderpushed to/pulled from Docker registry

DOCKERFILE/BUILDERDockerfile: Simple text file with instructions:

FROM ubuntu:latestMAINTAINER Johannes 'fish' Ziemke <fish@docker.com>RUN apt­get ­yq updateRUN apt­get ­yq install nginxENTRYPOINT [ "/usr/sbin/nginx" ]CMD [ "­g", "daemon off" ]

docker build creates image from DockerfileEach instruction creates new layerIf instruction hasn't changed, uses cached layerdocker push uploads image layers to registry

DOCKER REGISTRYhosting/delivery of imagesopen source projectsupports various storage backendshosted platform: hub.docker.com

COPY-ON-WRITE STORAGEProvides writable layer on top of (read-only) imagesPersists all changes done by running containerPluggable, supported drivers:

aufsbtrfsdevicemapperoverlayfsvfs

ISOLATED EXECUTIONPlugable, supported: lxc, nativeUsing kernel features:

namespacesIsolation by scopingAvailable: pid, mnt, net, uts, ipc, user

cgroups (control groups)limit, account and isolate resourcesCPU, memory, I/O and general devices

Future: solaris zones, BSD jails, full blown virtualization

DOCKER APIRESTish API, defaults to UNIX socketOptional TLS client and server authenticationThe system APINo need for any other remote accessReady for building your infrastructuredeployment/automation on top

FUTUREComposition: Multi-containerScheduling: Multi-hostTools

Docker nativefigKubernetesMesos

DEMO TIME!

THANKS!QUESTIONS?

Johannes 'fish' Ziemke/

GitHub/Twitter/Facebook: @discordianfishfish@docker.com fish@freigeist.org