Container Crash CourseInterop Las Vegas 2015

Processes and FilesystemsInterop Las Vegas 2015

Zen and the Art of System Management

Interop Las Vegas 2015

Building Cloud Native Architectures

Interop Las Vegas 2015

Agenda• Introduction

• Containers as a metaphor

• Containers from first principles

• Linux Containers

• Managing Containers

• Really Managing Containers

• What’s next?

Andrew Clay Shafer

Andrew Clay Shafer

@littleidea

What is a container?

History

The Box

Someone thought this was interesting enough to write 400 pages.

The system, developed after World War II, dramatically reduced transport costs, supported

the post-war boom in international trade, and was a major element in globalization.

Timeline• Origins in 18th Century English Coal Mining

• By 1830, Railroads Carried Boxes Designed For Other Modes

• Early 1900s, Closed Boxes Designed for Both Road and Rail

• 1933 Bureau International des Containers et du Transport Intermodal

• 1951 Purpose Built Ships

• 1955, Modern Intermodal Container Developed

• 1968: ISO_668 Defined the Terminology, Dimensions and Ratings

• Innovation Since 1970 Optimized Logistics

McLean had initially favored the construction of "trailerships"—taking trailers from large trucks

and stowing them in a ship's cargo hold.

Containers• “Intermodal”

• Have Standardized Dimensions

• Transported Without Being Opened

• Handling Completely Mechanized

• All Containers Numbered and Tracked

Impact• Did Away With Most Manual Sorting

• Significantly Reduced Warehousing

• Reduced Port Congestion

• Shortened Shipping Time

• Reduced Loss From Damage and Theft

• Displaced Thousands of Dock Workers

As of 2009, approximately 90% of non-bulk cargo worldwide is moved by

containers stacked on transport ships

few initially foresaw the extent of the influence of containerization on the

shipping industry.

Impact wasn’t from the details of ‘containers’ but from the infrastructure

and logistic optimization

where were we?

What makes a process?• executable code

• machine registers

• heap

• stack

• file descriptors

• environment variables

a process is context

information maintained about and for an executing program

the abstraction of a physical processor

Some Context• Process ID (pid)

• Parent process ID (ppid)

• Real User ID

• Effective User ID

• Current Directory

• File Descriptor Table

• ENV

PID 0sched

actually part of the kernel

PID 1init

UID PID PPID CPU PRI NI VSZ RSS WCHAN STAT TT TIME COMMAND 0 1 0 0 48 0 2540232 14288 - Ss ?? 49:39.12 /sbin/launchd

every process has a parent who is supposed to take care of it

Why am I telling you this?

a ‘container’ is just a process pretending it’s all alone

Containers are Not VMs“trailerships”

Well, what is a VM?

Hypervisors & Virtual Machines

• software emulation of hardware

• must boot another kernel

• another layer between process and silicon

moving forward

On to containers

a glimpse of the future

How did we get here?

‘container’ history• 1979 Unix v7 chroot system call

• 2000 FreeBSD Jails

• 2001 Linux Vserver

• 2004 Solaris Zones

• 2005 OpenVZ

• 2006 Process Containers

• 2007 cgroups merged into mainline

‘container’ history• 2008 LXC

• 2011 Cloud Foundry Warden

• 2013 LMCTFY

• 2013 Docker

• 2014 Rocket

• ??

Not New

features of the kernel

namespaces and groups

• namespaces limit visibility

• cgroups limit access

The purpose of each namespace is to wrap a particular global system resource in an abstraction that makes it

appear to the processes within the namespace that they have their own isolated instance of the global resource.

Namespaces• mount - filesystem hierarcy

• UTS - hostnames

• IPC - interprocess communication

• PID - process ID (different namespaces can have same PID)

• network - each namespace has own devices, IP, routing tables

• user - isolate the user and group id number spaces

grouping/partitioning of processes, with newly forked processes ending up in the

same group (cgroup) as their parent process

cgroups• cpu

• memory

• cpuacct

• cpuset

• devices

• freezer

• net_cls

• ns

But what is actually running?

chroot is an operation that changes the apparent root directory for the current running process and their children. A program that is run in such a modified environment cannot access files and commands outside that environmental directory tree. This

modified environment is called a chroot jail.

files systems

abstraction representing data, ownership and permissions

When people talk about ‘containers’ right now they are often conflating the runtime

isolation and the packaging of filesystems

Blame Docker

Docker, Docker, Docker

Docker made it easy to make and share filesystem images

Docker made containers accessible to the average developer

the best thing about docker is the social sharing and workflows

the worst thing about docker is the social sharing and workflows

:)

Now you have 1000s of containers deployed

Oh Wait… how?

Intermodalwe just need the trains and cranes

Purpose Built Ships

• role based access to resources

• run specified bits on demand

• coordinate cross service configurations

• route public requests to running bits

• read and write persistent data

• add and remove resources

• record internal and external events

• isolate resources and failures

• measure performance/health

• detect and determine failure (plan & provoke failure)

• recover failures

• work tomorrow

Problems to solve…

better get to work

Resources• namespaces - http://lwn.net/

Articles/531114/

• cgroups - https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt

• http://lattice.cf/

• http://cloudfoundry.org/

• https://linuxcontainers.org/

• https://www.docker.com/

• https://coreos.com/blog/rocket/

• http://kubernetes.io/

• http://mesos.apache.org/