Dec, 2018 Yoann Juet Information Technology Department ... · containerd containerd : runtime daemon Core container runtime for Docker Manage the complete container lifecycle (stop,

Dockerize Your IT !

1

Centrale NantesInformation Technology Department

Yoann JuetDec, 2018

A Brief History of Containers

2

1979 2000 2001 2002

2006 2008

UNIXCHROOT

BSDJAIL

LINUXVSERVER

LINUXNAMESPACES

2005

LINUXOPENVZ

LINUXCGROUPS

LINUXLXC

2013

DOCKER

2015

KUBERNETES

Docker Containers != Virtual Machines

3

VMs have a full copy of an OS, leading to cpu, memory, network overhead ; very good isolation between VMs

Containers share host ressources (kernel, cpu, memory, network) ; good isolation between Containers

Docker architecture

4

● docker : the Docker user CLI● dockerd : engine daemon

○ Create image, pass it to containerd

● containerd : runtime daemon○ Core container runtime for

Docker○ Manage the complete

container lifecycle (stop, start, transfer, supervision, storage, network)

● This model gives the ability to restart or upgrade Docker Engine without breaking the running containers

Docker images,

5

An image is a read-only group of layers of other images. It includes everything an application needs to run: binaries, libraries, config files…

Each image is made of a base image (e.g. debian, ubuntu, alpine) plus a collection of diffs - intermediate images/layers - that adds the required features (e.g. emacs, apache).

Images can be stored on public, private repos, on any host machine that has previously pulled the package from a repo

Docker Containers

6

A container is a running instance - read-write - of an image

You can run containers on Linux, Windows 10, Windows Server 2016, Cloud (AWS, Google…)

Containers should be as ephemeral as possible. You should expect them to go down at any time and lose all data stored inside:

● Don’t store data in containers● Don’t run more than one process in a single

container● Use custom created volumes or system

mounts

7

Docker Image != Docker Container

App Image

App 1Container

Public or PrivateRegistry

Remember…

● A Docker Image is similar to a read-only template● A Docker Container is an writable instance of a

Docker Image ● Each Docker Container has its own read-write

layer - thus its own data - that sits on one or more Docker imageApp 2

Container

pull

r/w Instance

Commit

r/w

Inst

ance

push

8

Lightweight, low resources consumptions

Isolation model, default set of capabilities

Low attack surface

Networking can be tricky

Fast boot, removal, reproducibility : the same code runs everywhere

Orchestration complexity

Large ecosystem - lot of official and unofficial images -

Ideal for development team (test, pre-prod, prod)

Docker Containers - Pros and Cons

9

Docker repository (version 18.09)user@host:~$ sudo apt-get update && apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-common

user@host:~$ sudo curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -

user@host:~$ sudo apt-key fingerprint 0EBFCD88

pub rsa4096 2017-02-22 [SCEA]

9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88

uid [ unknown] Docker Release (CE deb) <[email protected]>

sub rsa4096 2017-02-22 [S]

user@host:~$ sudo add-apt-repository "deb https://download.docker.com/linux/debian stretch stable"

user@host:~$ sudo apt-get update && apt-get install docker.ce

user@host:~$ sudo usermod -aG docker <user> && newgrp docker

Installing Docker Community Edition (CE)On debian 9 (stretch) host - from the netinst image -

10

Check for correct installationuser@host:~$ docker info...Server Version: 18.09.0Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true...Runtimes: runcDefault Runtime: runcKernel Version: 4.9.0-8-amd64Operating System: Debian GNU/Linux 9 (stretch)...Docker Root Dir: /var/lib/docker...Product License: Community Engine

Installing Docker Community Edition (CE)On debian 9 (stretch) host - from the netinst image -

11

Search for an official nginx image...user@host:~$ docker search nginxNAME DESCRIPTION STARS OFFICIAL AUTOMATEDnginx Official build of Nginx. 10386 [OK] jwilder/nginx-proxy Automated Nginx reverse proxy for docker con… 1473 [OK]richarvey/nginx-php-fpm Container running Nginx + PHP-FPM capable of… 652 [OK]...

Download the image from the official Docker Registryuser@host:~$ docker pull nginxUsing default tag: latestlatest: Pulling from library/nginxa5a6f2f73cd8: Pull complete 1ba02017c4b2: Pull complete 33b176c904de: Pull complete Digest: sha256:5d32f60db294b5deb55d078cd4feb410ad88e6fe77500c87d3970eca97f54dbaStatus: Downloaded newer image for nginx:latest

Run Your First Docker ApplicationLet’s say an nginx http server !

In the OFFICIAL column, OK indicates an image built by the company/community behind the project

12

Verify user@host:~$ docker image lsREPOSITORY TAG IMAGE ID CREATED SIZEnginx latest 568c4670fa80 46 hours ago 109MB

Run a container based on this imageuser@host:~$ mkdir ~/mydir && echo "Hello, France!" > ~/mydir/index.html && docker run --name myweb -v ~/mydir:/usr/share/nginx/html -p 8080:80 -d nginx8807f82f280b6dde9848029c414b22b9ac7b77362b39d7c58c94e15d2eedc905

user@host:~$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES8807f82f280b nginx "nginx -g…" 40 seconds ago Up 39 seconds 0.0.0.0:8080->80/tcp myweb


13

Connect to your Appuser@host:~$ wget -q localhost:8080 -O /dev/stdout Hello, France!

Enter in your Appuser@host:~$ docker exec -it myweb /bin/bashroot@8807f82f280b:/# ls /etc/nginx/ && exitconf.d fastcgi_params koi-utf koi-win mime.types modules nginx.conf ...

Stop and Remove your Appuser@host:~$ docker rm -f mywebmyweb

root@host:/tmp# docker ps --allCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES


Focus on Docker Images...

14

docker push image[:TAG]docker image push image[:TAG]

docker imagesdocker image ls [--all]List all installed images

Pull an image from a registry docker pull imagedocker image pull image

Push an image to a registry

Remove an imagedocker rmi imagedocker image rm imagedocker image prune [--all]

Backup and restore an image (tarball)docker save -o archive.tar imagedocker load < archive.tar

Inspect the content of an image docker inspect image

Focus on Docker Containers...

15

docker stop containerdocker container stop container

docker ps [--all]docker container ls [--all]List all containers

[re]Start an inactive container docker [re]start containerdocker container [re]start container

Stop an active container

[stop and] Remove a container docker rm [-f] containerdocker container rm [-f] container

Execute a command in an active container / Get a shell console

docker exec container commanddocker exec -it container /bin/bash

Inspect the content of a container docker inspect container

Focus on Docker Containers...

16

docker create imagedocker run image

docker commit container [REPO[:TAG]]Create an image from a container

Show the logs of a container docker logs container

Create a new container

Backup and restore a container (tarball) docker export container > archive.tardocker import - container < archive.tar

17

1 - Get alpine imageuser@host:~$ docker pull alpine

2 - Run ituser@host:~$ docker run -d -it --name myapp alpine3 - Edit a file in the containeruser@host:~$ docker exec myapp \sh -c 'echo "Hello, France!" > /root/msg.txt'4 - Stop the containeruser@host:~$ docker stop myapp5 - Start the containeruser@host:~$ docker start myapp6 - Checkuser@host:~$ docker exec myapp less /root/msg.txtHello, France

What about my data ?

7 - Stop then remove the containeruser@host:~$ docker rm -f myapp8 - Run again a containeruser@host:~$ docker run -d -it --name myapp alpine9 - Checkuser@host:~$ docker exec myapp less /root/msg.txtmore: can't open '/root/msg.txt': No such file or directory

Remember that each container has is own read-write layer

When you instantiate an image, the new container starts with a clean filesystem

18

Run it user@host:~$ echo "Hello, Nantes!" > ~/msg.txt && docker run -d -it -v ~/msg.txt:/root/hop.txt --name mydata alpineCheckuser@host:~$ docker exec mydata ls /roothop.txtuser@host:~$ docker exec mydata less /root/hop.txtHello, Nantes!


Run another docker based on alpineuser@host:~$ docker run -d -it -v ~/msg.txt:/root/hop.txt --name mydatb alpineEdit the messageuser@host:~$ echo "Hello, Centrale!" > ~/msg.txtCheckuser@host:~$ docker exec mydata less /root/hop.txtHello, Centrale!user@host:~$ docker exec mydatb less /root/hop.txtHello, Centrale!

Sharing data in your docker host with containers

Use the -v option : -v [HOST-DIR]:[CONTAINER-DIR]It creates a bind mount exposing the host directory/host file to the container directory/file

19

Get information about volumesuser@host:~$ docker inspect mydata... "Mounts": [ { "Type": "bind", "Source": "/home/user/msg.txt", "Destination": "/root/hop.txt", "Mode": "", "RW": true, "Propagation": "rprivate" } ],...


By default, Docker mounts the volume in read-write mode

Sharing data in your docker host with containers

Instruction to share data in read-only modeuser@host:~$ docker run -d -it -v ~/msg.txt:/root/hop.txt:ro --name mydata alpine

20

Run the yoyo container user@host:~$ docker run -d -it -v /sharing --name yoyo alpineuser@host:~$ docker inspect yoyo... "Mounts": [ { "Type": "volume", "Name": "fdc5bd0de90688d590b38f9f931eb011c4de9c4032d49...", "Source": "/var/lib/docker/volumes/fdc5bd0de90...a54b/_data", "Destination": "/sharing", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ],


If the host directory/file is omitted, a data container (read-write mode per default) is created

The volume specified, here /sharing, is created inside the container

user@host:~$ touch /var/lib/docker/volumes/fdc5bd0d.../_data/msg.txt

user@host:~$ docker exec yoyo ls /sharingmsg.txt

Sharing data between containers

21

Run the dong container user@host:~$ docker run -d -it --volumes-from yoyo --name dong alpineuser@host:~$ docker inspect dong... "Mounts": [ { "Type": "volume", "Name": "fdc5bd0de90688d590b38f9f931eb011c4de9c40...4a54b", "Source": "/var/lib/docker/volumes/fdc5bd0de90...54b/_data", "Destination": "/sharing", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ],


user@host:~$ docker exec yoyo ls /sharingMsg.txt

user@host:~$ docker exec dong ls /sharingMsg.txt

user@host:~$ docker exec dong touch /sharing/msg2.txt

user@host:~$ docker exec yoyo ls /sharingmsg.txtMsg2.txt

user@host:~$ ls /var/lib/docker/volumes/fdc5bd0de90688d5.../_datamsg2.txt msg.txt

Sharing data between containers

22

user@host:~$ docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZEalpine latest 196d12cf6ab1 2 months ago 4.41MBuser@host:~$ docker run -d -it --name ding alpineuser@host:~$ docker exec -it ding /bin/sh/ # apk update && apk upgradefetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gzv3.8.1-133-g80b45d6920 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]v3.8.1-133-g80b45d6920 [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]OK: 9546 distinct packages availableOK: 4 MiB in 13 packages/ # exituser@host:~$ docker commit ding alpine:201812sha256:4acef3925b22e668e0e1755a3fa5ab6004889cacee28c8a487c5754ea105be70user@host:~$ docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZEalpine 201812 4acef3925b22 27 seconds ago 5.71MBalpine latest 196d12cf6ab1 2 months ago 4.41MB

Committing Changes in a ContainerTurning a container into an image

23

user@host:~$ docker diff dingC /libC /lib/apkC /lib/apk/db...C /var/cacheC /var/cache/apkA /var/cache/apk/APKINDEX.adfa7ceb.tar.gzA /var/cache/apk/APKINDEX.efaa1f73.tar.gzC /rootA /root/.ash_history

Committing Changes in a Container

First column: A means that the directory/file was added, C means that a change was made, D means that it was removed.

You see all changes applied to the read-write layer, the container itself - useful before executing the commit instruction

Display container changes

Docker Networking Services

24

Bridge Host

Share host interfaces

No more network isolation between the host and containers

Should be used very carefully

Macvlan Bridge

Unique MAC address

Allows you to configure slave/sub-interfaces of a parent, physical ethernet interface, each with its

own unique MAC address

None

No network !

Overlay

Multiple hosts with only L3 connectivity

Combines local bridges and VXLAN, GRE to

overlay container-to-container

Useful to offer network connectivity between containers on multiple

hosts usingtheir own IP addresses

25

user@host:~$ docker run --name alice -p 8080:80 -d nginxuser@host:~$ docker run --name bob -p 8081:80 -d nginx

user@host:~$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESffe72798ac2d nginx "nginx -g 'daemon of…" … … 0.0.0.0:8081->80/tcp bob1a2f2f83e044 nginx "nginx -g 'daemon of…" … … 0.0.0.0:8080->80/tcp alice

Going deeper in docker bridge default mode

Expose unique ports on the host - here 8081 and 8080 -

26

user@host:~$ brctl showbridge name bridge id STP enabled interfacesdocker0 8000.02422d1a2630 no veth856faee

vetha53c999user@host:~$ ip add...3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:2d:1a:26:30 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 valid_lft forever preferred_lft forever7: vetha53c999@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default link/ether be:71:06:64:64:8f brd ff:ff:ff:ff:ff:ff link-netnsid 111: veth856faee@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default link/ether b2:6c:95:08:43:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 2

...


27

user@host:~$ docker inspect bob "NetworkSettings": { "Bridge": "",

... "Gateway": "172.17.0.1", "IPAddress": "172.17.0.4", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:04",

...}

user@host:~$ docker inspect alice "NetworkSettings": { "Bridge": "",

... "Gateway": "172.17.0.1", "IPAddress": "172.17.0.3", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:03",

...}


Docker assigns a dynamic IP address for both containers

28

user@host:~$ iptables -L -n -v ...Chain DOCKER (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 172.17.0.3 tcp dpt:80 0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 172.17.0.4 tcp dpt:80...

user@host:~$ iptables -t nat -L -n -vChain DOCKER (2 references) pkts bytes target prot opt in out source destination 1 84 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0 0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 to:172.17.0.3:80 0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8081 to:172.17.0.4:80


29

user@host:~$ docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZEalpine latest 196d12cf6ab1 2 months ago 4.41MB

user@host:~$ vi /tmp/DockerfileFROM alpine:latestRUN apk update && apk upgradeENTRYPOINT ["/bin/echo","Hello, France!"]

user@host:~$ cd /tmp && docker build . -t hello

Sending build context to Docker daemon 18.94kBStep 1/3 : FROM alpine:latest ---> 196d12cf6ab1Step 2/3 : RUN apk update && apk upgrade ---> Running in 116d94cb96fafetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gzv3.8.1-115-ge3ed6b4e31 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]v3.8.1-112-g45bdd0edfb [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]OK: 9546 distinct packages availableOK: 4 MiB in 13 packagesRemoving intermediate container 116d94cb96fa ---> fd105816bcb9Step 3/3 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in 8756eb9e1d37Removing intermediate container 8756eb9e1d37 ---> b4277a54d78bSuccessfully built b4277a54d78bSuccessfully tagged hello:latest

Build your own image

30

user@host:~$ docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZE<none> <none> fd105816bcb9 5 minutes ago 5.71MBhello latest b4277a54d78b 5 minutes ago 5.71MBalpine latest 196d12cf6ab1 2 months ago 4.41MB

Sending build context to Docker daemon 18.94kBStep 1/3 : FROM alpine:latest ---> 196d12cf6ab1Step 2/3 : RUN apk update && apk upgrade ---> Running in 116d94cb96fafetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gzv3.8.1-115-ge3ed6b4e31 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]v3.8.1-112-g45bdd0edfb [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]OK: 9546 distinct packages availableOK: 4 MiB in 13 packagesRemoving intermediate container 116d94cb96fa ---> fd105816bcb9Step 3/3 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in 8756eb9e1d37Removing intermediate container 8756eb9e1d37 ---> b4277a54d78bSuccessfully built b4277a54d78bSuccessfully tagged hello:latest

31

user@host:~$ vi /tmp/DockerfileFROM alpine:latestRUN apk update && apk upgradeRUN apk add opensshENTRYPOINT ["/bin/echo","Hello, France!"]

user@host:~$ cd /tmp && docker build . -t hello

Sending build context to Docker daemon 5.632kBStep 1/4 : FROM alpine:latest ---> 196d12cf6ab1Step 2/4 : RUN apk update && apk upgrade ---> Using cache ---> fd105816bcb9Step 3/4 : RUN apk add openssh ---> Running in c0fbeb4c8ea8fetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gz(1/6) Installing openssh-keygen (7.7_p1-r3)...(6/6) Installing openssh (7.7_p1-r3)Executing busybox-1.28.4-r1.triggerOK: 8 MiB in 19 packagesRemoving intermediate container c0fbeb4c8ea8 ---> 3ba8527195d0Step 4/4 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in ead00e7c9adaRemoving intermediate container ead00e7c9ada ---> 1d451e42e9e2Successfully built 1d451e42e9e2Successfully tagged hello:latest


32

Sending build context to Docker daemon 5.632kBStep 1/4 : FROM alpine:latest ---> 196d12cf6ab1Step 2/4 : RUN apk update && apk upgrade ---> Using cache ---> fd105816bcb9Step 3/4 : RUN apk add openssh ---> Running in c0fbeb4c8ea8fetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gz(1/6) Installing openssh-keygen (7.7_p1-r3)...(6/6) Installing openssh (7.7_p1-r3)Executing busybox-1.28.4-r1.triggerOK: 8 MiB in 19 packagesRemoving intermediate container c0fbeb4c8ea8 ---> 3ba8527195d0Step 4/4 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in ead00e7c9adaRemoving intermediate container ead00e7c9ada ---> 1d451e42e9e2Successfully built 1d451e42e9e2Successfully tagged hello:latest

root@host:/tmp# docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZE<none> <none> 3ba8527195d0 5 minutes ago 11MBhello latest 1d451e42e9e2 5 minutes ago 11MB<none> <none> b4277a54d78b 24 hours ago 5.71MB<none> <none> fd105816bcb9 24 hours ago 5.71MBalpine latest 196d12cf6ab1 2 months ago 4.41MB

root@host:/tmp# docker image pruneWARNING! This will remove all dangling images.Are you sure you want to continue? [y/N] yDeleted Images:deleted: sha256:b4277a54d78bc870ba2c1e971f076da6a324cc73f97dead788eb0823592c9e5e

Total reclaimed space: 0B


Thanks for your attention

Documents

Dec, 2018 Yoann Juet Information Technology Department ... · containerd containerd : runtime daemon Core container runtime for Docker Manage the complete container lifecycle (stop,