28
dockersh Tomas Doran @bobtfish 20141014

Dockersh and a brief intro to the docker internals

Embed Size (px)

DESCRIPTION

Dockersh is a new tool to give a login shell into per-user Docker containers. (https://github.com/Yelp/dockersh) This talk will be an illustrated tour of what dockersh does, and why it might be useful to you. During this journey we’ll dive into the Go programming language, + libcontainer (the technologies Docker is built on) in addition to the facilities Docker uses in the kernel (Namespaces, Cgroups and Capabilities), how these work, and how normal mortals can (ab)use them for fun and profit

Citation preview

Page 1: Dockersh and a brief intro to the docker internals

dockershTomas  Doran  @bobtfish  2014-­‐10-­‐14

Page 2: Dockersh and a brief intro to the docker internals

2

Page 3: Dockersh and a brief intro to the docker internals

Shared (personal) bounce host

• Multiple users • Persistent ssh sessions • ‘Playground’

• Fair split of resources? • Isolation? Security?

3

Page 4: Dockersh and a brief intro to the docker internals

VMs are expensive

• 12 tmux sessionsvs

• 12 Virtual machines

4

Page 5: Dockersh and a brief intro to the docker internals

Containers are cheap

• Container as lightweight VM • One persistent container per user • /home/myuser from host • /etc/passwd from host • Let the user supply own container? • sshd per container = 1 port per user

5

Page 6: Dockersh and a brief intro to the docker internals

Containers are cheap

• Container as lightweight VM • One persistent container per user • /home/myuser from host • /etc/passwd from host • Let the user supply own container? • sshd per container = 1 port per user

6

Page 7: Dockersh and a brief intro to the docker internals

Containers are cheap

• One persistent container per user • Even let the user supply the container

• sshd per container = 1 port per user • Container as lightweight VM? • Need to edit ~/.ssh/config

7

Page 8: Dockersh and a brief intro to the docker internals

Can we do better?

8

Page 9: Dockersh and a brief intro to the docker internals

nsenter• Exec a process in an existing namespace • Debug running containers as root

9

Page 10: Dockersh and a brief intro to the docker internals

nsenter• Exec a process in an existing namespace • Debug running containers as root

10

Page 11: Dockersh and a brief intro to the docker internals

dockersh.sh

11

Page 12: Dockersh and a brief intro to the docker internals

12

Page 13: Dockersh and a brief intro to the docker internals

What’s a Docker container?cat /var/lib/docker/execdriver/native/d910d20082fed3763b377a2d46e30da5def9fdd7863a0642ea154er.json \ | jq .

13

Page 14: Dockersh and a brief intro to the docker internals

14

Page 15: Dockersh and a brief intro to the docker internals

Capabilities

• Pluggable in Docker 1.2.0 • —drop_cap

• Scary default capabilities: • SUID • SGID • MKNOD

15

Page 16: Dockersh and a brief intro to the docker internals

cgroups

• Memory groups • CPU groups • IO groups

16

Page 17: Dockersh and a brief intro to the docker internals

/sys/fs/cgroup

17

Page 18: Dockersh and a brief intro to the docker internals

/sys/fs/cgroup

18

Page 19: Dockersh and a brief intro to the docker internals

/sys/fs/cgroup

19

Page 20: Dockersh and a brief intro to the docker internals

/sys/fs/cgroup

20

Page 21: Dockersh and a brief intro to the docker internals

Reuse capabilities

21

Page 22: Dockersh and a brief intro to the docker internals

Namespaces

• Per container separation

• UTS - hostnames • IPC - sysvipc • PID - processes • NET - network

22

Page 23: Dockersh and a brief intro to the docker internals

PID Namespaces From inside

23

Page 24: Dockersh and a brief intro to the docker internals

PID Namespaces From outside

24

Page 25: Dockersh and a brief intro to the docker internals

NET Namespace

• Per container IP stack • Bandwidth limits per container

25

Page 26: Dockersh and a brief intro to the docker internals

Reuse namespaces

26

Page 27: Dockersh and a brief intro to the docker internals

Todo

• Ptys • scp • Better agent forwarding

27

Page 28: Dockersh and a brief intro to the docker internals

Thanks!

• We’re hiring! http://www.yelp.co.uk/careers?jvi=ogVTXfwL

• https://github.com/Yelp/dockersh • http://engineeringblog.yelp.com/2014/08/

hack209-dockersh.html

28