43
Docker cluster With swarm, consul, registrator and consul-template jmaitrehenry

Docker cluster with swarm, consul, registrator and consul-template

Embed Size (px)

Citation preview

Page 1: Docker cluster with swarm, consul, registrator and consul-template

Docker clusterWith swarm, consul, registrator and consul-

template

jmaitrehenry

Page 2: Docker cluster with swarm, consul, registrator and consul-template

2

Overview

• Create a swarm cluster

• Manage a swarm cluster

• Service discovery

• Service auto-configuration

2015-06-15

Page 3: Docker cluster with swarm, consul, registrator and consul-template

3

1 linux server…

2015-06-15

An host running Linux

Page 4: Docker cluster with swarm, consul, registrator and consul-template

4

With docker…

2015-06-15

An host running Linux

Docker daemon

Page 5: Docker cluster with swarm, consul, registrator and consul-template

5

2 linux servers with docker…

2015-06-15

An host running Linux

Docker daemon

An host running Linux

Docker daemon

Page 6: Docker cluster with swarm, consul, registrator and consul-template

6

How can I manage them?

• Individually? Why not if you like that

• On a cluster with a manager? Yeah!

2015-06-15

Page 7: Docker cluster with swarm, consul, registrator and consul-template

7

Docker swarm

2015-06-15

Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual host.

https://github.com/docker/swarm

Page 8: Docker cluster with swarm, consul, registrator and consul-template

8

How do I add a node to swarm ?

• Swarm need to register node IP and docker tcp port somewhere

• Consul have a Key/Value store and swarm can register node on it

2015-06-15

Page 9: Docker cluster with swarm, consul, registrator and consul-template

9

What is Consul?

Consul is a distributed, highly available system and provides:

• Service discovery

• Failure detection

• Key/Value store

• Bonus: supports multiple datacenter

2015-06-15

Page 10: Docker cluster with swarm, consul, registrator and consul-template

10

Create a swarm cluster

2015-06-15An host running Linux

Docker daemon

An host running Linux

Docker daemon

Consul host

Consuldaemon

Swarm cluster

Page 11: Docker cluster with swarm, consul, registrator and consul-template

11

How do I register an node to consul ?

• With swarm !

docker run -d swarm:0.3.0-rc2 join --addr=10.1.1.231:2375 consul://10.1.8.32:8500/dc1

2015-06-15

Page 12: Docker cluster with swarm, consul, registrator and consul-template

12

How do I register an node to consul ?

• swarm:0.3.0-rc2 ? Support docker login

• --addr=<public host IP>:<port of docker http API>

• consul://<IP of consul server>:<port>/<datacenter>

docker run -d swarm:0.3.0-rc2 join --addr=10.1.1.231:2375 consul://10.1.8.32:8500/dc1

2015-06-15

Page 13: Docker cluster with swarm, consul, registrator and consul-template

13

Create a swarm cluster

2015-06-15An host running Linux

Docker daemon

An host running Linux

Docker daemon

Consul host

Consuldaemon Register node to consul K/V store

Swarm cluster

Page 14: Docker cluster with swarm, consul, registrator and consul-template

14

How can I manage my swarm cluster?

2015-06-15

• With swarm !

docker run -d -p 2375:2375 swarm manage consul://10.1.8.32:8500/dc1

Page 15: Docker cluster with swarm, consul, registrator and consul-template

15

How can I manage my swarm cluster?

# export DOCKER_HOST= tcp://<manager IP>:<port>

# docker <docker cmd>

Without TLS enabled (dev only please ) : # unset DOCKER_TLS_VERIFY

2015-06-15

Page 16: Docker cluster with swarm, consul, registrator and consul-template

16

How can I manage my swarm cluster?

2015-06-15

Page 17: Docker cluster with swarm, consul, registrator and consul-template

17

Manage a swarm cluster

2015-06-15

An host

Docker daemo

n

Consul host

Consuldaemon

Register node

My laptopDocker daemo

n

Docker daemo

n

An host

manager

Swarm cluster

Page 18: Docker cluster with swarm, consul, registrator and consul-template

18

Manage a swarm cluster

2015-06-15

An host

Docker daemo

n

Consul host

Consuldaemon

Register node

My laptopDocker daemo

n

Docker daemo

n

An host

managerGet node list

Swarm cluster

Page 19: Docker cluster with swarm, consul, registrator and consul-template

19

Manage a swarm cluster

2015-06-15

An host

Docker daemo

n

Consul host

Consuldaemon

Register node

My laptopDocker daemo

n

Docker daemo

n

An host

manager

Manage hosts

Get node list

Swarm cluster

Page 20: Docker cluster with swarm, consul, registrator and consul-template

20

Ok, fine, I have 3 docker containers but how they know each other?

• First, we need to register service somewhere

• Second, each service can ask somewhere where are other service

2015-06-15

Page 21: Docker cluster with swarm, consul, registrator and consul-template

21

Service discovery

Service discovery is a key component of most distributed systems and service oriented architectures. The problem seems simple at first: How do clients determine the IP and port for a service that exist on multiple hosts?

2015-06-15

Page 22: Docker cluster with swarm, consul, registrator and consul-template

22

Oh, consul is a service discovery!

• Yes it is!

Service list: http://10.1.8.32:8500/v1/catalog/services

2015-06-15

Page 23: Docker cluster with swarm, consul, registrator and consul-template

23

Consul• Service details:

http://10.1.8.32:8500/v1/catalog/service/statsd-8125

2015-06-15

Consul server

Page 24: Docker cluster with swarm, consul, registrator and consul-template

24

Consul• Service details:

http://10.1.8.32:8500/v1/catalog/service/statsd-8125

2015-06-15

Consul server

Docker node

Page 25: Docker cluster with swarm, consul, registrator and consul-template

25

Consul• Service details:

http://10.1.8.32:8500/v1/catalog/service/statsd-8125

2015-06-15

Consul server

Docker node

Service address

Page 26: Docker cluster with swarm, consul, registrator and consul-template

26

How can docker register services in consul?

• With a service registry bridge for Docker like Registrator

2015-06-15

Page 27: Docker cluster with swarm, consul, registrator and consul-template

27

Registrator

• Service registry bridge for Docker

• « Registrator automatically register/deregisters services for Docker containers based on published ports and metadata from the container environment.  »

• https://github.com/gliderlabs/registrator

2015-06-15

Page 28: Docker cluster with swarm, consul, registrator and consul-template

28

Registrator

2015-06-15

On each swarm node

Page 29: Docker cluster with swarm, consul, registrator and consul-template

29

Registrator

2015-06-15

On each swarm node

Local docker socket

Page 30: Docker cluster with swarm, consul, registrator and consul-template

30

Registrator

2015-06-15

Public node IP

On each swarm node

Local docker socket

Page 31: Docker cluster with swarm, consul, registrator and consul-template

31

Demo

2015-06-15

Page 32: Docker cluster with swarm, consul, registrator and consul-template

322015-06-15

Page 33: Docker cluster with swarm, consul, registrator and consul-template

332015-06-15

Page 34: Docker cluster with swarm, consul, registrator and consul-template

34

Ok, I have some nginx running, how can I have a single entry point?

• With a load balancer (haproxy) with an auto update configuration

2015-06-15

Page 35: Docker cluster with swarm, consul, registrator and consul-template

35

Consul template

« The daemon consul-template queries a Consul instance and updates any number of specified templates on the filesystem. As an added bonus, consul-template can optionally run arbitrary commands when the update process completes »

https://github.com/hashicorp/consul-template

2015-06-15

Page 36: Docker cluster with swarm, consul, registrator and consul-template

36

Consul template - command

2015-06-15

Page 37: Docker cluster with swarm, consul, registrator and consul-template

37

Consul template – where to run it?

• Where you want, the only condition is to have access to consul server.

• You could have it in a container

• You could have it somewhere else

2015-06-15

Page 38: Docker cluster with swarm, consul, registrator and consul-template

38

Consul template – template file

2015-06-15

Service name

Page 39: Docker cluster with swarm, consul, registrator and consul-template

39

Consul template – template file

2015-06-15

Service name

Service IP

Page 40: Docker cluster with swarm, consul, registrator and consul-template

40

Consul template – template file

2015-06-15

Service name

Service IP

Service port

Page 41: Docker cluster with swarm, consul, registrator and consul-template

41

Consul template - testing

2015-06-15

Page 42: Docker cluster with swarm, consul, registrator and consul-template

42

Consul template - testing

2015-06-15

Dump generated template and exit

Page 43: Docker cluster with swarm, consul, registrator and consul-template

432015-06-15

Thanks!