Clustering RabbitMQ using Consul

Valentin Fadeev

DevOps Engineer @ Student.com

@ValFadeev HashiCorp Meetup, London 20 Jun 2017 1 / 10

Clustering in RabbitMQ

A RabbitMQ broker is a logical grouping of one or several Erlang nodes,each running the RabbitMQ application and sharing users, virtual hosts,queues, exchanges, bindings, and runtime parameters. Sometimes we referto the collection of nodes as a cluster1.

1https://www.rabbitmq.com/clustering.html@ValFadeev HashiCorp Meetup, London 20 Jun 2017 2 / 10

Autocluster Plugin1

automated clustering

declarative configuration

pluggable backends

1https://github.com/rabbitmq/rabbitmq-autocluster@ValFadeev HashiCorp Meetup, London 20 Jun 2017 3 / 10

Startup Sequence v1

start with a random delay

discover registered nodes

choose a node to cluster with and join

register with the backend

begin updating own health check

@ValFadeev HashiCorp Meetup, London 20 Jun 2017 4 / 10

Startup Sequence v2

acquire startup lock

discover registered nodes

choose a node to cluster with and join

register with the backend

begin updating own health check

release startup lock

@ValFadeev HashiCorp Meetup, London 20 Jun 2017 5 / 10

Why Consul?

DNS-based service discovery

decentralised health checking

configuration store

synchronisation primitives

event propagation

@ValFadeev HashiCorp Meetup, London 20 Jun 2017 6 / 10

Locking in Consul1

create a session associated with a node

begin refreshing session TTL

try to acquire the session for a key in KV store

if lock is held by someone else watch for changes

acquire session

release session

1https://www.consul.io/docs/guides/leader-election.html@ValFadeev HashiCorp Meetup, London 20 Jun 2017 7 / 10

Plugin Enhancements

use long node names

allow health checks in warning status

use start up locking instead of randomized delay

@ValFadeev HashiCorp Meetup, London 20 Jun 2017 8 / 10

Deployment

Consul Server

Consul ServerConsul ServerRabbitMQ Consul Client

RabbitMQ Consul Client

RabbitMQ Consul Client

@ValFadeev HashiCorp Meetup, London 20 Jun 2017 9 / 10

DNS Configuration

Docker Bridge

BIND

ConsulApp

Host network

Container network

@ValFadeev HashiCorp Meetup, London 20 Jun 2017 10 / 10