Microservices, Docker & Service Discovery with Smartstack - English version

techblog.newsweaver.com

github.com/PierreVincent

@PierreVincent

pierrevincent

Microservices, Docker & Service Discovery

with Smartstack

Meetup - Docker RennesISTIC, May 28th 2015

This work is licensed under a Creative Commons Attribution 4.0 International License.

newsweaver

MicroservicesDeployment

Build environment

Integration tests

Dev environment

Microservices are small, autonomous services that work together.

Sam NewmanBuilding Microservices

MonolithicApplication

Anchored in Technology

Complex code base

Refactoring is difficult

Disruptive deployment

Production issues are hard to isolate

Scaling options are limited

Small size=

High cohesion

Autonomy & isolation

=Low coupling

TechnologyHeterogeneity

Error isolation=

Resilience

Deployments with lower impact

Tailored scaling

+

Microservices

Cohesion1 container = 1 microservice

Technology HeterogeneityLanguage specific to the container

Delivery- Docker images (whatever the tech)- Docker registry

Low couplingEach container is independent

Simpler deploymentsOnly needs Docker

Orchestration solutions- More complex deployments- Swarm, Compose, Kubernetes, Mesos...

Test / Validation

Distributed Systems

ContinuousDelivery

MonitoringInterfaces between

services

What microservices boundaries

Everything on microservices Some reading...

Service DiscoveryDynamically wiring Microservices

Variable number of services Short life-span of services

How to locate the available services?

How to balance the requests to these

services?

What to do when a service is no longer

available?

❏ Registration / Lookup

❏ Load Balancing

❏ Transparency for clients

❏ Proactive error management

❏ Resistance to external problems

❏ No “Single Point of Failure”

Checklist

Recommendations Service

Viewing HistoryService

C*

.../users/123/recommendations

.../users/123/viewingHistory

{ “watched”: [ “Breaking Bad”, … ]}

{“recommendations”: [ { “watch”: “Better call saul!”, “because”: [“Breaking Bad”] }, ...]}

Recommendations Service

Viewing HistoryService

http://1.1.1.1/users/123/recommendations

services.properties

viewing_history_url = http://1.1.1.1

1.1.1.1

Solution 1: Config file

Clear list of services and instances

Possible to use shared volume with containers

Not really dynamic :- service must take changes into account- configuration must be updated

+

-

Recommendations Service

Viewing HistoryService

http://viewing-history/users/123/recommendations

/etc/hosts

1.1.1.1 viewing-history

1.1.1.1

Solution 2: Docker links

docker run --named=viewing-history viewing-history-service:latest

docker run --named=recommendations --link viewing-history:viewing-history recommendations-service:latest

Simple for the clients

Dependencies are clear

Possible deployment using docker-compose

Still not dynamic :- predefined order- harder for multiple instances

+

-

viewing-history.service → 1.1.1.1

Recommendations Service

Viewing HistoryService

http://viewing-history.service/users/123/recommendations

1.1.1.1

Solution 3: DNS

DNS

Simple for clients

Round-robin DNS:- multiples instances- load balancing

Propagation issues

+

-

[ viewing-history: 1.1.1.1 ...]

Recommendations Service

Viewing HistoryService

http://1.1.1.1/users/123/recommendations

1.1.1.1

Solution 4: Publisher / Subscriber

Key-valueStore

Dynamic :- each instance registers itself- clients discover the instances

Setup of a Key-value store

More complex for the services- Discovery logic coded in the services

+

-

[ viewing-history: 1.1.1.1 ...]

Recommendations Service

Viewing HistoryService

1.1.1.1

Solution 4: Publisher / Subscriber + Ambassador

Key-valueStore

http://1.1.1.1/users/123/recommendations

Very dynamic

Transparency for Clients and Providers

Setup of a Key-value Store

+

-

SmartstackDiscovery framework and integration with Docker

github.com/airbnb/nerve

github.com/airbnb/synapse

Zookeeper

Application

Viewing History Service

API Application

Recommendations Service

SynapseHAProxyNerve

Publication

Discovery

SmartstackOverview

Application

Nerve

ZK

1.1.1.1

nerve.confname = viewing_historyip = 1.1.1.1port = 8080healthCheck = /health

Publicatio

n

SmartstackNerve

API(8080)

/health

Viewing History Service

Application

Synapse

ZK

1.1.1.2

synapse.confviewing_history → 9000

Discov

ery

SmartstackSynapse

HAProxy

1.1.1.1

haproxy.confviewing_history: frontend: localhost:9000 backends: [1.1.1.1:8080]

GEThttp://localhost:9000/users/123/viewingHistory

8080 Servicesviewing_history: [1.1.1.1:8080]

Recommendations Service

ping

Distribution de base(ex. Ubuntu)

ruby

synapse (gem)

HA Proxy

nerve (gem)

service.jarsynapse.conf / nerve.conf

startup script

server.js + autres

startup script

Base Smartstack

startSynapse.sh

startNerve.sh

+ Tech

synapse.conf / nerve.conf + Service code+ Config

FROM newsweaver/smartstack-java

# DiscoveryADD nerve.conf.json /etc/ADD synapse.conf.json /etc/

# JARADD service.jar /opt/service/

# StartupADD start.sh /opt/start.shENTRYPOINT ["/opt/start.sh"]

Dockerfile

#!/bin/bash

/opt/startSynapse.sh/opt/startNerve.sh

java -jar /opt/service/service.jar

start.sh

$ docker build -t my-service ....$ docker run -d -e ZK_HOSTS=zk1:2181,zk2:2181 my-service

color-service

color-service

color-serviceGET .../v1/color

color-ui-service

blue

{ "name": "blue", "hex": "0000FF"}

{ "name": "red", "hex": "FF0000"}

{ "name": "green", "hex": "00FF00"}

Demo !github.com/PierreVincent/discovery-demo-color

techblog.newsweaver.com

Questions ?

@PierreVincent