56
NATS in action A Real time Microservices Architecture handled by NATS 1

NATS in action - A Real time Microservices Architecture handled by NATS

Embed Size (px)

Citation preview

Page 1: NATS in action - A Real time Microservices Architecture handled by NATS

NATS in actionA Real time Microservices Architecture handled by NATS

1

Page 2: NATS in action - A Real time Microservices Architecture handled by NATS

Who am I?Raül PérezSenior Engineer at Carrenza Ltd.

Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]

2

Page 3: NATS in action - A Real time Microservices Architecture handled by NATS

Who am I?Raül PérezSenior Engineer at Carrenza Ltd.

Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]

3

● Almost 4 years working on devop & distributed projects.

Page 4: NATS in action - A Real time Microservices Architecture handled by NATS

Who am I?Raül PérezSenior Engineer at Carrenza Ltd.

Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]

4

● Almost 4 years working on devop & distributed projects.● Still more a dev than op dude.

Page 5: NATS in action - A Real time Microservices Architecture handled by NATS

Who am I?Raül PérezSenior Engineer at Carrenza Ltd.

Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]

5

● Almost 4 years working on devop & distributed projects.● Still more a dev than op dude.● Proud to be a Gopher after a looong PHP/Ruby/Python/Node.js past

experience. :P

Page 6: NATS in action - A Real time Microservices Architecture handled by NATS

Who am I?Raül PérezSenior Engineer at Carrenza Ltd.

Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]

6

● Almost 4 years working on devop & distributed projects.● Still more a dev than op dude.● Proud to be a Gopher after a looong PHP/Ruby/Python/Node.js past

experience. :P● Based in Barcelona.

Page 7: NATS in action - A Real time Microservices Architecture handled by NATS

Who am I?Raül PérezSenior Engineer at Carrenza Ltd.

Twitter: @repejotaGithub: https://github.com/repejotaEmail: [email protected]

7

● Almost 4 years working on devop & distributed projects.● Still more a dev than op dude.● Proud to be a Gopher after a looong PHP/Ruby/Python/Node.js past

experience. :P● Based in Barcelona.● Love startups & love remote work!

Page 8: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices

8

Page 9: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices● Small autonomous services that work together.

9

● Divide and conquer approach.

Page 10: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices● Small autonomous services that work together.● Communicate with lightweight mechanisms.

10

● Divide and conquer approach.● HTTP/REST microservices. ( Request / Response )

Page 11: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices● Small autonomous services that work together.● Communicate with lightweight mechanisms.

11

● Divide and conquer approach.● HTTP/REST microservices. ( Request / Response )● Message Systems for asynchronous communication. ( Publish/Subscribe )

Page 12: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices● Small autonomous services that work together.● Communicate with lightweight mechanisms.● The platform is composed of one or more stateless processes.

12

● Divide and conquer approach.● HTTP/REST microservices. ( Request / Response )● Message Systems for asynchronous communication. ( Publish/Subscribe )● There is no shared data between different processes.

Page 13: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices● Small autonomous services that work together.● Communicate with lightweight mechanisms.● The platform is composed of one or more stateless processes.● Designed to be distributed.

13

● Divide and conquer approach.● HTTP/REST microservices. ( Request / Response )● Message Systems for asynchronous communication. ( Publish/Subscribe )● There is no shared data between different processes.● Scaling is easy, each microservice can be executed in a different server.

Page 14: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices … Why?● It is quick to (re) write and to deploy.

14

● Simple application can be handled by less developers.

Page 15: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices … Why?● It is quick to (re) write and to deploy.

15

● Simple application can be handled by less developers.● Developers do not need to know the entire platform to be productive.

Page 16: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices … Why?● It is quick to (re) write and to deploy.● It knows that failures exist and how to handle them.

16

● Simple application can be handled by less developers.● Developer do not need to know the entire platform to be productive.● Defensive programming, services should not break.

Page 17: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices … Why?● It is quick to (re) write and to deploy.● It knows that failures exist and how to handle them.● KISS: Keep It Simple and Stupid.

17

● Simple application can be handled by less developers.● Developer do not need to know the entire platform to be productive.● Defensive programming, services should not break.● It must provide one functionality.

Page 18: NATS in action - A Real time Microservices Architecture handled by NATS

Microservices … Why?● It is quick to (re) write and to deploy.● It knows that failures exist and how to handle them.● KISS: Keep It Simple and Stupid.● It must provide an API that other services can use.

18

● Simple application can be handled by less developers.● Developer do not need to know the entire platform to be productive.● Defensive programming, services should not break.● It must provide one functionality.● Rely on best practices.

Page 19: NATS in action - A Real time Microservices Architecture handled by NATS

NATS

19

Page 20: NATS in action - A Real time Microservices Architecture handled by NATS

What is NATS?● Cloud-native message platform.

20

● Designed to natively support modern cloud architectures.

Page 21: NATS in action - A Real time Microservices Architecture handled by NATS

What is NATS?● Cloud-native message platform.● Highly performant.

21

● Designed to natively support modern cloud architectures.● It is fast! Can handle millions of messages per second.

Page 22: NATS in action - A Real time Microservices Architecture handled by NATS

What is NATS?● Cloud-native message platform.● Highly performant.● Extremely lightweight.

22

● Designed to natively support modern cloud architectures.● It is fast! Can handle millions of messages per second.● A single binary, easy to deploy without unnecessary complexity.

Page 23: NATS in action - A Real time Microservices Architecture handled by NATS

What is NATS?● Cloud-native message platform.● Highly performant.● Extremely lightweight.● Support for various messaging models and use cases.

23

● Designed to natively support modern cloud architectures.● It is fast! Can handle millions of messages per second.● A single binary, easy to deploy without unnecessary complexity.● Request/Response, Publish/Subscribe, Queues.

Page 24: NATS in action - A Real time Microservices Architecture handled by NATS

What is NATS?● Cloud-native message platform.● Highly performant.● Extremely lightweight.● Support for various messaging models and use cases.

24

● Designed to natively support modern cloud architectures.● It is fast! Can handle millions of messages per second.● A single binary, easy to deploy without unnecessary complexity.● Request/Response, Publish/Subscribe, Queues.● Simple text based protocol.

Page 25: NATS in action - A Real time Microservices Architecture handled by NATS

ErnestIAAS+PAAS Hybrid Cloud

Platform

25

Page 26: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)

26

● Manage servers, networks, balancers etc ... of your platform.

Page 27: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)● Manage your platform. (PAAS)

27

● Manage servers, networks, balancers etc ... of your platform.● Deploy and upgrade your application to the desired servers.

Page 28: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)● Manage your platform. (PAAS)● Scaling.

28

● Manage servers, networks, balancers etc ... of your platform.● Deploy and upgrade your application to the desired servers.● Automate everything.

Page 29: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)● Manage your platform. (PAAS)● Scaling.

29

● Manage servers, networks, balancers etc ... of your platform.● Deploy and upgrade your application to the desired servers.● Automate everything.● Evolve your platform when you need and redeploy your platform.

Page 30: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : IAAS+PAAS Hybrid Cloud Platform● Manage your infrastructure. (IAAS)● Manage your platform. (PAAS)● Scaling.● Hybrid (Multiple Providers)

30

● Manage servers, networks, balancers etc ... of your platform.● Deploy and upgrade your application to the desired servers.● Evolve your infrastructure when you need and redeploy your platform.● Automate everything.● Do not rely on a single provider, use different ones with a single interface.

Page 31: NATS in action - A Real time Microservices Architecture handled by NATS

Architecture of Ernest

31

Provider

Provider

Provider

MicroservicesNATS

FSM

LogsDB

PublicAPI

&

UI

Monitor

Provider

Page 32: NATS in action - A Real time Microservices Architecture handled by NATS

Architecture of Ernest

32

Provider

Provider

Provider

MicroservicesNATS

FSM

LogsDB

PublicAPI

&

UI

Monitor

Provider

REST API

Page 33: NATS in action - A Real time Microservices Architecture handled by NATS

Architecture of Ernest

33

Provider

Provider

Provider

MicroservicesNATS

FSM

LogsDB

PublicAPI

&

UI

Monitor

Provider

REST API

Events + State

Page 34: NATS in action - A Real time Microservices Architecture handled by NATS

Architecture of Ernest

34

Provider

Provider

Provider

MicroservicesNATS

FSM

LogsDB

PublicAPI

&

UI

Monitor

Provider

REST API

Events + State

Business LogicReal Time (almost)

Page 35: NATS in action - A Real time Microservices Architecture handled by NATS

Architecture of Ernest

35

Provider

Provider

Provider

MicroservicesNATS

FSM

LogsDB

PublicAPI

&

UI

Monitor

Provider

REST API

Events + State

Business LogicReal Time (almost)

VMWare

Page 36: NATS in action - A Real time Microservices Architecture handled by NATS

Architecture of Ernest

36

Provider

Provider

Provider

MicroservicesNATS

FSM

LogsDB

PublicAPI

&

UI

Monitor

Provider

REST API

Events + State

Business LogicReal Time (almost)

VMWare

AWS

Page 37: NATS in action - A Real time Microservices Architecture handled by NATS

Architecture of Ernest

37

Provider

Provider

Provider

MicroservicesNATS

FSM

LogsDB

PublicAPI

&

UI

Monitor

Provider

REST API

Events + State

Business LogicReal Time (almost)

VMWare

AWS

Others...

Page 38: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest + NATS

38

Page 39: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest + NATS● NATS is the central communication system for Ernest.

39

● Ernest core is a series of microservices talking each other through NATS.

Page 40: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest + NATS● NATS is the central communication system for Ernest.● Ernest executes workflows defined by the user to build environments.

40

● Ernest core is a series of microservices talking each other through NATS.● Users posts a workflow written in a YAML file to be processed.

Page 41: NATS in action - A Real time Microservices Architecture handled by NATS

Workflow example

41

---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run

● Simple example.

Page 42: NATS in action - A Real time Microservices Architecture handled by NATS

Workflow example

42

---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run

● Simple example.● Each environment has an unique name.

Page 43: NATS in action - A Real time Microservices Architecture handled by NATS

Workflow example

43

---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run

● Simple example.● Each environment has an unique name.● Action: Create a network on our provider.

Page 44: NATS in action - A Real time Microservices Architecture handled by NATS

Workflow example

44

---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run

● Simple example.● Each environment has an unique name.● Action: Create a network on our provider.● Action: Create an instance type for our app.

Page 45: NATS in action - A Real time Microservices Architecture handled by NATS

Workflow example

45

---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run

● Simple example.● Each environment has an unique name.● Action: Create a network on our provider.● Action: Create an instance type for our app.● Count determines the number of servers.

Page 46: NATS in action - A Real time Microservices Architecture handled by NATS

Workflow example

46

---name: my-awesome-examplenetworks: - name: network-web subnet: 10.2.0.0/24instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run

● Simple example.● Each environment has an unique name.● Action: Create a network on our provider.● Action: Create an instance type for our app.● Count determines the number of servers.● Action: Deploy your application and run it.

Page 47: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest + NATS● NATS is the central communication system for Ernest.● A workflow composed by multiple actions is executed by the user.● A finite state machine handles the state for the platform.

47

● Ernest core is a series of microservices talking each other through NATS.● Users posts a workflow written in a YAML file to be processed.● A list of actions are executed, one per microservice.

Page 48: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest + NATS● NATS is the central communication system for Ernest.● A workflow composed by multiple actions is executed by the user.● A finite state machine handles the state for the platform.● Publish Subscribe is the main pattern used.

48

● Ernest core is a series of microservices talking each other through NATS.● Users posts a workflow written in a YAML file to be processed.● A list of actions are executed, one per microservice.● Actions are published to NATS ( by a topic ) and microservices are

subscribed to execute them.

Page 49: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : Numbers● Less than 5 developers.

49

● A developer “owns” a component / microservice.

Page 50: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : Numbers● Less than 5 developers.● Started as a prototype in Ruby, moving almost everything to Go.

50

● A developer “owns” a component / microservice.● Fast to develop, less errors, less dependencies.

Page 51: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : Numbers● Less than 5 developers.● Started as a prototype in Ruby, moving almost everything to Go.● Almost 50 microservices.

51

● A developer “owns” a component / microservice.● Fast to develop, less errors, less dependencies.● Unix philosophy, do one thing and do it well done.

Page 52: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : Numbers● Less than 5 developers.● Started as a prototype in Ruby, moving almost everything to Go.● Almost 50 microservices.● Everything specified. Everything tested.

52

● A developer “owns” a component / microservice.● Fast to develop, less errors, less dependencies.● Unix philosophy, do one thing and do it well done.● Using TDD on each part of the platform.

Page 53: NATS in action - A Real time Microservices Architecture handled by NATS

Ernest : Numbers● Less than 5 developers.● Started as a prototype in Ruby, moving almost everything to Go.● Almost 50 microservices.● Everything specified. Everything tested.

53

● A developer “owns” a component / microservice.● Fast to develop, less errors, less dependencies.● Unix philosophy, do one thing and do it well done.● Using TDD on each part of the platform.● Documentation on APIs, documentation on payloads.

Page 54: NATS in action - A Real time Microservices Architecture handled by NATS

Resources● http://microservices.io● https://en.wikipedia.org/wiki/Cloud_computing ● http://nats.io

54

● http://carrenza.com● http://apcera.com

Page 55: NATS in action - A Real time Microservices Architecture handled by NATS

Questions?

55

Page 56: NATS in action - A Real time Microservices Architecture handled by NATS

Thank you!

56