4finance microservices 2015.06 Prague JUG

@mgrzejszczak

Microservices at 4financeIT

08/06/2015, Prague

@mgrzejszczak

About meSoftware Architect atAuthor of "Mockito Instant", "Mockito Cookbook" booksCo-author of the Groovy core’s @Builder AST Co-founder of the Warsaw Groovy User GroupCo-author of "micro-infra-spring" libCo-author of “spring-cloud-zookeeper” and “spring-cloud-sleuth”

Twitter: @MGrzejszczakBlog: http://toomuchcoding.blogspot.comHomepage: http://marcin.grzejszczak.pl

http://toomuchcoding.blogspot.com

http://marcin.grzejszczak.pl

@mgrzejszczak

Agendashort intro to microserviceshow we did it in 4financeITmicroservice challenges

@mgrzejszczak


@mgrzejszczak

Conway’s Law

Conway, Melvin E. (April 1968), How do Committees Invent?, Datamation 14 (5): 28–31, retrieved 2009-04-05

Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations

— M. Conway

@mgrzejszczak

A single codebase

Conway’s Law in practice

@mgrzejszczak

A single codebase

Conway’s Law in practice

@mgrzejszczak

Conway’s Law in practiceConcept:

one team

two countries

one codebase

@mgrzejszczak

Conway’s Law in practiceReality:

two teams

two countries

one codebase

@mgrzejszczak

Conway’s Law in practiceEffect:

two different solutions

solving same stuff

one codebase

@mgrzejszczak

Conway’s Law - siloed teams

extract from http://martinfowler.com/articles/microservices.html

@mgrzejszczak

Conway’s Law - cross functional teams


@mgrzejszczak

Business flow

AccountingBack officeFront office

@mgrzejszczak

Common problematic code flow

Dto Service

Impl Entity

Accounting Accounting

Accounting Accounting

Front office Front office

Front office Front office

Back office Back office

Back office Back office

monolith

many programmers

big organization

@mgrzejszczak

Common problematic code flow

Looks familiar?

http://www.foodnetwork.com/topics/spaghetti-recipes.html

@mgrzejszczak

Code flow

REST REST

Autonomous

Business oriented

PolyglotLeightweight

Front officebounded context

JARS

Back officebounded context

JARS

Accountingbounded context

JARS

@mgrzejszczak

Small and lightweight

single application as a suite of small applications

each running in its own process

communicating with lightweight mechanisms


@mgrzejszczak

Business oriented

http://www.ops.fhwa.dot.gov/publications/ics_guide/

@mgrzejszczak

Business orientedbuilt around business capabilities

products not projects

cross business not technical boundaries


@mgrzejszczak

Polyglot

@mgrzejszczak

Autonomous

@mgrzejszczak

Autonomous

independently deployable / upgradeable

deployed via fully automated deployment machinery


@mgrzejszczak

Autonomous

bare minimum of centralized management (do not share your data, minimum set of standards)

embrace failure (resilience - recover fast)


@mgrzejszczak

Deployment - provisoning (Ansible)

http://labs.qandidate.com/blog/2013/11/21/installing-a-lamp-server-with-ansible-playbooks-and-roles/

@mgrzejszczak

Deployment - pipeline (Jenkins)

@mgrzejszczak

Deployment - job scheduling (Rundeck)

http://rundeck.org/

@mgrzejszczak

Fail fast

@mgrzejszczak


@mgrzejszczak

Explain one simple truth

@mgrzejszczak

Microservices are complex!who can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose and view my metrics

@mgrzejszczak

@mgrzejszczak

So we wrote a framework...

@mgrzejszczak

… having management on our side

@mgrzejszczak

Microservices are complex!who can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose and view my metrics

@mgrzejszczak

The big picture

@mgrzejszczak

The big picture

HEY GUYS LET’S TELL ZOOKEEPER WHERE WE ARE

@mgrzejszczak

The big picture

MY IP: …MY PORT: ….

@mgrzejszczak

The big pictureI WOULD LOVE TO TALK TO SERVICE X

@mgrzejszczak

The big pictureWHERE IS SERVICE X ? GIVE ME ANY INSTANCE

@mgrzejszczak

The big pictureTHERE YOU GO!IP: ….PORT: ….

@mgrzejszczak

The big pictureGREAT NOW I CAN SEND HIM A REQUEST

@mgrzejszczak

Microservice descriptor{

"pl": {

"this": "some/path/my-name",

"dependencies": {

"an-alias": {

"path": "some/path/his-name"

}

}

}

@mgrzejszczak


"pl": {


"dependencies": {

"an-alias": {


}

}

}

REALM

@mgrzejszczak


"pl": {


"dependencies": {

"an-alias": {


}

}

}

MY NAME IN ZOOKEEPER

@mgrzejszczak


"pl": {


"dependencies": {

"an-alias": {


}

}

}

HIS NAME IN ZOOKEEPER

@mgrzejszczak

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

@mgrzejszczak

How can I communicate easily?serviceRestClient.forService('an-alias')

.post()

.onUrl('/api/whatever/123')

.body('''{"some":"json"}''')

.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

@mgrzejszczak

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhere do I take my properties fromwhat’s my APIhow to be part of a bigger system: logshow do I expose and view my metrics

@mgrzejszczak

Where do I take my props from?

@mgrzejszczak


@mgrzejszczak


@mgrzejszczak


@mgrzejszczak


@mgrzejszczak

your microservice publishes its API by itself!

What’s my API?

@mgrzejszczak

What’s my API?

@mgrzejszczak

What’s my API?

RESTCONTROLLERS

@mgrzejszczak

What’s my API?

METHOD TYPES

@mgrzejszczak

What’s my API?

METHOD URLS METHOD NAMES

@mgrzejszczak

What’s my API? - Try me out!

@mgrzejszczak

What’s my API? - Try me out!

ENTER PARAMETERS

SEND A REQUEST

@mgrzejszczak


@mgrzejszczak

How to be part of a bigger system: logs?You’re not going to grep your logs from 10 servers, right?

@mgrzejszczak

Correlation ID

a header value in a request

in each microservice this header is the same for the same business action

you can see a flow of messages between microservices in a chronological order

@mgrzejszczak

How to be part of a bigger system: logs?

FIRST SERVICE

REQUEST

No correlationId

CorrelationId set to X


RESPONSE

SECOND SERVICE

REQUEST



RESPONSE

ANOTHER SERVICE

REQUEST

CorrelationId set to XCorrelationId

set to X

RESPONSECorrelationId set to X

YET ANOTHER SERVICE

REQUEST

CorrelationId set to XCorrelationId

set to X

RESPONSE

@mgrzejszczak

How to be part of a bigger system: logs?Log example:

2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |

c.o.c.v.w.a.AbstractBackofficeApplication |

Initializing new backoffice application

@mgrzejszczak


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



TIMESTAMP IN ISO8601 FORMAT

@mgrzejszczak


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



LOGGING LEVEL

@mgrzejszczak


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



CORRELATION ID

@mgrzejszczak


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



THREAD NAME

@mgrzejszczak


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



CLASS FULLY QUALIFIED NAME (SHORTENED)

@mgrzejszczak


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |


Initializing new backoffice applicationLOG MESSAGE

@mgrzejszczak


@mgrzejszczak


SAME CORRELATION ID

@mgrzejszczak


DIFFERENT MICROSERVICES

@mgrzejszczak

View the logs in Kibana

@mgrzejszczak


@mgrzejszczak

How do I expose my metrics?just have to update your metrics:

// treat this code more of a pseudocode ;)Counter transferBalanceRequests = metricRegistry.counter(‘transfer.requests.balance’)

void doSomethingUponTransferBalanceRequest() {

// do sth with the request for transfer balance…

// and update the counter metrictransferBalanceRequests.inc()

}

@mgrzejszczak

How do I view my metrics?

@mgrzejszczak

How do I view my metrics?

@mgrzejszczak

Ok, so how do I start?But, but, but… it sounds so complicated!

@mgrzejszczak

Perform 3 steps...You just have to:

add ONE dependency - 4finance micro-infra-spring to your project

add ONE annotation - @EnableMicroservice

create ONE file - microservice.json

https://github.com/4finance/micro-infra-spring

@mgrzejszczak

One dependency with Spring Boot

And for Spring-Boot….

@mgrzejszczak

It’s even easier with Spring Boot!You just have to:

add ONE dependency - 4finance micro-infra-spring-boot to your project

create ONE file - microservice.json

https://github.com/4finance/micro-infra-spring

@mgrzejszczak

Ok, so how do I start?

Question:what should I do to start working on a new microservice?

@mgrzejszczak


Answer:

Clone a repo

@mgrzejszczak


Without UI:https://github.com/4finance/boot-microservice

With UI:https://github.com/4finance/boot-microservice (branch boot-microservice-gui)

https://github.com/4finance/boot-microservice

https://github.com/4finance/boot-microservice

https://github.com/4finance/boot-microservice-gui

@mgrzejszczak

Ok, so how do I start?those repos:

contain completely set up microservice

are written in Groovy and built with Gradle

are JDK8 compliant

contain an example of a business scenario

@mgrzejszczak

Where are we?

Majority of microservice related features are there…

We are moving our stuff to Spring-Cloud

We have automated / standardized plenty of things already

NOW

Move more stuff to Spring-Cloud

Leave 4financeIT related stuff in micro-infra-spring

Automate more

Standardize more stuff

NEXT

New templates for other JVM languages

Scale dynamically our production servers (Mesos, Marathon etc.)

Sky is the limit...

FUTURE

@mgrzejszczak

Working closely with Pivotal

@mgrzejszczak

Working closely with Pivotal

@mgrzejszczak

3 steps to eternal fame1. Get account at Github

2. Pick an issue, fix it and create a pull request

3. Once merged profit from eternal fame

@mgrzejszczak

The setup - overview

@mgrzejszczak


@mgrzejszczak

Code reusedo not abstract everything

sometimes copy paste gives you code decoupling

no - copy paste is not a solution to all problems ;)

@mgrzejszczak

Too many technology stackspick a right tool for the job but don’t exaggerate

do you really want to support a microservice written in Brainfuck or Whitespace?

@mgrzejszczak

Managing dozens of servicesmonitoring, monitoring, monitoring!

Zabbix, Nagios, Kibana, Graphite etc.

do not write nanoservices - who will support it?

@mgrzejszczak

Microservice envythose guys wrote a microservice and I’m diving in the abyss of a monolith

I will write a microservice per feature!

one bounded context - 100 microservices with duplicated data - WRONG!

@mgrzejszczak

Global changes (big features)you’ll have to work piece by piece and communicate

you can use feature toggling and API versioning

easier A/B testing - load balancing via Service Discovery

@mgrzejszczak

Testingcontract changes - how to test that?

how do I know what’s the API of other services?

Consumer Driven Contracts to the rescue!Accurest - https://github.com/Codearte/accurestPact - https://github.com/DiUS/pact-jvm

@mgrzejszczak

Management issueshave to invest time and effort to build foundations

have to invest in infrastructure and devops

feature delivery pace will decrease for some time

@mgrzejszczak

Questions?

Documents

4finance microservices 2015.06 Prague JUG