4finance microservice pilot in Poland

Microservices

PILOT

Agendaproblems being solvedhow to use it

Problems of a microservice

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

Who can I talk to?solved by:

4finance micro-infra-spring-base

Apache Zookeeper

Who can I talk to?you just have to:

create a microservice metadata file (i.e. microservice.json)

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

src/main/resources/microservice.json

Who can I talk to?you just have to:

add 4finance micro-infra-spring-base dependency to your project

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-base:0.5.2'

}

Who can I talk to?you just have to:

add service discovery module

@Configuration

@Import([ServiceDiscoveryConfiguration])

class MyConfiguration {

}

Microservice metadata file

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

src/main/resources/microservice.json

Microservice metadata file

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

Microservice context (realm)

Microservice metadata file

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

name of services I talk with

Microservice metadata file

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

paths to services I talk within context (realm)

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

Where are my collaborators?{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

COLLABORATOR’S PATH IN ZOOKEEPER

pl/com/ofg/payments/pl/bluecash/bluecash-vivus-pl

Where are my collaborators?

PLpl/com/ofg/loans/payment-processor

pl/com/ofg/loans/risk-analyzer

LTlt/com/ofg/loans/payment-processor

lt/com/ofg/loans/risk-analyzer

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

How can I contact others?if you want to call your collaborator (e.g. ‘bluecash-pl’) just:

add micro-infra-spring-base dependency to your project

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-base:0.5.2'

}

How can I contact others?if you want to call your collaborator (e.g. ‘bluecash-pl’) just:

add Service discovery and Abstraction over RestTemplate modules

@Configuration

@Import([ServiceDiscoveryConfiguration, ServiceRestClientConfiguration])

class MyConfiguration {

}

How can I contact others?if you want to call your collaborator (e.g. ‘bluecash-pl’) just:

use 4Finance ServiceRestClient bean

How can I contact others?serviceRestClient.forService('bluecash-pl')

.post()

.onUrl('/api/whatever/123')

.body('''{"some":"json"}''')

.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

COLLABORATOR’S ALIAS FROM MICROSERVICE.

JSON

How can I contact others?serviceRestClient.forService('bluecash-pl')

.post()

.onUrl('/api/whatever/123')

.body('''{"some":"json"}''')

.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

HTTP METHOD

How can I contact others?serviceRestClient.forService('bluecash-pl')

.post()

.onUrl('/api/whatever/123')

.body('''{"some":"json"}''')

.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

URL ON COLLABORATOR TO CALL

How can I contact others?serviceRestClient.forService('bluecash-pl')

.post()

.onUrl('/api/whatever/123')

.body('''{"some":"json"}''')

.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

BODY TO SEND

How can I contact others?serviceRestClient.forService('bluecash-pl')

.post()

.onUrl('/api/whatever/123')

.body('''{"some":"json"}''')

.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

HEADERS TO SEND

How can I contact others?serviceRestClient.forService('bluecash-pl')

.post()

.onUrl('/api/whatever/123')

.body('''{"some":"json"}''')

.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

RETURN A RESPONSE OF A GIVEN TYPE

How can I contact others?serviceRestClient.forService('bluecash-pl')

.post()

.onUrl('/api/whatever/123')

.body('''{"some":"json"}''')

.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

How can others contact me?solved by:

4finance micro-infra-spring-base

Apache Zookeeper

Netflix Exhibitor - UI for Zookeeper

How can others contact me?you just have to:

create a microservice metadata file (i.e. microservice.json)

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

src/main/resources/microservice.json

How can others contact me?you just have to:

add 4finance micro-infra-spring-base dependency to your project

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-base:0.5.2'

}

How can others contact me?you just have to:

add service discovery module

@Configuration

@Import([ServiceDiscoveryConfiguration])

class MyConfiguration {

}

How can others contact me?

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

How can others contact me?

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

Microservice context (realm)

How can others contact me?

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

my path in context (realm)

How can others contact me?{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

MY PATH

pl/com/ofg/loans/pl/backoffice-vivus-pl

How can others contact me?when microservice boots up it registers itself in Zookeeper under specified path

you can register multiple instances under same path

Zookeeper supports load balancing (Round Robin, Random and Sticky)

How can others contact me?

/pl/com/ofg/loans/pl/backoffice-vivus-pl

instance id (you can have multiple ones)

How can others contact me?

Data as String:{ "name": "com/ofg/loans/pl/backoffice-vivus-pl", "id": "15628f76-753e-4a25-83de-de132293a58d", "address": "192.168.0.1", "port": 8080, "sslPort": null, "payload": null, "registrationTimeUTC": 1410343209917, "serviceType": "DYNAMIC", "uriSpec": { ... }}

Instance entry in Zookeeper:

How can others contact me?

Data as String:{ "name": "com/ofg/loans/pl/backoffice-vivus-pl", "id": "15628f76-753e-4a25-83de-de132293a58d", "address": "192.168.0.1", "port": 8080, "sslPort": null, "payload": null, "registrationTimeUTC": 1410343209917, "serviceType": "DYNAMIC", "uriSpec": { ... }}

Instance entry in Zookeeper:THE PATH

WHERE MICROSERVICE

REGISTERS ITSELF IN GIVEN

REALM

How can others contact me?

Data as String:{ "name": "com/ofg/loans/pl/backoffice-vivus-pl", "id": "15628f76-753e-4a25-83de-de132293a58d", "address": "192.168.0.1", "port": 8080, "sslPort": null, "payload": null, "registrationTimeUTC": 1410343209917, "serviceType": "DYNAMIC", "uriSpec": { ... }}

Instance entry in Zookeeper:

MICROSERVICE REGISTERS

ITSELF IN ZOOKEPER

WITH ADDRESS AND PORT

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

What’s my API?solved by:

4finance micro-infra-spring-swagger

Wordnik Swagger

4finance micro-infra-spring-swagger-ui

Wordnik Swagger-UI

What’s my API?you just have to add dependencies to your project:

micro-infra-spring-swaggermicro-infra-spring-swagger-ui

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-swagger:0.5.2'

compile 'com.ofg:micro-infra-spring-swagger-ui:0.5.2'

}

What’s my API?from micro-infra-spring-swagger:

add Swagger module to Spring’s context

@Configuration

@Import([SwaggerConfiguration])

class MyConfiguration {

}

thanks to micro-infra-spring-swagger-ui:

swagger-ui will be present under /src/main/resources/static/swagger/

What’s my API?

and now your microservice publishes its API by itself!

What’s my API?

What’s my API?

What’s my API?

RESTCONTROLLERS

What’s my API?

METHOD TYPES

What’s my API?

METHOD URLS METHOD NAMES

What’s my API? - Try me out!

What’s my API? - Try me out!

ENTER PARAMETERS

SEND A REQUEST

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

How to be part of a bigger system: logs?Question:

How many lines of logs were produced on production by PL Cola during 1 hour?

How to be part of a bigger system: logs?Answer:

73 000 lines of logs

How to be part of a bigger system: logs?

You are not going to grep that, are you?

How to be part of a bigger system: logs?solved by:

logback log pattern

4finance micro-infra-spring-base

How to be part of a bigger system: logs?solved by:

logback log pattern

4finance micro-infra-spring-base

SO THAT WE HAVE A STANDARDISED WAY OF

LOGGING

How to be part of a bigger system: logs?solved by:

logback log pattern

4finance micro-infra-spring-base SO THAT CORRELATION ID IS ALWAYS SET FOR YOU

How to be part of a bigger system: logs?correlation ID:

a header value in a requestin each microservice this header is the same for the same business actionyou can see a flow of messages between microservices in a chronological order

How to be part of a bigger system: logs?

COLA

REQUEST

No correlationId

CorrelationId set to X

CorrelationId set to X

RESPONSE

FIRST SERVICE

REQUEST

CorrelationId set to X

CorrelationId set to X

RESPONSE

ANOTHER SERVICE

REQUEST

CorrelationId set to XCorrelationId

set to X

RESPONSECorrelationId set to X

YET ANOTHER SERVICE

REQUEST

CorrelationId set to XCorrelationId

set to X

RESPONSE

How to be part of a bigger system: logs?you just have to:

add 4finance micro-infra-spring-base dependency to your project

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-base:0.5.2'

}

How to be part of a bigger system: logs?you just have to:

add CorrelationId module to Spring’s context

@Configuration

@Import([CorrelationIdConfiguration])

class MyConfiguration {

}

How to be part of a bigger system: logs?Application logs in a specified format:

"%d{yyyy-MM-dd HH:mm:ss.SSSZ, Europe/Warsaw} |

%-5level |

%X{correlationId} |

%thread |

%logger{1} |

%m%n"

How to be part of a bigger system: logs?Log example:

2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |

c.o.c.v.w.a.AbstractBackofficeApplication |

Initializing new backoffice application

How to be part of a bigger system: logs?Log example:

2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |

c.o.c.v.w.a.AbstractBackofficeApplication |

Initializing new backoffice application

TIMESTAMP IN ISO8601 FORMAT

How to be part of a bigger system: logs?Log example:

2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |

c.o.c.v.w.a.AbstractBackofficeApplication |

Initializing new backoffice application

LOGGING LEVEL

How to be part of a bigger system: logs?Log example:

2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |

c.o.c.v.w.a.AbstractBackofficeApplication |

Initializing new backoffice application

CORRELATION ID

How to be part of a bigger system: logs?Log example:

2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |

c.o.c.v.w.a.AbstractBackofficeApplication |

Initializing new backoffice application

THREAD NAME

How to be part of a bigger system: logs?Log example:

2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |

c.o.c.v.w.a.AbstractBackofficeApplication |

Initializing new backoffice application

CLASS FULLY QUALIFIED NAME (SHORTENED)

How to be part of a bigger system: logs?Log example:

2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |

c.o.c.v.w.a.AbstractBackofficeApplication |

Initializing new backoffice applicationLOG MESSAGE

How to be part of a bigger system: logs?

How to be part of a bigger system: logs?

SAME CORRELATION ID

How to be part of a bigger system: logs?

DIFFERENT MICROSERVICES

How to be part of a bigger system: logs?you can see collected logs from all services in kibana

How to be part of a bigger system: logs?

How to be part of a bigger system: logs?with kibana you can:

query your logs

create dashboards with filtered data

collect logs from different services

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

How do I expose my metrics?solved by:

4finance micro-infra-spring-base

Dropwizard Metrics

How do I expose my metrics?you just have to:

add 4finance micro-infra-spring-base dependency to your project

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-base:0.5.2'

}

How do I expose my metrics?you just have to:

add metrics publishing module to Spring’s context

@Configuration

@Import([MetricsConfiguration])

class MyConfiguration {

}

How do I expose my metrics?we have a template of metric path in Graphite:

apps.(env).(country).(service-name).(dot-separated-metric)

apps.test.pl.bluecash-adapter.transfer.requests.balance.count

“Bluecash adapter’s” count of “transfer” type requests

How do I expose my metrics?you just have to provide (as a property):

# Environment to which you deploy application ('test', 'prod', etc.) - default value: ‘test’

metrics.path.environment=test

# Country for which you deploy your app - default value: ‘pl’

metrics.path.country=pl

# Your app name - default value: ‘service-name’

metrics.path.app=bluecash-adapter

How do I expose my metrics?having properties set you just have to update your metrics:

// treat this code more of a pseudocode ;)Counter transferBalanceRequests = metricRegistry.counter(‘transfer.requests.balance’)

void doSomethingUponTransferBalanceRequest() {

// do sth with the request for transfer balance…

// and update the counter metrictransferBalanceRequests.inc()

}

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

How do I view my metrics?solved by:

Graphite

Grafana - UI for Graphite

JMX - jconsole, jvisualvm, etc.

How do I view my metrics?you can view your metrics in Graphite

you can create dashboards in Grafana

your applictaion metrics are exposed via JMX

How do I view my metrics?

How do I view my metrics?

Ok, so how do I start?

But, but, but… it sounds so complicated!

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-swagger-ui:0.5.2'

}

Don’t want to add separate modules?@Configuration@Import([ServiceDiscoveryConfiguration])

class MyConfiguration {

}

@Configuration

@Import([SwaggerConfiguration])

class MyConfiguration {

}

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-swagger:0.5.2'

}

@Configuration

@Import([MetricsConfiguration])

class MyConfiguration {

}

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-inf

ra-spring-base:0.5.2'

}

@Configuration

@Import([ServiceDiscoveryConfiguration])

class MyConfiguration {

}

Don’t want to add separate modules?

Don’t want to add separate modules?

Good! We’ll do that for you!!You just have to:

add ONE dependency - 4finance micro-infra-spring to your project

import ONE configuration - WebAppConfiguration

create ONE file - microservice.json

One dependency - micro-infra-spring

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring:0.5.2'

}

One config - WebAppConfiguration@Configuration

@Import(WebAppConfiguration)

class MyConfiguration {

}

If you want you can import only needed pieces or override configuration defaults

One file - microservice.json

{

"pl": {

"this": "com/ofg/your/microservice",

"dependencies": {

"collaborator": "com/ofg/microservice/collaborator"

}

}

}

Ok, so how do I start?

Question:what should I do to start working on a new microservice?

Ok, so how do I start?

Answer:

Clone a repo

Ok, so how do I start?

Without UI:https://github.com/4finance/boot-microservice

With UI:https://github.com/4finance/boot-microservice-gui

Ok, so how do I start?those repos:

contain completely set up microservice

are written in Groovy

are built with Gradle (set up for 4finance)

are JDK8 compliant

contain an example of a business scenario

Ok, so how do I start?our libs:

have Javadocs

Ok, so how do I start?our libs:

have ReadMe’s

Where are we?what we have managed to achieve:

Microservice setup (correlationId setting, metrics support, rest template abstraction etc.)

CorrelationId setting for Spring Reactor

Where are we?Libs on production in:

COLA-VIVUS

WEB-VIVUS

BLUECASH-ADAPTER

MYMOID-ADAPTER

WEB-ZAPLOCOLA-ZAPLOBROKERS

Where are we?

Microservice setup (correlationId setting, metrics support, rest template abstraction etc.)

CorrelationId setting for Spring Reactor

CorrelationId setting for Camel

NOW

Fix bugs

Consumer driven contracts (4finance StubRunner)

Circuit breaking (Netflix Hystrix)

Store properties in Zookeeper (Netflix Archaius)

Retry policies for RestTemplate

Resolve issues with circular obligatory dependencies

NEXT

Investigate Riemann and Suro as event servers / data pipelines

New templates for other JVM languages

Sky is the limit

FUTURE

Want to contribute?

Want to contribute? ISSUES!

3 steps to eternal fame1. Ask any of the existing 4Finance organization members

to add you to that organization

2. Pick an issue, fix it and create a pull request

3. Once merged profit from eternal fame

Questions?