HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect

© 2016 MapR Technologies 1© 2016 MapR Technologies 1MapR Confidential

© 2016 MapR Technologies

Fabian WilckensEMEA Solutions Architect

THINK DIFFERENTLYStream-based Microservice Architecture for Next-Generation Applications





The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.

- Martin Fowler

MICROSERVICE ARCHITECTURAL STYLESINGLE APPLICATION

SUITE OF SMALL SERVICESCOMMUNICATING WITH LIGHTWEIGHT MECHANISMS

BUILT AROUND BUSINESS CAPABILITIESFULLY AUTOMATED

DEPLOYMENTBARE MINIMUM OF CENTRALIZED MANAGEMENTWRITTEN IN DIFFERENT PROGRAMMING LANGUAGES

DIFFERENT DATA STORAGE TECHNOLOGIES


WAIT A SECOND

SOA?SERVICE ORIENTED ARCHITECTURE

IS THERE A DIFFERENCE?


Integrates different applications as a set of services

Architect a single application as a set of services

APP 1 APP 2

APP 4 APP 5

ENTERPRISE SERVICE BUS

APP 2

LOGISTICSSERVICE

CUSTOMERSERVICE

BILLINGSERVICE

USER INTERFACE SERVICE

MICROSERVICE-STYLE

CLASSIC SOA-STYLE


A few years back …


App Server

WAR/EAR

Backend

Logistics

Ordering

User Interface

Easy to develop? Easy to deploy? Easy to scale?


CHALLENGES AHEAD


LARGE CODE & BIG DATAHARD TO UNDERSTAND & CHANGE

OVERLOADED IDE

OVERLOADED WEB CONTAINERS

DEVELOPMENT SLOWS DOWN

EVEN SMALL APPLICATION CHANGES REQUIRE FULL REBUILD, TEST AND DEPLOYMENT

IMPACT ANALYSIS IS A HUGE EFFORT AND TAKES TIME

DIFFICULT TO FREQUENTLY CHANGE & DEPLOY


OTHER CHALLENGESLONG-TERM COMMITMENT TO TECHNOLOGY STACK

RE-WRITE = COMPLETE RE-WRITE

MODULE DEPENDENCIES FORCE RE-WRITE

FAILURE IN MONOLITHIC APPLICATIONS BRINGS IT DOWN


HORIZONTAL SCALING

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface


VERTICAL SCALING

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App Server

WAR/EAR

Backend

Logistics

Ordering

User Interface


DATA SCALING

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface



MICROSERVICES SCALING

Container

Ordering

Container

Ordering

Container

Ordering

Ordering Database

Container

User Interface

Container

Logistics

LogisticsDatabase

Container

Backend

BackendDatabase


MICROSERVICES TEAM SCALING

Container

Ordering

Container

Logistics

Container

Backend


LET’S TALK BENEFITS (PROS)ONLY MAKE APIS TALK TO EACH OTHER

ADD MODULES BASED UPON BUSINESS REQUIREMENTS

CHANGE MODULES INDEPENDENTLY

DIFFERENT TEAMS WORK ON THEIR OWN PROBLEMS

FASTER TIME-TO-MARKET FOR NEW SERVICES

CAN STILL INTEGRATE WITH SOA

SERVICES/TEAMS SCALE INDEPENDENTLY

…




The Challenge with Data Pipelines

Filtering & Aggregation

Alerting Processing


Streams Simplify Data Movement

Filtering & Aggregation

Alerting Processing

StreamsReliable publish/subscribe transport between sources and destinations.


Legacy Systems: Message QueuesIBM MQ, TIBCO, RabbitMQ

OrdersFront End

Order Processing

Order Processing

Usage/Requirements●Tight, transactional conversations between systems

●1:1 or Few:Few

●Low data rates

●Mission-critical delivery

Approach

●Queue-oriented design

●Each message replicated to N output queues

●Messages popped when read

●Scale-up, master/slave

Doesn’t Do

●High message rates (>100K/s)

●Slow consumers

●Queue replay/rewind


Evolving “big data” Event Streams: Distributed LogsKafka, Hydra, DistributedLog

Usage/Requirements●High throughput data transferred from decoupled systems

●Many->1

●1->Many

●Different speeds

Approach

●Log-oriented design

●Write messages to log files

●Consumers pull messages at their own pace

●Scale-out

Doesn’t Do

●Global applications

●Message persistence

●Integrated analytics (data movement required)

DB_Changes

Stream Processing

Search/EDW

DB


Introducing MapR StreamsGlobal Publish/Subscribe Event Streaming

Producers Publish Billions of

messages/sec to a topic

ConsumersReliable delivery to all

consumers. Immediately

GlobalTie together geo-dispersed

clusters. Worldwide


MapR Converged Data Platform

Tenant 2 Tenant 3

Tenant 1


Container

Ordering

Container

User Interface

Container

Logistics

Container

Backend


Global Applications

Container

Ordering

Container

Backend

Container

Logistics

Container

User Interface

Container

User Interface

Container

User Interface


Are Microservices here to stay?

We don’t know …Monolithic Apps are dead


Content & Pictures• http://enterprise.vnews.com/wp-content/uploads/2015/07/EP-SeanSmithCover-ns-vn-0728151.jpeg• http://www.rodmatthews.com.au/_assets/images/blog/Positional%20Images/Complexity%20Small%20

Version.jpg• http://www.coliquio-insights.de/wp-content/uploads/2015/02/5141328136_16d8f73ee7_b.jpg

Technology

HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect