If you are synchronous, you are not resilient....© 2016 Gregor Hohpe Resilient –Fail Operational 5 Failure Accepted Fail-safe Fail-operational Continue (degraded) service

If you are synchronous,

you are not resilient.

Gregor HohpeChief Architect, Allianz SE

www.EnterpriseIntegrationPatterns.com

leanpub.com/37things

@ghohpe

www.EnterpriseIntegrationPatterns.com© 2016 Gregor Hohpe 2

Robust Resilient Antifragile

Resist Disturbance Absorb Disturbance Benefit from Disturbance

www.EnterpriseIntegrationPatterns.com© 2016 Gregor Hohpe

Robust

3

• Maximize MTBF

• Infrastructure-based

• Scale-up

• Mainframe

• Correctness

• Models

• Verification

• Testing

Terminator T-800

Colossus

T-800


Resilient

4

• Minimize MTTR

• Application-based

• Scale-out

• Cluster, High-Availability

• Redundancy

• Monitoring

• Automation

• Fail operational

Deadpool

T-1000


Resilient – Fail Operational

5

Fail-operationalFailure Accepted Fail-safe

Continue (degraded) service

Something will always be

broken

Some Things must still work

in case of failure

Safe state would result in a

(global) shutdown

“Close apps and restart”

Accepted in consumer IT

Not where life is

endangered, IoT

Stop on failure and enter a

safe state

Common practice in

industry, but burns money

Not for process industry,

avionics, power, …

System unusable on

frequent incidents


Antifragile – Not the Opposite of Fragile

6

Fragile Robust Anti-Fragile

Efficiency Redundancy Degeneracy

Theory Phenomenology Heuristics

Sword of Damocles Phoenix Hydra

Model-based Decisioning Heuristic-based Convex heuristic

Academic, Executive Train conductor Artist, writer

Middle Class Minimum-wage Bohemian, aristocracy

Academic Knowledge Expertise Erudition

Rules Principles Virtue

Debt Equity Venture capital

Nassim Taleb:

Antifragile


Antifragile

7

• Zero MTTR

• System-based

• Networked

• Fully Distributed

• Always Failing

• Chaos Monkey / Simian Army

• Learn & ImproveDeadpool

Hydra


The Lines Are More Interesting than the Boxes

8

A

B

C

D

A B

C D


Structure defines Behavior

9

Non-inverting amplifierInverting amplifier

Bill-of Materials


Coupling = Measure of Dependency


Coupling

“How do you make two systems loosely coupled?

Don't connect them.”

David Orchard, BEA


Tightly Coupled Systems

• Make many assumptions about each other

• Require coordinated change

• Well suited for internal communication where you have control over

all sides of the interaction

• Generally more efficient, easier to develop and debug

• More support during development

• Brittle

12

tight looseCoupling


Loosely Coupled Systems

• Make fewer assumptions about each other

• Allows systems to vary independently from each other

• Well suited for “far” communication where you don’t have control

over all systems

• Generally less efficient, more difficult to develop and debug

• More responsibilities for the developer:

• Correlation, Exception handling

• State management, Sequencing

• Robust against change

13

tight looseCoupling


Don’t Control, But Observe!

14

A B

Control

Tight Coupling OK

A B

Observe!

Loose Coupling

Independent

Variability

Coordinated

Variability


Many Facets of Coupling

• Technology Dependency - Migratability

• Location Dependency - Composability

• Data Format Dependency - Integratability

• Semantic Dependency - Integratability

• Temporal Dependency - Resilience

• Interaction Style Dependency - Resilience

15


Asynchronous Messaging Style

Channels have logical, location-independent

addresses

Systems send messages

Placing a message into the Channel is quick (“fire-and-

forget”). The Channel queues messages until the

receiving application is ready to consume.

Simplified

Interaction

Temporal

Decoupling

Location

Decoupling

An "honest" architectural style that does not try to deny

the limitations of the underlying medium.

17

ReceiverSender

Message

Channel

(Queue)


Asynchronous Architectural Style

18

Web SiteNew Order

Order Mgmt

Inventory

Shipping

Confirm

Web SiteNew Order

Order Mgmt

Inventory

Shipping

Ack

Ack

New Order

Synchronous(Call Stack)

Asynchronous(Pipeline, Itinerary)

Idle

Ack

New Order


Asynchronous Messaging Benefits

Asynchrony

• Sender does not have to wait for receiver to process message

• Temporal decoupling

Limit Failure Propagation

• Sender not affected by intermittent failure

• Can be reliable over unreliable transports

Throttling

• Receiver can consume messages at its own pace

• Processing units can be tuned independently

Insertion of intermediaries (Pipes-and-Filters)

• Composability: Transformation, routing etc.

Throughput over latency

• “Wider bridges not faster cars”

19


Asynchronous Messaging Considerations

More left to do for the application

• Correlation, sequencing, state recovery

Bridging multiple architectural styles

• Object-Document mapping

• Sync / async translation

• Message to method translation

Exception Handling

• How to fix something I am not aware of and that might have already

happened (or not)?

Dynamic system management

• Tracking a moving target

20


Messaging Pattern Language

1. Transport messages ApplicationApplication

2. Design messages

3. Route the message to

the proper destination

Application

4. Transform the message

to the required format

5. Produce and consume

messages

ApplicationApplication

Application

6. Manage and Test the

System

21


Messaging Pattern Language

1. Transport messages

3. Route the message to

the proper destination

4. Transform the message

to the required format

5. Produce and consume

messages

6. Manage and Test the

System

Channel Patterns

Routing Patterns

Transformation Patterns

Endpoint Patterns

Management Patterns

2. Design messages Message Patterns

Application

22


Enterprise Integration Patterns

24

185,000 Words

700 pages

65,000 copies

English

Russian

Chinese

Korean


Bridging Architectural Styles

25

• Asynchronous

• Document oriented (coarse)

• Message State

• Concurrent

• Interface Contracts

• Synchronous

• Object-oriented (fine)

• Instance State

• Sequential

• APIs

Application

Application

Gateway

Communication Layer Application Layer

State Containment Behavior Containment


Messaging as First-Class Programming

Constructs

GoLang

• Channels in a CSP model

• Buffered and/or blocking

Erlang

• Active objects / Actor model, isolated with a state machine

• Reliable distributed control systems in the presence of errors

• Not a concurrent functional language, it’s a platform for building

reliable systems.

26

ch <- v // Send v to channel chv := <-ch // Receive from channel ch

www.EnterpriseIntegrationPatterns.com© 2016 Gregor Hohpe 27


Messaging and Conversations

29

Conversation PatternsMessaging Patterns

• Focus on message

• Follows the message

• One-way

• Deals with transport issues

• Focus on participants

• Follows time

• Two- / multi-way

• Deals with state / resources


Subscriptions

30

Publish-Subscribe Channel Subscribe-Notify

PublisherSubscriber

Subscriber

Subscriber

Subscribe

Notify

NotifyNotify

Subscriber Provider

…

How can the sender broadcast an event to

all interested receivers?

How can one participant receive

information from another participant if that

information cannot easily be packaged into

a single message?


Conversation Pattern Language

31

Discovery Basic Conversations Resource Management

Setting Up Participants Application-level

?

!

Initiation Intermediaries Ensuring Consistency


Conversation Pattern Language

32

?

!

Discovery Basic Conversations Resource Management

• Dynamic Discovery

• Advertise Availability

• Consult Directory

• Referral

• Leader Election

• Fire-and-Forget

• Asynchronous Req-Resp

• Req-Resp with Retry

• Polling

• Subscribe-Notify

• Quick Acknowledgment

• Incremental State

• Lease

• Renewal reminder

Initiation Intermediaries Ensuring Consistency

• Three-way Handshake

• Acquire Token First

• Rotate Tokens

• Verify Identity

• User Grants Access

• Proxy

• Relay

• Load Balancer

• Scatter Gather

• Ignore Error

• Compensating Action

• Tentative Operation

• Coordinated Agreement


Conclusions

• Messaging is here to stay

• Asynchrony and decoupling aid resilience

• Language support is welcome

• Respect the new programming model & layer

eaipatterns.com

@ghohpe, #eaipatterns

33

leanpub.com/37things

Documents

If you are synchronous, you are not resilient....© 2016 Gregor Hohpe Resilient –Fail Operational 5 Failure Accepted Fail-safe Fail-operational Continue (degraded) service