Addressing Messaging Challenges Using Spring and RabbitMQ

© 2012 SpringOne 2GX. All rights reserved. Do not distribute without permission.

Gary Russell, Staff Engineer, SpringSource; @gprussell

Addressing Messaging Challenges

•  Modern Application Trends •  Modularize with Spring Integration •  Distribute with AMQP •  Analyze with Hadoop

2

Place Order

Fulfill Order

Manage Inventory

Schedule Delivery

If any component is unavailable within this synchronous flow, all in-flight messages will be affected, regardless of what

component they are currently in -- and since state is maintained across the components, the system can be complex to reset

X X

X X

X X

X X

X

An exception occurred,

please resend

Synchronous Architecture is Brittle

3

In this asynchronous flow: •  all state is kept in the simple messages being passed •  each stateless component interacts only with the broker cloud •  if a component is lost, only the in-flight messages within that

component must be re-delivered •  components can be quickly provisioned on-demand

I have an Order

I can fulfill part of that

Order

I’ll schedule delivery of that

order

I can fulfill part of that

Order Your order will be delivered next Tuesday

The status of your order is

…

Asynchronous Architecture Rules the Web

4

Modern approaches to messaging embrace the unpredictable,

dynamic, and transient nature of the web

Traditional approaches to messaging imply predictable,

static interaction between well-known end-points

Modern Applications Need Modern Messaging

5

Modularize with Spring Integration

•  At the core, an embedded Message Bus –  Inspired by Gregor Hohpe and Bobby Woolf's Enterprise

Integration Patterns (2003) –  Runs within any Spring ApplicationContext –  All components are Spring-managed objects

•  Also, an Application Integration Framework –  Connects to other systems via adapters –  Unidirectional Channel Adapters –  Bidirectional Messaging Gateways

What is Spring Integration?

7

cafe

orders drinks

coldDrinks

hotDrinks

preparedDrinks deliveries

CafeDemoApp

Spring Integration Cafe

8

Café

http

Spring Integration Café – Simple HTTP Deployment

9

Distribute with AMQP

10

A  Protocol,  not  an  API  •  A  defined  set  of  messaging  

capabili2es  called  the  AMQ  model  

•  A  network  wire-­‐level  protocol,  AMQP   On  commodity  hardware  

•  10-­‐25  thousand  messages  per  second  is  rou@ne  *  

•  The  NIC  is  usually  the  boFleneck  

 *  Non-­‐persistent  messages  

Why AMQP?

11

AMQP  security  •  Brokers  support  separate  

virtual  hosts  •  Three  levels  of  permissions  •  AMQP  over  SSL  is  supported  

Designed  to  scale  horizontally  •  Dozens  of  clustered  brokers  

are  common  

Why AMQP?

12

new.order

queue

M3 M1 M2 new.order binding

new.order routing key

AMQP Architecture

13

café NA deliveries

queue

café deliveries

queue

M3 M1 M2

M3 M1 M2

café WW deliveries

queue

M3 M1 M2

AMQP Architecture

14

hot_drinks

queue

cold_drinks

queue

1 2

2

3

1

drink.cold

Message Routing Keys:

1.  drink.hot

2.  drink.cold

3.  drink.warm

all_drinks

queue

AMQP Architecture

15

Spring-AMQP

•  AMQP core abstraction plus RabbitMQ implementation (built on rabbit client libraries)

•  Higher level patterns for clients: admin, producers and consumers –  RabbitAdmin – declare exchanges, queues, bindings –  RabbitTemplate – convenience methods for send and receive –  MessageListenerContainer – POJO message handler, asynchronous

•  Spring Integration support

http://www.springsource.org/spring-amqp

16

Spring-AMQP – What’s New (1.1.x)

•  Support for HA Queues –  Broker Failover

•  Publisher Confirms –  Callback when the message is secured in the broker

•  Publisher Returns –  Callback if message cannot be deposited in at least one queue

•  Explicit Reply Queue –  Avoids creation of temporary queue for request/reply scenarios

•  Dead Letter Exchange/Queue Support

17

Demo – Distributing the Spring Integration Café Demo with AMQP

Spring Integration Café – Distributed Deployment with AMQP

19

amqp

Café Ops Café Storefront

http

amqp

Spring Integration Café – Distributed Deployment with Hadoop

CRM Data Marts ERP

hdfs

map reduce

unstructured event data

structured information EDW

20

amqp

Café Ops Café Storefront

http

amqp