EMEA 2 Enterprise Integration Erik Doernenburg ThoughtWorks, Inc. edoernen@thoughtworks.com

EMEAEMEA

Enterprise IntegrationEnterprise Integration

Erik DoernenburgThoughtWorks, Inc.edoernen@thoughtworks.com

EMEAEMEA33 © Copyright ThoughtWorks, Inc.® 2005© Copyright ThoughtWorks, Inc.® 2005

AgendaAgenda

Challenges in enterprise application development

Design patterns

Enterprise Integration patterns

Interop technologies

Conclusion & Resources

The challengeThe challenge

Enterprises depend on a growing number of IT

systems

The systems must provide an integrated solution

for the enterprise

The systems must interoperate with each other

Architectural trends and “waves” of technologies

Changing business needs and requirements

Existing Systems

New System

Finance Operator ApplicationFinance Operator Application

Workflow items are stored on Tibco RV/CM queue Tibco Integration Manager

organises workflow handles imports from document recognition handles interaction with existing ERP systems

Reference data imported using linked servers

fax/telex

mail documentrecognition

app web

ERP ref. data

New application

Existing

Portfolio Management ToolPortfolio Management Tool

Front-end uses Office 2003 code-behind technology

communicates with server using WebSphere MQ queues

Message format defined with XSD schema

Application uses existing services

accesses analytics library with COM bridge

accesses stored deals through deal manager service

accesses service bus through messaging abstraction library

analytics

library

Existing

dealmanage

Existing

messaging library

technology

adaptor

Excelfront-end

application

EMEAEMEA

Design PatternsDesign Patterns

Why design patterns?Why design patterns?

Code reuse remains difficult

but…

Knowledge reuse can be very valuable

Patterns encapsulate knowledge of successful designs

Using technology successfullyUsing technology successfully

Syntax Basic language mechanism A necessity but not a guarantee

Constructs Classes, Interfaces, Inheritance, Polymorphism, etc. Helpful but not sufficient for good design

Principles Separation of concerns, Dependency Injection, etc. Steer us towards a better solution

Patterns Model-View-Controller, Observer, Decorator Concrete design guidance

PatternsPatterns

“Mind sized” chunk of information (Ward Cunningham)

Shows a good solution to a common problem within a

specific context

Observed from actual experience

Has a distinct name

Not copy-paste

ThoughtWorks collaborates with Microsoft to document

design patterns for the .NET platform

Enterprise Integration PatternsEnterprise Integration Patterns

Gregor Hohpe defined a visual pattern language describing message-based enterprise integration solutions

Pattern language comprises 65 patterns in 6 categories

Application

MessageChannel

Router TranslatorEndpoint Endpoint

MonitoringMessagingMessagingEndpointsEndpoints

MessagingMessagingChannelsChannels

MessageMessageConstructioConstructio

MessageMessageRoutingRouting

MessageMessageTransformationTransformation

SystemSystemManagemenManagemen

Pattern: Pattern: Request-ResponseRequest-Response

Service Consumer and Provider (similar to RPC) Channels are unidirectional Two asynchronous point-to-point channels Separate request and response messages

Request Channel

Response

Request

Reply Channel

ProviderConsumer

Multiple consumersMultiple consumers

Each consumer has its own reply queue But how does the provider send the response?

Could send to all consumers (very inefficient) Hard code (violates principle of context-free

service)

Responses

Requests

RequestsRequest Channel

Reply Channel 1

Reply Channel 2

ProviderConsumer 1

Consumer 2

Pattern: Pattern: Return AddressReturn Address

Consumer specifies Return Address (the reply channel) in the request message

Service provider sends response message to specified channel

Responses

ReplyChannel 1

ReplyChannel 2

Request Channel

Reply Channel 1

Reply Channel 2

ProviderConsumer 1

Consumer 2

Load-balanced service providersLoad-balanced service providers

Request message can be handled by multiple providers Point-to-point channel supports competing services Only one service receives each request message But what if the response messages are out of order?

Request Channel

Reply Channel

Provider 1

Provider 2

Consumer

Pattern: Pattern: Correlation IdentifierCorrelation Identifier

Consumer assigns a unique identifier to each message Identifier can be an arbitrary ID, a GUID, a business

key Provider copies the ID to the response message Consumer can match request and response

MessageIdentifier

CorrelationIdentifier

Request Channel

Reply Channel

Provider 1

Provider 2

Consumer

Multiple specialised providersMultiple specialised providers

Each provider can only handle a specific type of message Route the request to the "appropriate" provider. But how?

Do not want to burden sender with decision Letting providers "pick out" messages requires coordination

OrderEntryOrderEntry

Widget Inv.Widget Inv.

??Order Messages

Gadget Inv.Gadget Inv.

Pattern: Pattern: Content-Based RouterContent-Based Router

Insert a content-based router Routers forward incoming messages to different

channels Message content not changed Mostly stateless, but can be stateful, e.g. de-duper

Widget Inv.Widget Inv.Order Messages

Gadget Inv.Gadget Inv.ContentBasedRouter

Composite messagesComposite messages

How can we process a message that contains multiple elements?

OrderEntryOrderEntry ??

OrderMessage

Pattern: Pattern: Splitter & RouterSplitter & Router

Use a splitter to break out the composite message into a series of individual messages

Then use a router to route the individual messages as before

Note that two patterns are composed

RouterSplitter

OrderMessage

OrderItems

OrderItem 1

OrderItem 2

Producing a single responseProducing a single response

How to combine the results of individual but related messages? Messages can be out-of-order, delayed Multiple conversations can be intermixed

BillingBilling

OrderItem 1 Response 1

OrderItem 2

Response 2

ConfirmedOrder

Pattern: Pattern: AggregatorAggregator

Use a stateful filter, an Aggregator Collects and stores messages until a complete set

has been received (completeness condition) Publishes a single message created from the

individual messages (aggregation algorithm)

BillingBilling

Aggregator

OrderItem 1 Response 1

OrderItem 2

Response 2

ConfirmedOrder

Communicating with multiple partiesCommunicating with multiple parties

How to send a message to a dynamic set of recipients?

And return a single response message?

RequestFor Quote

Vendor AVendor A

Vendor BVendor B

Quotes

AggregatorBest

Vendor CVendor C??

Pattern: Pattern: Scatter-GatherScatter-Gather

Send message to a pub-sub channel Interested recipients subscribe to a "topic" Aggregator collects individual response messages

may not wait for all quotes, only returns one quote

RequestFor Quote

Vendor AVendor A

Vendor BVendor B

Quotes

AggregatorBest

Vendor CVendor C

Pub-Subchannel

Complex compositionComplex composition

Receive an order message Use splitter to create one message per item Send to scatter/gather which returns "best quote"

message Aggregate to create quoted order message

Quote requestfor each item

Vendor AVendor A

Vendor BVendor B

Quotes

AggregatorBest Quote

for each item

Vendor CVendor C

Aggregator

Splitter

Pub-Subchannel

EMEAEMEA

Interop technologiesInterop technologies

Resource based

RPC style / distributed objects

Message based / document-oriented

Major interop technologiesMajor interop technologies

DB files

RMI-IIOP(CORBA)

in-processbridge

WSRemotin

g(DCOM)

Interop technology characteristicsInterop technology characteristics

point-to-point

transient messagesdurable message

server lifetime-management

bridgeRemoting(DCOM)

RMI-IIOP(CORBA)

WSMOMDB/files

routable

clustering

in-proc.NET

serverJ2EE

serverplatformneutral

WSMOM WS-*

MessagingMessaging

Channels are separate from applicationsRemoves location dependencies

Channels are asynchronous & reliableRemoves temporal dependencies

Data is exchanged in self-contained messagesRemoves data format dependencies

Loosely coupled integration enables independent variation

Why Web Services?Why Web Services?

Web services provide all benefits of messaging solution loose-coupling service oriented reliable communication

Why web services? composable protocol vendor neutral suitable for access through Internet

Web Services developmentWeb Services development

Web Services are expected to become the default Messaging solution in the future

WS-* standards are evolving rapidly, most important WS-Security WS-Policy WS-Addressing

WS-I defines profiles, sample applications and testing tools Profiles are guidelines for using WS specifications Use Basic profile 1.1 to build interoperable solutions

Further research on WS is being carried out

ConclusionConclusion

Enterprise systems are becoming more complex

We have patterns to help us with the design

We have technologies to implement our designs

Building for interoperability and integration is key

Enterprise Solution Patterns using Microsoft .NETMicrosoft Patterns & Practices 2003

Recommended booksRecommended books

Patterns of Enterprise Application Architecture Martin Fowler Addison-Wesley, 2003

Enterprise Integration Patterns Gregor Hohpe, Bobby WoolfAddison-Wesley, 2004 Integration Patterns

Microsoft Patterns & Practices2004

Enterprise SOA Dirk Krafzig, Karl Banke, Dirk SlamaPrentice Hall, 2004

Please complete your sessionfeedback form

THANK YOU

Please complete your sessionfeedback form

THANK YOU

EMEA 2 Enterprise Integration Erik Doernenburg ThoughtWorks, Inc. edoernen@thoughtworks.com

Documents

ThoughtWorks Approach 2009

TECHNOLOGY RADAR VOL - thoughtworks.com

ThoughtWorks Technology Radar Roadshow - Sydney

Ruby for Java Developers Neal Ford Architect ThoughtWorks nford@thoughtworks.com Blog: memeagora.blogspot.com

© ThoughtWorks, 2007 The TW Agile template for VSTS Microsoft Architect Insight Conference – 2007 Nick Hines ThoughtWorks

Rajeev.Singh@ThoughtWorks.com · Rajeev.Singh@ThoughtWorks.com . Is Agile: The best way to run a project? OR The way best people want to run a project?

Thoughtworks Tech Radar Jul 2014

Domain Specific Languages in Erlang Dennis Byrne dbyrne@thoughtworks.com

Rajeev.Singh@ThoughtWorks.com · Rajeev.Singh@ThoughtWorks.com . MANAGEMENT Lack of courage it often demonstrates ORGANIZATION …

Thoughtworks Rebrand and Rollout of Marketo

Thoughtworks Brochure - January 2016

ThoughtWorks Technology Radar Roadshow - Brisbane

ThoughtWorks JC Session - BDD

UjjainiAlam, Shabbir Bawaji(ThoughtWorks), SurajitMondal

[ThoughtWorks] Perforce for Release Engineers

ThoughtWorks Digital - Estratégia de Dados

The Curious Clojurist - Neal Ford (Thoughtworks)

ThoughtWorks - Huodongjia.com

Retail Innovation - Ruth Harrison, ThoughtWorks

PERSPECTIVES - ThoughtWorks