1© 2005 Sonic Software Corporation The BAA Terminal 5 Project – The reality of XML, ESB and Web services

1 © 2005 Sonic Software Corporation

The BAA Terminal 5 Project – The reality of XML, ESB and Web services


The reality of serving 35 million people per year

37 million man hours to build T5

6.5 million cubic metres of earth works

15,000 cubic metres of concrete per week

16 major projects, 100 sub-projects

Sub projects cost between £30M and £150M

60,000 people involved in the build

The IT infrastructure must operate entirely new level of

speed, efficiency and availability (they land planes!)


The IT landscape – must be integrated

6000 display systems, 400 COTS apps,197 line of business apps, 35 operationalIT platforms, over 1000 servers

One hour server failure has Europe-wide impact on flights, more than one hour has global impact

There have been two failures of over 6 hours in the last 8 months – these even caused several mile tailbacks on major surrounding roads

£250M fine for latedelivery of T5 !!!

CONNECT EVERYTHING. ACHIEVE ANYTHING.™

Why not Web services alone?

If we asked you to solve this problem with WS-*, SOAP and XML – what would happen?


How would we get there?

CRM ERP

PARTNER SYSTEMS FINANCE

Today’s architecture rigid, costly and difficult to operate

Proprietary technologies and skill sets

Multiple communication infrastructures

High cost of license, consulting and operation

Lots of turf control and organizational issues

ORDERENTRY


J2EE™ APPLICATION

PACKAGED APPLICATION

& LEGACY SYSTEMS

.NET™APPLICATION

PARTNER SYSTEM

WEBSERVICE

In walks MOM – and Web services

Hiding implementation details enables reuse

XML-based data easily exchanged

Designed for remote access, across heterogeneous platforms

Can be easily passed over HTTP(S), JMS, CORBA, Sockets, MQ, RV and almost any other messaging layer

Standard Interfaces are Major Step Forward

TCP/IP

WEB SERVICESINTERFACE

XML

XML


WEB SERVICESINTERFACE

J2EE™ APPLICATION

PACKAGED APPLICATION

& LEGACY SYSTEMS

.NET™APPLICATION

PARTNER SYSTEM

WEBSERVICE

Web Services

Is it reliable, scalable and secure?

How do you change business processes?

How do you manage and monitor distributed services?

But Have We Solved The Whole Problem?

Web services are interoperable communications stacks and don’t offer routing, service deployment, management, format transformation, guaranteed delivery, etc.

You are building standards based spaghetti !

TCP/IP


Why not Web Services alone?

WS-Reliable Messaging creates reliable point-to-point connections– But still 100’s or 1,000’s of them – where’s the manageability?– How do you configure reliability to suit your needs?

WS-Security creates security “Swiss cheese”…– Each secure Web service needs to authenticate incoming

messages– All accessing corporate security server? At the same time?– Creates 100’s of security holes? Can you run your Web services in

the DMZ?– Many customers we work with simply do not allow external Web

services

WS still lacks federated enterprise features


Where is the Strategic Inflection Point?

When the balance of forces shifts from the old structure, from the old ways of doing business and the old ways of competing, to the new.

Before the strategic inflection point, the industry simply was more like the old.

After it, it is more like the new. It is a point where the curve has subtly but profoundly changed, never to change back again.

- Andy Grove, Only the Paranoid Survive, 1996

By 2008, Gartner predicts that SOA will be a prevailing software-engineering practice, ending the 40-year domination of monolithic software architecture.


So, back to BAA…


BAA – The Strategy

“… our strategy is to minimize the interdependencies between products, using open standards to increase operational flexibility and make sure that applications are responsive to change. Therefore a Service Oriented Architecture approach is inevitable.

Our first challenge was to find a platform that would

work well in our very demanding environment, and could orchestrate the services that will drive T5 operations. Sonic Enterprise Service Bus is a very natural fit."

Nick GainsHead of ITBAA


Why did BAA choose SOA?

New technology, regulations, re-organizations, and market demands

For years, change velocity has outstripped IT capacity

Legacy integration approaches failed Costs -- license and services -- exceeded plans Broker / platform stacks: costly, closed, complex Infrastructure never scaled to the extent of the enterprise For example, not one single terminal has opened successfully in over 25 years The last attempt was Seoul with CORBA – it was a complete failure

Very high development costs, particularly with integration Time delays where projects are always behind the business needs Lack of visibility and understanding of systems

Typical Result:


Service Oriented Architectures lead to:

Reduced costs — Simplifies the integration process by making application interoperability "plug-and-play". By utilising open standards, there is less software infrastructure to purchase and maintain.

Faster time to market — An extended enterprise will be able to respond more quickly to market changes than its competitors, as its business is more agile.

Greater operating efficiencies — Companies will be able to reuse existing application components and utilise new services, rather then manually duplicating them in-house.

Increased customer satisfaction — Through tighter integration of the business value chain and less manual intervention in business processes, suppliers and customers will have greater reuse of data, and more reliable and timely information.


INCREMENTAL DEPLOYMENT

BAA’s Enterprise SOA Vision

BROAD-SCALE INTEROPERABILITY

MODULARITY / REUSE

FLEXIBILITY

APPLICATION SERVER

USER-DEFINED SERVICE

LEGACY APPLICATION

PROCESS SERVER

RELATIONAL DATABASE

BATCH SYSTEM

PORTAL SERVICE

Benefits address long-standing IT dilemmas

What most people concentrate on are the endpointsBut the problem area is the “white-space” of SOA


Architectural Challenges of SOA platform

Dependability – they land planes!– Reliable, high-performance communications between services– High Availability = Business continuity– Security

Flexibility – this is a changing environment– Mechanism to orchestrate process through the network– Ability to dynamically re-configure services to new uses– Ability to normalise in-flight documents between services– Bridge multiple low-level middleware technologies

Reach and scale– Connect any resources regardless of where they are deployed– Scale from initial phases to arbitrarily large deployment– Retain visibility and control of distributed infrastructure


INCREMENTAL DEPLOYMENT

Enterprise SOA

BROAD-SCALE INTEROPERABILITY

MODULARITY / REUSE

FLEXIBILITY

APPLICATION SERVER

USER-DEFINED SERVICE

LEGACY APPLICATION

PROCESS SERVER

RELATIONAL DATABASE

BATCH SYSTEM

PORTAL SERVICE

BAA picked an ESB as their SOA framework

BAA chose an ESB to address this SOA “white-space”

HIGH AVAILABILITY


WITHOUT AN ESB

The Purpose of an ESB

WITH AN ESB

Connect, Mediate and Control

Connect Control

Mediate


Connect applications and servicesMultiple on-ramps, dependable communications

Connect old and new– Legacy applications, RDBMS– J2EE, .Net– Web services– B2B protocols

Link services and processes across the extended enterprise

Establish robust, scalable and secure communications

Connect

All connected resources are first-class citizens

Examples:– Securely link internal processes

with those of business partners.– Portal integration


Mediate servicesBridge and extend incompatible technologies

Mediate

Reconcile system incompatibilities– Communication Protocol– Interaction model

Transform and enrich data– Map between data formats– Split, aggregate and enrich data

Provide flexible routing and process flow

– Decoupled, event-driven services– Intelligent routing– Support stateful process management

Eliminate service interdependencies

Examples:– Aggregate data from multiple SAP systems– Regulation compliance logging


Control service interactionDeploy, configure, manage

Dynamically configure, deploy and upgrade hosted services

Establish and alter process flows, routing, Quality of Service

Gain control and visibility over services and their interaction

Control

Configured, not coded

Examples:– Deploy and upgrade 1000s of end-points

from a single location.– Detect faults and diagnose problems in

complex deployment.


Global reach, global scalabilityEnd-to-end SOA

ESB spans clusters and security infrastructure to form federated environment

Bus topology obviates hub-and-spoke bottlenecks Deploy what you need, where and when you need it


What are the technical problems?

The Devil is in the detail…


How do you manage a project this big?

How do they leverage their existing IT portfolio?

What will this cost?

What would be the impact of– Changes?

– Expansion?

– New security threats?

– Regulation changes?

How will they accommodate future requirements?

Business Process Definitions


Imagine Project Managing the Internet

…would you scope the project?

…would you consider all future needs

…would you handle training?

…would you manage change?

…would ensure interoperability?

…would you manage the technical differences?

…would you manage the scale?

…would you manage the risk?

How…


So how was Internet Successful?

Built around a few sacred principles

Evolved from selected technology standards

Deployment abstracted from design

Incremental deployment

Tactical execution

What made it work?


Strategy Versus Tactics

This is how BAA are making T5 a success

Everything is broken down into manageable tasks

Matrix management

Evolutionary project management

– NOT Waterfall project management

An IT back-bone and architecture from

the very start

Industry patterns are being exploited

What makes some ideas work where others fail?


Design patterns

I encourage you to work with patterns – check out:– http://www.enterpriseintegrationpatterns.com/

– http://www.oreilly.com/catalog/esb/index.html

Working with patterns…– Some examples…


Scenario from a European Airline

Printing


Solution Scenarios - CITP


CITP in Pattern language


Micro Patterns as Services

Print Req.

ESB Infrastructure

PDS

JMS

Web

JCA

MDB

EJB

SSB

Servlet

Portlet

P2P

P2P

P2PCITP

MQ

1

2 3 4

5

5

5

1. Print Request arrives at CITP2. Request crosses the MQ Series Bridge3. Print Token is resolved in PDS4. Request is routed via CBR5. Request is consumed in Terminal


Reservation Applications

Using a re-factoring pattern

Agent Central CRM Finance Applications

SOAPB

row

se

r

XML SOAP

DB

Registration AppAudit

Adapter

Integration Broker

eP

oS

Cli

en

t

Siebel

Adapter

Mainframe

MQ

S

erie

s


Incremental Staged Deployment

Registration App

Bro

ws

er

DB

ESB

Enterprise SOA – one step at a time

Service Containers

eP

oS

SOAPHTTP

WS

SOAP

Integration Broker

Audit

Adapter

COM

Cli

en

t

Siebel

Adapter

MQ

S

erie

s

Mainframe

Reservation ApplicationsAgent Central CRM Finance Applications


Widely Distributed Enterprises

Registration App

Bro

ws

er

ESBESB

eP

oS

SOAP

Integration Broker

Audit

Adapter

SOAPHTTP

WS

Cli

en

t

Siebel

Adapter

MQ

S

erie

s

Mainframe

DB



Leave and Layer

Registration App

Bro

ws

er

DB

ESBESB

eP

oS

SOAPHTTP

WS

Integration Broker

Cli

en

t

Siebel

Adapter

Audit

Adapter

ESB

MQ

S

erie

s

Mainframe



Partner Integration

Registration App

Bro

ws

er

DB

ESBESB

eP

oS

SOAP

MQ

S

erie

s

Siebel

Adapter

Audit

Adapter

SOAPHTTP

WSESB

Integration Broker

Mainframe

Cli

en

t



Incremental Adoption

I like to say: “Think Strategically, Act Tactically”

Registration App

Bro

ws

er

DB

Mainframe

Cli

en

t

ESBESB

Phase 1Phase 2

Phase n

MQ

S

erie

s

ESB

eP

oS

Siebel

Adapter

Audit

Adapter

Partner ESB

Integration Broker



Some architecture tricks to be aware of…

– Dealing with: Trapped messages Out of Order messages


Machine failure causes trapped messages

Two machines, a brokereach and clustered

Server

Client

1 2

3 4

Machine fails!


Clustering means system carries on straight away


Server

Client1 2

Machine fails!

3 4


Machine recovers and messages delivered


Server

Client1 2

Machine fails!

3 4

Recovery of messages takes:1. Machine reboot2. OS restart3. Software reload/restart4. Database recovery5. Re-send of messages

= several minutes…


Can you wait several minutes?

Trader trying to execute a buy

Retailer trying to process your credit card at a till

Bank trying to process mortgages before the end of the day

Airport trying to route baggage

Telco network usage just before billing run at end of month

Many other situations…

It gets worse…

Now add a requirement for guaranteed message ordering


Typical scenario – an airport display board

Flight Information(four messages)

1 2

3 4

Messages:1 = BMI256 last call gate 32 = BA35 to gate 193 = BA35 to gate 23 (correction)

4 = BA765 last call gate 15

Machine fails!


Messages:1 = BMI256 last call gate 32 = BA35 to gate 193 = BA35 to gate 23 (correction)

4 = BA765 last call gate 15

Message ordering causes “pile-up” in other broker


1 2

3 4

Machine fails!

Where are mymessages!

Due to guaranteed message ordering,can’t deliver other messages!

And remember, recoverytakes several minutes…

(I can see potential for missed flights!)

Think of complexity if you add in XA transactions


Turn to Continuous Availability Architecture

CAA

Conventional hardware Seconds to recover Simple and flexible Maintains transactional integrity Easy to use with message

ordering

CAAHOT-HOT deployment

Increases overall throughput

• Each broker has a hot backup broker on another machine

• Transparent to client• Transactional integrity during failover• Once and only once delivery ensured• Fully scalable as we can also add clustering!


P

Remember our airport?


11 22

33 44

Machine fails!

Secondary becomes primary and sends messages


Sonic ESBSonic ESB

Overview architecture for BAA T5

Sonic Cluster

BrokerBroker A’A’

BrokerBroker BB

BrokerBroker AA

BrokerBroker B’B’

Sonic Cluster

BrokerBroker A’A’

BrokerBroker BB

BrokerBroker AA

BrokerBroker B’B’

Geographical Site 1 Geographical Site 2

AODB

Sonic ESBcustom JavaservicesAODB

Sonic ESBcustom Javaservices

Clie

nt

Ap

p

Subscribe

Adapter

Clie

nt

Ap

p

Subscribe

Adapter


Summary – moving to “One Architecture” I suggest you look into the following:

– Evolutionary project management Things like extreme programming techniques

– Design and architecture patterns www.enterpriseintegrationpatterns.com

– Service Oriented Architecture This is one of the most sought after roles in IT right now

– (There were 280 architect roles on monster yesterday) Enterprise Service Bus concepts – read the ESB book!

SOA has gone past the critical inflection point– Learn SOA, understand the nuances (the detail)– Serious money is being spent on this new technology– Web services is important, but is only one part of the hybrid

architecture that companies are moving toward

Documents

1© 2005 Sonic Software Corporation The BAA Terminal 5 Project – The reality of XML, ESB and Web services