Enterprise ArchitectureConsulting Group
How to Deliver Complex SOA Projects Successfully: A look at Do’s and Don’t’s from Real Projects
Impact 2009Session #1532
Thomas Bohn, Jan Thielscher
Las Vegas, May 5th, 2009
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 2
Agenda
Story of the Project
Ingedients of Success
Some more Secrets to Success
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 3
The Customer and his Business
� One of the three long-established, very well known german mailorder „giants“
� ~3500 employees, HQ in Frankfurt
� Traditional catalog business dropped to 50%
� Web-based business gains importance, but systems are more and more limited
Project Goals
� Establish new eCommerce platform
� approx. 700 000 000 EUR turnover per year
� 20.000 tx (Customer Orders) per day
� Timeframe: 6 months
� Environment:
� several backend systems based on IMS v8.1
� Current shop hosted on 128 servers
� Replaced with Demand Ware
� New product data management based on SAP MDM
rolled out
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 5
The first thing to think about:
Have a SOA vision – and keep talking about it
� Google knows about 261k SOA visions
� How about yours?
� SOA means change.
…The way a business works.
…The way people work.
Your SOA vision should be as unique as your business.
� Your vision should cover your needs – not
the latest trends
� Think about what you need to bring
forward your business – and what you
already have in plcae
� No „Bullshit Bingo“ – use your companies /
peoples terminology
Talk Business in a way
your management will
understand.
Talk Tech in a way your
techies will identify.
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 6
In the project first a Clear SOA Vision has been
identified
as the main Governing Thought
Identify the Golden Guiding Principles you will never overrule again
� Understand domains required and their
differences
� Define how business objects will
interact / be maintained concerning
application in services
� Define a tailoring principle for service to
prevent monster size services
Accept SOA is not healing your existing applications
� SOA will grow slowly from a single seed
� SOA lives from its design principles it
does not heal badly designed
applications!
� Use SOA principles to force clear
design decisions and assign
responsibilities
Never ever accept bad behaviour in your new SOA design
� Encapsulate all bad behaviour
� Use processes or covering services
while connecting to badly behaving
applications
� Build compensations where
necessary
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 7
Project with dedicated Senior Sponsorship has
been chosen
Search the impacting projects
� Implementing SOA successfully, you will need power and influence
� Only projects impacting the business will have the right senior sponsorship
� Don´t base your SOA strategy on a small, unimportant project. Rather look for real business impact.
� Set the vision with line-of-business and talk about the big picture
Escalation never talks Tech
� Deciding for a power project, you automatically have senior attention
� You will escalate due to business reasons not for “Tech” reasons � Understanding assured
� But remember: You are measured delivering the business benefit not the SOA
Manage your Sponsor throughout the Project
� Be aware that your Sponsor knows how to escalate
� Align with the Sponsors other needs and goals
� Keep continuous track of your reporting, alert early!
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 8
A Hands-On-Approach has been selected
Why a hands on approach?� Integration of new eShop with existing backend was very attractive candidate
project due to high business impact
� Timeframe (6 months) challenging short
� CIO sponsorship has been assured
� Strong team with excellent resources available
Artificial Approach
� Convince first, deliver later
� Spend much time on collecting and
designing Principles
� Argue why SOA is better than
standard integration way
� Write papers, saying how optimal it
would be if …
Hands on Approach
� Deliver first, convince later
� Start with a small but strong set of
principles, grow if necessary
� Don’t argue, escalate due to
business reasons!
� Implement a solution that serves the
organisations requirements and sell it
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 9
SOA Reference Architecture
applied completely as „Basic Stack“
Business Application
Services
Strategy and Planning Services
Enterprise Service Bus
Access Services
Partner Services
Process Services
Information Services
Interaction Services
Business Events and Services
Lifecycle Services
Asset and Registry Services
Development Services
Management Services
Infrastructure Services
WebSphere Process Server
WebSphere ESB
WebSphere Service
Registry & Repository
WebSphere MQ
WebSphere
Partner Gateway
WebSphere Business Modeler
WBM Publish Server
WebSphere
Integration Developer
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 10
Make sure your team covers the essential roles
Business Analyst/Service Designer
� Deep business understanding
� Identifying real requirements
� Serving to deliver business
benfits by suitable service
deifnitions
Service Developer� Highly skilled Java developers
� deep understanding of
WID, WAS, XML, JAXP, JMS
� knowledge of required
back end technologies
Chief Architect� Being involved with overall architecture of project(s)
� Responsible for architecture conformity
� Enabling & supporting Service Designers & Developers
Project Manager� Capability to drive & escalate
complex multi-project structures
� Understand and communicate
SOA Vision
� Power to sell on Senior
Management
Test Manager� Planning, implements & executing
function and performance tests
� Coordinate with system context
Supplying systems
� Identifying Service Metrics and
organize Measurement
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 11
Mediation Module
How we prepared to launch the rocket…
� Mass Data Transformation� Mailorder Company… But no unified
catalog availbale� It´s an eShop… 24x7 � Unreliable Backends� We don´t wanna loose any order� You can not touch the Host
You have to guarantee a response time of 5
sec. Because the customer is waiting in front of
the Browser… However the backend is not
„reliable“ and might even be gone completly.
Ah, and by the way… We are talking about
orders… So they can not get lost by any
chance!
„“„“My friend, the business consultant
WebSphere JDBC Adapter
outbound
WebSphere JDBC Adapter
inbound
Mediation Module
eShopeShop
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 12
dmc4Control
Architecture Overview ESBSystems View
SAP KAA
ETL
Demandware
VpGate
WSGW
AVS RKMDM
NES NBFh
NBFu
Internet
E1
E2
E3
• Mass Data Transformation
• Complex Business Rules
• Integration
• Service Provision
• Process Management
ESB
• Integration external
Services
• Provide Services to external
Users
• Monitoring & Reporting of
Partner Communication
CrossGate
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 13
Enterprise Service Bus & Sub-SystemsHW Allocation
E1
E2
E3
Development
Loadbalancer
nxesb01
ESBCell1
nxesb02
nxesb03
nxesb04
nxesb09
nxesb10
TSM HA Cluster (DB2/MQ)
nxdb201 nxdb202 nxdb203
dev
ETL QS
nxsfa05 nxsfa02
ETLCell1
nxsfa01
nxsfa03
nxsfa04
ESB QS
nxesb07 nxesb08WaWiCell1
nxesb11
nxesb12
dev
SAN(File Shares)
ESBCell2
nxesb05
nxesb06
WSGW
nxpgw01 nxpgw02
Status: 1.4.2009
NAS(File Shares)
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 14
Make your architecture visible and consumable
� Design a Solution Architecture and document it well
but brief
� Outline the architectural layers, principles and
approaches chosen
� Stay pragmatic, let feasibillity guide you
� If you are successfull, many people will want to know
what you do and how
this will save you a lot of time later ;-)
Write down architectural decision –
you will need them again!
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 15
ASBO – GBO – ASBO
… or the importance of your (cannonical)
datamodel
ExportASBO – GBO
Mediation[Routing] ESB Service
GBO - ASBOMediation
[Routing] Import Backend
Inbound Interface Provider Outbound Interface ProviderESB ServiceGBOASBO GBO ASBO
ESB Service BackendService
Requestor
<<composite>>ESB
Service
X
[GBO – GBOVersion
Mapping]
[GBO – GBOVersion
Mapping]
[Routing]
[Routing]
<<any>>ESB
Service
Y
<<any>>ESB
Service
Z
GBOV.Y
GBOV.Z
GBOV.X
GBOV.X
<<atomic>>
ESB Service
X
GBOV.Y
GBOV.Y
composite atomic
<<ISvc, MM>>
DW_ESB_GetCustomer
<<CSvc, MM>>
Customer_GetCustomer
<<CSvc, MM>>
ESB_KAA_GetCustomer
<<ISvc, MM>>
DW_ESB_GetCustomer
<<CSvc, MM>>
Customer_GetCustomer
<<CSvc, MM>>
ESB_KAA_GetCustomer
<<ISvc, MM>>
DW_ESB_ProcessOrder
<<CSvc, MM>>
Customer_GetCustomer
<<CSvc, MM>>
ESB_KAA_GetCustomer
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 16
Derive developers cookbook
� Translate and extend solution architecture into
architectural guidelines document for
developers
� Write an architectural guidance document in
cookbook style for developers in order to keep
them on track from the beginning
� This will be your key enablement tool
Craig Hayman, Rachel Reinitz, Jon
Richter said two out of the three
biggest inhibitors of SOA is lack and
availability of Skill…
This document helped us to staff and
do project with mostly „plain J2EE“
resources.
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 17
Pay attention to the small things
For example: Naming conventions
SOA Projects often suffer fromchaos – the longer the projectlasts the worse it get´s
� SOA implementations usuallyproduce a huge amount of artifacts
� Often time get´s critical and so documentation suffers
Consistent naming conventionsare simple but effective to masterthe complexity
ESB-inbound Service[SERVICE_NAME] =
[PARTNER_SYSTEM_NAME]_ESB_
[SERVICE_NAME]ISvc
ESB-outbound Service[SERVICE_NAME] =
ESB_[PARTNER_SYSTEM_NAME]_
[SERVICE_NAME]OSvc
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 18
Pay attention to the small things
For example: Mapping in Sourcecode Control
System
ASBO
ESB
Service
X
[GBO – GBO
VersionMapping]
[GBO – GBO
VersionMapping]
[Routing]
[Routing]
ESB
ServiceY
ESB
ServiceZ
GBO
V.Y
GBO
V.Z
GBO
V.X
GBO
V.X
GBOV.X
Root
InterfaceProviderService
[ESB_PARTNER_SYSTEM_NAME]
Inbound
Outbound
[SERVICE_NAME]ISvc
[SERVICE_NAME]OSvc
CommonService
[SERVICE_NAME]ISLib
[SERVICE_NAME]OSLib
[DOMAIN_NAME]
[SERVICE_NAME]CSvc
CommonLibrary[DOMAIN_NAME]
[NAME]CLib_[VERSION]
CommonLibraryMap
[DOMAIN_NAME]
[CLib_NAME]CLibMap_[CLib_SOURCE_VERSION]_[CLib_TARGET_VERSION]
Technology / Application
Adapter
External Service
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 19
Pay attention to the small things
…more examples
XML Namespaces
� Clear design of XML namespaces
makes live so much easier
� Most of service versionning depends on
XML namespaces - at least be prepared
Versioning
If you are sucessfull you have to deal with
versioning
� But think about what you have to
version! Will you have to version
Common Services, or only interfaces?
Testing
If you do integration, testing can become
VERY painfull!
� Invite backend system owners to the
table
� Coordinate with backend system owners
early, so you have test data
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 20
Service Identification &
Definition Document
� Document-Template to standardizethe definition & documentation of Services.
� To-be-done before and while serviceis developed
� Provides the documentation for later
Use both, verbal and formal
description.
UML 2.x provides a great syntax for
standardized formal description.
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 21
…Let me introduce you to Frank,
„King of the knobs“
WebSphere Enterprise Service Bus allows you to createyour own Mediation Primitives and add them to WIDspalette.
� Extremly usefull & efficiant for re-ocurring, small-grained tasks like logging, checking, reading datafrom sources… and many more!
� Developing custom mediations for WebSphere Enterprise Service Bus(http://www.ibm.com/developerworks/websphere/library/techarticles/0601_daniels/0601_daniels.html)Building your own mediation primitive with WebSphere Integration Developer and WebSphere Enterprise Service Bus(http://www.ibm.com/developerworks/websphere/library/techarticles/0705_butek/0705_butek.html)
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 22
Align your Approach with the Maturity of your
Organisation
Correct SOA is a highly mature Concept� Service Design requires a high degree of
discipline� Services modularize applications, therefore the
number of components to manage increases� Unmanaged SOAs will die fast
Understand the maturity of your IT� Identify where your Organisation is located� Identify the gaps� Take care the projects covers the gaps
(but only within the project, there it is possible!)
New Capabilities will seed a new level of maturity� Enabling the team comes first� Deliver and ride on the success� Transfer knowledge into organization by
demonstrating benefits � Slowly open up to new teams,
make it desirable to be mature!
Organizational Maturity
Impact 2009 DeliverComplexSOA.ppt EACG GmbH 23
Planning the approach
� Adopt existing (formal & informal) organizational structures andresponsibilities
Develop and apply governance principles
� Functions conducted by humans are thespinal discs of business processes
Identify services that will add value to your business processes
� Invite stakeholders to the table EARLY!Set implementation priorities and assess the impact of the new SOA environment on current legacy systems
� People to be SOA enabled
� Leverage and transform knowledge into thenew approach
Map existing assets to business processes
� Identify what are the assests that can bereused for the future.
� Identify technical stakeholders and makethem part of the project.
Determine how your IT environment supports your business strategy
… and what should not be missingBuilding blocks of planning
Enterprise ArchitectureConsulting Group
Jan Thielscher
Enterprise Architecture Consulting Group – EACG GmbH
Opernturm, 16.OG
60306 Frankfurt am Main
Germany
T: +49 69 66774 8184
F: +49 69 66774 8450
E-mail: [email protected]
Thomas Bohn
IBM Deutschland
Woerthstrasse 13-15
97082 WuerzburgGermany
T: +49 931 808 0
E-mail: [email protected]