A Pragmatic Approach for Message Modeling

Mina Boström Nakićenović

A Case study in an agile organization

BACKGROUNDSUNGARD Front Arena

SUNGARD FRONT ARENABACKGROUND

Large, world-wide financial services software companySoftware and processing solutions for financial servicesMore than 25 000 customers and 70 countriesSolutions for:

BanksBrokerage & ClearanceCapital marketsCorporationsGlobal TradingTradingInvestment BankingEtc.

OUR CLIENTS2010

2009 FINANCIAL NEWS EXCELLENCE IN I.T. TECHNOLOGY & TECHNOLOGY: Best Systems Vendor.

2009 WATERS RANKINGS: Best Counterparty Risk Solution Provider

2009 BANKING TECHNOLOGY “READERS’ CHOICE” AWARDS:Winner: Adaptiv: Best Risk Management Product (third year running).Winner: Front Arena: Trade processing.Runner up: Front Arena: Best Trading Platform / Order Management Product.

2009 CHARTIS RESEARCH RISKTECH100 RANKINGS:SunGard placed first for Functionality and Trading and Capital Markets: ranked 3rd placed overall.

2008 CHARTIS RESEARCH RISKTECH100 RANKINGS:SunGard ranked first place overall - 3rd consecutive year: Taking top honours in 5.

2009 FINTECH 100: SUNGARD TOP PROVIDER FOR CAPITAL MARKETS(ranked 2rd in FinTech 100 top providers of IT solutions overall).

2008 RISK TECHNOLOGY RANKING (3RD CONSECUTIVE YEAR):SunGard placed first for Functionality and Trading and Capital Markets: ranked 3rd placed overall.

2008 CREDIT MAGAZINE’S TECHNOLOGY INNOVATION AWARD:SunGard placed first for Functionality and Trading and Capital Markets: ranked 3rd placed overall.

OUR AWARDS

INTRODUCTIONTNP Message Protocol

TNP MESSAGE PROTOCOLINTRODUCTION

Our system includes functionality for order management and deal capture for instruments traded on electronic exchanges.

Market access is based on client/server architecture.

Client and server components: communicate using an internal financial message protocol: Transaction Network Protocol (TNP)

TNP: a protocol for transaction handling built on top of TCP/IP.

The TNP protocol uses its own messages, implementing :TransactionsTransaction RepliesUpdates (multicasts)

The TNP API and TNP structures provide the standard method of communication between clients and server components

TNP MESSAGE PROTOCOLINTRODUCTION

TNP MESSAGE DEFINITIONINTRODUCTION

The TNP messages contain one or more TNP message records.

The TNP records: organised hierarchically (parent, child, sibling)

PROBLEM DESCRIPTION

THE PRESENT ARCHITECTURE OF THE TNP MESSAGE DEFINITIONPROBLEM DESCRIPTION

Components that use the TNP protocol: implemented in different programming languages must use the same TNP message definition.

The same TNP message definition is defined in several different files:

none contained the complete definition some information was duplicated.

Architecture had several critical problems:An unnecessary duplication of the data definition (risk for data inconsistency)A programming language dependency (one of the definition files was a C++ file)

Currently there are more than 1000 defined TNP records and the number of these records grows constantly.

THE C++ DEFINITION FILEPROBLEM DESCRIPTION

// TNPGETORDERBOOKSREPLY - Parent to zero or more TNPINSTRUMENTTYPEINFO records

// TNPGETORDERBOOKSREPLY - Parent to zero or more TNPORDERBOOK records

// TNPGETORDERBOOKSREPLY - Parent to zero or one TNPTEXT record

 typedef struct {

DWORD dwType;DWORD dwSize;DWORD dwResultCode;

} TNPGETORDERBOOKSREPLY, * PTNPGETORDERBOOKSREPLY;

DESIGNING A NEW ARCHITECTUREPROBLEM DESCRIPTION

A new architecture should be:CentralizedProgramming language independentWithout risk for data inconsistency

Multiple software artifacts, defining the same financial message definition in different languages, should be generated from the same platform-independent model.

DESIGNING A NEW ARCHITECTURE PROBLEM DESCRIPTION

The right solution for our problem: to introduce the MDA concept

NON-TECHNICAL CONSTRAINTS PROBLEM DESCRIPTION

Short time-frame

Developers without MDA experience

No possibility for investment in change management

Agile company culture where short time-to-market constraints are an important issue

THE MAIN QUESTION PROBLEM DESCRIPTION

How to create a new architecture for the TNP message definition with an MDA philosophy taking mentioned factors into account?

Introducing MDD in the company

COMPANY MANAGEMENT LIMITATIONSINTRODUCING MDD IN THE COMPANY

Short time-frame

No new tools or licenses

No investment in change management

ORGANIZATIONAL MATURITYINTRODUCING MDD IN THE COMPANY

The MDA approach has not been used within the company

A previous attempt of introducing MDA failed

The large number of developers/architects has never heard of this concept

UML modeling was not used in general

Eclipse Modeling Framework (EMF) was not used at all

Developers were familiar with XML

Application of Agile principles

THE MAIN PROBLEMSAPPLICATION OF AGILE PRINCIPLES

The long starting curve for MDA should be shortened (short time-frame)

MDA tools cannot be used (requires learning time -> no investment in change management )

The organization is not mature enough for the full-scale MDA

OUR OWN PRAGMATIC MDD APPROACHAPPLICATION OF AGILE PRINCIPLES

Two main Agile principles:"Don't plan for future" - we needed a solution just for the current problem"Simplicity is essential" - we wanted to have as simple solution as possible

The MDA concept:

ModelsTransformationsCode generatorsReverse engineering

“light” Agile principles + “complex” MDA concept

= our own pragmatic MDD approach

Pragmatic MDD approach

MODELPRAGMATIC MDD APPROACH

XML was good enough The tradeoff between the familiarity of the XML and abstraction benefits of UML and related frameworks was crucial for keeping within the time-frame of this project.

No separate PIM and PSMs Pragmatic solution: just one model, with entangled platform specific information.

CODE GENERATORS PRAGMATIC MDD APPROACH

XSL Transformations

Common standard

Common ground for C++, Java and .NET developers

“Collective code ownership“ is achieved

REVERSE AND ROUND-TRIP ENGINEERING PRAGMATIC MDD APPROACH

One-time activity (to convert the existing C++ data structures to XML format)

Our own tool

Scrum development: working in sprints

Vertical slice approach (“Spike” principle)Start with simple scenariosDo the whole chain (model-code generators-reversing back to the model) early in the projectIn every sprint expand scenarios to the more complex ones

The results of the reverse engineering helped us with the model specification.

Agile practices in the pragmatic MDD

“LIGHT” Agile principles + “HEAVY” MDA’s standards

= PRAGMATIC MDD approach

PRAGMATIC MDD AND AGILE PRINCIPLES AGILE PRACTICES IN THE PRAGMATIC MDD

“Simplicity is essential.” We have simplified the full scale MDA and adjusted MDA principles to our needs, company and organizational requirements. We got a very useful pragmatic MDD approach.

“Welcome changing requirements, even late in development.”

We presented an iterative development, which allowed late model changes. We worked in sprints, according to the Vertical slice approach.

“Business people and developers must work together daily throughout the project.”

Working with the XML model made management people more involved in the project, since they have some understanding of XML.

PRAGMATIC MDD AND XP AGILE PRACTICES IN THE PRAGMATIC MDD

”Never duplicate your code” (DRY principle)This principle is the heart of the MDD – to have one central input point, model (or set of models) from which everything else is generated.

“Simple design”We simplified the MDA in order to get an applicable solution. Instead of the UML modeling language we used the XML format. The PIM and PSMs were merged in order to work only with one model and to avoid maintenance of several models and transformations among them.

“Collective Code Ownership”We extended this principle to include common ownership of formal models and code generators.

"Don’t build for tomorrow“If you leave the design simple, it wouldn’t be difficult to improve that in the future.

"Spike solution“Vertical slice approach: first to evaluate the chosen approach and second to investigate the application of the MDD within this Vertical slice.

Lessons learned

LESSON #1LESSONS LEARNED

Agile principles can make the starting curve for MDD shorter

Through the application of the agile principles the long learning curve and introduction gap of MDD methods and tools could be avoided.

Stepwise development contributed to the easier acceptance of the MDD approach.

LESSON #2LESSONS LEARNED

The XP's Spike principle (Vertical slice) can be very helpful

We have started with the simple stories and used them to implement the whole chain, the whole round-trip for our MDD approach.

The spike principle helped us with the quick development of the reverse engineering also.

Spike principle had a contribution to the faster introduction of the MDD.

LESSON #3LESSONS LEARNED

The pragmatic MDD approach could be used instead of the full scale MDA

The pragmatic MDD approach could be successfully used as a replacement for the full scale MDA.

“The difference between the theory and practice is much greater in the practice than in the theory”. Some principles of the MDA had to be changed and some new, more pragmatic, principles had to be adopted, in order to get the system working in practice.

LESSON #4LESSONS LEARNED

Organizational maturity can matter for MDD implementation

Depending on the organizational maturity the architectural decisions were made: should the full scale MDA or the MDD approach be used, in which way the MDD approach would be introduced, etc.

LESSON #5LESSONS LEARNED

The PIM- PSM concept doesn't always have to be used

Since the PIM model didn't differ too much from the other PSM models, we merged all models in one. In that way we avoided unnecessary complexity with maintaining several models and the transformations among them.

LESSON #6LESSONS LEARNED

Drawbacks of the pragmatic MDD approach

Reduced abstraction and expressiveness due to low-level details in the model

A lack of good Separation of Concerns

The advantages of the MDA tools were not used

The constant improvement of the GUI appliaction

CONCLUSIONS

Through the application of agile principles a pragmatic MDD approach was produced and used instead of the full scale multi-model MDA, which was unacceptable for the organization.

The pragmatic MDD approach relaxes the recommendations defined by the OMG’s standards.

MDA's principles were simplified by merging the PIM and PSMs to one model expressed in a custom XML dialect. The MDD became less abstract, simpler and applicable in our organization.

The introduction of the pragmatic MDD could fit within the short time-frame. A rough estimate of the time: 200 developer hours spread out on a calendar period of 6 months.

By using agile principles the learning curve and introduction gap of MDD methods and tools were avoided.