A university for the world real R © 2009, Chapter 19 Process Integration Lachlan Aldred

a university for the worldrealR

WW LLLYYY AA

© 2009, www.yawlfoundation.org YYY

Chapter 19 Process Integration

Lachlan Aldred


2WW LLLYYY AA

YYYYY

© 2009, www.yawlfoundation.org

Overview

• Introduction


3WW LLLYYY AA

YYYYY


Introduction

• Process Integrations (PIs) need to couple (connect) differently depending on the purpose of integrating.

– E.g. Feedback or Fire-and-forget– Terms such synchronous/ asynchronous, point-to-point publish

subscribe refer to this coupling but have many meanings.

• Industry solutions create adapters in earnest. One of the most compelling Process Integration products claims over 350 adapters.

– No real support for aligning coupling with intent.– It is as if the what (the thing being integrated with) has

drowned out the how (online/offline).


4WW LLLYYY AA

YYYYY


Introduction …

• Languages for process integration are not taken seriously enough.

– Important efforts such as BPEL and BPMN are not given the mindshare they deserve.

– Neither is formally defined.– Neither can handle correlated event processing.

• This lecture will not solve process integration.– Merely lay some conceptual groundwork.– Rediscover and redefine some old ideas.– Perhaps show some attributes we’ll see in future languages.


5WW LLLYYY AA

YYYYY


Process Integration as a Field of Endeavour

• Workflow Reference Model (WRM) defined 5 interfaces– Interface 3 addresses invoking remote applications from a

process.– Interface 4 addresses being integration with remote processes.

– Large effort - requires programming to these interfaces.

• BPM systems and ESBs superseded WRM.– moved/reduced the need to program to APIs.– Represented a major improvement for business users.

• BPEL enabled SOA paradigm applied to Business Processes.

– Interfaces 3 & 4 of WRM became invoking a Web services and exposing a process as a Web service – simple.

– Sending incoming requests to the right process instance was addressed cleanly through Correlation-sets.


6WW LLLYYY AA

YYYYY


Why do we Care?

• BPEL is a language for Process Integration so why improve it?

• Expressive– Can’t step outside of the Web service paradigm.– Can’t handle correlated event processing.– Models cannot discriminate between tightly coupled and

loosely coupled middleware (middleware is hidden in the WSDL binding).

• Conceptualisation– No publish-subscribe construct.– Over the wire message format exposed to process layer.


7WW LLLYYY AA

YYYYY


Why do we Care …?

• BPEL is…

• Suitability– Difficult to model batch message processing.– Difficult to model message filtering.

• Understandable– No graphical representation.– One sided: can’t model two integrated processes together.

• Precise– Rules of BPEL written in English : ambiguity, interpretation.


8WW LLLYYY AA

YYYYY


Process Integration is not Trivial

• Heterogeneity – of process engines, middleware, applications (most challenging)

• Uncertainty/trust– 2 Generals paradox – Cannot trust that other general will co-attack based on

messages.– There is no message protocol that can guarantee intent.

• Reliability– Latency of messages (delays, stale data)– Lost messages, duplicates deliveries.

• Events – Don’t wait for them to occur – just handle it if it occurs.– Composite events : Don’t do anything unless getX(Event_A) =

getX(Event_B)


9WW LLLYYY AA

YYYYY


Process Integration is not Trivial …

• Batch Messaging– handle/create large numbers of messages.– Avoid clumsy loops and lists

• Conversations– Avoid clumsy correlation IDs.– Correlation ID variables– Technology agnostic– Nested conversations

• Channel passing– Processes learn of new message source/sinks through

message contents.– Technology agnostic


10WW LLLYYY AA

YYYYY


Conceptualizing PI

• Conceptual modelling community have learned that conceptualising a domain has pitfalls.

• Avoiding pitfalls - guiding principles:– 100 percent principle:

• Every aspect of process integration should be describable.

• Full expressiveness. No need to express aspects of process integration outside the language.

– Conceptualization principle:• Only need to express relevant aspects.

• Avoid needing to express every low-level detail (e.g. access credentials, message formats).


11WW LLLYYY AA

YYYYY


Conceptualizing PI …

• Software development community have learned that abstracting a domain can fail:

• e.g. Law of Leaky abstractions:– “all non-trivial abstractions, to some degree, are leaky“

(Spolsky).– Abstract APIs can save huge amounts of development time,

but leaky ones cost just as much in learning time.


12WW LLLYYY AA

YYYYY


Why Patterns?

• Patterns express what the key static and dynamic aspects of the domain are.

• Patterns, collectively, express what needs to be modelled.

– Blueprint for a process integration language


13WW LLLYYY AA

YYYYY


Pattern Categories

• Patterns of Coupling– Dimension of Threading, Time, and Space

• Batch Patterns– Multiple message sends/receives, filtering.

• Bi-directional Interactions– Various patterns for receiving feedback – within the interaction.

• Composed Interactions– Patterns achieving state-aware conversations, across many

interactions.

• Event-based Process Patterns– Patterns for handling unsolicited messages in-process


14WW LLLYYY AA

YYYYY


Pattern Categories …

• Transformations– Patterns that transform messages and message data.

• Process Discovery– Patterns that dynamically change during runtime to perform

new actions.


15WW LLLYYY AA

YYYYY


Coupling Pattern: Non-blocking receive

• Occurs when the receiving process is not waiting for the message to arrive.

• Observable as foundation of most asynchronous architectures.

• Enabler for Event-handling.

• e.g. reallocate seats for airline cancellations.


16WW LLLYYY AA

YYYYY


Coupling Pattern: Time Decoupled

• No indication back to sender about success of interaction.

• ‘Fire and forget’ – send message and start doing something else while the interaction is running.

• e.g. Reconciling inter-bank payments overnight.


17WW LLLYYY AA

YYYYY


Composed Interaction: Property-based Conversation

• Modelled by expressing functions over messaging tasks• You only need to model the function logic and identify

the tasks involved.• The process environment does the matching.• When a match is found the message is directed to the

right process instance.


18WW LLLYYY AA

YYYYY


Conclusions

• Patterns have been used in software engineering to express conceptual solutions to re-occurring problems.

• Process Integration Patterns, on the other hand, are requirements patterns.

– They express what process integration needs to b able to do.

• They indicate what is relevant and what is less relevant (Conceptualisation Principle).

• They indicate what a fully expressive PI language should be capable of (100 Percent principle).

Documents

A university for the world real R © 2009, Chapter 19 Process Integration Lachlan Aldred