Advanced Web Service Transactions Management Peter Dolog [email protected] CS Department Intelligent Web Information Systems ://,

Advanced Web Service Transactions Management

Peter [email protected] DepartmentIntelligent Web Information Systemshttp://www.cs.aau.dk, http://iwis.cs.aau.dk

http://www.cs.aau.dk/

http://iwis.cs.aau.dk/

2Web Engineering, Lecture 14, Advanced Transactions Management

OutlineCompensations Design

• General problem• Business transactions• Middleware for advanced compensations• Service provider and client feature modelling• Matchmaking and restriction model• Further Challenges

Distributed Transactions and Waiting Cycles• Motivation• Related Work• Transaction Dependencies Management• Protocol• Cycle Detection• Experiments• Conclusions and Further Work


Outline

General problemBusiness transactionsMiddleware for advanced compensationsService provider and client feature modellingMatchmaking and restriction modelFurther Challenges


Open Web Service Environment

Service Providers• A number of autonomous service providers exist• They can provide similar functionality• They can dis-/appear any time• Each wants to maximize its profit for executing provided

services by external consumersService Consumers

• Number of consumers with similar requirements exist• They want to achieve high value for their expense• To maximize their service• By composing matched available services from different

providers


Software Product Lines

Software Providers• Number of reusable software assets exist• They may vary in its functionality• They want to maximize its profit by providing the

assets in an application in a family mostly from one company

Software Consumers• Number of consumers with similar requirements• They want to achieve high value for their expense• To maximize their service• By composing a final application from the reusable

assets

Feature Modelling as a Technique in Software Product Lines



Outline



Print and send payslipTransfer salaryTransfer tax

Payroll ScenarioCompany Employee

Bank

Transfersalary

Transfertax

Print andmail payslip

Transfer carinstalment

Wait for paymentTransfer monthly instalment for the new car


Service Oriented Payroll ScenarioCompany Employee

Bank

Transfertax

Print andmail payslip

Transfer carinstalment

To reach mutually-agreed outcome (commit/cancel)In environment with concurrent access

Transfersalary


Transactions

Control the execution of the required operations on the external services.

Consist of a set of operations (e.g. database operations) that are performed by multiple participants.

Control the collective outcome of the operations.

Distributed transactions control the execution of operations on multiple providers.• Participant• Coordinator


Error CompensationDifferent transaction specifications exist for different purposesBackward recovery

Normally, predefined rollback operations are executed in order to restore the state before the transaction. Time and money is lost Dependent transactions also have to roll back (domino

effect)

Forward recoveryAims at changing pro-actively the state of the participant or transaction to enable a successful execution after a failure. Complex Can normally only be performed semi-automatically


Traditional WS-Transaction Coordin. Structure

Company: Business Process

Transaction Coordinator

Transfer salary

Web Service 1

1. Create new transaction

2. Return coordination context

3. Invoke service, send coordination context

4. Register with coordination context

5. Confirm registration

8. Abort transaction

6. Process request

7. Send failure notification

→ Failure

7. Send request result

Normal request processingRequest failure handling


C

T1

WS4

WS1

WS2 WS3

abstract state diagram

WS – Tx / Business Activity Coordination Type


Payroll Processing

Calculate salary

Perform payment

Company

Pay bills

Employee

AccountsCompany

Employee

Tax

Car Dealer

1. Transfer of the salary to the employee‘s account

Transfer tax

Web Service 2

2. Transfer of the tax to the tax authority‘s account3. Specify the salary details, print and send the payslip

…

Transfer instalment to the car dealer‘s account

Transaction

Transfer salary

Web Service 1

Print and send payslip

Web Service 3


Motivating Scenario – Problem

A service fails due to an internal error.The error can only be compensated by aborting the complete transaction.Why should the transaction be aborted, if a different service exists that can perform the same operations?

Calculate salary

Perform payment

Company

Pay bills

Employee

Transfer tax

Web Service 2 …

Transaction

Transfer salary

Web Service 1

Print and send payslip

Web Service 3


Outline



Extended Transaction Coordination Structure



Transfer salary

Web Service 1

Abstract Service

Adapter

1. Create new transaction

2. Return coordination context

3. Invoke abstract service, send coordination context

9. Register with adapter context


11. Process request


5. Register with coordination context

4. Request adapter context


7. Return adapter context

8. Invoke concrete service, send adapter context



New Components - Abstract Service

Does not directly implement functionalities.Manages a list of concrete services.Is a mediator between the client and the concrete service.Manages and performs compensation actions.Interfaces:

• Service• Event (internal

compensation handling)• Compensation (external

compensation handling)• Contract exchange

Abstract Service

Management

Concrete service list

Concrete service wrappers

Compensation rules repository

Contract repository

Abstract service

Even

t

CompensationC

on

trac

t ex

chan

ge


Compensation Activities and Types


Example: Internal Compensation Rule

<cmp:InternalCompensationRule identifier="internalFailureLastRequestResending"> <cmp:CompensationCondition> <cmp:ParticipantEvent eventCode=

"http://sourceforge.net/projects/frogs/AdapterInteraction/ParticipantFault"/> <cmp:ParticipantState stateType='http://schemas.xmlsoap.org/ws/2004/10/wsba/Faulting' /> <cmp:ReplacementService exists="true" isDirectReplacement="true" /> <cmp:RequestSequence> <cmp:Request identifier="transferSalaryMethod" /> </cmp:RequestSequence> </cmp:CompensationCondition> <cmp:CompensationPlan> <cmp:Compensation> <cmp:ServiceReplacement/> </cmp:Compensation> <cmp:Compensation> <cmp:RequestResending lastN="1" /> </cmp:Compensation> </cmp:CompensationPlan></cmp:InternalCompensationRule>

The condition of the compensation rule

Condition 1: The internal event must have been a failure of the concrete service

Condition 2: The state in which the concrete service has to be

Condition 3: A direct replacement concrete service has to exist

Condition 4: The last request must have called this method

The execution plan of the compensation rule

Step 1: Replace the current concrete service

Step 2: Resend the last request


New Components - Adapter

Encapsulates coordinator-specific functionality.Functions as a coordinator for the concrete service.Manages messaging:

• Forwards normal messages between the real coordinator and the concrete service.

• Intercepts failure messages and informs the abstract service.

• Creates additional notifications as part of a compensation process.

Adapter

Coordinator Capabilities

Concrete service

Coordinator

Adapter ManagementAbstract service

Inst

ruct

ion

s


Internal Compensation Handling – No Action


Concrete service fails.Abstract service checks its compensation rules and contract.Compensation is not possible.Normal transaction abort.


Transfer salary

Web Service 1

Abstract Service

Adapter

11. Process request

12. Signal failure

13. Report event

14. Fail

15. Forward failure notification 16. Abort

transaction


Internal Compensation Handling – Replacement


Concrete service fails.Abstract service checks its compensation rules and contract.Concrete service is replaced.Coordinator was not notified!


Transfer salary

Web Service 1

Abstract Service

Adapter

11. Process request


12. Signal failure


13. Report event

14. Forget participant

15. Confirm failure

Transfer salary

Web Service 4

19. Process request

16. Resend request

17. Register with adapter context



Evaluation

Multiple scenarios for internal and external compensation handling have been implemented and tested.

An evaluation model has been created, which calculates net values for the standard environment and the abstract service environment.Allows an assessment whether the utilization of

the new design is economical and beneficial.Experiment performed on a simalated environmentMore in ACM TWEB paper


Outline



Compensation Types

<< Concept >>Compensation

Action

<< Concept >>Replacement

<< Concept >>Forwarding

<< Concept >>AdditionalRequest

<< Concept >>AdditionalService

<< Concept >>SessionRestart

<< Concept >>CompensationPlan

<< Concept >>Repetition

<< Concept >>NoCompensation

1 1..*<< Concept >>Compensation 1 1


Compensation Features


<< OptionalFeature >>ExternalCompensation

Handling

<< MandatoryFeature >>InternalCompensation

Handling

<< OptionalFeature >>AdditionalRequest

<< OptionalFeature >>AdditionalService

<< MandatoryFeature >>ServiceAbort

<< OptionalFeature >>Repetition

<< OptionalFeature >>Replacement

<< VariationPoint >>{Kind = AND}

<< MandatoryFeature >>RequestSequence

Change

<< VariationPoint >>{Kind = OR}

<< OptionalFeature >>AllRequestRepetition

<< MandatoryFeature >>LastRequest

Repetition

<< MandatoryFeature >>ResultResending

<< OptionalFeature >> SessionRestart

<< OptionalFeature >>AdditionalActions

<< MandatoryFeature >>NoCompensation

<< OptionalFeature >>Forwarding

<< OptionalFeature >>PartialRequest

Repetition


Capability Feature Model

Consists of:• functionality feature model • compensation feature model

The compensation feature model can contain custom features.

<< Concept >>SalaryTransferService

<< MandatoryFeature >>TransferSalary

<< MandatoryFeature >>GetAccountBalance


Handling


Handling







<< MandatoryFeature >>LastRequestRepetition





Repetition

<< MandatoryFeature >>TelephoneCall


Service Capabilities


Handling


Handling








Change




Repetition


<< OptionalFeature >> SessionRestart





Repetition




<< MandatoryFeature >>TelephoneCall


Consumer Requirements

<< MandatoryFeature >>ExternalCompensation

Handling


Handling




<< MandatoryFeature >>Repetition

<< MandatoryFeature >>Replacement



Change




Repetition






Repetition




<< MandatoryFeature >> SessionRestart


Outline



Matchmaking between service and consumer feature models

Compatibility score calculationIteratively compares feature modelsFeatures must appear at the same place in the graphMandatory features must all match but do not

contribute to the compatibility scoreIf a mismatch is found in a mandatory feature,

algorithm stops and a negative score is returnedOptional features add to the compatibility score when a

match is found (in our case +1)Additional features may contribute with different scores


Restriction Feature Model


<< Feature >>InternalCompensation

Handling

<< Feature >>Repetition

<< Feature >>Replacement

<< Feature >>AllRequestRepetition

<< Feature >>LastRequest

Repetition

<< Feature >>ResultResending

<< Feature >>NoCompensation

<< Feature >>PartialRequest

Repetition


Example: Internal Compensation Rule

<cmp:InternalCompensationRule identifier="internalFailureLastRequestResending"> <cmp:CompensationCondition> <cmp:ParticipantEvent eventCode=

"http://sourceforge.net/projects/frogs/AdapterInteraction/ParticipantFault"/> <cmp:ParticipantState stateType='http://schemas.xmlsoap.org/ws/2004/10/wsba/Faulting' /> <cmp:ReplacementService exists="true" isDirectReplacement="true" /> <cmp:RequestSequence> <cmp:Request identifier="transferSalaryMethod" /> </cmp:RequestSequence> </cmp:CompensationCondition> <cmp:CompensationPlan> <cmp:Compensation> <cmp:ServiceReplacement/> </cmp:Compensation> <cmp:Compensation> <cmp:RequestResending lastN="1" /> </cmp:Compensation> </cmp:CompensationPlan></cmp:InternalCompensationRule>


Feature Model<feature name="Compensation" type="NONE" id="compensation"> <feature name="InternalCompensationHandling" type="NONE”

id="internalCompensationHandling">…

<feature name="PartialRequestRepetition" type="NONE" id="reference3IXIpartialRequestRepetition">

<feature name="ResultResending" type="NONE" id="reference3IXIreferenceIXIresultResending">

</feature> </feature> </feature> <feature name="Replacement" type="NONE" id="replacement"> <feature name="LastRequestRepetition" type="NONE"

id="reference4IXIlastRequestRepetition"> </feature> <feature name="PartialRequestRepetition" type="NONE"

id="reference5IXIpartialRequestRepetition"> <feature name="ResultResending" type="NONE"

id="reference5IXIreferenceIXIresultResending"> </feature> </feature> <feature name="AllRequestRepetition" type="NONE"

id="reference6IXIallRequestRepetition"> <feature name="ResultResending" type="NONE"

id="reference6IXIreferenceIXIresultResending"> </feature> </feature> </feature> </feature>

…</feature>


Layers of Abstraction

and

xor

xor

Physical Services and Workflow Variants

Capability and CompensationConcepts

Capability and Compensation Features and Configurations

Restriction Profiles

Navigation and Interaction


Outline

IWIS group and backgroundGeneral problemBusiness transactionsMiddleware for advanced compensationsService provider and client feature modellingMatchmaking and restriction modelFurther Challenges


Workflows vs. Middleware

Compensations and adaptations can be specified at the design level in workflows

Copensations and adaptations can be encoded in an intelligent middleware

How to combine themHow to compose themHow to ensure consistency…


References

• M. Schäfer and P. Dolog: Feature-Based Engineering of Compensations in Web Service Environment. ICWE 2009: 197-204

• M. Schäfer, P. Dolog, W. Nejdl: An Environment for Flexible Advanced Compensations of Web Service Transactions. ACM TWEB, 2(2), 2008

• P. Dolog, W. Nejdl: Using UML-based feature models and UML collaboration diagrams to information modelling for web-based applications. UML 2004.

Distributed coordination of concurrent transactions


Web Engineering, Lecture 14, Advanced Transactions Management

Outline

MotivationRelated WorkTransaction Dependencies ManagementProtocolCycle DetectionExperimentsConclusions and Further Work

41


WS4

WS1

WS2 WS3

Business Activity

Loosly coupledLong-runningWith user interaction

WS1: Conference WS1: Flight Ticket WS3: Hotel Room WS4: Car rental

42


Transactionas

Business Activity

T1

WS4

WS1

WS2 WS3

Transactional Business Activity

WS1: ConferenceWS1: Flight TicketWS3: Hotel RoomWS4: Car rental

To reach mutually-agreed outcome (commit/cancel)

43


Transactional

Business Activity

T1

WS4

WS1

WS2 WS3

Transactional Business Activity

WS1: ConferenceWS1: Flight TicketWS3: Hotel RoomWS4: Car rental

To reach mutually-agreed outcome (commit/cancel)In environemnt with concurrent acces

T2

WS4T2

WS2T2 WS3T2

WS3T2

44


ACID Properties RelaxationCurrent Standards and Open Specifications:

• Atomicity Relaxation• Allow partial sets of participants to commit

or abort (e.g. without having some services complete succesfully)

• Isolation Relaxation• Allow concurrent users to observe partial

results (e.g. after having some services executed, temporal results comming from their execution may be shown)

45


Existing Open SpecificationsWS-Coordination/WS-Transaction (IBM, Microsoft,

and BEA …)OASIS Business Transaction Protocol (HP, Oracle)…All support coordination activation, participant

registration, transaction contextProtocols with limited abilities for concurrency under

relaxed ACID properties

46


C

T1

WS4

WS1

WS2 WS3


WS – Tx / Business Activity Coordination Type

47


C

T1

WS4

WS1

WS2 WS3


WS – Tx / Business Activity Coordination TypeStandards do not prescribe how to

handle conflicts in concurent access

It is all a matter of compensation which is considered part of a business logic

We propose an approach where concurency control is part of the protocol based on completion dependency graph and cycle detection as a new coordination type

48


Completion Dependency Graphs

Within a transaction: two operations in conflictBetween transactions:

• Assuming that T1 and T2 are two different transactions, a Completion Dependency T1 T2 occurs if there are two Web Service operations O1 and O2 (possibly from different Web Services) where O1 belongs to T1 and O2 belongs to T2 and:

• O1 reads data updated by O2 • O2 still may compensate (roll-back is a special case of

compensation)

• T1 is called: dependent transaction• T2 is called: dominant transaction

49


P1 P2

WS1 WS3 WS2WS4

WS6

WS3 WS2

WS4

WS6

T2T1

WS3

WS1

WS2 WS4

T1T2

Concurrency Control for WSMaintain an operation

conflict matrixKeep track of

invocationsRecognize completion

dependenciesDo not commit before

dominant and cycle is resolved

50


C1

T1

WS4

WS1

WS2 WS3

Chaining Coordinators

C2

T2

WS4

WS1

WS2 WS3

XCoordinatorPortType

XParticipantPortTypeS. Choi, H. Jang, H. Kim, J. Kim, S.

Kim, J. Song, and Y. Lee. Maintaining consistency under isolation relaxation of web services transactions. WISE 2005,

K. Haller, H. Schuldt, and C. Türker. Decentralized coordination of transactional processes in peer to peer environments. CIKM 2005

51


C1

T1

WS4

WS1

WS2 WS3

Chaining Coordinators

C2

T2

WS4

WS1

WS2 WS3

XCoordinatorPortType

XParticipantPortTypeS. Choi, H. Jang, H. Kim, J. Kim, S.

Kim, J. Song, and Y. Lee. Maintaining consistency under isolation relaxation of web services transactions. WISE 2005,

K. Haller, H. Schuldt, and C. Türker. Decentralized coordination of transactional processes in peer to peer environments. CIKM 2005

Information about other transactions might be considered sensitiveCreates similar problem asin EAI – a need for publish/subscribe or message queueing middleware

52


waiting

compensating

completedwait

State Diagram with Waiting State

53


Cycle Detection in Waiting State

Participant sends CycleCheck with ID token to a direct dominant transactions coordinators

The coordinator checks with participants in the schedule about dependencies

Participants forward the CycleCheck as it is in coordinators of direct dominant transactions

If there is no dominant transaction the participant sends back NoCycle token

If NoCycle token received back, transaction may endIf the CycleCheck token with the same ID received,

there is a cycle, transaction may end

54


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

WS5WS6

WS: Web ServiceP: Peer hosting Web ServicesT: Transactional ProcessC: WS Coordinator

C2C3

P12 P34

P56

WS1WS2

WS4

WS3WS4

Example

55


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

T3T1

C2C3

P12 P34

P56

T1T2

WS4

T2T3

Complete?

Protocol


56


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

wait

Protocol

T3T1

T1T2

T2T3


57


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

Close/rollback

Protocol

T3T1

T1T2

T2T3


58


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

T3T1

C2C3

P12 P34

P56

T1T2

WS4

T2T4

Completed/rolled back

Protocol


59


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

Complete?

Waiting Cycle

T3T1

T1T2

T2T3


60


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

wait

Waiting Cycle

T3T1

T1T2

T2T3


61


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

checkCycle

Waiting Cycle

T3T1

T1T2

T2T3


62


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

checkCycle

Waiting Cycle

T3T1

T1T2

T2T3


63


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

checkCycle

Waiting Cycle

T3T1

T1T2

T2T3


64


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

checkCycle

Waiting Cycle

T3T1

T1T2

T2T3


65


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

checkCycle

Waiting Cycle

T3T1

T1T2

T2T3


66


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

checkCycle

Waiting Cycle

T3T1

T1T2

T2T3


67


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

checkCycle

Waiting Cycle

T3T1

T1T2

T2T3


68


C1

WS1WS6

T1

WS5WS3WS2

T2 T3

C2C3

P12 P34

P56

WS4

checkCycle

Waiting Cycle

T3T1

T1T2

T2T3


69


Experimental Set Up

num of Tx = 100 num of Providers = 1num of WS (10000, 8000, 6000, 4000, 2000) length of each transaction = [4,12]each WS is executed in 2 secondsthere are 2 seconds delay between every 2

ws invocations at the client sideIBM Thinkpad T6x, Genuine Intel CPU, T2500 @ 2.00

GHz, 2.00 GB RAM, Microsoft Windows XP Professional, Version 2002, Service Pack 2, JVM: J2SE 1.5.0_07

70


Throughput in Running Time

Throughput with 100Tx

0

0,01

0,02

0,03

0,04

0,05

0,06

0,07

0,08

0,09

0,00

31,0

0

60,0

0

91,0

0

121,

00

152,

00

182,

00

213,

00

244,

00

274,

00

305,

00

335,

00

366,

00

397,

00

425,

00

456,

00

486,

00

517,

00

547,

00

578,

00

609,

00

639,

00

670,

00

700,

00

731,

00

762,

00

790,

00

821,

00

851,

00

882,

00

912,

00

943,

00

974,

00

1004

,00

1035

,00

1065

,00

1096

,00

1127

,00

1155

,00

1186

,00

Running Time

nu

m.

of

term

inat

ed t

ran

sact

ion

s p

er s

ec.

Throughput

71


Throughput and Nr. of WS

Throughput

0

0,2

0,4

0,6

0,8

1

1,2

1,4

1,6

1,8

6000

5800

5600

5400

5200

5000

4800

4600

4400

4200

4000

3800

3600

3400

3200

3000

2800

2600

2400

2200

2000

1800

1600

1400

1200

1000 800

600

400

200

Number of Webservices

Nu

m o

f T

erm

inat

ed T

ran

sact

ion

s p

er s

eco

nd

Throughput

MaxThroghput

minThroughput

72


Throughput vs. Tx LenghtThroughput

0

0,5

1

1,5

2

2,5

4 - 8 6 - 10 8 - 12

Transaction Length (num of WS)

Nu

m o

f te

rmin

ated

Tra

nsa

ctio

ns

per

sec

on

d

Throughput

73


Tx Execution Times vs. Nr.of WS

Distribution of Transactions Execution Time

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

0 - 3

0s

30 -

39s

40 -

49s

50 -

59s

60 -

69s

70 -

79s

80 -

89s

90 -

99s

100

- 200

s

200

- 500

s

500

- 800

s

Transaction Execution Time(sec.)

nu

m o

f T

ran

sact

ion

s (%

)

3000WS6000WS1000WS

74


Conclusions and Further Work

We have proposed and experimented with a protocol for concurrency control in WS environment• Without direct communication between

coordinators• With information kept on the participant side• With cycle detection• Experiments shows that it is promising

We will look at the compensations, partial roll-backs, or redo for some operations in combination with our protocol

75

Documents

Advanced Web Service Transactions Management Peter Dolog [email protected] CS Department Intelligent Web Information Systems ://,