Deciding Choreography Reliazability

Deciding Choreography Reliazability

Samik Basu

Iowa State University

Tevfik Bultan

University of California at Santa Barbara

Meriem Ouederni

University of Malaga

Motivation 1: Web Services

• Web services support basic client/server style interactions

• Example: Amazon E-Commerce Web Service (AWS-ECS)

Service Requester

Service Provider

Request

Response

SOAP

WSDL

Client Server

Service Composition via Choreography

• Can we compose a set of services to construct a new service?• For example:

– If we are building a bookstore service, we may want to use both Amazon’s service and Barnes & Noble’s service in order to get better prices

Choreography: A specification of how the individual services that participate to a composite service should interact with each other

• A choreography is a global specification of interactions among services

• Web Services Choreography Description Language (WS-CDL)

Motivation 2: Singularity OS

• Experimental OS developed by Microsoft Research to explore new ideas for operating system design focusing on dependability

• Software Isolated Processes (SIPs)– Closed code space (no dynamic code loading or code generation)– Closed object space (no shared memory)

• Inter-process communication occurs via message passing over channels

• Singularity channels allow 2-Party asynchronous communication via FIFO message queues– Sends are non blocking– Receives block until a message is at the head of a receive queue

Singularity Channel Contracts

• Written in Sing #• Contracts specify two things:

1. The messages that may be sent over a channel• out message are sent from the

Server endpoint to the Client endpoint (SC)

• in messages are sent from the Client endpoint to the Server endpoint (CS)

– The set of allowed message sequences• out message marked with !• in messages marked with ?

public contract KeyboardDeviceContract { out message AckKey( uint key ); out message NakKey(); out message Success();

in message GetKey(); in message PollKey();

state Start { Success! -> Ready; }

state Ready { GetKey? -> Waiting; PollKey? -> (AckKey! or NakKey!) -> Ready; }

state Waiting { AckKey! -> Ready; NakKey! -> Ready; }}

Motivation 3: Erlang

• Erlang is a general purpose programming language developed initially at Ericsson for improving dependability of telephony applications

• In Erlang distributed processes do not share memory and only interact with each other via exchanging messages asynchronously

• UBF(B) is a language for specifying communication contracts in distributed Erlang programs.

• UBF(B) specifications list

transitions between states where

each transition is identified with a

request (the message received)

and response (the message sent)

+NAME(“IRC SERVER”)... +STATE start logon() => ok() & active

| error() & stop

+STATE active ls() => files() & active getFile() => fileSent() & active

| noFileErr() & stop...

Common: Asynchronous Messaging

• Sender does not have to wait for the receiver– Message is inserted to a message queue– Messaging platform guarantees the delivery of the message

• Why support asynchronous messaging?– Otherwise the sender has to block and wait for the receiver – Sender may not need any data to be returned– If the sender needs some data to be returned, it should only wait

when it needs to use that data– Asynchronous messaging can alleviate the latency of message

transmission through the Internet– Asynchronous messaging can prevent sender from blocking if the

receiver service is temporarily unavailable• Rather then creating a thread to handle the send, use

asynchronous messaging

Common: Conversations

• Specifications of message-based asynchronous communication– Web Service Choreography Specifications: Global specification

of interactions for composition of services– Singularity Channel Contracts: Coordinating inter-process

communication in Singularity OS– Erlang Communication Contracts: Coordinating interactions

among distributed processes

• All these specifications can be modeled as state machines and they all specify sequences of send actions (aka, conversations):

Conversation: A sequence of send actions

Conversation Protocol (aka Choreography): Specifies a set of conversations

public contract KeyboardDeviceContract { out message AckKey( uint key ); out message NakKey(); out message Success();

in message GetKey(); in message PollKey();

state Start { Success! -> Ready; }

state Ready { GetKey? -> Waiting; PollKey? -> (AckKey! or NakKey!) -> Ready; }

state Waiting { AckKey! -> Ready; NakKey! -> Ready; }}

• A Singularity channel contract corresponds to a finite state machine• Each message causes a deterministic transition from one state to

another state

KeyboardDeviceContract

Example Singularity Channel Contract

Start

Ready$0ReadyWaiting

SC:Success

SC:AckKey

SC:AckKey

CS:GetKey

CS:PollKey

SC:NakKey

SC:NakKey

Implicit State

• Each contract state machine specifies a set of conversations, i.e., it is a conversation protocol:

KeyboardDeviceContract

Example Singularity Channel Contract

Start

Ready$0ReadyWaiting

SC:Success

SC:AckKey

SC:AckKey

CS:GetKey

CS:PollKey

SC:NakKey

SC:NakKey

Success(GetKey(AckKey|NakKey)|PollKey(AckKey|NakKey))*

Conversation set:

Going to Lunch at UCSB

• At UCSB Samik, Meriem and I were using the following protocol for going to lunch:

– Sometime around noon one of us would call another one by phone and tell him where and when we would meet for lunch.

– The receiver of this first call would call the remaining peer and pass the information.

• Let’s call this protocol the First Caller Decides (FCD) protocol.

• At the time we did not have answering machines or voicemail due to budget cuts at UC!

FCD Protocol Scenarios

• Possible scenario

1. Tevfik calls Samik with the decision of where and when to eat

2. Samik calls Meriem and passes the information• Another scenario

1. Samik calls Tevfik with the decision of where and when to eat

2. Tevfik calls Meriem and passes the information• Yet another scenario

1. Tevfik calls Meriem with the decision of where and when to eat• Maybe Samik also calls Meriem at the same time with a

different decision. But the phone is busy.• Samik keeps calling. But Meriem is not going to answer

because according to the protocol the next thing Meriem has to do is to call Samik.

2. Meriem calls Samik and passes the information

FCD Protocol: Tevfik’s Behavior

Tevfik calls Samik with the lunch decision

Let’s look at all possible behaviors of Tevfik based on the FCD protocol

Tevfik is hungry

Tevfik calls Meriem with the lunch decision

Tevfik receives a call from Samik passing him the

lunch decision

Tevfik receives a call from Meriem passing him the

lunch decisionTevfik receives a call from Meriem telling

him the lunch decision that Tevfik has to pass

to Samik

FCD Protocol: Tevfik’s Behavior

!T->S:D

!T->M:D

?S->T:P

?M->T:P

?M->T:D?S->T:D

!T->S:P!T->M:P

T->S:D Tevfik calls Samik with the lunch decision

Message Labels:

! send

? receiveS->M:P

Samik calls Meriem to pass the decision

!T->S:D

?M->T:D

!T->M:D

?S->T:D

!T->M:P

Tevfik

!T->S:P

?S->T:P

?M->T:P

!M->S:D

?T->M:D

!M->T:D

?S->M:D

!M->T:P

Meriem

!M->S:P

?S->M:P

?T->M:P

!S->T:D

?M->S:D

!S->M:D

?T->S:D

!S->M:P

Samik

!S->T:P

?T->S:P

?M->S:P

State machines for the FCD Protocol

• Three state machines characterizing the behaviors of Tevfik, Meriem and Samik according to the FCD protocol

FCD Protocol Has Voicemail Problems

• After the economy started to recover, the university installed a voicemail system FCD protocol started causing problems– We were showing up at different restaurants at different times!

• Example scenario: – Tevfik calls Meriem with the lunch decision – Samik also calls Meriem with the lunch decision

• The phone is busy (Meriem is talking to Tevfik) so Samik leaves a message

– Meriem calls Samik passing the lunch decision• Samik does not answer (he already left for lunch) so Meriem

leaves a message– Samik shows up at a different restaurant!

• Message sequence is: T->M:D S->M:D M->S:P– The messages S->M:D and M->S:P are never consumed

• This scenario is not possible without voicemail!

A Different Lunch Protocol

• To fix this problem, I suggested that we change our lunch protocol as follows:

– As the most senior researcher among us I would make the first call to either Meriem or Samik and tell when and where we would meet for lunch.

– Then, the receiver of this call would pass the information to the other peer.

• Let’s call this protocol the Tevfik Decides (TD) protocol

?M->S:P?T->S:D

!S->M:P

Samik MeriemTevfik

?S->M:P?T->M:D

!M->S:P

!T->S:D !T->M:D

State machines for the TD Protocol

• TD protocol works fine with voicemail!

T->S:D

T->M:D

M->S:P

M->T:D M->S:DS->T:D

S->M:D

S->M:PT->S:P S->T:P T->M:P

M->T:P

FCD Protocol

T->S:D T->M:D

S->M:P M->S:P

TD Protocol

FCD and TD Conversation Protocols

Conversation set: { T->M:D M->S:P, T->S:D S->M:P, M->T:D T->S:P, M->S:D S->T:P, S->T:D T->M:P,

S->M:D M->T:P }

Conversation set: { T->S:D S->M:P, T->M:D M->S:P}

Observation & Question

• The implementation of the FCD protocol does not obey the FCD protocol if asynchronous communication is used

• Implementation of the TD protocol obeys the TD protocol even if asynchronous communication used

– Given a conversation protocol can we figure out if there is an implementation which generates the same conversation set?

Realizability

• Conversation protocols identify the global communication behavior– How do we implement processes that conform to the conversation

protocol?

• Realizability question:– Given a conversation protocol, are there processes whose

communication behavior in terms of conversations (i.e., send sequences) is equal to the set of conversations (i.e., send sequences) specified by the conversation protocol?

• The FCD protocol is unrealizable• The TD protocol is realizable

Conversations generated by some processes

Conversations specified by the conversation protocol

?

ConversationProtocol(ChoreographySpecification)

F(S->M:P M->S:P)? LTL property

InputQueue

...Conversation?

LTL property

Peer T Peer XPeer J

T->S:D T->M:D

S->M:P M->S:P

F(S->M:P M->S:P)

!T->S:D

!T->M:D

?M->S:P

?T->S:D

!S->M:P

?S->M:P?T->M:D

!M->S:P

Top-Down Verification

Unrealizable Conversation Protocols

AB: m1

CD: m2

AB: m1BA: m2

AC: m3

BA: m2

AB: m1

• There are unrealizable conversation protocols:

AB: m1

CA: m2

Unrealizable Examples

• Some conversation protocols are unrealizable!

AB: m1

CD: m2

Conversationprotocol

Conversation “m2 m1m2 m1” will be generated by all implementations which follow the protocol

!m1 ?m1 !m2 ?m2

Peer A Peer B Peer C Peer D

Projections of the protocol to the processes


• Some conversation protocols are unrealizable!

AB: m1

CA: m2

Conversationprotocol

Conversation “m2 m1m2 m1” will be generated by all implementations which follow the protocol

!m1?m1 !m2

?m2

Peer A

Peer B Peer C

Projections of the protocol to the processes


m2 m1 m3

m1

m2

m3

AB: m1BA: m2

AC: m3

BA: m2

AB: m1

A

B

C

m1m2

m3

Conversation:

Generated conversation:

B A, C

Challenge & Contribution

• Finite state processes that communicate with FIFO message queues can simulate Turing Machines– Checking conformance to a conversation protocol is undecidable

• We show that conversation protocol realizability problem is decidable

• We implemented the realizability check and applied it to many specifications – Demonstrated that realizability can be checked efficiently in

practice

Refining Realizability

• Just looking at equivalence of the conversation sets is not enough

Conversations generated by some processes

Conversations specified by the conversation protocol

?

Another Conversation Protocol

aP1->P2 aP1->P2

bP2->P1 cP2->P1 bP2->P1

A conversation protocol for 2 processes: P1 and P2

Projections on P1 and P2

!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Synchronous Communication

!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

aP1->P2

bP2->P1 cP2->P1


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

aP1->P2 aP1->P2

bP2->P1 cP2->P1 bP2->P1


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

aP1->P2 aP1->P2

bP2->P1 cP2->P1 bP2->P1


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

aP1->P2 aP1->P2

bP2->P1 cP2->P1 bP2->P1


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

aP1->P2 aP1->P2

bP2->P1 cP2->P1 bP2->P1

BLOCKED


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

aP1->P2 aP1->P2

bP2->P1 cP2->P1 bP2->P1

Conversations sets are equal but processes may get stuck

Asynchronous Communication

!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Queue:

Queue:


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Queue:

Queue: a

aP1->P2


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Queue:

Queue:

aP1->P2


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Queue:

Queue:

aP1->P2

bP2->P1


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Queue:

Queue: a

aP1->P2 aP1->P2

bP2->P1


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Queue:

Queue:

aP1->P2 aP1->P2

bP2->P1


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Queue: c

Queue:

aP1->P2 aP1->P2

cP2->P1 bP2->P1


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Queue: c

Queue:

aP1->P2 aP1->P2

cP2->P1 bP2->P1

Cannot consume c


!a !a

?b ?c ?b

?a ?a

!b !c !b

Process P1

Process P2

Queue: c

Queue:

aP1->P2 aP1->P2

cP2->P1 bP2->P1

Cannot consume c

Realizability Requirements

We refine our realizability condition to eliminate such cases.

We have two requirements for realizability:

1.Conversations specified by the conversation protocol = Conversations generated by the asynchronous system

2.Asynchronous system is well-formed:

All sent messages can be eventually consumed

Conversation protocol is realizable if and only if there exists such an asynchronous system

Summary of our contribution

Conversation protocol: C

Asynchronous System• with unbounded buffer: I

• with k size communication buffer: Ik

Synchronous System: I0

C is realizable if and only C is equivalent to determinized I1 obtained from projections of C

Determinizing Projections

?a ?a

!b !c !b

Process P2

!a !a

?b ?c ?b

Process P1

Determinizing Projections

!a !a

?b ?c ?b

?a ?a

!b !c !b

Peer P1

Peer P2

!a

?c ?b

?a

!c !b

Observation 1: Behavioral Order

• Behavior exhibited by projections when communicating synchronously is larger than the conversation

• Behavior exhibited by projections when communicating asynchronously is larger than that exhibited by projections when communicating synchronously

C ≤ I0 ≤ I1 ≤ I2 ≤ … ≤ I

Observation 1: Behavioral Ordering

aP1->P2

cP2->P1

bP3->P4!a ?c ?a !c

P2

!b

P3

?b

P4P1


aP1->P2

cP2->P1

bP3->P4!a ?c

Synchronous System

?a !c

P2

!b

P3

?b

P4

aP1->P2

bP3->P4

cP2->P1

bP3->P4

bP3->P4

cP2->P1 aP1->P2

P1


aP1->P2

cP2->P1

bP3->P4!a ?c

Synchronous System

?a !c

P2

!b

P3

?b

P4

aP1->P2

bP3->P4

cP2->P1

bP3->P4

bP3->P4

cP2->P1 aP1->P2

aP1->P2

cP2->P1

Asynchronous System

P1

Observation 2: Synchronizability

A system is synchronizable if and only if its behaviors are identical for asynchronous and synchronous communication

For synchronizable systems:

Forall k ≥ 0: Ik is equivalent to I

I is synchronizable iff I0 is equivalent to I1

[WWW’11: Choreography Conformance via Synchronizability]

Observation 2: Synchronizability

?a ?a

!b !c !b

Peer P2

This system is synchronizablesince the asynchronous andsynchronous versions areequivalent in term of sequences of send actions but it is not well-formed

!a !a

?b ?c ?b

Peer P1

Observation 3: Synchronizability & Determinism

A synchronizable system that consists of deterministic processes is well-formed (all sent messages are eventually consumed)

Observation 3

?a ?a

!b !c !b

Peer P2Synchronizablebut not well-formed

Synchronizableand well-formed !a

?c ?b

?a

!c !b

!a !a

?b ?c ?b

Peer P1

Outline of the Realizability Check

• Project conversations to processes

• Determinize peers

• Check equivalence between conversation C and I1

– C = I1 if and only if I is synchronizable [Obs 1, 2] and C = I

– C = I1 implies I is well-formed [Obs 3]

• C = I1 if and only if C is realizable

Implementation

• Implemented using CADP toolbox– Automatically generate a LOTOS specification for the conversation

protocol– Generate determinized projections (in LOTOS)– Check equivalence of the 1-bounded asynchronous system and the

conversation protocol

• Checked realizability of – 9 web service choreography specifications

• 8 are realizable– 9 collaboration diagrams

• 8 are realizable– 86 Singularity channel contracts

• 84 are realizable

• Realizability check takes about 14 seconds on average

Related Work

• Sufficient conditions for realizability:– [Fu et al. TCS’04] Conversation Protocols

• [Honda et al. POPL’08] has similar conditions for session types– Arbitrary Initiators are not allowed: Conversation protocol cannot

have two different peers initiating send actions from the same state– [Stengel and Bultan ISSTA’09]: Application of sufficient realizability

conditions to checking Singularity channel contracts– [Halle and Bultan FSE’10]: more relaxed sufficient condition that

allows arbitrary initiators • [Kazhamiakin, Pistore FORTE’06]: Realizability for restricted

communication models• [Lohmann, Wolf ICSOC’11]: Shows decidability of realizability with

unbounded asynchronous communication when messages are not ordered (i.e., FIFO requirement is dropped)!

Related Work

• Message Sequence Charts (MSC)– [Alur, Etassami, Yannakakis ICSE’00, ICALP’01] Realizability of

MSCs and MSC Graphs• Defines similar notion of realizability

– [Uchitel, Kramer, Magee ACM TOSEM 04] Implied Scenarios in MSCs

– Different conversation model

Related Work

Results on synchronizability:• [Fu et al. TSE’05]: Sufficient conditions for synchronizability • [Basu and Bultan WWW’11]: Necessary and sufficient condition for

synchronizability• [Basu, Bultan, Ouderni VMCAI’12]: Synchronizability considering send

sequences + reachability of synchronized states • [Manohar, Martin MPC 98] Slack elasticity

– Presents conditions under which changing the size of communication queues does not effect the behavior of the system

– Behavior definition also takes the decision points into account in addition to message sequences

– It gives sufficient conditions for slack elasticity and discusses how to construct systems to ensure slack elasticity

Related Work

• Singularity:– [Hunt, Larus SIGOPS ‘07] Singularity: rethinking the software stack– [Fähndrich, Aiken, Hawblitzel, et. al SIGOPS/Eurosys ‘07]

Language support for fast and reliable message-based communication in singularity os.

– Influenced by work on Session Types• [Honda, Vasconcelos, Kubo ESOP ’98] Language primitives

and type discipline for structured communication-based programming

– Source code and RDK: http://codeplex.com/singularity

Future Directions

• Choreography realizability for other communication models

• Analyzing failure of realizability– Correcting unrealizable choreographies with minimal changes to

the choreography

THE END

Documents

Deciding Choreography Reliazability