1

Performance Analysis of the novel

Virtually Synchronous Group Communication Service

VS GCS

…

2

• Group Abstraction– processes interact in a group

– dynamic: fail/join/partition/merge

• Group Multicast– enforces certain ordering and reliability

• Group Membership – tells each process who it is connected to: current “view”

• Virtual Synchrony – integrates Group Multicast and Group Membership

– synchronizes message and view deliveries

Group Communication -- Useful “Building Block”

MBRSHPMCAST

VSVSview view

recv

3

Example: Group Communication

p

r

q

tim e

views

sendm deliver

m

m3,{p,q,r}

4 ,{q ,r}

4 ,{q ,r}

4 ,{p}

m3,{p,q,r}

3 ,{p ,q,r}

4

Virtual Synchrony

• Synchronization of Messages and Views:

• Before delivering new view v to its client, process has to synchronize with others

• Powerful abstraction for replication• Semantics: VS [Birman, Joseph 87], EVS, SVS

Processes that transition together through same views, deliver same sets of messages.

5

Example: Virtual Synchrony

p

r

q

time

views

sendm

m

m3,{p,q,r}

4 ,{q ,r}

4 ,{q ,r}

4 ,{p}

3,{p ,q,r}

3 ,{p ,q,r}

m

deliver

VS algorithm executes: r learns it missed m and delivers m

6

Performance Challenges in WANs• Clients care how fast the GCS delivers new views

after a network event (NE) occurs• After NE: MBRSHP forms view, VS synchronizes

View FormationVS AlgorithmNE

View

Delivered

• Existing systems (were designed for LANs):– Both MBRSHP and VS several rounds of msg exchange– Once begin, unable to respond to NEs -- “obsolete views”

• They are inappropriate for WANs, which typically have

– high and unpredictable message latency, – frequent connectivity changes

7

Novel Algorithm for Virtual Synchrony[Keidar, Khazan 00]

• Single message exchange among procs of new view• In parallel to MBRSHP forming the new view• No obsolete views -- reacts to membership changes

View FormationVS AlgorithmNE

View

Delivered

• Scalable architecture: MBRSHP is decoupled from VS

• Formal modeling: specs, algs, safety & liveness proofs

• Can work with novel MBRSHP service [Keidar, et al 00]

– View delivery in one message exchange in almost all cases

8

Challenges of Formal Performance Analysis

• The GCS system is a composition of building blocks– Multicast service, Membership service, VS end-points

• Clients care about performance of the whole system– How fast after a network event new views are delivered

• But our design focuses on the novel VS algorithm– Reduces the number of communication rounds to one,

which are in parallel with MBRSHP forming new view

• How can analyze improvement due to novel VS?– If MBRSHP and MCAST services are only specified

• How to compare performance with existing GCSs?– If existing GCSs blend MBRSHP and VS together

9

Performance Analysis: The Plan

Analyze the VS layer only– in terms of its inputs

– and timing assumptions

State reasonable performanceproperties of MBRSHP and MCAST

Compose with to yield conditional Corollaries– “Provided holds, the whole system performs like this…”

Next step: Compare with existing VS GCSs

MBRSHPMCAST

VSVS

view view

recv

10

Analysis of the VS layer• Assume component C stabilizes after some time on:

– MBRSHP delivers same views to VS end-points of C Let Tmax[MBRSHP.start] and Tmax[MBRSHP.view] be last events

– MCAST provides reliable and timely communication Let be message latency

• Liveness proof establishes that VS delivers views• Upper-bound the times when VS outputs views• In terms of the times when MBRSHP outputs occur• Conditional on timing assumptions (local speed: )

Tmax[MBRSHP.start] + + x +

Tmax[MBRSHP.view]

Tmax[VS.view] max

11

Illustration

MBRSHP algorithm

VS AlgorithmNE

start

view

One msg latency + xlast

view

lastlast

Tmax[MBRSHP.start] Tmax[MBRSHP.view]

Tmax[VS.view]

start

first

12

Bounds on MBRSHPReasonable bounds on the times of MBRSHP events

MBRSHP algorithm

NE

start

view

lastlast

start

first

~One msg latency ~One msg latency

One all-to-all msg latency

These bounds correspond to Fast-Path of [Keidar, et al 00]

observed empirically in almost all cases

Tmax[MBRSHP.start] Tmax[MBRSHP.view]Tmin[MBRSHP.start]

13

Compose MBRSHP and VS boundsBounds on the whole system, conditional on MBRSHP

MBRSHP algorithm

NE

start

view

lastlast

start

first

~One msg latency ~One msg latency

One all-to-all msg latency

Tmax[MBRSHP.start] Tmax[MBRSHP.view]Tmin[MBRSHP.start]

VS AlgorithmOne msg latency + x

last

view

Tmax[VS.view]

14

Next Step: Comparison with existing GCSs

• Existing VS algorithms all use similar ideas– Pre-agree on common identifiers. Deliver obsolete views

• Formulate a high-level description of existing algs– Requires looking at inherent costs (e.g., all-to-all latency)

• Analyze under the same scenarios and conditions

• Express performance advantages due to:– Faster VS algorithm that doesn’t pre-agree on common ids

– Not wasting time on obsolete views