Several sets of slides by Prof. Jennifer Welch will be used in this course. The slides are mostly identical to her slides, with some minor changes.

Set 1: Introduction

1

DISTRIBUTED ALGORITHMS

Spring 2014Prof. Jennifer Welch

CSCE 668

Set 1: Introduction

2

Distributed Systems

Set 1: Introduction

3

Distributed systems have become ubiquitous: share resources communicate increase performance

speed fault tolerance

Characterized by independent activities (concurrency) loosely coupled parallelism (heterogeneity) inherent uncertainty

Uncertainty in Distributed Systems

Set 1: Introduction

4

Uncertainty comes from differing processor speeds varying communication delays (partial) failures multiple input streams and interactive

behavior

Reasoning about Distributed Systems

Set 1: Introduction

5

Uncertainty makes it hard to be confident that system is correct

To address this difficulty: identify and abstract fundamental problems state problems precisely design algorithms to solve problems prove correctness of algorithms analyze complexity of algorithms (e.g., time,

space, messages) prove impossibility results and lower bounds

Potential Payoff of Theoretical Paradigm

Set 1: Introduction

6

careful specifications clarify intent increased confidence in correctness if abstracted well then results are

relevant in multiple situations indicate inherent limitations

cf. NP-completeness

Application Areas

Set 1: Introduction

7

These areas have provided classic problems in distributed/concurrent computing: operating systems (distributed) database systems software fault-tolerance communication networks multiprocessor architectures

Newer application areas: cloud computing mobile computing, …

Basic Models

Set 1: Introduction

8

Introduce two basic communication models: message passing shared memory

and two basic timing models: synchronous asynchronous

Basic Models

Set 1: Introduction

9

Message passing Shared memory

synchronous

asynchronous

Yes No

Yes Yes

(Synchronous shared memory model is PRAM)

Relationship of Theory to Practice

Set 1: Introduction

10

time-shared operating systems: issues relating to (virtual) concurrency of processes such as mutual exclusion deadlock

also arise in distributed systems MIMD multiprocessors:

no common clock => asynchronous model common clock => synchronous model

loosely coupled networks, such as Internet, => asynchronous model

Relationship of Theory to Practice

Set 1: Introduction

11

Failure models: crash: faulty processor just stops.

Idealization of reality. Byzantine (arbitrary): conservative

assumption, fits when failure model is unknown or malicious

self-stabilization: algorithm automatically recovers from transient corruption of state; appropriate for long-running applications

Message-Passing Model

Set 1: Introduction

12

processors are p0, p1, …, pn-1 (nodes of graph)

bidirectional point-to-point channels (undirected edges of graph)

each processor labels its incident channels 1, 2, 3,…; might not know who is at other end

Message-Passing Model

Set 1: Introduction

13

1 1

2

2

1 1

3

2

p3

p2

p0

p1

Modeling Processors and Channels

Set 1: Introduction

14

Processor is a state machine including local state of the processor mechanisms for modeling channels

Channel directed from processor pi to processor pj is modeled in two pieces: outbuf variable of pi and inbuf variable of pj

Outbuf corresponds to physical channel, inbuf to incoming message queue

Modeling Processors and Channels

Set 1: Introduction

15

inbuf[1]

p1's localvariables

outbuf[1] inbuf[2]

outbuf[2]

p2's localvariables

Pink area (local vars + inbuf) is accessible state for a processor.

Configuration

Set 1: Introduction

16

Vector of processor states (including outbufs, i.e., channels), one per processor, is a configuration of the system

Captures current snapshot of entire system: accessible processor states (local vars + incoming msg queues) as well as communication channels.

Deliver Event

Set 1: Introduction

17

Moves a message from sender's outbuf to receiver's inbuf; message will be available next time receiver takes a step

p1 p2m3 m2 m1

p1 p2m3 m2 m1

Computation Event

Set 1: Introduction

18

Occurs at one processor Start with old accessible state (local vars

+ incoming messages) Apply transition function of processor's

state machine; handles all incoming messages

End with new accessible state with empty inbufs, and new outgoing messages

Computation Event

Set 1: Introduction

19

c d eoldlocalstate

a

newlocalstate

b

pink indicates accessible state: local vars and incoming msgswhite indicates outgoing msg buffers

Execution

Set 1: Introduction

20

Format is config, event, config, event, config, … in first config: each processor is in initial

state and all inbufs are empty for each consecutive (config, event,

config), new config is same as old config except: if delivery event: specified msg is transferred

from sender's outbuf to receiver's inbuf if computation event: specified processor's

state (including outbufs) change according to transition function

Admissibility

Set 1: Introduction

21

Definition of execution gives some basic "syntactic" conditions. usually safety conditions (true in every finite prefix)

Sometimes we want to impose additional constraints usually liveness conditions (eventually something

happens) Executions satisfying the additional constraints

are admissible. These are the executions that must solve the problem of interest. Definition of “admissible” can change from context to

context, depending on details of what we are modeling

Asynchronous Executions

Set 1: Introduction

22

An execution is admissible for the asynchronous model if every message in an outbuf is eventually delivered every processor takes an infinite number of steps

No constraints on when these events take place: arbitrary message delays and relative processor speeds are not ruled out

Models reliable system (no message is lost and no processor stops working)

Example: Flooding

Set 1: Introduction

23

Describe a simple flooding algorithm as a collection of interacting state machines.

Each processor's local state consists of variable color, either red or green

Initially: p0: color = green, all outbufs contain M others: color = red, all outbufs empty

Transition: If M is in an inbuf and color = red, then change color to green and send M on all outbufs

Example: Flooding

Set 1: Introduction

24

p1

p0

p2

M M

p1

p0

p2

MM

deliver eventat p1 from p0

computationevent by p1

deliver eventat p2 from p1

p1

p0

p2

M

M

M M

p1

p0

p2

M

M

computationevent by p2

Example: Flooding (cont'd)

Set 1: Introduction

25

deliver eventat p1 from p2

computationevent by p1

deliver eventat p0 from p1

etc. to deliverrest ofmsgs

p1

p0

p2

M

M M

M

p1

p0

p2

M

M M

M

p1

p0

p2

M

M M

p1

p0

p2

M

M

M

Nondeterminism

Set 1: Introduction

26

The previous execution is not the only admissible execution of the Flooding algorithm on that triangle.

There are several, depending on the order in which messages are delivered.

For instance, the message from p0 could arrive at p2 before the message from p1 does.

Termination

Set 1: Introduction

27

For technical reasons, admissible executions are defined as infinite.

But often algorithms terminate. To model algorithm termination, identify

terminated states of processors: states which, once entered, are never left

Execution has terminated when all processors are terminated and no messages are in transit (in inbufs or outbufs)

Termination of Flooding Algorithm

Define terminated processor states as those in which color = green.

Set 1: Introduction

28

Synchronous Message PassingSystems

Set 1: Introduction

29

An execution is admissible for the synchronous model if it is an infinite sequence of "rounds"

What is a "round"? It is a sequence of deliver events that

move all messages in transit into inbuf's, followed by a sequence of computation events, one for each processor.

Synchronous Message Passing Systems

Set 1: Introduction

30

The new definition of admissible captures lockstep unison feature of synchronous model.

This definition also implies every message sent is delivered every processor takes an infinite number of

steps. Time is measured as number of rounds

until termination.

Example of Synchronous Model

Set 1: Introduction

31

Suppose flooding algorithm is executed in synchronous model on the triangle.

Round 1: deliver M to p1 from p0

deliver M to p2 from p0

p0 does nothing (as it has no incoming messages) p1 receives M, turns green and sends M to p0 and

p1

p2 receives M, turns green and sends M to p0 and p1

Example of Synchronous Model

Set 1: Introduction

32

Round 2: deliver M to p0 from p1 deliver M to p0 from p2

deliver M to p1 from p2

deliver M to p2 from p1

p0 does nothing since its color variable is already green

p1 does nothing since its color variable is already green

p2 does nothing since its color variable is already green

Example of Synchronous Model

Set 1: Introduction

33

p1

p0

p2

M

MM

M

p1

p0

p2

M M

p1

p0

p2

round 1 events

round 2events