Deterministic Distributed Resource Discovery

Shay Kutten TechnionDavid Peleg Weizmann Inst.Uzi Vishkin Univ. of Maryland & Technion

Distributed Resource Discovery [HLL99]

pointer

The corresponding directed graph (on nodes A,B,C):

(assumption: weakly connected)Problem: compute the connected component

router

A

C

B

A B C

Distributed Resource Discovery-[Harcol-Balter, Leighton, Lewin 1999]

“)Names Dropper” protocol(

Licensed to AKAMAI, a leading Web caching provider

The resource discovery task: the set of servers changes Find the current set (towards deciding which Akamai server to assign to each user)

Akamai servers need to learn how to reach each other via the Internet (towards deciding which Akamai server is “close” to the user).

Akamai Server

Internet

user

CNN

Think Distributed

A B

a pointer used by the algorithm (one per node, initially pointing at the node itself)

A Address “known” to Node A (“knowledge graph”)

Mr. and Ms. B132.12.16.64 IP street

Interland

a pointer used by the algorithm

A?

When the Algorithm is started

A

C

D

B

Weakly connected directed graph

a pointer used by the algorithm (one per node, initially pointing at the node itself)

A Address “known” to Node A (“knowledge graph”)

Algorithm Actions

A

C

D

BC

(1) Node A learns C’s addr. From B, thus, directed edge (A C) is added to knowledge graph.

Algorithm Actions

A

C

D

BC

(1) Node A learns C’s addr. From B, thus, directed edge (A C) is added to knowledge graph.

(2) Node D who knows of C, change PTR to point at C, thus, pointer graph changes.

D

Algorithm Actions

A

C

D

BC

(1) Node A learns C’s addr. From B, thus, directed edge (A C) is added to knowledge graph.

(2) Node D who knows of C, change PTR to point at C, thus, pointer graph changes.

D

change

Algorithm’s result

pointers form star, knowledge graph is strongly connected,Plus, possibly, additional links (possibly clique) .

Model [HLL99]- synchronous (not necessary for our alg)

- simultaneously start (for our alg: just “close”)

- complete comm. Graph

Complexity: #Msgs (connections [HLL99]),

#bits (#pointers*log n), time (#rounds)

Motivation (2) GnutellaNapster server (mp3 music files)

I have “Love Song”I want “Love Song”

I have “Madonna”

Motivation (2) Gnutella

Napster server

I have Love SongI want Love Song

A

Ask A

Motivation (2) Gnutella

Napster server

I want Love SongA

Ask A

Motivation (2) Gnutella

Napster server

A

Ask A

Love Song

Motivation (2) Gnutella vs. Napster

Napster serverIf the court closes The service collapsed

Motivation (2) Gnutella vs. Napster

Napster serverIf the court closes The service collapsed

Motivation (2) Gnutella vs. Napster

Napster server If the court closes the service collapses

Motivation (2) Gnutella vs. Napster

Napster serverIf the court closes the service collapses

Gnutella: attempts tosolve by having everyclient=server

Same problem: user knows some others find more increase connectivity to withstand disconnections

Motivation(2) Gnutella

Users get disconnected often

Bob

Carol

Initially Alice knows only Bob & Carol (personally, or from ICQ, or from Google…) and can connect via themTo others they know.

Alice

David

Motivation(2) gnutellaUsers get disconnected often

Bob

Carol

Initially Alice knows only Bob & Carol (personally, or from ICQ, or from Google…) Alice learned from Carol how to reach David, so that when Bob & Carol are not online Alice is still connected.

Alice

David

Motivation: “Peer to Peer” Applications(P2P)

(1) Akamai http://www.akamai.com 1999(2) Gnutella http://gnutella.wego.com (originally

Nullsoft (Winamp)) 2000 (3) Gpulp: general Purpose Location Protocol http://www.gpulp.com (4) Genny http://www.gnutelladev.com (5) Freenet http://freenet.sourceforge.net 2000(6) JXTA http://www.jxta.org (Sun Microsystems) April 2001(7) Retsina: Discovery of Infrastructure in Multi-Agent

Systems (CMU)

This paper [HLL99]

deterministic randomized termination detection no detection O(log n) rounds O(log n)

O(n log n) messages O(n log n)

O(E log n) bits O(n log n)

2

2

2 320

Algorithm strategies

D

E

F

G

H

I

(1) Shrink tall trees to make them stars [SV82]

12

5

(2) Merge stars to get one tree (changed from [SV82])

(3) Carefully connect weakly connected(explained later; one of the differences from]Shiloach, Vishkin 82[)

Example of a technique taken from ]SV82[

(Handling tall trees in “pointer graph”) D

E

F

G

H

I

C

B

AShortcuts ]SV82[

D

E

F

G

H

I

C

B

A

Handing tall trees in “pointer graph” (cont.)

Shortcuts (]SV82[

Each child learnsthe pointer to its grandparent

D

E

F

G

H

I

C

B

A

Shortcuts

Handing tall trees in “pointer graph” (cont.)

D

E

F

G

H

I

C

B

A

Algorithm: finding who to merge with ]SV82[

C

E,E,F,G,H,I,C

node I, on becoming a child of root D, tells D about C

Add edges to knowledge graph:

Algorithm (2) handling active star roots

12

5join

3

join

Star roots ask to connect

24jo

in

Algorithm (2) handling active star roots

12

5join

3

join

-An undirected subgraph is created by joinMessages. -Each root connects to local minima (ID) rootOn undirected subgraph.

24jo

in

Algorithm (2) handling active star roots

12

5

3-An undirected subgraph is created by joinMessages. -Each root connects to local minima (ID) rootOn undirected subgraph.

24

Algorithm (2) handling active star roots

9

7 24

20

12

join

(note differences from [SV82])

Non-stars do not join

Algorithm (2) handling active star roots

9

7 24

20

12

5join

3

joinjoin

Join smallest id-prevents cycles

5 joins 12!!! : smallest

id neighbor

(only stars join, as opposed to [SV82])

Algorithm (2) handling active star roots

12

9

7 24

20

5join

3

join

join

16join

Algorithm (2) handling active star roots

12

9

7 24

20

5join

3

join

join

16join

Algorithm (2) handling active star roots

12

9

7 24

20

5join

3

join

join

16join

Algorithm handling passive star roots

24

5join

9

Node 9 is “passive” since it does not know any Node it cannot initiate any joining

Algorithm handling passive star roots

12

9

7 24

20

5join

3

join

8join

Node 12 is passive:

It does not know Any non-child.

It cannot Initiate any

progress.

Algorithm handling passive star roots

12

9

7 24

20

5join

3

join

8join

Node 12 joins its lowest “suitor” 3.Suitors of 12 join 3 too.

Algorithm: An Example of a Techniquedifferent than [SV82]

(weakly connected directed graph)Without an additional technique, may take (n) time:

A

Worst case (only A is active, its star grows by 1per phase):

Phase 1

(weakly connected diagraph)

Without an additional technique, may take (n) time:

A

Worst case (only A is active, its star grows by 1per phase):

phase 2

(weakly connected diagraph)

Without an additional technique, may take (n) time:

A

Worst case (only A is active, its star grows by 1per phase):

phase n-1

(weakly connected diagraph)

Idea: star root A picks 2 neighbors (not just 1) in phase i.(In phase i there are only n/2 stars)

i

Prevent (n) time, and still do not send too many messages:

Example: i=2 2 =4

A

i

i

(weakly connected diagraph)

Idea: A picks 2 neighbors (not just 1) in phase i.i

Prevent (n) time:

Some delicate points: resolving collisions, mixed topologies prove convergence in O(log n) phases

A

CorrectnessLemma: No cycles are created (hook on smaller + non-star does not connect)

Assured by a new technique:

37Possible because ofWeak connectivity!!

14

3 7 50

30

200

No connection No cycle

Lemma: always some active star

Complexity

Two kinds of progress in each phase:

(1) A tall tree gets shallower by a constant factor.

Obstacles:- # of star trees may grow because of (1)- Trees may get taller because of (2)

But (with the right combination) there is a progress in the combination.

(2) Active star roots merge.

Conclusion

- improved complexity in all measures (verifies [HLL99] conjecture, forecasting a simple algorithm with these complexities).- Deterministic.- Terminates (answers an open problem of Lipton).

Further research

- Adaptive algorithms (vs. “one shot”) - Lower bounds? (in [CGK95] O(n) msgs for undirected case).