Upload
horace-brown
View
219
Download
1
Tags:
Embed Size (px)
Citation preview
29 July 2005 [email protected] 3
Background
• DKS System (http://dks.sics.se/)
– Were initially replicating on successor-lists (Similar to Chord/Koorde)
– Turned out to have several problems…
29 July 2005 [email protected] 4
Outline
• Background• Proposed scheme: symmetric
replication• Extensions• Related Work & Summary
29 July 2005 [email protected] 5
What is Replication on the Successor-List?
• In one sentence:
– Replicate node n’s data to its f closest
neighbors for replication degree f
29 July 2005 [email protected] 6
Successor-list Example: Chord
A Chord-alike system, with nodes 0, 3, 5, 10, 11
0 12
1514
13 3
12
11
4
5
6
9 87
10
29 July 2005 [email protected] 7
Successor-list Example: Chord
With a replication factor of f=1
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11
Data: 6, 7, 8, 9, 10
29 July 2005 [email protected] 8
Successor-list Example: Chord
With a replication factor of f=2
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11
Data: 6, 7, 8, 9, 10
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 5: 4, 5
Node 10: 6, 7, 8, 9, 10
Node 11: 11
29 July 2005 [email protected] 9
Successor-list Example: Chord
With a replication factor of f=3
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11
Data: 6, 7, 8, 9, 10
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 5: 4, 5
Node 10: 6, 7, 8, 9, 10
Node 11: 11
Node 11: 11
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 5: 4, 5
Node 10: 6, 7, 8, 9, 10
29 July 2005 [email protected] 10
Successor-list Example: Chord
With a replication factor of f=4
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11
Data: 6, 7, 8, 9, 10
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 5: 4, 5
Node 10: 6, 7, 8, 9, 10
Node 11: 11
Node 11: 11
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 5: 4, 5
Node 10: 6, 7, 8, 9, 10
Node 11: 11
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 5: 4, 5
Node 10: 6, 7, 8, 9, 10
29 July 2005 [email protected] 11
Problem: Dynamism
What happens in case of leaves or failures?f=4
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11, 10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 10: 6, 7, 8, 9, 10
Node 11: 11
Node 11: 11
Node 0: 12, 13, 14, 15, 0
Node 5: 4, 5
Node 10: 6, 7, 8, 9, 10
Node 11: 11
Node 3: 1, 2, 3
Node 5: 4, 5
Node 10: 6, 7, 8, 9, 10
29 July 2005 [email protected] 12
Problem: Dynamism
What happens in case of leaves or failures?f=4
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11, 10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 11: 11,10,9,8,7,6
Node 11: 11,10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 5: 4, 5
Node 11: 11Node 3: 1, 2, 3
Node 5: 4, 5
29 July 2005 [email protected] 13
Problem: Dynamism
Need to move items to restore replication degree!
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11, 10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 11: 11,10,9,8,7,6
Node 11: 11,10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 5: 4, 5
Node 11: 11,10, 9, 8, 7, 6
Node 3: 1, 2, 3
Node 5: 4, 5
Get items
Data: 4, 5
Node 5: 4, 5
29 July 2005 [email protected] 14
Problem: Dynamism
Need to move items to restore replication degree!
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11, 10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 11: 11,10,9,8,7,6
Node 11: 11,10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 5: 4, 5
Node 11: 11,10, 9, 8, 7, 6
Node 3: 1, 2, 3
Node 5: 4, 5
Get items
Data: 1, 2, 3
Node 5: 4, 5
Node 3: 1, 2, 3
29 July 2005 [email protected] 15
Problem: Dynamism
Need to move items to restore replication degree!
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11, 10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 11: 11,10,9,8,7,6
Node 11: 11,10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 5: 4, 5
Node 11: 11,10, 9, 8, 7, 6
Node 3: 1, 2, 3
Node 5: 4, 5
Get itemsData: 12, 13, 14, 15, 0
Node 5: 4, 5
Node 3: 1, 2, 3
Node 0: 12,13,14,15,0
29 July 2005 [email protected] 16
Problem: Dynamism
Need to move items to restore replication degree!
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 12, 13, 14, 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 11, 10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 3: 1, 2, 3
Node 11: 11,10,9,8,7,6
Node 11: 11,10, 9, 8, 7, 6
Node 0: 12, 13, 14, 15, 0
Node 5: 4, 5
Node 11: 11,10, 9, 8, 7, 6
Node 3: 1, 2, 3
Node 5: 4, 5
Get items
Data: 10, 9, 8, 7, 6
29 July 2005 [email protected] 17
Problem: Dynamism
• Every join/leave/failure requires: (f) messages, to restore a replication
degree of f
– Need coordination of the restoration• What if the coordinator fails?
29 July 2005 [email protected] 18
Problem: Performance & Security
• How do we contact 2nd, 3rd, 4th… replica? – Have to contact the 1st replica!
• The 1st replica is– Bottleneck – Can fail– Can be malicious (distributed voting
useless)
29 July 2005 [email protected] 19
Outline
• Background• Proposed scheme: symmetric
replication• Extensions• Related Work & Summary
29 July 2005 [email protected] 20
Symmetric Replication
Goal of Symmetric Replication:– Simple algorithms
• Each join/leave should require O(1) messages
– Enable concurrent requests• Load-balancing without a bottleneck
– Increase security• Use distributed voting
– Add proximity information• Route through closest replica
29 July 2005 [email protected] 21
Symmetric Replication
Idea:– For replication degree f, partition the
identifier space into m equivalence classes:
• Use the congruence classes modulo f
– The cardinality of each class is f
– Each node replicates the equivalence class of every identifier it is responsible for
29 July 2005 [email protected] 22
Symmetric replication
Replication degree f=4, Id={0,…,15}• 4 equivalence classes:
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 14, 13, 12, 11
Data: 6, 7, 8, 9, 10
0, 4, 8, 12
1, 5, 9, 13
2, 6, 10, 14
3, 7, 11, 15
29 July 2005 [email protected] 23
Symmetric replication
Replication degree f=4, Id={0,…,15}• 4 equivalence classes:
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 14, 13, 12, 11
Data: 6, 7, 8, 9, 10
0, 4, 8, 12
1, 5, 9, 13
2, 6, 10, 14
3, 7, 11, 15
Data: 10, 9, 8, 7
Data: 11, 12
Data: 13, 14, 15
Data: 0, 1
Data: 2, 3, 4, 5, 6
Data: 6, 5, 4, 3
Data: 2, 1, 0, 15Data: 7, 8
Data: 3, 4
Data: 9, 10, 11
Data: 5, 6, 7
Data: 12, 13
Data: 8, 9
Data: 14, 15, 0, 1, 2
Data: 10, 11, 12, 13, 14
29 July 2005 [email protected] 24
Symmetric replication: joins!
Replication degree f=4, Id={0,…,15}• 4 equivalence classes:
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 14, 13,
Data: 6, 7, 8, 9, 10
0, 4, 8, 12
1, 5, 9, 13
2, 6, 10, 14
3, 7, 11, 15
Data: 10, 9
Data: 11, 12
Data: 13, 14, 15
Data: 0, 1
Data: 2, 3, 4, 5, 6
Data: 6, 5
Data: 2, 1Data: 7, 8
Data: 3, 4
Data: 9, 10, 11
Data: 5, 6, 7
Data: 12, 13
Data: 8, 9
Data: 14, 15, 0, 1, 2
Data: 10, 11, 12, 13, 14
Data: 11, 12
Data: 7, 8
Data: 3, 4
Data: 0, 15
Get items
Data: 11, 12, 7, 8, 3, 4, 0, 15
29 July 2005 [email protected] 25
Symmetric replication: failures
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 14, 13,
Data: 6, 7, 8, 9, 10
0, 4, 8, 12
1, 5, 9, 13
2, 6, 10, 14
3, 7, 11, 15
Data: 10, 9
Data: 11, 12
Data: 13, 14, 15
Data: 0, 1
Data: 2, 3, 4, 5, 6
Data: 6, 5
Data: 2, 1Data: 7, 8
Data: 3, 4
Data: 9, 10, 11
Data: 5, 6, 7
Data: 12, 13
Data: 8, 9
Data: 14, 15, 0, 1, 2
Data: 10, 11, 12, 13, 14
Successor has to fetch [11], [12] from closest
replica
29 July 2005 [email protected] 26
Restoring Replication Degee: failures
• Assume a node responsible for (10, 14) fails– Classes [10]={1, 10, 19} [14]={5, 14, 23}
• We decide to fetch replicas from (1, 5)
• Use restricted broadcast to fetch (1, 5)– [ansary-iptps02, ghodsi-pdcs02]– Reliable, O(log R) time complexity to reach R nodes
• Several nodes might be responsible for (1, 5)– Given uniform distribution of node identifiers
E[peers in range (1,5)] = 1
29 July 2005 [email protected] 27
Outline
• Background• Proposed scheme: symmetric
replication• Extensions• Related Work & Summary
29 July 2005 [email protected] 28
Extensions to Symmetric Replication
•Augment the routing information to:–Add proximity awareness
•Example: node 2, f=3 [2]={2, 5, 8}
Routing Table: node 2
Finger 1 Succ(2+20)
Finger 2 Succ(2+21)
Finger 3 Succ(2+22)
Finger 4 Succ(2+23)
29 July 2005 [email protected] 29
Extensions to Symmetric Replication
•Augment the routing information to:–Add proximity awareness
•Example: node 2, f=3 [2]={2, 5, 8}
Routing Table: node 2
Finger 1 Succ(2+20
)Succ(5+20
)Succ(8+20
)
Finger 2 Succ(2+21
)Succ(5+21
)Succ(8+21
)
Finger 3 Succ(2+22
)Succ(5+22
)Succ(8+22
)
Finger 4 Succ(2+23
)Succ(5+23
)Succ(8+23
)
29 July 2005 [email protected] 30
Extensions: Erasure Codes
– Symmetric Replication with Erasure Codes
– Encode data into n blocks, only k needed to reconstruct original data
• Example: n=10, k=5
– Replication degree=n (# encoded blocks)
fetch any k blocks to reconstruct data
29 July 2005 [email protected] 31
Outline
• Background• Proposed scheme: symmetric
replication• Extensions• Related Work & Summary
29 July 2005 [email protected] 32
Related Work
• Several systems use multiple hash functions– Content Addressable Networks (CAN)– Tapestry – Simple Load Balancing in Chord
29 July 2005 [email protected] 33
Problem with Multiple Hash Functions
• Hash1(“music.mp3”)=11
• Hash2(“music.mp3”)=2
0 12
1514
13 3
12
11
4
5
6
9 87
10
Data: 15, 0
Data: 1, 2, 3
Data: 4, 5
Data: 14, 13,
Data: 6, 7, 8, 9, 10
Node 12 crashed,We need the inverse of
Hash1-
1(11)=“music.mp3”
29 July 2005 [email protected] 34
Summary
• A general replica placement scheme– Algorithms for join, leave, fail, insert in paper
• O(1) messages to restore replication degree f instead of (f)
• Can access any replica without a bottleneck:– Concurrent Requests– Load balance queries– Distributed Voting
• Current Use:– Implemented in DKS, http://dks.sics.se/