CHAPTER 12

Parallel and

Distributed Algorithms

Algorithm 12.1.1 Maximum

This algorithm returns the maximum value in array a[0], ... , a[n - 1]. The value of a.length is the number of elements in the array.

Input Parameters: aOutput Parameters: Nonemaximum(a) { n = a.length current_max = a[0] for i = 1 to n - 1 if (a[i] > current_max) current_max = a[i] return current_max}

Algorithm 12.2.2 Parallel Search

Given an array a of size n and a number x, this algorithm searches for x in a and returns true if there is an i such that a[i] = x and false otherwise.Input Parameters: a,xOutput Parameters: Noneparallel_search(a,x) { n = a.length found = false for i = 0 to n - 1 in parallel if (a[i] == x) found = true return found}

Algorithm 12.2.4 Locate

Given an array a and a value x, the algorithm returns an index i such that a[i] = x if there is one, and -1 otherwise.

Input Parameters: a,xOutput Parameters: Nonelocate(a,x) { n = a.length

location = -1 found = false for i = 0 to n - 1 in parallel if (a[i] == x) location = i return location}

Cache Coherence Problem

What value does a[i] contain, after performing this code?

n = a.lengthx = 0for i = 0 to n in parallel

if (i == n) x = 1

else a[i] = x

Algorithm 12.2.5 ER Broadcast

This algorithm takes as input a value x and an array a and assigns x to all elements of the array.

Input Parameters: a,xOutput Parameters: Noneer_broadcast(a,x) {

n = a.lengtha[0] = xfor i = 0 to lg n - 1

for j = 0 to 2i - 1 in parallelif (2i + j < n)

a[2i + j ] = a[j]}

Algorithm 12.2.11 Sequential -Sum

This algorithm computes given an array a as input.

Input Parameters: a,xOutput Parameters: Nonesequential__sum(a) {

n = a.lengthcurrent_sum = a[0]for i = 1 to n - 1

current_sum = current_sum a[i]return current_sum

}

][10 ian

i

Algorithm 12.2.12 Parallel -Sum

This EREW algorithm computes given an array a as input.

Input Parameters: a,xOutput Parameters: Noneparallel__sum(a) {

n = a.lengthfor i = 0 to lg n - 1

for j = 0 to n/2i+1 - 1 in parallelif (j * 2i+1 + 2i < n)

a[j * 2i+1 + 2i] = a[j * 2i+1 + 2i] a[j * 2i+1 + 2i]

return a[0]}

][10 ian

i

Algorithm 12.2.16 Optimal -Sum

This algorithm computes given an array a as input.][10 ian

i

Input Parameters: aOutput Parameters: Noneoptimal__sum(a) {

n = a.lengthblocksize = 2lg lg n // first, compute sum of all blocks of // size blocksize ≈ lg nfor i = 0 to n/blocksize in parallel {

base = i * blocksize for j = 1 to blocksize - 1 if (base + j < n) a[base] = a[base] a[base + j]

}// second, compute the sum of the remaining // n/blocksize elementsfor i = lg blocksize to lg n - 1

for j = 0 to n/2i+1 - 1 in parallelif (j * 2i+1 + 2i < n)

a[j * 2i+1 + 2i] = a[j * 2i+1 + 2i] a[j * 2i+1 + 2i]

return a[0]}

Algorithm 12.2.20 Optimal ER BroadcastThis algorithm takes as input a value x and an array a and assigns x to all elements of the array.

Input Parameters: a,xOutput Parameters: Noneoptimal_er_broadcast(a,x) {

n = a.lengthn = n/lg na[0] = x// Use er_broadcast to fill the first n’ cells of a with xfor i = 0 to lg n’ - 1

for j = 0 to 2i - 1 in parallelif (2i + j < n’)

a[2i + j] = a[j]// Spread the contents using n processors// each performing the sequential algorithmfor i = 1 to lg n

for j = 0 to n’ - 1 in parallelif (i × n’ + j < n)

a[i × n’ + j] = a[j]}

Algorithm 12.2.25 Sequential Prefix

This algorithm computes the prefixes for all 0 ≤ k < n given an n-element array a as input.

Input Parameters: aOutput Parameters: Nonesequential_prefix(a) {

n = a.lengthsum[0] = a[0]for i = 0 to n - 1

sum[i] = sum[i – 1] a[i]}

][0 iaki

Algorithm 12.2.26 Parallel PrefixThis algorithm computes the prefixes for all 0 ≤ k < n given an n-element array a as input.

Input Parameters: aOutput Parameters: Noneparallel_prefix(a) {

n = a.lengthfor i = 0 to n - 1

sum[j] = a[j]for i = 0 to lg n

for j = 0 to n - 1 in parallelif (j + 2i < n)

sum[j + 2i] = sum[j] sum[j + 2i]}

][0 iaki

Algorithm 12.2.31 Sequential List RankingThis algorithm takes as an input the first node u of a linked list. Each node has fields next, the next node of the list or null for the last node in the list, and rank, which is assigned the distance of the node from the end of the list.Input Parameters: uOutput Parameters: Nonesequential_list_ranking(u) { n = 0 // counter for total number of nodes node = u while (node ! = null) { n = n + 1 node = node.next } node = u // start again at the beginning while (node ! = null) { node.rank = n n = n - 1 node = node.next }}

Algorithm 12.2.32 List RankingThis algorithm takes as an input the first node u of a linked list L(u). Each node has fields next, the next node of the list or null for the last node in the list, and rank, which is assigned the distance of the node from the end of the list.

Input Parameters: uOutput Parameters: Nonelist_ranking(u) { for each node in L(u) in parallel node.rank = 1 done = false do { for each node in L(u) in parallel if (node.next == null) done = true

else { node.rank = node.rank + node.next.rank node.next = node.next.next } } while (!done)}

Algorithm 12.4.11 Minimum Label

This algorithm takes as input a two-dimensional array a with indexes ranging from 0 to n + 1. Every element of the array contains fields color and label. The color field of elements in rows and columns with indexes 0 or n + 1 is assumed to be zero. The algorithm computes a component labeling for a and stores it in the label field.

Input Parameters: a,nOutput Parameters: Noneminimum_label(a,n) {

// initialize the label field of // every black pixel to a unique value for i,j {1, ... , n} in parallel if (a[i,j].color == 1) a[i,j].label = i × n + j else a[i,j].label = 0 // perform the second step on // every label n2 times for k = 1 to n2

for i,j {1, ... , n} in parallel for x = i - 1 to i + 1 for y = j - 1 to j + 1 if (a[x,y].color == 1) a[i,j].label =

min(a[i,j].label, a[x,y].label)}

Algorithm 12.5.2 Ring BroadcastThis algorithm is run on a ring network. The initiator runs init_ring_broadcast, and the noninitiators run ring_broadcast. All processors terminate successfully after having received a message from the initiator.

init_ring_broadcast() { send token to successor receive token from predecessor terminate}

ring_broadcast() { receive token from predecessor send token to successor terminate}

Algorithm 12.5.4 BroadcastThis algorithm works on any (fixed) connected network. The initiator runs init_broadcast and the noninitiators broadcast. All processors terminate successfully after having received a message from the initiator. In the algorithms we assume that the current machine we are running the code on is called p.

init_broadcast() { N = {q | q is a neighbor of p} for each q N send token to neighbor q terminate}

broadcast() { receive token from neighbor q N = {q | q is a neighbor of p} for each q N send token to neighbor q terminate}

Algorithm 12.5.5 EchoThis algorithm works on any (fixed) connected network. The initiator runs init_echo and the noninitiators echo. All processors terminate successfully after having received a message from the initiator. The initiator terminates after all processors have received its message. The machine on which we are running the code is called p.

init_echo() { N = {q | q is a neighbor of p} for each q N send token to neighbor q counter = 0 while (counter < |N|) { receive token counter++ } terminate}

echo() { receive token from neighbor q parent = q N = {q | q is a neighbor of p} - {parent}

for each q N send token to neighbor q counter = 0 while (counter < |N|) { receive token counter++ } send token to neighbor parent terminate}

Algorithm 12.5.9 Leader ElectionThis algorithm runs on the unidirectional ring. The initiators run init_election and the noninitiators run election. All processors terminate successfully, and exactly one initiator has its leader attribute set to true. In the algorithms, we assume that the current machine we are running the code on is called p. Every processor has a next neighbor, p.next, to which it can send messages.

init_election() {send token,p.ID to p.next

min = p.ID receive token,I while (p.ID != I) { if (I < min) min = I send token,I to p.next receive token,I

} if (p.ID == min) p.leader = true else p.leader = false terminate}election() { p.leader = false // noninitiator is never chosen as leader do { receive token,I send token,I to p.next } while (true)}