Chapter 25: All Pairs Shortest Paths. Contextjohnsoj5/CS 405/Lecture...Chapter 25: All Pairs...

Chapter 25: All Pairs Shortest Paths.

Context.

1. Throughout the chapter, we assume that our input graphs have no negativecycles.

2. [G = (V,E), w : E → R] is a weighted, directed graph with adjacency matrix[wij]. Wolog, vertices are 1, 2, . . . , V .

wij =

0, i = j∞, i 6= j and (i, j) 6∈ Efinite, i 6= j and (i, j) ∈ E.

Note: Output is already O(V 2), because for every vertex pair (u, v), the algo-rithm must report the weight of the shortest (weight) path from u to v.

Since the output requires O(V 2) work, there is no significant increase in usingan O(V 2) input scheme.

3. The output consists of two matrices:

∆ = δij = the weight of the shortest path i j

Π = πij =

null, if i = j or there is no path i j

predecessor of j on a shortest i j, otherwise.

4. Row i of [πij] defines a shortest path tree rooted at i. That is,

Gπ,i = (Vπ,i, Eπ,i)

Vπ,i = j : πij 6= null ∪ iEπ,i = (πij, j) : j ∈ Vπ,i\i.

1

5. Can recover a shortest path i j with

Print-Shortest-Path(Π, i, j) if (i = j)

print i;else if (πij = null)

print(“no shortest path from i to j”);else

Print-Shortest-Path(Π, i, πij);print j

Obvious approaches.

1. If negative edges exist (but no negative cycles), we can run multiple Bellman-Ford Single-Source-Shortest-Paths, once for each v ∈ V as the Bellman-Fordsource node. Since Bellman-Ford is O(V E), this approach is O(V 2E), whichis O(V 4) for dense graphs (E = Ω(V 2)).

2. If there are no negative edges, we can run multiple Dijkstra’s, one for eachv ∈ V as the Dijkstra source node. Since Dijkstra is O(E lg V ), this approachis O(V E lg V ), which is O(V 3 lg V ) for dense graphs (E = Ω(V 2)).

Improvements in this chapter, all of which work in the presence of negative edges,provided there are no negative cycles.

1. Via DP (Dynamic Programming), we can obtain Θ(V 3 lg V ).

2. Via Warshall’s Algorithm, we can obtain Θ(V 3).

3. Johnson’s Algorithm delivers O(V 2 lg V + V E). (Donald B. Johnson, 1977)

This approach is better for sparse graphs, specifically for E = O(V lg V ), webreak the V 3 level with O(V 2 lg V ).

2

The DP Algorithm.

Let V = 1, 2, . . . , n, and let δij be the weight of the shortest-weight path from ito j.

Suppose p : i j is a shortest path from i to j containing at most m edges. Asthere are no negative cycles, m is finite.

Let k be the predecessor of j on this path. That is, p : i k → j. Then p′ : i kis a shortest path from i to k by the optimal substructure property. Moreover, thelength of p′ is at most m− 1 edges.

Let δ(m)ij denote the minimum weight of any path from i to j with at most m edges.

We have

δ(0)ij =

0, i = j

∞, i 6= j,

the lower result because, if i 6= j, the set of paths from i to j containing at mostzero edges is the empty set. Recall that the minimum over a set of numbers is thelargest number that is less than or equal to every member of the set

For m ≥ 1,

δ(m)ij = min

δ(m−1)ij , min

1≤k≤n

δ(m−1)ik + wkj

= min

1≤k≤n

δ(m−1)ik + wkj

. (∗)

The last reduction follows because wjj = 0 for all j:

[δ(m−1)ik + wkj

∣∣∣k=j

= δ(m−1)ij + wjj = δ

(m−1)ij .

Now, in the absence of negative cycles, a shortest-weight path from i to j cancontains at most n− 1 edges. That is

δij = δ(n−1)ij = δ

(n)ij = δ

(n+1)ij = . . . .

3

We envision a three-dimensional storage matrix, consisting of planes 0, 1, . . . , n−1.In each plane, say k, is an n × n matrix of δ

(k)ij values. Envision i = 1, 2, . . . , n as

the row index and j = 1, 2, . . . , n as the column index in each plane. Recall thatwe are computing δ

(m)ij in plane m from the row δ

(m−1)i,? in plane m− 1.

Plane 0 is particularly easy to establish since

δ(0)ij =

0, i = j

∞, i 6= j.

Each subsequent plane fills each of n2 cells by choosing the minimum over n valuesderived from the previous plane. This procedure then requires O(n3) time per

plane. As we need n planes to reach the desired minimal paths lengths in δ(n−1)ij ,

we can accomplish the entire computation in O(n4) time.

But, Bellman-Ford repeated V times is more straightforward, and it also runs inO(n4) time, even for dense graphs (E = Ω(V 2)).

4

Extend-Shortest-Paths(∆ = [δij],W = [wij]) // Θ(n3)n = row count of ∆;∆′ = [δ′ij] = new n× n matrix;

for i = 1 to nfor j = 1 to n δ′ij =∞;

for k = 1 to nδ′ij = minδ′ij, δik + wkj;

return ∆′;

Slow-All-Pairs-Shortest-Paths(W = [wij]) // DP implementationn = row count of W ;∆ = [δij] = new n× n matrix;for i = 1 to n

for j = 1 to nδij =∞;

δii = 0;for k = 1 to n− 1 // Θ(n4)

∆out = Extend-Shortest-Paths(∆,W );∆ = ∆out;

return ∆out;

5

Some helpful observations:

(a) The passage from plane 0 to plane 1 is particularly easy.

δ(0)ik =

∞, k 6= i0, k = i

δ(0)ik + wkj =

∞, k 6= iwij, k = i.

δ(1)ij = min

1≤k≤n

δ(0)ik + wkj

= min∞,∞, . . . ,∞, wij,∞, . . . ,∞ = wij.

Plane 1 is the input weight matrix.

(b) Moreover, we can modify the initial recursion argument as follows. If, for evenm, p : i j is a shortest path of at most m edges, then we can express thepath as p : i k j, for some intermediate vertex k, such that each of i kand k j contains at most m/2 edges. Then

δ(m)ij = min

δ(m/2)ij , min

1≤k≤n

δ(m/2)ik + δ

(m/2)kj

= min

1≤k≤n

δ(m/2)ik + δ

(m/2)kj

,

again since δ(m/2)ik + δ

(m/2)kj = δ

(m/2)ij when k = j.

Recall that δwhateverjj = 0 can not be improved in the absence of negative cycles.

(c) Note similarity to previous recursion:

δ(m)ij = min

1≤k≤n

δ(m−1)ik + wkj

= min

1≤k≤n

δ(m−1)ik + δ

(1)kj

.

So, Extend-Shortest-Paths(∆ = [δij],W = [wij]) can be used as

Extend-Shortest-Paths(∆(m/2)ij ,∆

(m/2)ij )

6

We can now compute as follows,

∆(1) = [δ(1)ij ] = [wij] = W

∆(2) = [δ(2)ij ] = Extend-Shortest-Paths(∆(1),∆(1))

∆(4) = [δ(4)ij ] = Extend-Shortest-Paths(∆(2),∆(2))

......

Since [δ(k)ij ] = [δ

(n−1)ij ] for all k ≥ n− 1, this computation will stabilize after at most

dlg ne iterations.

Fast-All-Pairs-Shortest-Paths(W = [wij]) n = row count of W ;∆ = W ;m = 1;while m < n− 1

∆ = Extend-Shortest-Paths(∆,∆);m = 2m;

return ∆;

This DP computation uses lg n passes through Extend-Shortest-Paths, which is aΘ(n3) algorithm. Therefore the Fast-All-Pairs-Shortest-Paths algorithm is Θ(n3 lg n).Consequently, this algorithm

1. beats the Θ(n4) required by multiple passes through the Bellman-Ford algo-rithm,

2. ties the Θ(n3 lg n) required by multiple passes through the Dijkstra algorithm(which is not available is the graph contains negative edges),

3. and accommodates negative edges (although not negative cycles).

7

The Floyd-Warshall Algorithm also accommodates negative edges, althoughthe input graph must be free of negative cycles. Let G = (V,E), where V =1, 2, . . . , n.

Define δ(k)ij to be the weight of the shortest path p : i j with all intermediate

vertices constrained to lie in the set 1, 2, . . . , k.

We proceed via induction from the basis δ(0)ij = wij, which reflects the constraint

that there be no intermediate vertices. The induction step is then

δ(k)ij = min

δ(k−1)ij , δ

(k−1)ik + d

(k−1)kj

,

reflecting the fact that the shortest path with all of 1, 2, . . . , k available as inter-mediates is either (a) a path that does not use vertex k as an intermediary, or (b)a path that does use k.

Preliminary-Floyd-Warshall(W = [wij]) // Θ(n3)n = row count of W ;∆ = [δij] = W ;∆′ = [δ′ij] = new n× n matrix;

for k = 1 to n for i = 1 to n

for j = 1 to nδ′ij = minδij, δik + δkj;

∆ = ∆′;return ∆′;

Thus Floyd-Warshall, a simple data-structures algorithm, beats our best dynamic-programming algorithm: Θ(n3) against Θ(n3 lg n). We now further develop Floyd-Warshall to capture the shortest paths themselves.

8

Recall that πij is the predecessor of j on a shortest path p : i j, null if no such

path exists. We define π(k)ij to be the predecessor of j on a shortest path p : i j

constrained such that all intermediate vertices lie in the set 1, 2, . . . , k. We have

π(0)ij =

null, i = j or wij =∞i, i 6= j and wij <∞

π(k)ij =

π(k−1)ij , δ

(k−1)ij ≤ δ

(k−1)ik + δ

(k−1)kj

π(k−1)kj , δ

(k−1)ij > δ

(k−1)ik + δ

(k−1)kj .

In the context of

Preliminary-Floyd-Warshall(W = [wij]) // Θ(n3)n = row count of W ;∆ = [δij] = W ;∆′ = [δ′ij] = new n× n matrix;

for k = 1 to n for i = 1 to n

for j = 1 to nδ′ij = minδij, δik + δkj;

∆ = ∆′;return ∆′;

the computation for π(k)ij , the upper option is used when δ

(k)ij is achieved via a path

that uses only intermediate vertices in 1, 2, . . . , k − 1. That is, the predecessor

of j is unchanged from π(k−1)ij .

The lower option is used when the new intermediate vertex k produces a shorterpath via k. That is, the new shortest path has the form p : i k j, in whichcase the new predecessor of j on this path is the predecessor of j on best pathq : k j constrained to intermediaries in 1, 2, . . . , k − 1.

9

Floyd-Warshall(W = [wij]) // Θ(n3)n = row count of W ;∆ = [δij] = W ;∆′ = [δ′ij] = new n× n matrix;

Π = [πij] = new n× n matrix;Π′ = [π′ij] = new n× n matrix;

for i = 1 to nfor j = 1 to n

if (i = j) or (wij =∞)πij = null;

elseπij = i;

for k = 1 to n for i = 1 to n

for j = 1 to nif δij ≤ δik + δkj δ′ij = δij;

π′ij = πij;

else δ′ij = δik + δkj;

π′ij = πkj;

∆ = ∆′;Π = Π′;

return (∆′,Π′);

10

Transitive closure via Warshall’s Algorithm.

Definition. Given a directed graph G = (V,E), the transitive closure of G isG∗ = (V,E∗), where

E∗ = (i, j) : there exists a path i j in G.

First Approach.

1. Redefine the edge set:

wij =

0, i = j

∞, i 6= j and (i, j) 6∈ E1, i 6= j and (i, j) ∈ E.

2. Run the Floyd-Warshall Algorithm ⇒ ∆ = [δij], a matrix of shortest pathsbetween each i, j pair.

3. Establish E∗ = (i, j) : δij <∞.

11

Second Approach: Rework Floyd-Warshall to use logical operations.

1. Redefine the edge set:

wij =

false, i 6= j and (i, j) 6∈ Etrue, i = j or (i, j) ∈ E.

2. Run the following revised Floyd-Warshall algorithm, which returns ∆′ = [δ′ij],a matrix of true-false values.

Floyd-Warshall(W = [wij]) // Θ(n3)n = row count of W ;∆ = [δij] = W ;∆′ = [δ′ij] = new n× n matrix;

for k = 1 to n for i = 1 to n

for j = 1 to nδ′ij = δij ∨ (δik ∧ δkj);

∆ = ∆′;return ∆′;

On conclusion, E∗ = (i, j) : δ′ij = true.

12

Johnson’s Algorithm.

We want to exploit Dijkstra’s single-source-shortest paths algorithm, but we needto circumvent the requirement that the graph cannot contain negative edge weights.

We continue to comply with the requirement that there be no negative cycles.

So, given a directed G = (V,E) with no negative weight cycles, we need a newweighting function, say w, with the following properties.

1. For all u, v ∈ V , p : u v is a shortest w-weighted path from u to v if andonly if p is a shortest w-weighted path from u to v.

2. w(u, v) ≥ 0 for all (u, v) ∈ E.

Note: We cannot obtain the new weighting by simply adding a constant to allweights, sufficiently large to boost all negative weights into positive territory:

13

Lemma 25.1: Given a directed weighted G = (V,E), w : E → R, Let h : V → Rbe an arbitrary function. Define w : E → R via

w(u, v) = w(u, v) + h(u)− h(v).

Then

1. p : u v is a shortest w-weighted path from u to v if and only if p is a shortestw-weighted path from u to v.

2. G has a negative weight cycle via w if and only if G has a negative weight cyclevia w.

Proof. Let p = (v1, v2, . . . , vk) be any path in G. We have

w(p) =k∑i=2

w(vi−1, vi) =k∑i=2

[w(vi−1, vi) + h(vi−1)− h(vi)]

=

(k∑i=2

w(vi−1, vi)

)+

(k∑i=2

(h(vi−1)− h(vi))

)= w(p) + h(v1)− h(vk)

w(p)− w(p) = h(v1)− h(vk), regardless of path.

The component h(v1)− h(vk) is constant across all paths p′ : v1 vk.

Now suppose that p is a shortest w-weighted path from v1 to vk, and suppose thereexists some path q : v1 vk with w(q) < w(p). Then,

w(p)− w(p) = h(v1)− h(vk) = w(q)− w(q)

w(p)− w(q) = w(p)− w(q) > 0

w(q) < w(p),

a contradiction. Therefore p is also a shortest w-weighted path from v1 to vk.

14

Similarly, if p is a shortest w-weighted path from v1 to vk and there exists somepath q : v1 vk with w(q) < w(p), we again derive a contradiction:

w(p)− w(p) = h(v1)− h(vk) = w(q)− w(q)

w(p)− w(q) = w(p)− w(q) > 0

w(q) < w(p).

Hence p is a shortest w-weighted path from v1 to v2 if and only if p is a shortedw-weighted path from v1 to vk.

For any cycle, say p = (v1, v2, . . . , vk = v1),

w(p) = w(p) + h(v1)− h(vk) = w(p) + h(v1)− h(v1) = w(p).

Hence if p is a negative cycle under the w weights, it is also a negative cycle underthe w weights, and vice versa.

15

Both path lengths shift by 3− 0 = 3, and therefore the lower remains the shorter.

16

Returning to Johnson’s Algorithm, we modify the input graph as suggested in thesketch.

Specifically, we construct graph G′ = (V ′, E ′), where

V ′ = V ∪ sE ′ = E ∪ (s, v) : v ∈ V .

We extend the weight function to E ′ by defining w(s, v) = 0 for all v ∈ V .

Observations.

1. s has in-degree zero. Consequently, no shortest paths in G′, other than thosewith source s, contain s.

2. G′ has a negative weight cycle if and only if G has a negative weight cycle.

17

Now, suppose G has no negative weight cycle. Consequently, G′ has no negativeweight cycle. Letting δ(s, v) denote the length of the shorted path from s to v inG′, we define h(v) = δ(s, v). Then, we have

δ(s, v) ≤ δ(s, u) + w(u, v) (triangle inequality)

h(v) ≤ h(u) + w(u, v)

w(u, v) = w(u, v) + h(u)− h(v) ≥ 0, for all (u, v) ∈ G′.

18

Johnson(G = (V,E), w : E → R)Compute G′ = (V ′, E ′); // Θ(V + E)if Bellman-Ford(G′, w, s) = false // Θ(V E)

print(“negative cycle”);return;

for v ∈ V ′v.h = v.d; // δ(s, v) from Bellman Ford

for (u, v) ∈ E ′ // Θ(V + E)(u, v).w = (u, v).w + u.h− v.h; // all non-negative

for u ∈ G.V Dijkstra(G, w, u); // O(V E lg V ) puts δ(u, v) into v.dfor v ∈ G.V

∆uv = v.d+ v.h− u.h; // Θ(V 2)Πu,v = v.π;

return (∆,Π);

Observations.

1. When ∆uv is assigned, we get

∆uv = δ(u, v) + h(v)− h(u) = [δ(u, v) + h(u)− h(v)] + h(v)− h(u) = δ(u, v),

the shortest distance from u to v as desired.

2. Since the naked Dijkstra is O(E lg V ), we have O(V E lg V ) for the loop inwhich that call is embedded.

3. The total is then O(V E lg V ), which beats V 3 for sparse graphs (E = O(V 2−ε)):

V 3−ε lg V

V 3=

lg V

V ε→ 0.

4. The author notes slightly better performance with Fibonacci heaps underlyingDijkstra’s minHeap.

19