45
November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.1 Introduction to Algorithms 6.046J/18.401J LECTURE 18 Shortest Paths II Bellman-Ford algorithm Linear programming and difference constraints VLSI layout compaction Prof. Erik Demaine

Cormen Algo-lec18

Embed Size (px)

Citation preview

Page 1: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.1

Introduction to Algorithms6.046J/18.401J

LECTURE 18Shortest Paths II• Bellman-Ford algorithm• Linear programming and

difference constraints• VLSI layout compaction

Prof. Erik Demaine

Page 2: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.2

Negative-weight cyclesRecall: If a graph G = (V, E) contains a negative-weight cycle, then some shortest paths may not exist.Example:

uu vv

< 0

Page 3: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.3

Negative-weight cyclesRecall: If a graph G = (V, E) contains a negative-weight cycle, then some shortest paths may not exist.Example:

uu vv

< 0

Bellman-Ford algorithm: Finds all shortest-path lengths from a source s ∈ V to all v ∈ V or determines that a negative-weight cycle exists.

Page 4: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.4

Bellman-Ford algorithmd[s] ← 0for each v ∈ V – {s}

do d[v] ←∞initialization

for i ← 1 to |V| – 1do for each edge (u, v) ∈ E

do if d[v] > d[u] + w(u, v)then d[v] ← d[u] + w(u, v)

for each edge (u, v) ∈ Edo if d[v] > d[u] + w(u, v)

then report that a negative-weight cycle exists

relaxation step

At the end, d[v] = δ(s, v), if no negative-weight cycles. Time = O(VE).

Page 5: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.5

Example of Bellman-Ford

AA

BB

EE

CC DD

–1

4

12

–3

2

5

3

Page 6: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.6

Example of Bellman-Ford

AA

BB

EE

CC DD

–1

4

12

–3

2

5

3

0 ∞

∞ ∞

Initialization.

Page 7: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.7

Example of Bellman-Ford

AA

BB

EE

CC DD

–1

4

12

–3

2

5

3

0 ∞

∞ ∞

1

2

34

5

7

8

6

Order of edge relaxation.

Page 8: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.8

Example of Bellman-Ford

AA

BB

EE

CC DD

–1

4

12

–3

2

5

3

0 ∞

∞ ∞

1

2

34

5

7

8

6

Page 9: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.9

Example of Bellman-Ford

AA

BB

EE

CC DD

–1

4

12

–3

2

5

3

0 ∞

∞ ∞

1

2

34

5

7

8

6

Page 10: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.10

Example of Bellman-Ford

AA

BB

EE

CC DD

–1

4

12

–3

2

5

3

0 ∞

∞ ∞

1

2

34

5

7

8

6

Page 11: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.11

Example of Bellman-Ford

∞−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30 ∞

∞ ∞

1

2

34

5

7

8

6

Page 12: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.12

Example of Bellman-Ford

∞4

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30 ∞

1

2

34

5

7

8

6

Page 13: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.13

Example of Bellman-Ford

4

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30 ∞

1

2

34

5

7

8

6

Page 14: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.14

Example of Bellman-Ford

42

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30 ∞

1

2

34

5

7

8

6

Page 15: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.15

Example of Bellman-Ford

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30 ∞

1

2

34

5

7

8

6

Page 16: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.16

Example of Bellman-Ford

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30 ∞

1

2

34

5

7

8

6

End of pass 1.

Page 17: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.17

Example of Bellman-Ford

∞1

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30

1

2

34

5

7

8

6

Page 18: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.18

Example of Bellman-Ford

1

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30

1

2

34

5

7

8

6

Page 19: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.19

Example of Bellman-Ford

∞1

1

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30

1

2

34

5

7

8

6

Page 20: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.20

Example of Bellman-Ford

1

1

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30

1

2

34

5

7

8

6

Page 21: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.21

Example of Bellman-Ford

1

1

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30

1

2

34

5

7

8

6

Page 22: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.22

Example of Bellman-Ford

1

1

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30

1

2

34

5

7

8

6

Page 23: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.23

Example of Bellman-Ford

1

1

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30

1

2

34

5

7

8

6

Page 24: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.24

Example of Bellman-Ford

1−2

1

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30

1

2

34

5

7

8

6

Page 25: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.25

Example of Bellman-Ford

−2

1

2

−1

AA

BB

EE

CC DD

–1

4

12

–3

2

5

30

1

2

34

5

7

8

6

End of pass 2 (and 3 and 4).

Page 26: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.26

CorrectnessTheorem. If G = (V, E) contains no negative-weight cycles, then after the Bellman-Ford algorithm executes, d[v] = δ(s, v) for all v ∈ V.

Page 27: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.27

CorrectnessTheorem. If G = (V, E) contains no negative-weight cycles, then after the Bellman-Ford algorithm executes, d[v] = δ(s, v) for all v ∈ V. Proof. Let v ∈ V be any vertex, and consider a shortest path p from s to v with the minimum number of edges.

v1v1

v2v2

v3v3 vk

vkv0v0

…s

v

p:

Since p is a shortest path, we haveδ(s, vi) = δ(s, vi–1) + w(vi–1, vi) .

Page 28: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.28

Correctness (continued)

v1v1

v2v2

v3v3 vk

vkv0v0

…s

v

p:

Initially, d[v0] = 0 = δ(s, v0), and d[v0] is unchanged by subsequent relaxations (because of the lemma from Lecture 14 that d[v] ≥ δ(s, v)).• After 1 pass through E, we have d[v1] = δ(s, v1).• After 2 passes through E, we have d[v2] = δ(s, v2).M

• After k passes through E, we have d[vk] = δ(s, vk).Since G contains no negative-weight cycles, p is simple. Longest simple path has ≤ |V| – 1 edges.

Page 29: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.29

Detection of negative-weight cycles

Corollary. If a value d[v] fails to converge after |V| – 1 passes, there exists a negative-weight cycle in G reachable from s.

Page 30: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.30

Linear programming

Let A be an m×n matrix, b be an m-vector, and cbe an n-vector. Find an n-vector x that maximizes cTx subject to Ax ≤ b, or determine that no such solution exists.

. ≤ .maximizingm

n

A x ≤ b c xT

Page 31: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.31

Linear-programming algorithms

Algorithms for the general problem• Simplex methods — practical, but worst-case

exponential time.• Interior-point methods — polynomial time and

competes with simplex.

Page 32: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.32

Linear-programming algorithms

Algorithms for the general problem• Simplex methods — practical, but worst-case

exponential time.• Interior-point methods — polynomial time and

competes with simplex.

Feasibility problem: No optimization criterion. Just find x such that Ax ≤ b.• In general, just as hard as ordinary LP.

Page 33: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.33

Solving a system of difference constraints

Linear programming where each row of A contains exactly one 1, one –1, and the rest 0’s. Example:

x1 – x2 ≤ 3x2 – x3 ≤ –2x1 – x3 ≤ 2

xj – xi ≤ wij

Page 34: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.34

Solving a system of difference constraints

Linear programming where each row of A contains exactly one 1, one –1, and the rest 0’s. Example:

x1 – x2 ≤ 3x2 – x3 ≤ –2x1 – x3 ≤ 2

xj – xi ≤ wij

Solution:x1 = 3x2 = 0x3 = 2

Page 35: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.35

Solving a system of difference constraints

Linear programming where each row of A contains exactly one 1, one –1, and the rest 0’s. Example:

x1 – x2 ≤ 3x2 – x3 ≤ –2x1 – x3 ≤ 2

xj – xi ≤ wij

Solution:x1 = 3x2 = 0x3 = 2

Constraint graph:

vjvjvi

vixj – xi ≤ wijwij

(The “A”matrix has dimensions|E | × |V |.)

Page 36: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.36

Unsatisfiable constraintsTheorem. If the constraint graph contains a negative-weight cycle, then the system of differences is unsatisfiable.

Page 37: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.37

Unsatisfiable constraintsTheorem. If the constraint graph contains a negative-weight cycle, then the system of differences is unsatisfiable.Proof. Suppose that the negative-weight cycle is v1 → v2 →L→ vk → v1. Then, we have

x2 – x1 ≤ w12x3 – x2 ≤ w23

Mxk – xk–1 ≤ wk–1, kx1 – xk ≤ wk1

Page 38: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.38

Unsatisfiable constraintsTheorem. If the constraint graph contains a negative-weight cycle, then the system of differences is unsatisfiable.Proof. Suppose that the negative-weight cycle is v1 → v2 →L→ vk → v1. Then, we have

x2 – x1 ≤ w12x3 – x2 ≤ w23

Mxk – xk–1 ≤ wk–1, kx1 – xk ≤ wk1

Therefore, no values for the xican satisfy the constraints.

0 ≤ weight of cycle< 0

Page 39: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.39

Satisfying the constraintsTheorem. Suppose no negative-weight cycle exists in the constraint graph. Then, the constraints are satisfiable.

Page 40: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.40

Satisfying the constraintsTheorem. Suppose no negative-weight cycle exists in the constraint graph. Then, the constraints are satisfiable.Proof. Add a new vertex s to V with a 0-weight edge to each vertex vi ∈ V.

v1v1

v4v4

v7v7

v9v9

v3v3

Page 41: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.41

Satisfying the constraintsTheorem. Suppose no negative-weight cycle exists in the constraint graph. Then, the constraints are satisfiable.Proof. Add a new vertex s to V with a 0-weight edge to each vertex vi ∈ V.

v1v1

v4v4

v7v7

v9v9

v3v3

s

0 Note:No negative-weight cycles introduced ⇒shortest paths exist.

Page 42: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.42

Proof (continued)Claim: The assignment xi = δ(s, vi) solves the constraints.Consider any constraint xj – xi ≤ wij, and consider the shortest paths from s to vj and vi:

ss

vjvj

vivi

δ(s, vi)

δ(s, vj) wij

The triangle inequality gives us δ(s,vj) ≤ δ(s, vi) + wij. Since xi = δ(s, vi) and xj = δ(s, vj), the constraint xj – xi≤ wij is satisfied.

Page 43: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.43

Bellman-Ford and linear programming

Corollary. The Bellman-Ford algorithm can solve a system of m difference constraints on nvariables in O(mn) time. Single-source shortest paths is a simple LP problem.In fact, Bellman-Ford maximizes x1 + x2 + L + xnsubject to the constraints xj – xi ≤ wij and xi ≤ 0(exercise).Bellman-Ford also minimizes maxi{xi} – mini{xi}(exercise).

Page 44: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.44

Application to VLSI layout compaction

Integrated-circuit features:

minimum separation λ

Problem: Compact (in one dimension) the space between the features of a VLSI layout without bringing any features too close together.

Page 45: Cormen Algo-lec18

November 16, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L18.45

VLSI layout compactiond1

11 2

x1 x2

Constraint: x2 – x1 ≥ d1 + λBellman-Ford minimizes maxi{xi} – mini{xi}, which compacts the layout in the x-dimension.