Dynamic Programming Lecture 9 Asst. Prof. Dr. İlker Kocabaş 1

Dynamic ProgrammingDynamic Programming

Lecture 9

Asst. Prof. Dr. İlker Kocabaş

1

Dynamic Programming HistoryDynamic Programming History

• Bellman. Pioneered the systematic study of dynamic programming in the 1950s.

• Etymology.– Dynamic programming = planning over time.– Secretary of Defense was hostile to mathematical research.– Bellman sought an impressive name to avoid confrontation.

• "it's impossible to use dynamic in a pejorative sense"

• "something not even a Congressman could object to"

Reference: Bellman, R. E. Eye of the Hurricane, An Autobiography.

2

IntroductionIntroduction

• Dynamic Programming(DP) applies to optimization problems in which a set of choices must be made in order to arrive at an optimal solution.

• As choices are made, subproblems of the same form arise.

• DP is effective when a given problem may arise from more than one partial set of choices.

• The key technique is to store the solution to each subproblem in case it should appear

3

Introduction (cont.)Introduction (cont.)

• Divide and Conquer algorithms partition the problem into independent subproblems.

• Dynamic Programming is applicable when the subproblems are not independent.(In this case DP algorithm does more work than necessary)

• Dynamic Programming algorithm solves every subproblem just once and then saves its answer in a table.

4

Dynamic Programming ApplicationsDynamic Programming Applications

• Areas. – Bioinformatics.

– Control theory.

– Information theory.

– Operations research.

– Computer science: theory, graphics, AI, systems, ….

• Some famous dynamic programming algorithms. – Viterbi for hidden Markov models.

– Unix diff for comparing two files.

– Smith-Waterman for sequence alignment.

– Bellman-Ford for shortest path routing in networks.

– Cocke-Kasami-Younger for parsing context free grammars.

5

Assembly-line SchedulingAssembly-line Scheduling

6

Assembly-line SchedulingAssembly-line Scheduling

• If we are given a list of which stations to use in line 1 and which to use in line 2, then T(n)=Θ(n).

• There are 2n possible ways to choose stations. Thus, the solution is obtained by enumarating all posible ways and computing how long each takes would take Ω(2n).

7

The steps of a dynamic The steps of a dynamic programmingprogramming

• Characterize the structure of an optimal solution• Recursively define the value of an optimal solution• Compute the value of an optimal solution in a

bottom-up fashion• Construct an optimal solution from computed

information

8

Step 1:The structure of the fastest way Step 1:The structure of the fastest way through the factorythrough the factory

• Consider the fastest possible way(fpw) to get from the starting point through station S1,j.

• If j=1, there is only one way that the chassis could have gone.

• For j=2, 3, ..., n there are two choices: – The chassis could have come from S1, j-1 and then

directly to station S1, j.

– The chassis could have come from S2, j-1 and then directly to station S1, j-1.

9

Step 1:The structure of the fastest way Step 1:The structure of the fastest way through the factory (cont.)through the factory (cont.)

• First, suppose that the fastest way(fw) through station S1, j is through station S1, j-1.

• The key observation is that the chassis must have taken a fw from the starting point through station S1, j-1.

10


• Similarly, suppose that the fw through station S2, j

is through station S2, j-1.

• We observe that the chassis must have taken a fw from tha starting point through station S2, j-1.

11


• We can say that for assembly-line scheduling, an optimal solution to a problem contains within it an optimal solution to sub-problems (finding the fw through either S1, j-1 or S2, j-1).

• We refer to this property as optimal substructure.

12

The fw through stations The fw through stations SS1, 1, jj

• The fw through station S1, j-1 and then directly through station S1, j.

or

• The fw through station S2, j-1 and then through station S1, j.

13

The fw through station The fw through station SS2, 2, jj

• The fw through station S2, j-1 and then directly through station S2, j.

or

• The fw through station S1, j-1 and then through station S2, j.

14


• To solve the problem of finding the fw through station j of either line, we solve the subproblems of finding the fw through station j-1 on both lines.

• Thus, we can build an optimal solution to an instance of the assemly line scheduling problem by building on optimal solutions to subproblems

15

Step 2: A recursive solutionStep 2: A recursive solution

• We define the value of an optimal solution recursively in tems of the optimal solutions to subproblems.

• Let fi[j] denote the fastest possible time to get a chassis from the starting point throuhg station Si, j.

• Let fi* denote the fastest time to get a chassis all

the way through the factory.

16

Step 2: A recursive solution (cont.)Step 2: A recursive solution (cont.)

)]1[,]1[min(][

)]1[,]1[min(][

,...,3,2For

]1[

]1[

1For

)][,][min(

,21,11,222

,11,22,111

1,222

1,111

2211*

jjj

jjj

atjfajfjf

atjfajfjf

nj

aef

aef

j

xnfxnff

17


2jif)]1[,]1[min(

1if][

2jif)]1[,]1[min(

1if][

,21,11,22

1,222

,11,22,11

1,111

jjj

jjj

atjfajf

jaejf

atjfajf

jaejf

18

Step 2: A recursive solution: An exampleStep 2: A recursive solution: An example

Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6


4

2

7 9 3 8

5

4

32 3 1 3 4

4 2 1 5 2 1

8 65 4 7

2

j 2 3 4 5 6

l1[j]

l2[j]

j 1 2 3 4 5 6

f1[j]

f2[j]f*=0 l* =0

Line 1

Line 2

Chassis Enters

19



49 9 3 8

5

4

32 3 1 3 4

2 1 5 2 1

12 65 4 7

2

9

12

j 2 3 4 5 6

l1[j]

l2[j]

j 1 2 3 4 5 6

f1[j]

f2[j]f*=0 l* =0

Line 1

Line 2

Chassis Enters

20





49 18 3 8

5

4

33 1 3 4

5 2 1

12 616 4 7

2

18 9

1612

1

1

j 2 3 4 5 6

l1[j]

l2[j]

j 1 2 3 4 5 6

f1[j]

f2[j]f*=0 l* =0

Line 1

Line 2

Chassis Enters

21

1




49 18 20 8

5

4

31 3 4

5 2 1

12 2216 4 7

2

18 9 20

1612 22

2 1

2 1

j 2 3 4 5 6

l1[j]

l2[j]

j 1 2 3 4 5 6

f1[j]

f2[j]f*=0 l* =0

Line 1

Line 2

Chassis Enters

22




249 18 20 8

5

4

33 4

2 1

12 2216 25 7

2

18 9 20 24

1612 22 25

2 1 1

2 1 1

j 2 3 4 5 6

l1[j]

l2[j]

j 1 2 3 4 5 6

f1[j]

f2[j]f*=0 l* =0

Line 1

Line 2

Chassis Enters

23



249 18 20 32

30

4

34

1

12 2216 25 7

2

18 9 20 24 32

1612 22 25 30

2 1 1 1

2 1 1 2

j 2 3 4 5 6

l1[j]

l2[j]

j 1 2 3 4 5 6

f1[j]

f2[j]f*=0 l* =0

Line 1

Line 2

Chassis Enters

24


Step 2: A recursive solution: An exampleStep 2: A recursive solution: An example( Correction !!! )( Correction !!! )


249 18 20 32

30

4

34

1

12 2216 25 7

2

18 9 20 24 32

1612 22 25 30

2 1 1 1

2 1 1 2

j 2 3 4 5 6

l1[j]

l2[j]

j 1 2 3 4 5 6

f1[j]

f2[j]f*=0 l* =0

Line 1

Line 2

Chassis Enters

25





9 18 20 35

3

12 2216 37

2

18 9 20 24 32 35

1612 22 25 30 37

2 1 1 1 1

2 1 1 2 2

j 2 3 4 5 6

l1[j]

l2[j]

j 1 2 3 4 5 6

f1[j]

f2[j]f*=0 l* =0

Line 1

Line 2

Chassis Enters

26

322420

3022 25




9 18 20 32 35

3

12 2216 37

2

Line 1

Line 2

Chassis Enters

27

18 9 20 24 32 35+3

1612 22 25 30 37+2

2 1 1 1 2

2 1 1 2 2

j 2 3 4 5 6

l1[j]

l2[j]

j 1 2 3 4 5 6

f1[j]

f2[j]f*=38 l* =1

2420

3022 25


• fi[j] values give the values of optimal solutions to subproblems.

• Let li[j] to be the line number 1 or 2, whose station j-1 is used in a fw throgh station Si, j.

• We also define l* to be the line whose station n is used in a fw through the entire factory.

28


The fw through the entire factory:

1,12

2,21

3,12

4,22

5,21

6,1*

station use weso and ,1]2[at look We.6





station use we,1 Since 1.

Sl

Sl

Sl

Sl

Sl

Sl

29

Step 3: Computing the fastest timesStep 3: Computing the fastest times

We can write a simple recursive algorithm based on the equations for f1[j] and f2[j] to compute the fw through the factory. However its running time is exponential.

Let ri( j ) be the number of references made to fi [ j ] in a recursive algorithm:

54211

43211

32211

21211

10211

021

2121

21

222)2()2()1(

222)3()3()2(

222)4()4()3(

)2(222)5()5()4(

222)6()6()5(

21)6()6(

6 :Example

)1()1()()(

1)()(

rrr

rrr

rrr

rrr

rrr

rr

n

jrjrjrjr

nrnr

n

30

Step 3: Computing the fastest Step 3: Computing the fastest times(cont.)times(cont.)

• We observe that for each value of fi[j] depends only on the values of f1[j-1] and f2[j-1].

• By computing the fi[j] values in order of increasing station numbers we can compute the fw through the factory in time.

2j

)(n

31

The fastest way procedureThe fastest way procedure

32

The fastest way procedure The fastest way procedure (cont.)(cont.)

line 1, station 6

line 2, station 5

line 2, station 4

line 1, station 3

line 2, station 2

line 1, station 11 station"", line""print 5

][4

23

station"", line""print 2

1

),STATIONS(-PRINT*

ji

jili

nj

ni

li

nl

do

dowtofor

33

Matrix-Chain multiplicationMatrix-Chain multiplication

• We are given a sequence

• And we wish to compute

nAAA ...,,2,1

nAAA ...21

34

Matrix-Chain multiplicationMatrix-Chain multiplication (cont.) (cont.)

• Matrix multiplication is assosiative, and so all parenthesizations yield the same product.

• For example, if the chain of matrices is then the product can be fully paranthesized in five distinct way:

4321 AAAA421 ...AAA

)))(((

)))(((

)))(((

)))(((

)))(((

4321

4321

4321

4321

4321

AAAA

AAAA

AAAA

AAAA

AAAA

35

Matrix-Chain multiplicationMatrix-Chain multiplication

MATRIX-MULTIPLY (A,B)

if columns [A] ≠ rows [B]

then error “incompatible dimensions”

else for i←1 to rows [A]

do for j←1 to columns [B]

do C[i, j]←0

for k←1 to columns [A]

do C[ i, j ]← C[ i, j] +A[ i, k]*B[ k, j]

return C 36


Cost of the matrix multiplication:

An example:

505:

5100:

10010:

3

2

1

321

A

A

A

AAA

37


tions.multiplicascalar 7500 of totalafor ,by matrix this

multiply totionsmultiplicascalar 250050510another plus

,product matrix 510 thecompute totionsmultiplicascalar

5000 510010 perform we))((multiply weIf

3

21

321

A

AA

AAA

tions.multiplicascalar 000 75 of totalafor matrix, by this

multiply totionsmultiplicascalar 000 505010010another plus

,product matrix 50100 thecompute totionsmultiplicascalar

000 52 505100 perform we))((multiply weIf

1

32

321

A

AA

AAA

38


• The problem:

Given a chain of n matrices, where matrix Ai has dimension pi-1x pi, fully paranthesize the product

in a way that minimizes the number of scalar multiplications.

nAAA ...,,2,1

nAAA ...21

39


• Counting the number of alternative paranthesization : bn

)2(

2 if

trixnly one mathere is o ,1 if11

1

nn

n

kknk

n

b

nbb

n

b

40


• Based on the above defination we have

142332415

1322314

12213

2

12

1 1

bbbbbbbbb

bbbbbbb

bbbbb

bb

b

41


• Substituting the initial value for b we obtain

14

5

2

1

1

4233245

32234

223

2

1

bbbbbbb

bbbbb

bbb

b

b

42


• Let B(x) be the generating function

5

5

4

4

3

3

2

5

5

4

4

3

3

2

21

1

)(

xbxbxbxx

xbxbxbxbxb

xbxBi

i

i

43

Matrix-Chain multiplicationMatrix-Chain multiplication (cont.) (cont.)• Substituting in B(x) we can

write that...,, 321 bbb

5

4

5

23

5

32

5

4

5

5

4

3

4

22

4

3

4

4

3

2

3

2

3

3

22

2

1

xbxbbxbbxbxb

xbxbbxbxb

xbxbxb

xxb

xxb

44B(x)

Matrix-Chain multiplicationMatrix-Chain multiplication (cont.) (cont.)• Summing over the both sides

2

3

3

2

2

3

3

2

2

3

3

2

2

3

3

3

3

2

2

2

2

22

2

1

)}({

))((

)()()(

)(

)()()(

xBx

xbxbxxBx

xBxbxBxbxxBx

xbxbxxb

xbxbxxbxxBxxB

xxb

xxb

45

Matrix-Chain multiplicationMatrix-Chain multiplication (cont.) (cont.)• One way to express B(x) in closed form is

Expanding into a Taylor series about x=0 we get

Comparing the coefficients of x we finally show that

2

411)(

xxB

32 8421)( xxxxB

n

nb 246


Step 1: The structure of an optimal paranthesization(op)

• Find the optimal substructure and then use it to construct an optimal solution to the problem from optimal solutions to subproblems.

• Let Ai...j where i ≤ j, denote the matrix product Ai Ai+1 ... Aj

• Any parenthesization of Ai Ai+1 ... Aj must split the product between Ak and Ak+1 for i ≤ k < j.

47


The optimal substructure of the problem:

• Suppose that an op of Ai Ai+1 ... Aj splits the product between Ak and Ak+1 then the paranthesization of the subchain Ai Ai+1 ... Ak within this parantesization of Ai Ai+1 ... Aj must be an op of Ai Ai+1 ... Ak

48


Step 2: A recursive solution:• Let m[i,j] be the minimum number of scalar

multiplications needed to compute the matrix Ai...j where 1≤ i ≤ j ≤ n.

• Thus, the cost of a cheapest way to compute A1...n would be m[1,n].

• Assume that the op splits the product Ai...j between Ak and Ak+1.where i ≤ k <j.

• Then m[i,j] =The minimum cost for computing Ai...k and Ak+1...j + the cost of multiplying these two matrices. 49


Recursive defination for the minimum cost of paranthesization:

.if}],1[],[min

if0],[

1 jipppjkmkim

jijim

jkijki

50


To help us keep track of how to constrct an optimal solution we define s[ i,j] to be a value of k at which we can split the product Ai...j to obtain an optimal paranthesization.

That is s[ i,j] equals a value k such that

kjis

pppjkmkimjim jki

],[

],1[],[],[ 1

51


Step 3: Computing the optimal costs

It is easy to write a recursive algorithm based on recurrence for computing m[i,j].

But the running time will be exponential!...

52


Step 3: Computing the optimal costs

We compute the optimal cost by using a tabular, bottom-up approach.

53

Matrix-Chain multiplicationMatrix-Chain multiplication ( (Contd.Contd.))

MATRIX-CHAIN-ORDER(p)n←length[p]-1for i←1 to n do m[i,i]←0for l←2 to n do for i←1 to n-l+1 do j←i+l-1 m[i,j]← ∞ for k←i to j-1 do q←m[i,k] + m[k+1,j]+pi-1 pk pj

if q < m[i,j] then m[i,j] ←q

s[i,j] ←kreturn m and s 54


An example: matrix dimension A1 30 x 35A2 35 x 15A3 15 x 5A4 5 x 10A5 10 x 20A6 20 x 25

7125

11375201035043754]5,5[]4,2[

7125205351002625]5,4[]3,2[

1300020153525000]5,3[]2,2[

min]5,2[

51

531

521

pppmm

pppmm

pppmm

m

55


15750

0

9375

7875

15125

11875

4375

2625

10500

7125

0 0 0

575

2500

0

5000 1000

3500

0

750

3

3

3

3 3

3 3

3 1 5

2 1 3 4 5

4

5

3

6

2

3

4

5

61

2

5

4

32

3

5

4

62

1

1

i

i

j

j

A1 A2 A3 A4 A5 A6

ms

1000

2500

56


Step 4: Constructing an optimal solution

An optimal solution can be constructed from the computed information stored in the table s[1...n, 1...n].

We know that the final matrix multiplication is

The earlier matrix multiplication can be computed recursively.

nnsns AA ...1],1[],1[...1

57


PRINT-OPTIMAL-PARENS (s, i, j)1 if i=j

2 then print “Ai”3 else print “ ( “4 PRINT-OPTIMAL-PARENS (s, i, s[i,j])5 PRINT-OPTIMAL-PARENS (s, s[i,j]+1, j)6 Print “ ) ”

58


RUNNING TIME:

Recursive solution takes exponential time.

Matrix-chain order yields a running time of O(n3)

59

Elements of dynamic programmingElements of dynamic programming

When should we apply the method of Dynamic Programming?

Two key ingredients:

- Optimal substructure

- Overlapping subproblems

60

Elements of dynamic programming Elements of dynamic programming (cont.)(cont.)

Optimal substructure (os):

A problem exhibits os if an optimal solution to the problem contains within it optimal solutions to subproblems.

Whenever a problem exhibits os, it is a good clue that dynamic programming might apply.

In dynamic programming, we build an optimal solution to the problem from optimal solutions to subproblems.

Dynamic programming uses optimal substructure in a bottom-up fashion.

61


Overlapping subproblems:

When a recursive algorithm revisits the same problem over and over again, we say that the optimization problem has overlapping subproblems.

In contrast , a divide-and-conquer approach is suitable usually generates brand new problems at each step of recursion.

Dynamic programming algorithms take advantage of overlapping subproblems by solving each subproblem once and then storing the solution in a table where it can be looked up when needed.

62


Overlapping subproblems: (cont.)

1..4

1..1 3..42..4 1..2

2..2

1..3

1..1

4..4

3..4

3..3

2..3

4..4

4..4

2..2

1..12..2 2..2 4..4

3..3

3..3 2..3 1..2 3..3

1..1 2..23..3

The recursion tree of RECURSIVE-MATRIX-CHAIN( p, 1, 4). The computations performed in a shaded subtree are replaced by a single table lookup in MEMOIZED-MATRIX-CHAIN( p, 1, 4). 63

Matrix-Chain multiplicationMatrix-Chain multiplication ((Contd.Contd.))

RECURSIVE-MATRIX-CHAIN (p, i, j)1 if i = j2 then return 03 m[i,j] ←∞4 for k←i to j-15 do q←RECURSIVE-MATRIX-CHAIN (p, i, k) + RECURSIVE-MATRIX-CHAIN (p, k+1, j)+ pi-1 pk pj

6 if q < m[i,j]7 then m[i,j] ←q8 return m[i,j]

64


Overlapping subproblems: (cont.)Time to compute m[ i,j] by RECURSIVE-MATRIX-CHAIN:

We assume that the execution of lines 1-2 and 6-7 take at least unit time.

1

1

1

1

)(2

1for )1)()((1)(

,1)1(

n

i

n

k

niT

nknTkTnT

T

65


Overlapping subproblems: (cont.)

WE guess that

Using the substitution method with

).2()( nnT 12)( nnT

1

1

2

0

1

1

2

)12(2

22

22)(

n

n

n

i

i

n

i

i

n

n

nT

66


MemoizationThere is a variation of dynamic programming that often offers the

efficiency of the usual dynamic-programming approach while maintaining a top-down strategy.

The idea is to memoize the the natural, but inefficient, recursive algorithm.

We maintain a table with subproblem solutions, but the control structure for filling in the table is more like the recursive algorithm.

67


• Memoization (cont.)• An entry in a table for the solution to each subproblem is

maintained. • Eech table entry initially contains a special value to indicate that

the entry has yet to be filled.• When the subproblem is first encountered during the execution

of the recursive algorithm, its solution is computed and then stored in the table.

• Each subsequent time that the problem is encountered, the value stored in the table is simply looked up and returned.

68


1 MEMOIZED-MATRIX-CHAIN(p)2 n←length[p]-13 for i←1 to n4 do for j←i to n do m[i,j] ←∞return LOOKUP-CHAIN(p,1,n)

69


Memoization (cont.)

LOOKUP-CHAIN(p,1,n) 1 if m[i,j] < ∞ 2 then return m[i,j] 3 if i=j 4 then m[i,j] ←0 5 else for k←1 to j-1 6 do q← LOOKUP-CHAIN(p,i,k) + LOOKUP-CHAIN(p,k+1,j) + pi-1 pk pj

7 if q < m[i,j] 8 then m[i,j] ←q 9 return m[i,j]

70

Longest Common Subsequence Longest Common Subsequence

71


72


73


74


75


76


77


78


79


80


81


82

Documents

Dynamic Programming Lecture 9 Asst. Prof. Dr. İlker Kocabaş 1