Zhiyang Yao

Reference:

T.H.Cormen, C.E.Leiserson and R.L.Riverst.Introduction to Algorithms. The MIT Press, 1997.

BackgroundDivide-and-conquer algorithms partition the problem intoindependent subproblems, solve the subproblems recursively, andthen combine their solutions to solve the original problem.What happens if subproblems are not independent, that is, what ifsubproblems depend on the solution to another problem? In thiscase, the divide-and-conquer technique does more work thannecessary, repeatedly solving common subproblems.

IntroductionDynamic programming is a metatechnique (not an algorithm) likedivide and conquer. It predates computer programming. Dynamicprogramming solves every subproblem just once, and saves thesubproblem answers in a table.

What is Dynamic Programming?

Dynamic Programming is similar to Divide & Conquer–Both are algorithm design methods–Both solve problems by combining solutions to subproblems

Dynamic Programming is different from Divide & Conquer–Dynamic Programming solves optimization problems• Goal is to find the best of many possible answers–Dynamic Programming is applied when subproblems are notindependent•“Not independent” means the same subproblem can show upmany times•Each subproblem is computed just once, then the answer issaved

Dynamic Programming vs. Devide-and-Conquer

Top-Down Bottom-Up

Steps of the development of a dynamic-programmingalgorithm

•Characterize the structure of an optimal solution

•Recursively define the value of an optimal solution

•Computer the value of an optimal solution in a bottom-upfashion

•Construct an optimal solution from computed information (ifonly the value of an optimal solution is required, this step canbe omitted.)

Implementing Dynamic Programming

Example: matrix-chain multiplication

A=[p×q], B=[q × r], then the scalar multiplications to computer AB is p × q × r

=

×

333231

232221

131211

43

33

23

13

42

32

22

12

41

31

21

11

34333231

24232221

14131211

ccc

ccc

ccc

b

b

b

b

b

b

b

b

b

b

b

b

aaaa

aaaa

aaaa

A1A2 A3 A4 = (A1(A2 (A3 A4)))

= (A1((A2 A3 )A4))

= ((A1A2 )(A3 A4))

= ((A1(A2 A3 )A4))

= (((A1A2 )A3 )A4)

A product of matrices is fully parenthesized if it is either a single matrix or the product oftwo fully parenthesized matrix products, surrounded by parentheses.

Different parenthesized pattern will product different number ofscalar multiplications(the less the better)

For example, A1=[10×5], A2=[5×20], A3=[20×10], A4=[10×5]:

Case 1:

(A1(A2 (A3 A4))): (A3 A4 )=[20×5], perform 20 ×10 ×5 =1000 multiplications

(A2 (A3 A4)) =[5×5], perform 5 ×20 ×5 =500 multiplications

(A1(A2 (A3 A4))) =[10×5], perform 10 ×5 ×5 =250 multiplications

______________________________________________________________________

total: 1000+500+250=1750

Case 2:

((A1A2 )(A3 A4)): (A1 A2) =[10×20], perform 10 ×5 ×20 =1000 multiplications

(A3 A4 )=[20×5], perform 20 ×10 ×5 =1000 multiplications

((A1A2 )(A3 A4)) =[10×5], perform 10 ×20 ×5 =1000 multiplications

______________________________________________________________________

total: 1000+1000+1000=3000

Example: matrix-chain multiplication(Con.)

The matrix-chain multiplication problem can be stated as follows:given a chain ⟨A1 A2 ... An ⟩ of n matrices, where for i=1,2,…,n,matrix Ai has dimension p i-1 × pi, fully parenthesize the product A1

× A2 ... × An in a way that minimizes the number of scalarmultiplication.

Exhaustively checking all possible parenthesizations does notyield an efficient algorithm: O(pn), where p>2.

Example: matrix-chain multiplication(Con.)

Using Dynamic Programming

Step 1: The structure of an optimal parenthesization

Observation:

Suppose an optimal parenthesization of the product A1…An splits the product between Ak andAk+1:

(A1…Ak)(Ak+1…An). The cost of this optimal parenthesization is the cost of computing(A1…Ak) plus cost of (Ak+1…An) plus the cost of multiplying the two together.

Then, the parenthesization of (A1…Ak) must be optimal!!! So does (Ak+1…An), WHY?????

If there were a less costly way to parenthesize (A1…Ak), substituting that partenthesization inthe optimal patenthesization of (A1…An) would produce another parenthesization of (A1…An)whose cost was lower than the optimum->a contradiction

Conclusion:

An optimal solution to an instance of the matrix-chain multiplicationproblem contains within it optimal solutions to subproblem instances.

This is the optimal structure of the applicability of dynamic programming

Using Dynamic Programming

Step 2: A recursive solution

Goal: define the value of an optimal solution recursively in terms ofthe optimal solutions to subproblems.

Set Ai…j=(Ai…Aj), m[i,j] be the minimum number of scalar multiplications needed tocompute the matrix Ai…j.

If we know how to get m[i,j], then the minimal cost of compute A1…n=(A1A2…An) ofm[1,n].

If we know ((Ai…Ak)(Ak+1…Aj)) produce the optimal parenthesization, then :m[i,j]=m[i,k]+m[k+1,j]+pi-1pkpj

For [i,j], we can try to assign k=i,i+1,…,j-1 and to find the optimal k value.

So the recursive definition of the m[i,j] can be written as:

+++=−

<≤}],1[],[{

0],[

1min jkijki

pppjkmkimjimIf i=j,

if i<j.

Using Dynamic Programming

Step 3: Computing the optimal costs by a bottom-up approach

(filling up the dynamic-programming table)

Observation: there exists overlapping subproblems

If we compute the cost recursively, we will end-up withexponential time. What we need to do is to build a table to savethose values during the computation.

m[i,j] A1

[10×5]A2

[5×20]A3

[20×10]A4

[10×5]

1 m[1,1]=0 m[2,2]=0 m[3,3]=0 m[4,4]=02 m[1,2]=

1000m[2,3]=1000

m[3,4]=1000

34

m[i,j] A1

[10×5]A2

[5×20]A3

[20×10]A4

[10×5]

1 0 0 0 0

2 1000 1000 1000

3 m[1,3]=1500 m[2,4]=1250

4

1500.300010201001000320]3,3[]2,1[

;15001051010000310]3,2[]1,1[min]3,1[ =

=××++=++=××++=++

=pppmm

pppmmm

Using Dynamic Programmingfilling up the dynamic-programming table

A1 A2 A3=(A1 (A2 A3))=((A1 A2 )A3)

A1

[10×5]A2

[5×20]A3

[20×10]A4

[10×5]

1 0 0 0 0

2 1000 1000 1000

3 1500 1250

4 m[1,4]=1500

1500

.20005000150051010]4,4[]3,1[

;300010001000100052010]4,3[]2,1[

;1500250125005510]4,2[]1,1[

min]4,1[ =

=++=××++=++=××++

=++=××++=

mm

mm

mm

m

Using Dynamic Programmingfilling up the dynamic-programming table

A1

[10×5]A2

[5×20]A3

[20×10]A4

[10×5]

1 0 0 0 0

2 1000(A1A2)[10×20]

1000(A2A3)[5×10]

1000(A3A4)[20×5]

3 1500(A1(A2A3))[10×10]

1250((A2A3)A4)[5×5]

4 1500(A1((A2A3)A4)[10×5]

Using Dynamic Programming

Step 4: Constructing an optimal solution

If we can save the optimal parenthesization in each step, then we caneasily give the final optimal solution.

In this example, the final solution is (A1((A2A3)A4), m[1,4]=1500

Comparation with the recursive method1…4

1…1 2…4 1…2 3…4 1…3 4…4

2…2 3…4 2…3 4…4 1…1 2…2 3…3 4…4 1…1 2…3 1…2 3…3

2…2 3…4 2…3 4…4 2…2 3…3 1…1 2…2

1…1 2…2 3…3 4…4

1…2 2…3 3…4

1…3 2…4

1…4

Time: O(pn)

Time: O(n3)

Dynamic Programming canbe used when:•The goal is to find anoptimum answer among manypossible answers•There is a simple recursivealgorithm that solves theproblem, but it takes too muchtime because recursive callsare repeated

Example applications•Matrix-chainmultiplication•Longest commonsubsequence•All-pairs shortest paths in agraph•Minimum weighttriangulation of a convexpolygon•Approximate stringmatching (finding the bestmatch when deletion andsubstitution are allowed)

Dynamic Programming Conclusion

Elements of DynamicProgramming:

•Optimal substructure

•Overlapping subproblems

Using Dynamic Programming in Multiple Cutter Selectionfor Milling Operations

Cutter Size/Coverable Area Chart

0.00

500.00

1000.00

1500.00

2000.00

2500.00

3000.00

5 10 15 20 25 30 35 40 45 50

Cutter Size

Co

ve

rab

le A

rea

Coverable Area

(a) Given Part (b) Chart

Problem statement: How to find an optimal set of cutterssuch that the whole region can be cutted within the shortesttime?

We have n cutting tools, the index numbers are 0, 1, …, n, in decreasing sequence, where the size ofthe 0th cutter is infinity, and the size of the 1st cutter is the upper bond of cutter size for this problem,the nth cutter is the lower bond of cutter size for this problem.

P0 is the initial stock, and Pn is the final part. Basically the cutting problem becomes transforming theinitial stock P0 to the final part Pn by using a sequence of cutters. The goal of our optimizationprogram is to find the best sequence from the upper (may or may not include it) to the lower (mustinclude it) bond.

i∀ , Pi = part produced from P0 by using tool i.TTC i is the tool changing time for cutter i;TC i(Pi,Pj) is the cutting time by transforming Pj into Pi using cutter iTL i is the tool loading time for cutter i.

Dynamic Programming Formula:

Objective function = );,(*nPnT

}),(),({),( *

0

* min ijiiijij

i TLPPTCTTCPjTPiT +++=<<=

; ;0 ni <=<

If TCi(Pi, Pj)=0, TTCi=0 & TLi=0; else, TTCI = average tool changing time & TLI =average tool loading time

.0),0( 0* =PT

Using Dynamic Programming in Multiple CutterSelection for Milling Operations(Con.)

40 35 30 25 20 15 10 540 1.45 2.93 4.43 6.43 8.13 10 12.2 17.435 1.77 3.27 5.27 6.97 8.84 11.04 16.2430 2.2 4.2 5.9 7.77 9.97 15.1725 3.2 4.9 6.77 8.97 14.1720 4.2 6.07 8.27 13.4715 5.9 8.1 13.3310 9.2 14.45 21.2

Dynamic Table

Time: O(n3)

Using Dynamic Programming in Multiple CutterSelection for Milling Operations(Con.)

What is Greedy Algorithms?

Given a problem with many possible solutions, we want to findthe best solution, called optimal (usually max or min).

Definition: A greedy algorithm is a procedure that makes anoptimal choice at each of its steps.

Optimizing at each step does not guarantee that an overalloptimal solution is produced. Greedy algorithms sometimesyield optimal solutions, but not to every problem. There maybe more than one optimal solution. We examine what sort ofproblems can be solved with greedy algorithms.

How to get lest number of coins fortotal of $9.43 ?

$1 25c 10c 5c 1c

9 1 1 1 3

Example---Scheduling Problem

Task StartingTime

EndingTime

1 1 3

2 2 5

3 4 6

4 6 8

5 8 9

1

2

3

4

5

0 1 2 3 4 5 6 7 8 9

How to schedule those tasks such that the maximal number oftasks can be performed without time overlapping?

1

2

3

4

5

0 1 2 3 4 5 6 7 8 9

1

31

31 4

531 4

Steps:

1. Sorting those tasks byincreasing finishing time.

2. Using Greedy-Algorithmto solve it and get the optimalsolution.

For this example, we canproof that the result IS theoptimal.

Example---Scheduling Problem(Con.)

A greedy algorithm obtains an optimal solution to a problemby making a sequence of choices. For each decision point inthe algorithm, the choice that seems est at the moment ischosen. This heuristic strategy does not always produce anoptimal solution, but sometimes it does.

Two ingredients:

•Greedy choice property

•a globally optimal solution can be arrived at by making alocally optimal(greedy) choice.

•Optimal substructure

•if an optimal solution to the problem contains within itoptimal solutions to subproblems.

How can we tell is a greedy algorithm will solve aparticular optimization problem?

Greedy Algorithms Dynamic Programming

Greedy-choice property: wemake whatever choice seemsbest at the moment and thensolve the subproblems arisingafter the choices so far, but itcannot depend on any futurechoices o on the solutions tosubproblems

We make a choice at eachstep, but the choice maydepend on the solutions tosubproblems.

Top-down fashion Bottom-up fashion

Greedy Algorithm vs. Dynamic Programming

Example: Knapsack problems

Fractional Knapsack Problem:

5

$6 $10 $12

W.

P.

Unite Value: 6 > 5 > 4

Knapsack

1 2 3$6

$10

$8

Total: $24

0-1 Knapsack Problem:

1 2 35

$6 $10 $12

W.

P.

Unite Value: 6 > 5 > 4

Knapsack

1

2

Total: $16

$6

$10

Gold Dust Gold Block

How to get as muchvalue as possible inone knapsack?

Conclusion

Applications:

minimum-spanning-tree algorithms

Dijkstra’s algorithm for shortest paths from asingle source

Chvatal’s greedy set-covering heuristic

Using Dijkstra’s algorithm to find the best set of cutters.

Time O(N2)

InitialPart P0

Part P1 Part Pi Part Pj FinalPart Pn

Cutter CiCutter C1

Cutter Cj Cutter Cn

Cutter Ci Cutter Cj

Cutter Cj

Cutter Cn

Cutter Cn

Cutter Cn