Advanced Algorithm Design and Analysis (Lecture 6) SW5 fall 2004 Simonas Šaltenis E1-215b [email protected]

Advanced Algorithm Design and Analysis (Lecture 6)

SW5 fall 2004Simonas Šaltenis [email protected]

AALG, lecture 6, © Simonas Šaltenis, 2004 2

Dynamic Programming

Goals of the lecture: to understand the principles of dynamic

programming; use the examples of computing optimal

binary search trees, approximate pattern matching, and coin changing to see how the principles work;

to be able to apply the dynamic programming algorithm design technique.


Coin changing

Problem: Change amount A into as few coins as possible, when we have n coin denominations:

denom[1] > denom[2] > … > denom[n] = 1 For example:

A = 12, denom = [10, 5, 1] 10 5 1

10 6 1

Greedy algorithm works fine (for this example) Prove greedy choice property

What if A =12, denom = [10, 6, 1]?


Dynamic programming

Dynamic programming: A powerful technique to solve optimization problems

Structure: To arrive at an optimal solution a number of choices are

made Each choice generates a number of sub-problems Which choice to make is decided by looking at all

possible choices and the solutions to sub-problems that each choice generates

• Compare this with a greedy choice. The solution to a specific sub-problem is used many

times in the algorithm


Questions to think about

Construction: What are the sub-problems? Which parameters

define each sub-problem? Which choices have to be considered in each

step of the algorithm? In which order do we have to solve sub-

problems? How are the trivial sub-problems solved?

Analysis: How many different sub-problems are there in

total? How many choices have to be considered in

each step of the algorithm?


Edit Distance

Problem definition: Two strings: s[0..m-1], and t[0..n-1] Find edit distance dist(s,t)– the smallest number

of edit operations that turns s into t Edit operations:

• Replace one letter with another• Delete one letter• Insert one letter

• Example: ghost delete g

host insert u houst replace t by e house


Sub-problmes

What are the sub-problems? Goal 1: To have as few sub-problems as

possible Goal 2: Solution to the sub-problem should be

possible by combining solutions to smaller sub-problems.

Sub-problem: di,j = dist(s[0..i], t[0..j])

Then dist(s, t) = dm-1,n-1


Making a choice

How can we solve a sub-problem by looking at solutions of smaller sub-problems to make a choice? Let’s look at the last symbol: s[i] and t[j]. Do

whatever is cheaper:• If s[i] = t[j], then turn s[0..i-1] to t[0..j-1], else replace

s[i] by t[j] and turn s[0..i-1] to t[0..j-1]• Delete s[i] and turn s[0..i-1] to t[0..j]• Insert insert t[j] at the end of s[0..i-1] and turn s[0..i]

to t[0..j-1]


Recurrence

In which order do we have to solve sub-problems?

How do we solve trivial sub-problems? To turn empty string to t[0..j], do j+1 inserts To turn s[0..i] to empty string, do i+1 deletes

1, 1

, 1,

, 1

0 if [ ] [ ]

1 else

min 1

1

i j

i j i j

i j

s i t jd

d d

d


Algorithm

EditDistance(s[0..m-1], t[0..n-1]) 01 for i -1 to m-1 do dist[i,-1] = i+102 for j 0 to n-1 do dist[-1,j] = j+1 03 for i 0 to m-1 do04 for j 0 to n-1 do05 if s[i] t[j] then 06 dist[i,j] = min(dist[i-1,j-1], dist[i-1,j]+1, dist[i,j-1]+1) 07 else 08 dist[i,j] = 1 + min(dist[i-1,j-1], dist[i-1,j], dist[i,j-1])09 return dist[m-1,n-1]

What is the running time of this algorithm?


Approximate Text Searching

Given p[0..m-1], find a sub-string of t (w = t[i,j]), such that dist(p, w) is minimal. Brute-force: compute edit distance between p

and all possible sub-strings of t. Running time? What are the sub-problems? adi,j= min{dist(p[0..i], t[l..j]) | 0 l j+1}

The same recurrence as for di,j! The edit distance from p to the best match then

is the minimum of adm-1,0,adm-1,1, … , adm-1,n-1 Trivial problems are solved different:

• Think how.


Optimal BST

Static database the goal is to optimize searches Let’s assume all

searches are successful

Node (ki)

Depth Probability (pi)

Contribution

A 1 0.1 0.2

B 0 0.2 0.2

C 3 0.16 0.64

D 2 0.12 0.36

E 3 0.18 0.72

F 1 0.24 0.48

Total: 1.00 2.6

1 1

Expected cost of search in T ( ( ) 1) 1 ( )n n

T i i T i ii i

depth k p depth k p

C

D

E

FA

B


Sub-problems

Input: keys k1, k2, …, kn

Sub-problem options: k1, k2, …, kj

ki, ki+1, …, kn

Natural choice: pick as a root kr (1 r n) Generates sub-problems: ki, ki+1, …, kj

Lets denote the expected search cost e[i,j]. If kr is root, then

1

where ( , )j

ll

w i j p

( , ) [ , 1] ( , 1) [ 1, ] ( 1, ) ,re i j p e i r w i r e r j w r j


Solving sub-problems

How do I solve the trivial problem?

if ( , ) min{ [ , 1] [ 1, ] ( , )} if

i

i r j

p i je i j e i r e r j w i j i j

Thus,

( , ) [ , 1] [ 1, ] ( , )e i j e i r e r j w i j

Observe that

( , ) [ , 1] [ 1, ].rw i j w i r p w r j

In which order do I have to solve my problems?


Finishing up

I can compute w(i,j) using w(i,j-1) w(i,j) = w(i,j-1) + pj

An array w[i,j] is filled in parallel with e[i,j] array

Need one more array to note which root kr gave the best solution to (i, j)-sub-problem

What is the running time?


Elements of Dynamic Programming

Dynamic programming is used for optimization problems A number of choices have to be made to arrive

at an optimal solution At each step, consider all possible choices and

solutions to sub-problems induced by these choices (compare to greedy algorithms)

The order of solving of the sub-problems is important – from smaller to larger

Usually a table of sub-problem solutions is used


Elements of Dynamic Programming

To be sure that the algorithm finds an optimal solution, the optimal sub-structure property has to hold the simple “cut-and-paste” argument usually

works, but not always! Longest simple path example –

no optimal sub-structure!


Coin Changing: Sub-problems

A =12, denom = [10, 6, 1]?

What could be the sub-problems? Described by which parameters?

How do we solve sub-problems?

( 1, ) if [ ]( , )

min{ ( 1, ),1 ( , [ ])} if [ ]

c i j denom i jc i j

c i j c i j denom i denom i j

10 6 1

How do we solve the trivial sub-problems? In which order do I have to solve sub-

problems?

Documents

Advanced Algorithm Design and Analysis (Lecture 6) SW5 fall 2004 Simonas Šaltenis E1-215b [email protected]