Upload
trevor-woods
View
229
Download
1
Embed Size (px)
Citation preview
Divide-and-Conquer
Recursive in structure – Divide the problem into several smaller
sub-problems that are similar to the original but smaller in size
– Conquer the sub-problems by solving them recursively. If they are small enough, just solve them in a straightforward manner.
– Combine the solutions to create a solution to the original problem
An Example: Merge Sort
Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each
Conquer: Sort the two subsequences recursively using merge sort.
Combine: Merge the two sorted subsequences to produce the sorted answer.
Merge-Sort (A, p, r)
INPUT: a sequence of n numbers stored in array A
OUTPUT: an ordered sequence of n numbers
1. if p < r
2. then q [(p+r)/2]
3. Merge-Sort (A, p, q)
4. Merge-Sort (A, q+1, r)
5. Merge (A, p, q, r)
Analysis of Merge Sort
Divide: computing the middle takes (1) Conquer: solving 2 sub-problem takes 2T(n/2) Combine: merging n-element takes (n)
Total:T(n) = (1) if n = 1
T(n) = 2T(n/2) + (n) if n > 1
T(n) = (n lg n) (CLRS/Chapter 4)
Recurrence Relations
Recurrences (Chapter 4)– Substitution Method– Iteration Method– Master Method
Arising from Divide and Conquer
(e.g. MERGE-SORT)T(n) = (1) if n c
T(n) = a T(n/b) + D(n) + C(n) otherwise
Substitution Method
Guessing the form of the solutions, then using mathematical induction to find the constants and show the solution works.
It works well when it is easy to guess. But, there is no general way to guess the correct solution.
An Example
Solve: T(n) = 3T(n/3) + n
T(n) 3c n/3 lg n/3 + n
c n lg (n/3) + n
= c n lg n - c n lg3 + n
= c n lg n - n (c lg 3 - 1)
c n lg n
* The last step is true for c 1 / lg3.
Making a Good Guess
Guessing a similar solution to the one that you have seen before– T(n) = 3T(n/3 + 5) + n similar to T(n) = 3T(n/3) + n
when n is large, the difference between n/3 and (n/3 + 5) is insignificant
Another way is to prove loose upper and lower bounds on recurrence and then reduce the range of uncertainty.– Start with T(n) = (n) & T(n) = O(n2) T(n) = (n log n)
Subtleties
When the math doesn’t quite work out in the induction, try to adjust your guess with a lower-order term. For example:
– We guess T(n) O(n) for T(n) = 3T(n/3)+ 4, but we have T(n) 3c n/3 + 4 = c n + 4
– New guess is T(n) c n - b, where b 0
T(n) 3(c n/3 - b)+4 = c n - 3b + 4 = c n - b - (2b-4)
Therefore, T(n) c n - b, if 2b - 4 0 or if b 2
Changing Variables
Use algebraic manipulation to turn an unknown recurrence similar to what you have seen before. – Consider T(n) = 2T(n1/2) + lg n
– Rename m = lg n and we have
T(2m) = 2T(2m/2) + m
– Set S(m) = T(2m) and we have
S(m) = 2S(m/2) + m S(m) = O(m lg m)
– Changing back from S(m) to T(n), we have
T(n) = T(2m) = S(m) = O(m lg m) = O(lg n lg lg n)
Avoiding Pitfalls
Be careful not to misuse asymptotic notation. For example:
– We can falsely prove T(n) = O(n) by guessing T(n) c n for T(n) = 2T(n/2) + n
T(n) 2c n/2 + n
c n + n
= O(n) Wrong!– The err is that we haven’t proved T(n) c n
Exercises
Solution of T(n) = T(n/2) + 1 is O(lg n)
Solution of T(n) = 2T(n/2 + 17) + n is O(n lg n)
Solve T(n) = 2T(n1/2) + 1 by making a change of variables. Don’t worry whether values are integral.
Iteration Method
Expand (iterate) the recurrence and express it as a summation of terms dependent only on n and the initial conditions
The key is to focus on 2 parameters– the number of times the recurrence needs to be
iterated to reach the boundary condition– the sum of terms arising from each level of the
iteration process Techniques for evaluating summations can
then be used to provide bounds on solution.
An Example
Solve: T(n) = 3T(n/4) + n T(n) = n + 3T(n/4) = n + 3[ n/4 + 3T(n/16) ] = n + 3[n/4] + 9T(n/16) = n + 3[n/4] + 9 [n/16] + 27T(n/64)
T(n) n + 3n/4 + 9n/16 + 27n/64 + … + 3log4 n(1) n (3/4)i + (nlog43)
= 4n+ o(n) = O(n)
Recursion Trees
Keep track of the time spent on the subproblems of a divide and conquer algorithm
A convenient way to visualize what happens when a recursion is iterated
Help organize the algebraic bookkeeping necessary to solve the recurrence
Merge Sort
n
n/2
n/4
n/4
n/2
n/4
n/4
Running times to merge two sublists
Running time to sort the left sublist
Running Time
n
n/2
n/4
n/4
n/2
n/4
n/4
n=n
2¢(n/2) = n
4¢(n/4) = nlg n
Total: n lg n
Recursion Trees and Recurrences
Useful even when a specific algorithm is not specified– For T(n) = 2T(n/2) + n2, we have
Recursion Trees
T(n) = (n2)
Recursion Trees
For T(n) = T(n/3) + T(2n/3) + n
T(n) = O(n lg n)
Master Method
Provides a “cookbook” method for solving recurrences of the form
T(n) = a T(n/b) + f(n)Assumptions:
– a 1 and b 1 are constants
– f(n) is an asymptotically positive function
– T(n) is defined for nonnegative integers– We interpret n/b to mean either n/b or n/b
The Master Theorem
With the recurrence T(n) = a T(n/b) + f(n) as in the previous slide, T(n) can be bounded asymptotically as follows:
1. If f(n)=O(nlogba-) for some constant > 0, then T(n)= (nlogba).
2. If f(n) = (nlogba), then T(n) = (nlogba lg n).
3. If f(n) = ( nlogba+ ) for some constant > 0, and if a f(n/b) c f(n) for some constant c < 1 and all sufficiently large n, then T(n)= (f(n)).
Simplified Master Theorem
Let a 1 and b > 1 be constants and let T(n) be the recurrence
T(n) = a T(n/b) + c nk
defined for n 0.
1. If a > bk, then T(n) = ( nlogba ).
2. If a = bk, then T(n) = ( nk lg n ).
3. If a < bk, then T(n) = ( nk ).
Examples
T(n) = 16T(n/4) + n– a = 16, b = 4, thus nlogba = nlog416 = (n2)– f(n) = n = O(nlog416 - ) where = 1 case 1.– Therefore, T(n) = (nlogba ) = (n2)
T(n) = T(3n/7) + 1– a = 1, b=7/3, and nlogba = nlog 7/3 1 = n0 = 1– f(n) = 1 = (nlogba) case 2.– Therefore, T(n) = (nlogba lg n) = (lg n)
Examples (Cont.)
T(n) = 3T(n/4) + n lg n– a = 3, b=4, thus nlogba = nlog43 = O(n0.793)– f(n) = n lg n = (nlog43 + ) where 0.2 case 3.– Therefore, T(n) = (f(n)) = (n lg n)
T(n) = 2T(n/2) + n lg n– a = 2, b=2, f(n) = n lg n, and nlogba = nlog22 = n– f(n) is asymptotically larger than nlogba, but not
polynomially larger. The ratio lg n is asymptotically less than n for any positive . Thus, the Master Theorem doesn’t apply here.
Exercises
Use the Master Method to solve the following: T(n) = 4T(n/2) + n
T(n) = 4T(n/2) + n2
T(n) = 4T(n/2) + n3