Embed Size (px)
Citation preview
Divide-and-Conquer 1
Divide-and-Conquer
7 2 9 4 2 4 7 9
7 2 2 7 9 4 4 9
7 7 2 2 9 9 4 4
Divide-and-Conquer 2
Outline and ReadingDivide-and-conquerReview Merge-sort Recurrence Equations Iterative substitution Recursion trees Guess-and-test The master method
Integer Multiplication
Divide-and-Conquer 3
Divide-and-ConquerDivide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in
two or more disjoint subsets S1, S2, …
Recur: solve the subproblems recursively
Conquer: combine the solutions for S1, S2, …, into a solution for S
The base case for the recursion are subproblems of constant sizeAnalysis can be done using recurrence equations
Divide-and-Conquer 4
Merge-Sort ReviewMerge-sort on an input sequence S with nelements consists of three steps: Divide: partition S into
two sequences S1 and S2of about n2 elements each
Recur: recursively sort S1and S2
Conquer: merge S1 and S2 into a unique sorted sequence
Algorithm mergeSort(S, C)Input sequence S with n
elements, comparator COutput sequence S sorted
according to Cif S.size() > 1
(S1, S2) partition(S, n/2) mergeSort(S1, C)mergeSort(S2, C)S merge(S1, S2)
Divide-and-Conquer 5
Recurrence Equation Analysis
The conquer step of merge-sort consists of merging two sorted sequences, each with n2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b.Likewise, the basis case (n < 2) will take at b most steps.Therefore, if we let T(n) denote the running time of merge-sort:
We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation. That is, a solution that has T(n) only on the left-hand side.
2if)2/(22if
)(nbnnTnb
nT
Divide-and-Conquer 6
Iterative SubstitutionIn the iterative substitution, or “plug-and-chug,” technique, we iteratively apply the recurrence equation to itself and see if we can find a pattern:
Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n. So,
Thus, T(n) is O(n log n).
ibnnT
bnnTbnnTbnnT
bnnbnTbnnTnT
ii
)2/(2...
4)2/(23)2/(22)2/(2
))2/())2/(2(2)2/(2)(
44
33
22
2
nbnbnnT log)(
Divide-and-Conquer 7
The Recursion TreeDraw the recursion tree for the recurrence relation and look for a pattern:
depth T’s size
0 1 n
1 2 n2
i 2i n2i
… … …
2if)2/(22if
)(nbnnTnb
nT
time
bn
bn
bn
…
Total time = bn + bn log n(last level plus all previous levels)
Divide-and-Conquer 8
Guess-and-Test MethodIn the guess-and-test method, we guess a closed form solution and then try to prove it is true by induction:
Guess: T(n) < cn log n.
Wrong: we cannot make this last line be less than cn log n
nbncnncnnbnncn
nbnnncnbnnTnT
logloglog)2log(log
log))2/log()2/((2log)2/(2)(
2iflog)2/(22if
)(nnbnnTnb
nT
Divide-and-Conquer 9
Guess-and-Test Method, Part 2
Recall the recurrence equation:
Guess #2: T(n) < cn log2 n.
if c > b.So, T(n) is O(n log2 n).In general, to use this method, you need to have a good guess and you need to be good at induction proofs.
ncnnbncnncnncn
nbnncnnbnnnc
nbnnTnT
2
2
2
2
logloglog2log
log)2log(loglog))2/(log)2/((2
log)2/(2)(
2iflog)2/(22if
)(nnbnnTnb
nT
Divide-and-Conquer 10
Master MethodMany divide-and-conquer recurrence equations have the form:
The Master Theorem:
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
nfbnafnfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
Master Method
Intuitively, depending on how f(n) compares to nlogba, Case 1 is when f(n) < nlogba
“polynomially smaller…”
Case 2 is when f(n) = nlogba
Case 3 is when f(n) > nlogba
“polynomially larger…”
But this is only roughly speaking and some f(n)are not supported!
e.g., T(n)=T(n/2) + log n Divide-and-Conquer 11
Divide-and-Conquer 12
Master Method, Example 1The form:
The Master Theorem:
Example:
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
nfbnafnfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
nnTnT )2/(4)(
Solution: logba=2, so case 1 says T(n) is O(n2).
Divide-and-Conquer 13
Master Method, Example 2The form:
The Master Theorem:
Example:
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
nfbnafnfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
nnnTnT log)2/(2)( Solution: logba=1,
so case 2 says T(n) is O(n log2 n).
Divide-and-Conquer 14
Master Method, Example 3The form:
The Master Theorem:
Example:
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
nfbnafnfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
nnnTnT log)3/()( Solution: logba=0,
so case 3 says T(n) is O(n log n).
Divide-and-Conquer 15
Master Method, Example 4The form:
The Master Theorem:
Example:
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
nfbnafnfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
2)2/(8)( nnTnT Solution: logba=3,
so case 1 says T(n) is O(n3).
Divide-and-Conquer 16
Master Method, Example 5The form:
The Master Theorem:
Example:
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
nfbnafnfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
3)3/(9)( nnTnT Solution: logba=2,
so case 3 says T(n) is O(n3).
Divide-and-Conquer 17
Master Method, Example 6The form:
The Master Theorem:
Example:
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
nfbnafnfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
1)2/()( nTnTSolution: logba=0,
so case 2 says T(n) is O(log n).
(binary search)
Divide-and-Conquer 18
Master Method, Example 7The form:
The Master Theorem:
Example:
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
nfbnafnfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
nnTnT log)2/(2)( Solution: logba=1,
so case 1 says T(n) is O(n).
(heap construction)
Divide-and-Conquer 19
Iterative “Proof” of the Master Theorem
Using iterative substitution, let us see if we can find a pattern:
We then distinguish the three cases as The first term is dominant Each part of the summation is equally dominant The summation is a geometric series
1)(log
0
log
1)(log
0
log
2233
22
2
)/()1(
)/()1(
. . .)()/()/()/(
)()/()/())/())/((
)()/()(
n
i
iia
n
i
iin
bb
bb
bnfaTn
bnfaTa
nfbnafbnfabnTanfbnafbnTabnbnfbnaTa
nfbnaTnT
Divide-and-Conquer 20
Integer MultiplicationAlgorithm: Multiply two n-bit integers I and J. Divide step: Split I and J into high-order and low-order bits
We can then define I*J by multiplying the parts and adding:
So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2). But that is no better than the algorithm we learned in grade
school.
ln
h
ln
h
JJJ
III
2/
2/
2
2
lln
hln
lhn
hh
ln
hln
h
JIJIJIJI
JJIIJI
2/2/
2/2/
222
)2(*)2(*
Divide-and-Conquer 21
An Improved Integer Multiplication Algorithm
Algorithm: Multiply two n-bit integers I and J. Divide step: Split I and J into high-order and low-order bits
Observe that there is a different way to multiply parts:
So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by
the Master Theorem. Thus, T(n) is O(n1.585).
ln
h
ln
h
JJJ
III
2/
2/
2
2
lln
hllhn
hh
lln
llhhhlhhlllhn
hh
lln
llhhhllhn
hh
JIJIJIJI
JIJIJIJIJIJIJIJI
JIJIJIJJIIJIJI
2/
2/
2/
2)(2
2])[(2
2]))([(2*
