ADVANCED ALGORITHMS LECTURE 6: DP, GREEDY ALGORITHMS
�1
LECTURE 6
ANNOUNCEMENTS
▸ Homework 1 due on Monday Sep 10, 11:59 PM
▸ Video issues last class
▸ Comment on notes — JeffE’s material
�2
LECTURE 6
LAST CLASS
�3
▸ Dynamic programming
▸ Recursive algorithms — observe that same sub-problems solved many times
▸ Key idea: avoid re-solving by storing answers!
▸ How many “different problem instances” arise?
▸ Examples: subset sum and coin change problems
tn s
I emojis
LECTURE 6
OVERLAP IN RECURSIVE CALLS
�4
A = {1, 2, 3, 5, 7, 11, 14, 15} , S = 20
Use first element Don’t use first element
A’ = {2, 3, 5, 7, 11, 14, 15} , S’ = 19
A’’ = {5, 7, 11, 14, 15} , S’ = 17 A’’ = {5, 7, 11, 14, 15} , S’ = 17
Key observation: every sub-problem has a suffix of A as the array, and some target sum in the range [0,…,S]. Thus only n(S+1) sub-problems!
if Afi are notintegers
If all betsafe
LECTURE 6
TODAY’S PLAN
�5
▸ Conclude DP — a couple more examples from a high level
▸ Greedy algorithms
LECTURE 6
RECAP: DYNAMIC PROGRAMMING
�6
▸ Design recursive algorithm
▸ Observe that number of distinct sub-problems is “small”
▸ Store answers! (memorize)
▸ Top-down vs. Bottom-up
▸ Challenge in using DP: formalizing sub-problems
more in Hwa
LECTURE 6
POPULAR E.G.: LONGEST INCREASING SUB-SEQUENCE
�7
▸ Natural recursive algorithm?
Problem: given a sequence of numbers A[0], A[1], …, A[n-1], find longest “sub-sequence” whose elements are increasing
A107 AfD A147 Alto
f 3g2 10
7y
Split into cases based on whether thefirstelement is in the sub sequence or not
recursive call on remaining list
procedureM's Afo All Aln B find target
increasing sub seg that includes Afo
Answer to original problem
miaxMls Afi Afia Aln D
all
4 3Mls 411,2 3
GI write recursion for Mls 1,2 3 3Mis l 2,3
Mls Ato Atl Aln B
His t Afi AH findMI Ali Alief AlnB
ouput i f m
Correctness increasing
AT Take anynsubseg that includes
Afo Let it be Afo Afif Afif Alia
We know that Afi Alo so it will beconsidered in the recursive procedure
Now we can argue inductively that
It Mislaid Ali HI rtl
r.u.i.byI sftaIhifheystudent'ssuggestiondoes Not work
a
I bsgMl o InD
LECTURE 6
#(SUB-PROBLEMS) AND RUN TIME
�8
sub problems n
Running time n
Moira defining MIS iskeyotherwiseprocedure isn'tcorrect
should be carefulabout defining the
right sub problem
LECTURE 6
TRAVELING SALESMAN PROBLEM
�9
Problem: given undirected graph with edge wts, find a path of least total weight that visits every vertex precisely once and returns to start.
LECTURE 6
TRAVELING SALESMAN PROBLEM
�10
▸ Trivial algorithm?
Problem: given undirected graph with edge wts, find a path of least total weight that visits every vertex precisely once and returns to start.
ni canp
check all permutations see
which one has leastcost
F 3,6 on fin Nn al D n
Dynamicorithn 2n nu
key define sub problem verycarefully
LECTURE 6
TRAVELING SALESMAN PROBLEM
�11
▸ Trivial algorithm?
▸ Many heuristics for geometric cases
Problem: given undirected graph with edge wts, find a path of least total weight that visits every vertex precisely once and returns to start.
LECTURE 6
RECURSIVE ALGORITHM
�12
LECTURE 6
SUB-PROBLEMS, RUNNING TIME
�13
LECTURE 6
SUB-PROBLEMS, RUNNING TIME
�14
LECTURE 6
CAN WE DO BETTER?
�15
▸ Likely not — NP hard problem
▸ We will see that finding “approximately good” solution possible!
LECTURE 6
GREEDY ALGORITHMS
�16
▸ Greedy == “myopic”
▸ Algorithm makes sequence of decisions — chooses best at each step, without taking future into account
Greedy algorithmsalmost neues work
often give insights abouthowto solve
sometimes theyactually
LECTURE 6
EXAMPLE
�17
Scheduling: given set of jobs with processing times p1, p2, …, pn, find an order of processing so that sum of completion times is minimized
1 i 2 4 i 3
C 1 C I 3 i Cz 7 Cg 10
Intuitive idea do jobs with small first
Process in order of increasing pi 9rdgfnh.fm
LECTURE 6
NATURAL HEURISTIC?
�18
LECTURE 6
CORRECTNESS
�19
Clem Processing jobs in this order isthe best
possibleContradict
Proof1 Suppose the optimalorder
v u Ucouldnothaveit iz im beenoptorderI suppose we swap j
ti tiz t
ti tti.tpizmtiztk.TKi Pi Epi i similarly pi Epi
LECTURE 6
CORRECTNESS — PROOF 2
�20
LECTURE 6
EXAMPLE — MATCHING
�21
Matching: suppose we have n children, n gifts and “happiness” values Hij. Assign gifts to children to maximize “total happiness”
C G Hyvalue ofgiftGj to
Cz 0 Ga child C
o ciii li y
C o x Gnn
LECTURE 6
GREEDY ALGORITHM
�22
Go throughchildren
Ci CzaC o O For each child assign
gift from the nomaininggifts that has the
most value
Xcx
Greedy does not alwaysG X10
find best solution
LECTURE 6
CORRECTNESS
�23
LECTURE 6
CORRECTNESS
�24