ADVANCED ALGORITHMS LECTURE 4: D&C, DYNAMIC PROGRAMMING

�1

LECTURE 4

ANNOUNCEMENTS

▸ Homework 0 grading (close to done)

▸ Homework 1 posted; due on Monday Sep 10, 11:59 PM

▸ Start early! (Post comments/questions on canvas…)

▸ Unless immediate, argue why algorithm is correct

▸ Three steps — algorithm description, proof of correctness, complexity analysis

�2

LECTURE 4

LAST CLASS

�3

▸ Divide and conquer

▸ Break instance into smaller pieces, solve pieces separately, combine solutions

▸ Instances disjoint in examples we saw — not necessary (today)

▸ Merge-sort algorithm

▸ Recurrences

▸ Guess and prove by induction

▸ Plug and chug

▸ Recurrence tree

link oniMaster theorem homepage

Akira Bazzi Theorem

LECTURE 4

DIVIDE AND CONQUER

�4

PROBLEM INSTANCE

Divide step

Solve sub-problems

FULL SOLUTIONCombine/“conquer”

LECTURE 4

TODAY’S PLAN

�5

▸ More examples — divide and conquer

▸ Continue integer multiplication

▸ Median finding

▸ Introduction — dynamic programming

LECTURE 4

EXAMPLE 2: MULTIPLYING INTEGERS

�6

Question: given two n digit numbers, how efficiently can we multiply them?

▸ Elementary school algorithm: takes time O(n2)

▸ Basic divide and conquer — recap

LECTURE 4

DIVIDE AND CONQUER

�7

A = B =X Y Z W

A = 10n/2X + YSome observations: B = 10n/2Z + W

AB = 10nXZ + 10n/2(YZ + XW ) + YW

▸ Divide and conquer alg: compute XZ, YZ, XW, YW, compute sum above

▸ Gives recurrence: T(n) = 4T(n/2) + O(n)

n

Mz Mz X OO O Mz Mz

El D 7L'mazn digits

LECTURE 4

SOLVING RECURRENCE

�8

Tht 4TH Oln

1Tfn 042 f

E g mastertheorem

Corl plug a chug

Can we do better than elementaryschool Arg

LECTURE 4

CAN WE DO BETTER?

�9

Reason to hope: we only need to find three quantities:

XZ, (XW + YZ), YW

LECTURE 4

CAN WE DO BETTER?

�10

Reason to hope: we only need to find three quantities:

XZ, (XW + YZ), YW

Kartsuba’s observation:

XW + YZ = (X + Y )(W + Z) − XZ − YW

r

a

we needonly THREE multiplications

TIM 3 TIZI Oln

LECTURE 4

OVERALL RUNNING TIME

�11

TIM 3M c n g

Cin 131C I 1347

log C n 1 z cn 1 32T ngN

I

cnfltZztfzIt.r.JCatEHHcn.fzfzreaani.s.s 37171

LECTURE 4

EVEN BETTER?

�12

▸ Can we do linear time?

▸ HW1 — will see something approaching linear

I 35N

mK ite2 N

a Fast Fourier Transform Ofmlog n time alg42

Open problem if we can do better

LECTURE 4

COMPUTING THE MEDIAN

�13

Question: given an unsorted list A[0], A[1], …, A[n-1], find the (n/2)th smallest element

Divide and conquer?

Sort thearray output Ithsmallest O n log n0cal

An

n

medians of the pieces don't givemuch meaningful information

i l

LECTURE 4

KEY INSIGHT: “APPROXIMATE MEDIAN” HELPS!

�14

AH AH Aln i

tfAf AIDANTAIND Ifs

Fddi 12 elements

t213 13

Observations

1 we can now restrict search to

a 2.13 sized array een

2 needtobe careful answer is

not median of sub arraybut f

1h

Smallest element

ila qb6n

Given an array some k find theKth

smallest elementSelect A K

1 a select A Z Select Bg Eall elements

in Eu

1 n T Iz Oh time takentofindsub array

Ent E t C n t B

n l t Zz t E t Ept c'n

B

t n

213

Select Ez Z

How does recursive callwork

B24313Bx

Let a be njrd smallestelement of B

myth smallestelement of B is the

smallest element of B

select B Z FRida select A K wilt

a find approx medianselect B k

tidesThaty

cursive call10h a

givesdetails new array B sat IBIL F IAI

LECTURE 4

MEDIAN OF MEDIANS IS APPROXIMATE MEDIAN!

�15

Trick

AIA Afi AIn

o

Tfn 1 Fo 101Mt Tff

LECTURE 4

DIVIDE AND CONQUER — OVERVIEW

�16

▸ Problem “cleanly” divides into not-too-many sub-problems

▸ Solutions easily (efficiently) combined

▸ Leads to simple, efficient solutions

▸ Analysis by induction, complexity by recurrences

LECTURE 4

OVERLAP IN RECURSIVE CALLS

�17

▸ Divide & conquer: usually sub-problems don’t interact “much”

▸ What if they do? Can run into wastefulness — solving same sub-problem many times…

▸ Motivation for next topic — dynamic programming

LECTURE 4

NEW PROBLEM: SUBSET SUM

�18

Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.

isAfol t Akid t AEsB

all subsets find if any sum to S

2 n time

LECTURE 4

DIVIDE AND CONQUER

�19

▸ Try 1: divide into two halves

I 4

Alo f AlnDT T T T

o s

is.tl1 n Sti TIE

ntogs

s 1logan

LECTURE 4

RUNNING TIME

�20

logn

LECTURE 4

TRY 2

�21

▸ See if first element is picked or not

Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.

Ilookfords in AID Aln B lookfor a sumof

a sumof s Afo in AID

Takes time 2hAlnD

LECTURE 4

TRY 2

�22

▸ See if first element is picked or not

▸ Is this really divide and conquer?

Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.

LECTURE 4

RUNNING TIME

�23

LECTURE 4

LOOKING MORE CLOSELY — EXAMPLE

�24

A[] = {1, 2, 3, 5, 7, 9, 10, 11}; S = 20

LECTURE 4

AVOIDING MULTIPLE SOLVES ON “SAME” INSTANCE

�25

▸ Store answers!

▸ How much are we storing?

▸ Running time