Lecture 14 - web.stanford.edu

Lecture14Greedyalgorithms!

Announcements

• HW6DueFriday!

• TONSOFPRACTICEONDYNAMICPROGRAMMING

Lastweek

Roadmap

Graphs!

Asymptotic

Analysis

Dynamic

ProgrammingGreedyAlgs

MIDTERM

Future!

Moredetailedscheduleonthewebsite!

Thisweek

• Greedyalgorithms!

• Buildsonourideasfromdynamicprogramming

Greedyalgorithms

• Makechoicesone-at-a-time.

• Neverlookback.

• Hopeforthebest.

• Onenon-exampleofagreedyalgorithm:

• Knapsackagain

• Threeexamplesofgreedyalgorithms:

• ActivitySelection

• JobScheduling

• HuffmanCoding

Non-example

• UnboundedKnapsack.

• (Frompre-lectureexercise)

• UnboundedKnapsack:

• SupposeIhaveinfinitecopiesofalloftheitems.

• What’sthemostvaluablewaytofilltheknapsack?

• “Greedy”algorithmforunboundedknapsack:

• TacoshavethebestValue/Weightratio!

• Keepgrabbingtacos!

Weight:

Value:

6 2 4 3 11

20 8 14 3513

Capacity:10

Totalweight:10

Totalvalue:42

Totalweight:9

Totalvalue:39

ExamplewheregreedyworksActivityselection

FrisbeePractice

Orchestra

CS161study

TheoryLunch

TheorySeminar

Combinatorics

Seminar

Underwaterbasket

weavingclass

Math51Class

CS161Class

CS166Class

Section

CS161Office

Swimming

lessons

Programming

teammeeting

Socialactivity

Youcanonlydooneactivityatatime,andyouwantto

maximizethenumberofactivitiesthatyoudo.

Whattochoose?

Activityselection

• Input:

• Activitiesa1,a2,…,an• Starttimess1,s2,…,sn• Finishtimesf1,f2,…,fn

• Output:

• Howmanyactivitiescanyoudotoday?

GreedyAlgorithm

• Pickactivityyoucanaddwiththesmallestfinishtime.

• Repeat.

GreedyAlgorithm

• Repeat.

GreedyAlgorithm

• Repeat.

GreedyAlgorithm

• Repeat.

GreedyAlgorithm

• Repeat.

GreedyAlgorithm

• Repeat.

GreedyAlgorithm

• Repeat.

GreedyAlgorithm

• Repeat.

Atleastit’sfast

• Runningtime:

• O(n)iftheactivitiesarealreadysortedbyfinishtime.

• OtherwiseO(nlog(n))ifyouhavetosortthemfirst.

Whatmakesitgreedy?

• Ateachstepinthealgorithm,makeachoice.

• Hey,Icanincreasemyactivitysetbyone,

• Andleavelotsofroomforfuturechoices,

• Let’sdothatandhopeforthebest!!!

• Hope thatattheendoftheday,thisresultsinagloballyoptimalsolution.

Threequestions

1. Doesthisgreedyalgorithmforactivityselectionwork?

2. Ingeneral,whenaregreedyalgorithmsagoodidea?

3. The“greedy”approachisoftenthefirstyou’dthinkof…

• Whyarewegettingtoitnow,inWeek8?

Answers

• Yes.

• Whentheyexhibitespeciallyniceoptimalsubstructure.

• Relatedtodynamicprogramming!(WhichwedidinWeek7).

• Provingthatgreedyalgorithmsworkisoftennotsoeasy.

(Seemsto:IPython notebook…) (Butnowlet’sseewhy…)

Whydoesitwork?

• Wheneverwemakeachoice,wedon’truleoutanoptimalsolution.

There’ssomeoptimalsolutionthat

containsournextchoiceOurnext

choicewould

bethisone:

Toseethis,consider

OptimalSubstructure

• Subproblem i :

• A[i]=NumberofactivitiesyoucandoafterActivityi finishes.

Wanttoshow:whenwemakeachoiceak,theoptimalsolution

tothesmallersub-problemkwillhelpussolvesub-problemi

• Letak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes.

• ThenA[i]=A[k]+1.

A[k]:howmany

activitiescanIdohere?

A[i]:howmanyactivitiescanIdohere?

Proof• Letak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes.

• ClearlyA[i]≥ A[k]+1• SincewehaveasolutionwithA[k]+1activities.

• SupposetowardacontradictionthatA[i]> A[k]+1.

• There’ssomebettersolutiontosubproblem(i)that

doesn’tuseak• Sayaj endsfirstafterai inthatbettersolution.

• Removeaj andaddak fromthebettersolution.

a3 a7a4

Thesetwodon’tcount

forsub-problem(i)so

let’sgreythemout.

• SupposetowardacontradictionthatA[i]> A[k]+1.

• There’ssomebettersolutiontosubproblem(i)that

doesn’tuseak• Sayaj endsfirstafterai inthatbettersolution.

• Removeaj andaddak fromthebettersolution.

• Nowyouhaveasolutionofthesamesize…

butitincludesak soitmusthavesize≤A[k]+1.ak

a3 a7a3

• ClearlyA[i]≥ A[k]+1• SincewehaveasolutionwithA[k]+1activities.

• Andwejustshowed A[i]≤ A[k]+1• Bycontradiction

• Thatprovestheclaim.

Weneverruleoutanoptimalsolution

• We’ve shown:

• Ifwechooseak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes,thenA[i]=A[k]+1.

• Thatis:

• Assumethatwehaveanoptimalsolutionuptoai• Byaddingak wearestillontracktohitthatoptimalvalue

Sothealgorithmiscorrect

• Weneverruleoutanoptimalsolution

• Attheendofthealgorithm,we’vegotasolution.

• It’snotnotoptimal.

• Soitmustbeoptimal.

LuckytheLackadaisicalLemur

• InductiveHypothesis:• Afteraddingthet’th thing,thereisanoptimalsolutionthatextendsthecurrentsolution.

• Basecase:• Afteraddingzeroactivities,thereisanoptimalsolutionextendingthat.

• Inductivestep:• TODO

• Conclusion:• Afteraddingthelastactivity,thereisanoptimalsolutionthatextendsthecurrentsolution.

• Thecurrentsolutionistheonlysolutionthatextendsthecurrentsolution.

• Sothecurrentsolutionisoptimal.

PluckythePedanticPenguin

Inductivestep

• Supposethatafteraddingthet’th thing(Activityi),thereisanoptimalsolution:

• XactivitiesdoneandA[i]activitiesleft.

• Thenweaddthe(t+1)’st thing(Activityk).

• A[k]=A[i]- 1(bytheclaim)

• Now:

• X+1activitiesdoneandA[i]– 1activitiesleft.

• Samenumberasbefore!

• Stilloptimal.

• InductiveHypothesis:• Afteraddingthet’th thing,thereisanoptimalsolutionthatextendsthecurrentsolution.

• Basecase:• Afteraddingzeroactivities,thereisanoptimalsolutionextendingthat.

• Conclusion:• Afteraddingthelastactivity,thereisanoptimalsolutionthatextendsthecurrentsolution.

• Thecurrentsolutionistheonlysolutionthatextendsthecurrentsolution.

• Sothecurrentsolutionisoptimal.

PluckythePedanticPenguin

Commonstrategyforgreedyalgorithms

• Makeaseriesofchoices.

• Showthat,ateachstep,ourchoicewon’truleoutanoptimalsolution attheendoftheday.

• Afterwe’vemadeallourchoices,wehaven’truledoutanoptimalsolution,sowemusthavefoundone.

Commonstrategy(formally)forgreedyalgorithms

• InductiveHypothesis:

• Aftergreedychoicet,youhaven’truledoutsuccess.

• Basecase:

• Successispossiblebeforeyoumakeanychoices.

• Inductivestep:

• TODO

• Conclusion:

• Ifyoureachtheendofthealgorithmandhaven’truledoutsuccessthenyoumusthavesucceeded.

DPviewofactivityselection

• Thisalgorithmismostnaturallyviewedasa

greedyalgorithm.• Makegreedychoices

• Neverruleoutsuccess

• But,wecouldviewitasaDPalgorithm• Takeadvantageofoptimalsub-structureandfill

inatable.

• We’lldothatnow.• Justforpedagogy!

• (Thisisn’tthebestwaytothinkaboutactivity

selection).

RecipeforapplyingDynamicProgramming

• Step1:Identifyoptimalsubstructure.

• Step2:Findarecursiveformulationforthevalueoftheoptimalsolution.

• Step3:Usedynamicprogrammingtofindthevalueoftheoptimalsolution.

• Step4:Ifneeded,keeptrackofsomeadditionalinfosothatthealgorithmfromStep3canfindtheactualsolution.

• Step5:Ifneeded,codethisuplikeareasonableperson.

Optimalsubstructure

• Subproblem i:

• A[i]=numberofactivitiesyoucandoafterActivityi finishes.

Wedidthatalready

• Letak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes.

A[k]:howmany

activitiescanIdohere?

A[i]:howmanyactivitiescanIdohere?

Top-downDP

• InitializeaglobalarrayAto[None,…,None]

• Makea“dummy”activitythatendsattime-1.

• def findNumActivities(i):

• IfA[i]!=None:

• Return A[i]

• LetActivitykbetheactivityIcanfitinmyscheduleafterActivityi withthesmallestfinishtime.

• If thereisnosuchactivityk,setA[i]=0

• Else,A[i]=findNumActivities(k)+1

• Return A[i]

• Return findNumActivities(0)

Thisisaterriblewaytowritethis!

Theonlythingthatmattershereisthatthe

highlightedlinesareourrecursiverelationship.

SeeIPython notebookfor

implementation

Top-downDP

• InitializeaglobalarrayAto[None,…,None]

• InitializeaglobalarrayNextto[None,…,None]

• Makea“dummy”activitythatendsattime-1.

• def findNumActivities(i):• IfA[i]!=None:

• Return A[i]• LetActivitykbetheactivityIcanfitinmyscheduleafterActivityi withthesmallestfinishtime.

• If thereisnosuchactivityk,setA[i]=0• Else,A[i]=findNumActivities(k)+1and Next[i]=k• Return A[i]

• findNumActivities(0)

• Stepthrough“Next”arraytogetschedule.

Thisisaterriblewaytowritethis!

Theonlythingthatmattershereisthatthe

highlightedlinesareourrecursiverelationship.

SeeIPython notebookfor

implementation

Let’sstepthroughit.(SeeIPython notebookforcodewithsomeprintstatements)

Thislooksprettyfamiliar!!

Let’sstepthroughit.

• Startwiththeactivitywiththesmallestfinishtime.

Let’sstepthroughit

• Nowfindthenextactivitystilldo-ablewiththesmallestfinishtime,andrecurse afterthat.

• Ta-da!

It’sexactlythesame*asthegreedysolution!

*ifyouimplementthetop-downDPsolutionappropriately.

Sub-problemgraphview

• Divide-and-conquer:

Bigproblem

sub-problemsub-problem

sub-sub-

problem

sub-sub-

problem

sub-sub-

problem

sub-sub-

problem

sub-sub-

problem

• DynamicProgramming:

Bigproblem

sub-problemsub-problem

sub-sub-

problemsub-sub-

problem

sub-sub-

problem

sub-sub-

problem

sub-problem

• Greedyalgorithms:

Bigproblem

sub-sub-

problem

sub-problem

• Greedyalgorithms:

Bigproblem

sub-sub-

problem

sub-problem

• Notonlyisthereoptimalsub-structure:• optimalsolutionstoaproblemaremadeup

fromoptimalsolutionsofsub-problems

• buteachproblemdependsononlyone

sub-problem.

Answers

• Yes.

• Whentheyexhibitespeciallyniceoptimalsubstructure.

• Relatedtodynamicprogramming!(WhichwedidinWeek7).

• Provingthatgreedyalgorithmsworkisoftennotsoeasy.

Let’sseeafewmoreexamples

Anotherexample:

Scheduling

Overcommitted

StanfordStudent

CS161HW!

Callyourparents!

MathHW!

EconHW!

Practicemusicalinstrument!

ReadCLRS!

Haveasociallife!

Sleep!

Administrativestuffforyourstudentclub!

Dolaundry!

Meditate!

Scheduling

• ntasks

• Taski takesti hours

• Everythingisalreadylate!

• Foreveryhourthatpassesuntiltaski isdone,payci

• CS161HW,thenSleep:costs10⋅ 2+(10+8)⋅ 3=74units• Sleep,thenCS161HW:costs8⋅ 3+(10+8)⋅ 2=60units

CS161HW!

Sleep!

10hours

8hours

Cost:2 unitsper

houruntilit’sdone.

Cost:3unitsper

Optimalsubstructure

• Thisproblembreaksupnicelyintosub-problems:

JobA JobB JobC JobD

Supposethisistheoptimalschedule:

Thenthismustbetheoptimal

scheduleonjustjobsB,C,D.

Optimalsubstructure

• Seemsamenabletoagreedyalgorithm:

JobA JobB JobC JobD

Takethebestjobfirst Thensolvethisproblem

JobBJobC JobD

Takethebestjobfirst Thensolvethisproblem

JobBJobD

Takethebestjobfirst

(Thatone’seasyJ )

Thensolvethisproblem

Whatdoes“best”mean?

• Recipeforgreedyalgorithmanalysis:

• Wemakeaseriesofchoices.

• Weshowthat,ateachstep,ourchoicewon’truleoutanoptimalsolution attheendoftheday.

• Afterwe’vemadeallourchoices,wehaven’truledoutanoptimalsolution,sowemusthavefoundone.

JobA JobB JobC JobD

“Best”means:won’truleoutanoptimalsolution.

Theoptimalsolutiontothisproblemextendsanoptimalsolutiontothewholething.

Head-to-head

• Ofthesetwojobs,whichshouldwedofirst?

• Cost(AthenB)=x⋅z+(x+y) ⋅ w• Cost(BthenA)=y ⋅w+(x+y) ⋅z

xhours

y hours

Cost:z unitsper

Cost:w unitsper

AthenBisbetterthanBthenAwhen:

𝑥𝑧 + 𝑥 + 𝑦 𝑤 ≤ 𝑦𝑤 + 𝑥 + 𝑦 𝑧𝑥𝑧 + 𝑥𝑤 + 𝑦𝑤 ≤ 𝑦𝑤 + 𝑥𝑧 + 𝑦𝑧

𝑤𝑥 ≤ 𝑦𝑧𝑤𝑦 ≤

𝑧𝑥

Whatmattersistheratio:

costofdelaytimeittakes

Dothejobwiththe

biggestratiofirst.

• GivenjobssothatJobi takestime ti withcostci ,

• Thereisanoptimalschedulesothatthefirstjobistheonethatmaximizestheratioci/ti

• Proof:

• SayJobBmaximizesthisratio,andit’snotfirst:

• SwitchAandB!Nothingelsewillchange,andweshowedonthepreviousslidethatthecostwon’tincrease.

• RepeatuntilBisfirst.

JobA JobB

cA/tA >=cB/tB

JobC JobD

JobAJobBJobC JobD

Choosegreedily:Biggestcost/timeratiofirst

• Jobi takestime ti withcostci

• Thereisanoptimalschedulesothatthefirstjobistheonethatmaximizestheratioci/ti

• Soifwechoosejobsgreedilyaccordingtoci/ti,weneverruleoutsuccess!

GreedySchedulingSolution

• scheduleJobs(JOBS):

• SortJOBSbytheratio:

• 𝒓𝒊 = 𝒄𝒊𝒕𝒊 =

costofdelayingjobitimejobitakestocomplete

• Saythatsorted_JOBS[i] isthejobwiththei’th biggestri• Return sorted_JOBS

TherunningtimeisO(nlog(n))

Nowyoucangoaboutyourschedule

peacefully,intheoptimalway.

Formally,useinduction!

• Inductivehypothesis:

• Thereisanoptimalorderingsothatthefirsttjobsaresorted_JOBS[:t].

• Basecase:

• Whent=0,thisreads:“Thereisanoptimalorderingsothatthefirst0jobsare[]”

• That’strue.

• InductiveStep:

• Boilsdownto:thereisanoptimalorderingonsorted_JOBS[t:]sothatsorted_JOBS[t]isfirst.

• ThisfollowsfromtheLemma.

• Conclusion:

• Whent=n,thisreads:“Thereisanoptimalorderingsothatthefirstnjobsaresorted_JOBS.”

• aka,whatwereturnedisanoptimalordering.

SLIDESKIPPEDINCLASS

Whathavewelearned?

• Agreedyalgorithmworksforscheduling

• Thisfollowedthesameoutlineasthepreviousexample:

• Identifyoptimalsubstructure:

• Findawaytomake“safe”choicesthatwon’truleoutanoptimalsolution.

• largestratiosfirst.

JobA JobB JobC JobD

OnemoreexampleHuffmancoding

• everyday english sentence• 01100101011101100110010101110010011110010110010001100001011110010010000001100101011011100110011101101100011010010111001101101000001000000111001101100101011011100111010001100101011011100110001101100101

• qwertyui_opasdfg+hjklzxcv• 01110001011101110110010101110010011101000111100101110101011010010101111101101111011100000110000101110011011001000110011001100111001010110110100001101010011010110110110001111010011110000110001101110110

OnemoreexampleHuffmancoding

• everyday english sentence• 01100101 0111011001100101 01110010011110010110010001100001011110010010000001100101 011011100110011101101100011010010111001101101000001000000111001101100101 011011100111010001100101 011011100110001101100101

• qwertyui_opasdfg+hjklzxcv• 01110001011101110110010101110010011101000111100101110101011010010101111101101111011100000110000101110011011001000110011001100111001010110110100001101010011010110110110001111010011110000110001101110110

ASCIIisprettywasteful.Ife

showsupsooften,weshould

haveamoreparsimoniousway

ofrepresentingit!

Supposewehavesomedistributiononcharacters

A B C D E F

Percentage

Letter

Forsimplicity,

let’sgowiththis

made-upexample

Howtoencodethemas

efficientlyaspossible?

Try0(likeASCII)

A B C D E F

Percentage

Letter

000 011001 010 100 101

• Everyletterisassignedabinarystring

ofthreebits.

Wasteful!

• 110and111areneverused.

• Weshouldhaveashorterwayof

representingA.

A B C D E F

Percentage

Letter

0 100 01 10 11

• Everyletterisassignedabinarystring

ofoneortwobits.

• Themorefrequentlettersgetthe

shorterstrings.

• Problem:

• Does000meanAAAorBAorAB?

Try2:prefix-freecoding

A B C D E F

Percentage

Letter

01 00101 110 111 100

• Everyletterisassignedabinarystring.

• Morefrequentlettersgetshorterstrings.

• Noencodedstringisaprefixofanyother.

10010101

Confusingly,“prefix-freecodes”arealsosometimes

called“prefixcodes”(includinginCLRS).

A B C D E F

Percentage

Letter

01 00101 110 111 100

10010101 F

A B C D E F

Percentage

Letter

01 00101 110 111 100

10010101 FB

A B C D E F

Percentage

Letter

01 00101 110 111 100

10010101 FBA

Question:Whatisthemost

efficientwaytodoprefix-free

coding?(Thisisn’tit).

Aprefix-freecodeisatree

D:16A:45

B:13F:5 C:12 E:9

100 101 110 111Aslongasalltheletters

showupasleaves,this

codeis prefix-free.

B:13belowmeansthat‘B’

makesup13%ofthe

charactersthateverappear.

Sometreesarebetterthanothers

D:16A:45

B:13F:5 C:12 E:9

100 101 110 111

• Imaginechoosingaletteratrandomfromthelanguage.

• Notuniform,butaccordingtoourhistogram!

• Thecostofatreeistheexpectedlengthoftheencodingofthatletter.

Expectedcostofencodingaletterwiththistree:

𝟐 𝟎. 𝟒𝟓 + 𝟎. 𝟏𝟔 + 𝟑 𝟎. 𝟎𝟓 + 𝟎. 𝟏𝟑 + 𝟎. 𝟏𝟐 + 𝟎. 𝟎𝟗 = 𝟐. 𝟑𝟗

K 𝑃 𝑥 ⋅ depth(𝑥)�

QRSTRUV P(x)isthe

probability

ofletterx

Thedepthinthe

treeisthelength

oftheencoding

Question

• GivenadistributionP onletters,findthelowest-costtree,where

cost(tree) = K 𝑃 𝑥 ⋅ depth(𝑥)�

XYZ[Y\V P(x)isthe

probability

ofletterx

Thedepthinthe

treeisthelength

oftheencoding

Optimalsub-structure

• Supposethisisanoptimaltree:

Thenthisisan

optimaltreeon

fewerletters.

Otherwise,wecould

changethissub-tree

andendupwitha

betteroveralltree.

Inordertodesignagreedyalgorithm

• Thinkaboutwhatlettersbelonginthissub-problem...

10What’sasafe

choicetomake

fortheselower

sub-trees?

Infrequent

elements!Wewantthemaslow

downaspossible.

Solutiongreedilybuildsubtrees,startingwiththeinfrequentletters

D:16A:45 B:13 F:5C:12 E:9

100 101 110

1110 1111

Expectedcostofencodingaletter:

𝟏 ⋅ 𝟎. 𝟒𝟓+

𝟑 ⋅ 𝟎. 𝟒𝟏+

𝟒 ⋅ 𝟎. 𝟏𝟒= 𝟐. 𝟐𝟒

Whatexactlywasthealgorithm?

• Createanodelikeforeachletter/frequency

• Thekeyisthefrequency(16inthiscase)

• LetCURRENT bethelistofallthesenodes.

• while len(CURRENT)>1:

• X andY← thenodesinCURRENT withthesmallestkeys.

• CreateanewnodeZ withZ.key =X.key +Y.key

• SetZ.left =X,Z.right =Y

• AddZ toCURRENT andremoveX andY

• returnCURRENT[0]

F:5 E:9

XD:16A:45 B:13 C:12

Doesitwork?

• Yes.

• Samestrategy:

• Showthatateachstep,thechoiceswearemakingwon’truleoutanoptimalsolution.

• Lemma:

• Supposethatxandyarethetwoleast-frequentletters.Thenthereisanoptimaltreewherexandyaresiblings.

D:16A:45 B:13 F:5C:12 E:9

Lemmaproofidea

• Saythatanoptimaltreelookslikethis:

• Whathappenstothecostifweswapxfora?• thecostcan’tincrease;awasmorefrequentthanx,andwejustmadeitsencodingshorter.

• Repeatthislogicuntilwegetanoptimaltreewithxandyassiblings.• Thecostneverincreasedsothistreeisstilloptimal.

Ifxandyarethetwoleast-frequentletters,there

isanoptimaltreewherexandyaresiblings.

Lowest-levelsibling

nodes:atleastoneof

themisneitherxnory

Lemmaproofidea

• Saythatanoptimaltreelookslikethis:

• Whathappenstothecostifweswapxfora?• thecostcan’tincrease;awasmorefrequentthanx,andwejustmadeitsencodingshorter.

• Repeatthislogicuntilwegetanoptimaltreewithxandyassiblings.• Thecostneverincreasedsothistreeisstilloptimal.

Lowest-levelsibling

nodes:atleastoneof

themisneitherxnory

Ifxandyarethetwoleast-frequentletters,there

isanoptimaltreewherexandyaresiblings.

Proofstrategyjustlikebefore

• Lemma:

D:16A:45 B:13 F:5C:12 E:9

Proofstrategyjustlikebefore

• Lemma:

That’senoughtoshowthatwe

don’truleoutoptimalityafter

thefirststep.

Whataboutoncewestart

groupingstuff?

D:16A:45 B:13 F:5C:12 E:9

Lemma2thisdistinctiondoesn’treallymatter

F:5E:9

C:12B:13

A:45 A:4555

G:25H:30

Thefirstthingisanoptimal

treeon{A,B,C,D,E,F}

ifandonlyif

thesecondthingisan

optimaltreeon{A,G,H}

• Foraproof:

• SeeCLRS,Lemma16.3

• Rigorousalthoughpresentedinaslightlydifferentway

• SeeLectureNotes14

• Abitsketchier,butpresentedinthesamewayashere

• Proveityourself!

• Thisisthebest!

Siggi theStudiousStork

Gettingallthedetails

isn’tthatimportant,but

youshouldconvince

yourselfthatthisistrue.

Lemma2thisdistinctiondoesn’treallymatter

Together

• Lemma1:

• Lemma2:

• WemayaswellimaginethatCURRENTcontainsonlyleaves.

• Theseimply:

• Ateachstep,ourchoicedoesn’truleoutanoptimaltree.

Thewholeargument

• Inductivehypothesis:• afterthet’th step,

• thereisanoptimaltreecontainingthecurrentsubtreesas“leaves”

• Basecase:• afterthe0’thstep,

• thereisanoptimaltreecontainingallthecharacters.

• Conclusion:• afterthelaststep,

• thereisanoptimaltreecontainingthiswholetreeasasubtree.

• aka,• afterthelaststepthetreewe’veconstructedisoptimal.

Afterthet’th step,we’vegotabunchofcurrentsub-trees:

Inductivehyp.asserts

thatoursubtreescanbe

assembledintoan

optimaltree:

Inductivestep

• Supposethattheinductivehypothesisholdsfort-1

• Aftert-1steps,thereisanoptimaltreecontainingallthecurrentsub-treesas“leaves.”

• Wanttoshow:

• Aftertsteps,thereisanoptimaltreecontainingallthecurrentsub-treesasleaves.

We’vegotabunchofcurrentsub-trees:

saythatxandyarethetwosmallest.

Inductivestep

• ByLemma2,mayaswelltreatas

Inductivestep

• ByLemma2,mayaswelltreatas

• Inparticular,optimaltreesonthisnewalphabetcorrespondtooptimaltreesontheoriginalalphabet.

Inductivestep

• Ouralgorithmwoulddothisatlevelt:

wa a=x+y

Inductivestep

wa a=x+y

Lemma1impliesthatthere’s

anoptimalsub-treethatlooks

likethis;aka,whatour

algorithmdidokay.

Inductivestep

wa a=x+y

Lemma2againsaysthat

there’sanoptimaltreethat

lookslikethis

Inductivestep

wa a=x+y

Lemma2againsaysthat

there’sanoptimaltreethat

lookslikethis

Thisiswhatwe

wantedtoshowfor

theinductivestep.

Inductiveoutline:

• Inductivehypothesis:• afterthet’th step,

• thereisanoptimaltreecontainingthecurrentsubtreesas“leaves”

• Basecase:• afterthe0’thstep,

• thereisanoptimaltreecontainingallthevertices.

• Conclusion:• afterthelaststep,

• thereisanoptimaltreecontainingthiswholetreeasasubtree.

• aka,• afterthelaststepthetreewe’veconstructedisoptimal.

Afterthet’th step,we’vegotabunchofcurrentsub-trees:

Inductivehyp.asserts

thatoursubtreescanbe

assembledintoan

optimaltree:

Whathavewelearned?

• ASCIIisn’tanoptimalwaytoencodeEnglish,sincethedistributiononlettersisn’tuniform.

• HuffmanCodingisanoptimalway!

• Tocomeupwithanoptimalschemeforanylanguageefficiently,wecanuseagreedyalgorithm.

• Tocomeupwithagreedyalgorithm:

• Identifyoptimalsubstructure

• Findawaytomake“safe”choicesthatwon’truleoutanoptimalsolution.

• Createsubtreesoutofthesmallesttwocurrentsubtrees.

RecapI

• Threeexamples:

• ActivitySelection

• SchedulingJobs

• HuffmanCoding

RecapII

• Ofteneasytowritedown

• Butmaybehardtocomeupwithandhardtojustify

• Thenaturalgreedyalgorithmmaynotalwaysbecorrect.

• Aproblemisagoodcandidateforagreedyalgorithmif:

• ithasoptimalsubstructure

• thatoptimalsubstructureisREALLYNICE

• solutionsdependonjustoneothersub-problem.

Nexttime

• GreedyalgorithmsforMinimumSpanningTree!

• Pre-lectureexercise:candidategreedyalgorithmsforMST

Before nexttime

Lecture 14 - web.stanford.edu

Documents

Chemical Engineering 160/260 Important Concepts, …web.stanford.edu/class/cheme160/review2.pdf · Chemical Engineering 160/260 Important Concepts, Lecture 9-16 Lecture 9: Introduction

Lecture 1 Course Overview Introduction to OOP/Java Manu …web.stanford.edu/class/cs193j/slidesSummer2003/Lecture 01 - CS193J... · • OOP in Java – inheritance, abstract superclasses,

Lecture 14 Diffraction Gratings - Yonseitera.yonsei.ac.kr/class/2020_2_1/lecture/Lecture 14... · 2020. 10. 27. · Microsoft PowerPoint - Lecture 14 Diffraction Gratings Author:

CS110 Lecture 09: Threads - web.stanford.edu

Designing Applications that See Lecture 7: Object …web.stanford.edu/class/cs377s/slides/cs377s-Jan29.pdf · Designing Applications that See Lecture 7: ... OpenCV tutorial ... Rapid

Lecture’14:’’ Tracking’mo3on’features’’ –op3cal’ﬂow’vision.stanford.edu/.../lecture14_optical_flow_cs131.pdfFei-Fei Li Lecture 14 - Lecture’14:’’ Tracking’mo3on’features’’

Natural Language Processing with Deep Learning …web.stanford.edu/class/cs224n/slides/cs224n-2020-lecture...Lecture Plan Lecture 11: ConvNetsfor NLP 1.Announcements (5 mins) 2.Intro

Lecture 9: Logistic regression (v2) Ramesh Johari …web.stanford.edu/~rjohari/teaching/notes/226_lecture9...Lecture 9: Logistic regression (v2) Ramesh Johari ramesh.johari@stanford.edu

Lecture 10 - web.stanford.edu · Lecture 10 Sinusoidal steady-state and frequency response • sinusoidal steady-state • frequency response • Bode plots 10–1. Response to sinusoidal

JavaScript Programming - web.stanford.edu · CS142 Lecture Notes - JavaScript Programming Functional Programming Imperative: ... CS142 Lecture Notes - JavaScript Programming Stepping

Graphs - web.stanford.edu

Lecture 7 Frequency Domain Methods - web.stanford.edu

CS107, Lecture 14 - web.stanford.edu · CS107, Lecture 14 Assembly: Function Calls and the Runtime Stack Reading: B&O 3.7. 2 Learning Assembly Moving data around Arithmetic and logical

Lecture 2: Making Sequences of Good Decisions Given a ...web.stanford.edu/class/cs234/slides/lecture2.pdf · Emma Brunskill (CS234 Reinforcement Learning)Lecture 2: Making Sequences

LECTURE NOTES ON GEOMETRIC FEATURES OF THE ALLEN…web.stanford.edu/~ochodosh/AllenCahnSummerSchoolUConn2018.p… · LECTURE NOTES ON GEOMETRIC FEATURES OF THE ALLEN{CAHN EQUATION

Lecture 14: MCTS =1With many slides from or derived from ...web.stanford.edu/class/cs234/slides/cs234_2018_l14_annotated.pdf · from experience fS 1;A 1;R 2;:::;S Tg This is a supervised

39 - web.stanford.edu

lecture project topics - web.stanford.edu

Lecture 14: MCTS =1With many slides from or derived from ...web.stanford.edu/.../cs234_2018_l14_annotated.pdf · Lecture 14: MCTS 2 Emma Brunskill CS234 Reinforcement Learning. Winter

CS107 Lecture 2 - web.stanford.edu