Heuristic Search (Informed Search) - … Search •The idea is to develop a domain specific heuristic function h(n). •h(n) guessesthe cost of getting to the goal from node n (the

1McIlraith&Allin,CSC384,UniversityofToronto,Winter2018

HeuristicSearch(InformedSearch)


HeuristicSearch

• Inuninformedsearch,wedon’ttrytoevaluatewhichofthenodesonthefrontier/OPENaremostpromising.Wenever“look-ahead”tothegoal.

E.g.,inuniformcostsearchwealwaysexpandthecheapestpath.Wedon’tconsiderthecostofgettingtothegoalfromtheendofthecurrentpath.

• Oftenwehavesomeotherknowledgeaboutthemeritofnodes,e.g.,goingthewrongdirectioninRomania.


HeuristicSearchMerit ofafrontier/OPENnode:differentnotionsofmerit.• Ifweareconcernedaboutthecostofthesolution,wemightwantanotionofmeritofhowcostlyitistogettothegoalfromthatsearchnode.

• Ifweareconcernedaboutminimizingcomputation insearchwemightwanttoconsiderhoweasyitistofindthegoalfromthatsearchnode.

• Wewillfocusonthe“costofsolution”notionofmerit.


HeuristicSearch

• Theideaistodevelopadomainspecificheuristicfunctionh(n).

• h(n)guesses thecostofgettingtothegoalfromnoden(thecostofcompletingthepaththatiscapturedbythestateofnoden).

• Therearedifferentwaysofguessingthiscostindifferentdomains.I.e.,heuristicsaredomainspecific.

“Asthecrowflies”– Straightlineheuristic

Onthemap,thenumbersbetweencitiesrepresentthedrivingdistancebetweencitiesonpotentiallywigglyroads,eventhoughtheyaredrawnasstraightlines.Contrastthistotheline-of-sight/``asthecrowflies”distancewhichignoreswigglesintheroad,cliffs,bridges,andassumesyoucanjustdriveinastraightlinefromonecitytoanother.



Planning a path from Arad to Bucharest, we can utilize the straight line distance from each city to our goal as a heuristic/guess of the actual distance. This lets us plan our trip by picking cities at each time point that minimize the distance to our goal.

Example:StraightLineDistance


HeuristicSearch

• Ifh(n1)<h(n2) thismeansthatweguessthatitischeapertogettothegoalfromn1 thanfromn2.

• Werequirethat• h(n)=0 foreverynodenwhosestatesatisfiesthegoal.• Zerocostofgettingtoagoalnodefromn.


Usingonlyh(n):Greedybest-firstsearch(GreedyBFS)

• Weuseh(n)torankthenodesonthefrontier/OPEN.• Alwaysexpandnodewithlowesth-value.

• Wearegreedilytryingtoachievealowcostsolution.

• However,thismethodignoresthecostofgettington,soitcanbeleadastrayexploringnodesthatcostalottogettobutseemtobeclosetothegoal:

S

n1

n2

n3

Goal

→ stepcost=10

→ stepcost=100h(n3)=50h(n1)=70

[S][n3,n1][Goal, n1]


Usingonlyh(n):Greedybest-firstsearch(GreedyBFS).

S

n1

n2

n3

Goal

→ step cost = 10

→ step cost = 100h(n3) = 50h(n1) = 70

(Greedy BFS is• Incomplete• not optimal)

100

100

10

10

10

• Weuseh(n)torankthenodesonthefrontier.• Alwaysexpandnodewithlowesth-value.

• Wearegreedilytryingtoachievealowcostsolution.

• However,thismethodignoresthecostofgettington,soitcanbeleadastrayexploringnodesthatcostalottogettobutseemtobeclosetothegoal:


Greedybest-firstsearchexample

Whenyou’reatSibiuandcontemplatingwhethertogotoFagarasorRV,theheuristicvalueofthesuccessornodes,i.e.,thehvalueguessofthecostis:h(Fagaras)=178andh(RV)=193),soFagaraslookslikethebetterchoice,but…

ActualCost(Arad-Sibiu-RV-Pitesli-Bucharest):140+80+97+101=140+278=418ActualCost(Arad-Sibiu-Fagaras-Bucharest): 140+99+211 =140+310 =450


A*search

• Takeintoaccountthecostofgettingtothenodeaswellasourestimateofthecostofgettingtothegoalfromn.

• Defineanevaluationfunctionf(n)f(n)=g(n)+h(n)• g(n)isthecostofthepathtonoden• h(n)istheheuristicestimateofthecostofgettingtoagoalnodefromn.

• Alwaysexpandthenodewithlowestf-valueonthefrontier.

• Thef-valueisanestimateofthecostofgettingtothegoalviathisnode(path).


A* examplef(n) =g(n)+h(n),

=actualcostton+heuristicestimateofcostfromntothegoal

















A*search

• Takeintoaccountthecostofgettingtothenodeaswellasourestimateofthecostofgettingtothegoalfromn.

• Defineanevaluationfunctionf(n)f(n)=g(n)+h(n)• g(n)isthecostofthepathtonoden• h(n)istheheuristicestimateofthecostofgettingtoagoalnodefromn.

• Alwaysexpandthenodewithlowestf-valueonthefrontier.

• Thef-valueisanestimateofthecostofgettingtothegoalviathisnode(path).


Conditionsonh(n)

• Wewanttoanalyzethebehavioroftheresultantsearch.• Completeness,timeandspace,optimality?

• Toobtainsuchresultswemustputsomefurtherconditionsontheheuristicfunctionh(n)andthesearchspace.


Conditionsonh(n):Admissible

• Wealwaysassumethatc(n1→n2)≥ε >0.Thecostofanytransitionisgreaterthanzeroandcan’tbearbitrarilysmall.

• Leth*(n)bethecostofan optimalpath fromntoagoalnode(¥ ifthereisnopath).Thenanadmissible heuristicsatisfiesthecondition

h(n)≤h*(n)admissibleheuristichalwaysunderestimatesthetruecosttoreach

thegoal.i.e.,itisoptimisticJ

• Hence• h(g)=0,foranygoalnote,g• h*(n)=¥ ifthereisnotpathfromntoagoalnode


Consistency(akamonotonicity)

• Isastrongerconditionthanh(n)≤h*(n).

• Amonotone/consistent heuristicsatisfiesthetriangleinequality(forallnodesn1,n2):

h(n1)≤c(n1→ n2)+h(n2)

• Notethattheremightbemorethanonetransition(action)betweenn1andn2,theinequalitymustholdforallofthem.

• Notethatmonotonicityimpliesadmissibility.• (foralln1,n2)h(n1)≤c(n1→ n2)+h(n2)è (foralln)h(n)≤h*(n)


Intuitionbehindadmissibility

h(n)≤h*(n)meansthatthesearchwon’tmissanypromisingpaths.• Ifitreallyischeaptogettoagoalvian(i.e.,bothg(n)andh*(n)arelow),thenf(n)=g(n)+h(n)willalsobelow,andthesearchwon’tignoreninfavourofmoreexpensiveoptions.

• Thiscanbeformalizedtoshowthatadmissibilityimpliesoptimality.


Intuitionbehindmonotonicity

h(n1)≤c(n1→n2)+h(n2)

• Thissayssomethingsimilar,butinadditiononewon’tbe“locally”mislead.Seenextexample.


Consistencyè Admissible• Assumeconsistency:h(n1)≤c(n1→n2)+h(n2)Proveadmissible:h(n)≤h*(n)

Proof:If nopathexistsfromntoagoalthenh*(n)=¥ andh(n)≤h*(n)Else letnà n1à …à n*beanOPTIMALpathfromntoagoal.Notethecostofthispathish*(n),andeachsubpath (nià …à n*)hascostequaltoh*(ni).

Proveh(n)≤h*(n)byinductiononthelengthofthisoptimalpath.

BaseCase:n=n* [optimalpathlength=0]Byourconditionsonh,h(n)=0≤h(n*)=0InductionHypothesis:h(n1)≤h*(n1)h(n)≤c(n → n1)+h(n1)[consistency]

≤c(n→n1)+h*(n1) [defn h*]=h*(n)


Example:admissiblebutnonmonotonic

Thefollowingh isnotconsistent(i.e.,notmonotone) sinceh(n2)>c(n2→n4)+h(n4).Butitisadmissible.

S

n1

n3

n2

Goal

→ stepcost=200→ stepcost=100

{S}→{n1[200+50=250],n2[200+100=300]}→{n2[100+200=300], n3[400+50=450]}→{n4[200+50=250],n3[400+50=450]}→{goal[300+0=300],n3[400+50=450]}

Wedofind theoptimalpathastheheuristicisstilladmissible.But wearemisleadintoignoringn2untilafterweexpandn1.

n4

h(n2)=200

h(n4)=50

h(n1)=50

h(n3)=50g(n)+h(n)=f(n)


Example:admissiblebutnonmonotonic

S

n1

n3

n2

Goal

→ stepcost=200→ stepcost=100

{S}→{n1[200+50=250],n2[100+200=300]}→{n2[100+200=300], n3[400+50=450]}→{n4[200+50=250],n3[400+50=450]}→{goal[300+0=300],n3[400+50=450]}

Wedofind theoptimalpathastheheuristicisstilladmissible.But wearemisleadintoignoringn2untilafterweexpandn1.

n4

h(n2)=200

h(n4)=50

h(n1)=50

h(n3)=50g(n)+h(n)=f(n)

100

100

100

200

200

200

Thefollowingh isnotconsistent(i.e.,notmonotone) sinceh(n2)>c(n2→n4)+h(n4).Butitisadmissible.

“Asthecrowflies”– Straightlineheuristic


• Mostadmissibleheuristicsarealsomonotone.(Indeedit’shardtofindanadmissibleheuristicthatisnotmonotone!)



Consequencesofmonotonicity

1. Thef-valuesofnodesalongapathmustbenon-decreasing.

Let<Start→n1→n2…→nk>beapath.Weclaimthat

f(ni)≤f(ni+1)

Proof:f(ni)=c(Start→…→ni)+h(ni)

≤c(Start→…→ni)+c(ni→ni+1) +h(ni+1)[monotonicity]=c(Start→…→ni→ni+1)+h(ni+1)=g(ni+1)+h(ni+1)=f(ni+1).



Proof(2cases):• Ifn2wasonthefrontier/OPENwhenn1wasexpanded,

thenf(n1)≤f(n2)otherwisewewouldhaveexpandedn2.• Ifn2wasaddedtothefrontier/OPENaftern1’sexpansion,thenletn beanancestorofn2thatwaspresentwhenn1wasbeingexpanded(thiscouldben1itself).Wehavef(n1)≤f(n)sinceA*chosen1whilen waspresentinthefrontier/OPEN.Also,sincen isalongthepathton2,byproperty(1)wehavef(n)≤f(n2).So,wehavef(n1)≤f(n2).

----------------------------1) Thef-valuesofnodesalongapathmustbenon-decreasing.2) Ifn2isexpandedaftern1,thenf(n1)≤f(n2)

2. Ifn2isexpandedaftern1,thenf(n1)≤f(n2)(thef-valueincreasesmonotonically)


ConsequencesofmonotonicityCorollary: thesequenceoff-valuesofthenodesexpandedbyA*isnon-decreasing.I.e,Ifn2isexpandedafter (notnecessarilyimmediatelyafter)n1,thenf(n1)≤f(n2)

(thef-valueofexpandednodesismonotonic non-decreasing)Proof:• Ifn2wasonfrontier/OPENwhenn1wasexpanded,

thenf(n1)≤f(n2)otherwisewewouldhaveexpandedn2.• Ifn2wasaddedtofrontier/OPENaftern1'sexpansion,then

letn beanancestorofn2thatwaspresentwhenn1wasbeingexpanded(thiscouldben1itself).Wehavef(n1)≤f(n)sinceA*chosen1whilen waspresentonfrontier/OPEN.Also,sincen isalongthepathton2,byproperty(1)wehavef(n)≤f(n2).So,wehavef(n1)≤f(n2).



• Proof: Assumebycontradictionthatthereexistsapath<Start,n0,n1,ni-1,ni,ni+1,…,nk>withf(nk)<f(n)andni isitslastexpandednode.• ni+1mustbeonthefrontier/OPENwhilenisexpanded,so

a)by(1) f(ni+1)≤f(nk)sincetheyliealongthesamepath.b)sincef(nk)<f(n)(given)sowehavef(ni+1)<f(n)(froma)c)by(2) f(n)≤f(ni+1)becausenisexpandedbeforeni+1.

• Contradictionfromb&c!-----------------------------------------------------------

1) Thef-valuesofnodesalongapathmustbenon-decreasing.2) Ifn2isexpandedaftern1,thenf(n1)≤f(n2)3) Whennisexpandedeverypathwithlowerf-valuehasalreadybeenexpanded.

3. Whennisexpandedeverypathwithlowerf-valuehasalreadybeenexpanded.


Consequencesofmonotonicity4. Withamonotoneheuristic,thefirsttimeA*expandsa

state,ithasfoundtheminimumcostpathtothatstate.Proof:• LetPATH1 =<Start,n0,n1,…,nk,n> bethefirst pathtonfound.

Wehavef(path1)=c(PATH1)+h(n).• LetPATH2=<Start,m0,m1,…,mj,n> beanotherpathtonfound

later.wehavef(path2)=c(PATH2)+h(n).• Byproperty (3)anditscorollary,f(path1)≤f(path2)• hence:c(PATH1)≤c(PATH2)

1) Thef-valuesofnodesalongapathmustbenon-decreasing.2) Ifn2isexpandedaftern1,thenf(n1)≤f(n2)3) Whennisexpandedeverypathwithlowerf-valuehasalreadybeenexpanded.Corollary:thesequenceoff-valuesofthenodesexpandedbyA*isnon-decreasing.I.e,Ifn2isexpandedafter (notnecessarilyimmediatelyafter)n1,thenf(n1)≤f(n2)

(thef-valueofexpandednodesismonotonic non-decreasing)


ConsequencesofmonotonicityComplete.

• Yes,consideraleastcostpathtoagoalnode• SolutionPath =<Start→n1→…→G>withcostc(SolutionPath)• Sinceeachactionhasacost≥ε >0,thereareonlyafinitenumberofpaths

thathavecost ≤c(SolutionPath).• Allofthesepathsmustbeexploredbeforeanypathofcost>

c(SolutionPath).• SoeventuallySolutionPath,orsomeequalcostpathtoagoalmustbe

expanded.TimeandSpacecomplexity.

• Whenh(n)=0,foralln,hismonotone.(avery*un*informativeheuristic!!!)• A*becomesuniform-costsearch!

• Itcanbeshownthatwhenh(n)>0forsomen,thenumberofnodesexpandedcanbenolargerthanuniform-cost.

• Hencethesameboundsasuniform-costapply.(Theseareworstcasebounds).Stillexponentialunlesswehaveaverygoodh!

• Inrealworldproblems,werunoutoftimeandmemory!IDA*cansometimesbeusedtoaddressmemoryissues,butIDA*isn’tverygoodwhenmany cyclesarepresent.


ConsequencesofmonotonicityOptimality§ Yes,by(4)thefirstpathtoagoalnodemustbeoptimal.

CycleChecking§ Wecanuseasimpleimplementationofcyclechecking

(multiplepathchecking)---justrejectallsearchnodesvisitingastatealreadyvisitedbyapreviouslyexpandednode.Byproperty(4)weneedkeeponlythefirstpathtoanode,rejectingallsubsequentpaths.

4. Withamonotoneheuristic,thefirsttimeA*expandsastate,ithasfoundtheminimumcostpathtothatstate.


Searchgeneratedbymonotonicity

Insideeachcounter,thefvaluesarelessthanorequaltocountervalue!

• Foruniformcostsearch,bandsare“circular”.• Withmoreaccurateheuristics,bandsstretchoutmoretowardthegoal.


AdmissibilitywithoutmonotonicityWhen“h”isadmissiblebutnotmonotonic.

• TimeandSpacecomplexityremainthesame.Completenessholds.• Optimalitystillholds(withoutcyclechecking),butneedadifferent

argument:don’tknowthatpathsareexploredinorderofcost.

Proof(bycontradiction)ofoptimality(withoutcyclechecking):• Assumethegoalpath<S,…,G>foundbyA*hascostbiggerthanthe

optimalcost:i.e.C*(G)<f(G).• Theremustexistsanoden intheoptimalpaththatisstillinthe

frontier.• Wehave: f(n)=g(n)+h(n)≤g(n)+h*(n)=C*(G)<f(G)

• Therefore,f(n)musthavebeenselectedbefore GbyA*.contradiction!


Admissibilitywithoutmonotonicity

WhataboutCycleChecking?• Nolongerguaranteedwehavefoundanoptimalpathtoanodethefirst

time wevisitit.

• So,cyclecheckingmightnotpreserveoptimality.• Tofixthis:forpreviouslyvisitednodes,mustremembercostof

previouspath.Ifnewpathischeapermustexploreagain.

• contoursofmonotonicheuristicsdon’thold.


SpaceProblemswithA*

• A*hasthesamepotentialspaceproblemsasBFSorUCS

• IDA*- IterativeDeepeningA*issimilartoIterativeDeepeningSearchandsimilarlyaddressesspaceissues.


IDA*- IterativeDeepeningA*Objective:reducememoryrequirementsforA*• Likeiterativedeepening,butnowthe“cutoff”isthef-value(g+h)rather

thanthedepth• Ateachiteration,thecutoffvalueisthesmallestf-valueofanynodethat

exceededthecutoffonthepreviousiteration• Avoidsoverheadassociatedwithkeepingasortedqueueofnodes• Twonewparameters:

• curBound (anynodewithabiggerf-valueisdiscarded)• smallestNotExplored (thesmallestf-valuefordiscardednodesina

round)whenfrontier/OPENbecomesempty,thesearchstartsanewroundwiththisbound

• Easiertoexpandallnodeswithf-valueEQUALtothef-limit.Thiswaywecancompute“smallestNotExplored” moreeasily.

.


ConstructingHeuristics


BuildingHeuristics:RelaxedProblem

• Oneusefultechniqueistoconsideraneasierproblem,andleth(n)bethecostofreachingthegoalintheeasierproblem.

8-Puzzle

• CanmoveatilefromsquareAtoBif• Aisadjacent(left,right,above,below)toB• and Bisblank



8-Puzzlemoves(continued)• CanmoveatilefromsquareAtoBif

• Aisadjacent(left,right,above,below)toB• and Bisblank

• Canrelaxsomeoftheseconditions1. canmovefromAtoBifAisadjacenttoB(ignorewhetherornot

positionisblank)2. canmovefromAtoBifBisblank(ignoreadjacency)3. canmovefromAtoB(ignorebothconditions).


BuildingHeuristics:RelaxedProblem• #3“canmovefromAtoB(ignorebothconditions)”.

leadstothemisplacedtiles heuristic.• Tosolvethepuzzle,weneedtomoveeachtileintoitsfinalposition.• Numberofmoves=numberofmisplacedtiles.• Clearlyh(n)=numberofmisplacedtiles≤theh*(n)thecostofanoptimal

sequenceofmovesfromn.

• #1“canmovefromAtoBifAisadjacenttoB(ignorewhetherornotpositionisblank)”leadstothemanhattan distance heuristic.• Tosolvethepuzzleweneedtoslideeachtileintoitsfinalposition.• Wecanmoveverticallyorhorizontally.• Numberofmoves=sumoverallofthetilesofthenumberofverticaland

horizontalslidesweneedtomovethattileintoplace.• Againh(n)=sumofthemanhattan distances≤h*(n)

• inarealsolutionweneedtomoveeachtileatleastthatfarandwecanonlymoveonetileatatime.



Theoptimal costtonodesintherelaxedproblemisanadmissible heuristic fortheoriginalproblem!

ProofIdea:theoptimalsolutionintheoriginalproblemisasolutionforrelaxedproblem,thereforeitmustbeatleastasexpensiveastheoptimalsolutionintherelaxedproblem.

Soadmissibleheuristicscansometimesbeconstructedbyfindingarelaxationwhoseoptimalsolutioncanbeeasilycomputed.



Depth IDS A*(Misplaced)h1 A*(Manhattan)h210 47,127 93 3914 3,473,941 539 11324 --- 39,135 1,641

Leth1=Misplaced,h2=Manhattan• Doesh2always expandfewernodesthanh1?

• Yes!Notethath2dominatesh1,i.e.foralln:h1(n)≤h2(n).• Therefore,amongseveraladmissibleheuristictheonewithhighest

valueexpandsthefewestnodes.Isitthefastest?

Comparison ofIDSandA*(averagetotalnodesexpanded):


BuildingHeuristics:Patterndatabases.

•By searching backwards from these goal states, we can compute the distance of any configuration of these tiles to their goal locations. We are ignoring the identity of the other tiles.

•For any state n, the number of moves required to get these tiles into place form a lower bound on the cost of getting to the goal from n.

• Admissibleheuristicscanalsobederivedfromsolutiontosubproblems:Eachstateismappedintoapartialspecification,e.g.in15-puzzleonlypositionofspecifictilesmatters.

• Herearegoalsfortwosub-problems(calledCornerandFringe)of15-puzzle.

• NotethelocationofBLANK!


BuildingHeuristics:Patterndatabases.

• Theseconfigurationsarestoredinadatabase,alongwiththenumberofmovesrequiredtomovethetilesintoplace.

• Themaximum numberofmovestakenoverallofthedatabases canbeusedasaheuristic.

• Onthe15-puzzle• Thefringedatabaseyieldsabouta345folddecreaseinthesearchtreesize.

• Thecornerdatabaseyieldsabout437folddecrease.

• Sometimesdisjointpatterns canbefound,thenthenumberofmovescanbeadded ratherthantakingthemax(ifweonlycountmovesofthetargettiles).


LocalSearch

• Sofar,wekeepthepathstothegoal.• Forsomeproblems(like8-queens)wedon’tcareaboutthepath,weonlycareaboutthesolution.ManyrealproblemlikeScheduling,ICdesign,andnetworkoptimizationsareofthisform.

• Localsearch algorithmsoperateusingasinglecurrentstateandgenerallymovetoneighborsofthatstate.

• Thereisanobjectivefunction thattellsthevalueofeachstate.Thegoalhasthehighestvalue(globalmaximum).

• AlgorithmslikeHillClimbing trytomovetoaneighbourwiththehighestvalue.

• Dangerofbeingstuckinalocalmaximum.Sosomerandomnessisaddedto“shake”outoflocalmaxima.

• SimulatedAnnealing:Insteadofthebestmove,takearandommoveandifitimprovesthesituationthenalwaysaccept,otherwiseacceptwithaprobability<1.

• [IfinterestedreadthesetwoalgorithmsfromtheR&Nbook].

Documents

Heuristic Search (Informed Search) - … Search •The idea is to develop a domain specific heuristic function h(n). •h(n) guessesthe cost of getting to the goal from node n (the