©CopyrightJASSS

MatthewOremlandandReinhardLaubenbacher(2014)

OptimizationofAgent-BasedModels:ScalingMethodsandHeuristicAlgorithms

JournalofArtificialSocietiesandSocialSimulation 17(2)6<http://jasss.soc.surrey.ac.uk/17/2/6.html>

Received:28-Jun-2013Accepted:08-Dec-2013Published:31-Mar-2014

Abstract

Questionsconcerninghowonecaninfluenceanagent-basedmodelinordertobestachievesomespecificgoalareoptimizationproblems.Inmanymodels,thenumberofpossiblecontrolinputsistoolargetobeenumeratedbycomputers;hencemethodsmustbedevelopedinordertofindsolutionsthatdonotrequireasearchoftheentiresolutionspace.Modelreductiontechniquesareintroducedandastatisticalmeasureformodelsimilarityisproposed.Heuristicmethodscanbeeffectiveinsolvingmulti-objectiveoptimizationproblems.Aframeworkformodelreductionandheuristicoptimizationisappliedtotworepresentativemodels,indicatingitsapplicabilitytoawiderangeofagent-basedmodels.Resultsfromdataanalysis,modelreduction,andalgorithmperformanceareassessed.

Keywords:Agent-BasedModeling,Optimization,StatisticalTest,GeneticAlgorithms,Reduction

Introduction

1.1 Agent-basedmodels(ABMs)areoftencreatedinordertosimulatereal-worldsystems.Inmanycases,ABMsactasinsilicolaboratorieswhereinquestionscanbeposedandinvestigated;suchquestionsoftenarisenaturallyinthecontextofthesysteminquestion.Forexample,anABMofafinancialnetworkmightbeusedtodeterminewhichpoliciesleadtomaximizedprofit,whileanABMmodelingsocialnetworksmightbestudiedtodeterminethemosteffectivemeansoftransmittinginformation.QuestionsconcerninghowonecaninfluenceanABMinordertobestachievesomespecificgoalareoptimizationproblems.Inothercontexts,optimizationmayrefertoparametersormodeldesign.Itisimportanttoreiteratethatthemeaningoftheterminthisstudyisdifferent–itreferstotheoptimalchoiceofasequenceofexternalinputstoamodelinordertoachieveaparticulargoal.ThestochasticityinherentinmanyABMsmeansthatcaremustbetakenwhenattemptingtosolveoptimizationproblems.Underfixedinitialconditions,datafromindividualsimulationreplicationsoftenvary.Thus,carefulanalysisofABMdynamicsisaprerequisiteforthedevelopmentofoptimizationtechniques.Inparticular,statisticalmethodsmustbebroughttobearontheinterpretationofsimulationresults.

1.2 Inthisstudy,statisticalandoptimizationtechniquesarepresentedwhichcanbeapplieddirectlytoABMs:translationofthemodeltoanequation-basedformisnotnecessary.Thereareseveraladvantagestothisapproach–suchtechniquescanbeappliedtovirtuallyanyABM,andthereisnoneedfortransformationofeitherthemodelorthecontrols.Repeatedsimulationisusedtoobtainreliableresults,andcontrolsareapplieddirectlytotheABMs.Whiletheremaybemodelsforwhichthisapproachfails,thesufficientlybroadexamplesprovidegoodevidencethatforlargeclassesofABMs,meaningfulresultscanbeobtainedbydirectanalysisandoptimization.

1.3 Thegoalofthispaperistointroduceandillustrateaframeworkforsolvingoptimizationproblemsusingagent-basedmodels.Ingeneral,thenumberofpossiblesolutionstoanoptimizationproblemisfartoolargeforenumeration.Thus,heuristicmethodsmustbeemployedtoanswersuchquestions.Computationalefficiencyisakeyfactorinthisprocess;assuch,theuseofscaledapproximationscanbeinvaluable.Aslongasascaledmodelfaithfullymaintainsthedynamicsoftheoriginal,itcanbeusedtosolvetheoptimizationproblem,resultinginareductionofruntimeandcomputationalcomplexity.

1.4 Thepaperisorganizedasfollows:standardsfordataanalysisareestablishedandastatisticalmeasureformodelsimilarityisproposed.AheuristictechniqueknownasParetooptimizationisproposedasameansforsolvingoptimizationproblems.TheframeworkispresentedviatheuseoftwomodelsactingasrepresentativesoflargeclassesofABMs,whichoughttoholdinterestforresearchersfromawidevarietyofdisciplines.Briefmodeldescriptionsareoutlinedinthetext,anddetailedmodeldescriptionsfollowingtheOverview,DesignConcepts,andDetails(ODD)protocolforagent-basedmodels(Grimmetal.2006;Grimmetal.2010)areprovidedintheappendices.Thesedescriptionsoughttoprovideenoughdetailthatthemodel(andresults)canbereconstructedandverifiedbyindependentresearch.

Relatedwork

1.5 Optimizationproblemsofthetypepresentedherehavebeenstudiedinmodelsofinfluenzaandepidemics(Kasaieetal.2010;Yangetal.2011),cancertreatment(Lollinietal.1998;Swierniaketal.2009),andthehumanimmunesystem(Bernaschi&Castiglione2001;Rapinetal.2010),tonameafew.Previousstudieshaveinvestigatedtheeffectofvariousmodelfeaturesonoutcomes–forexample,subwaytravelonthespreadofepidemics(Cooleyetal.2011),mobilityandlocationinamolecularmodel(Klannetal.2011),molecularcomponentsinacancermodel(Wangetal.2011),andstrategiesformitigatinginfluenzaoutbreaks(Mao2011)–whilenotquiteposingformaloptimizationproblems.AstudyontheeffectofABMsindeterminingmalariaeliminationstrategies(Ferreretal.2010)suggeststhatresultsfromagent-basedmodelsareinvaluableintheanalysisofinterventions.

1.6 Inotherstudies,ABMshavebeentransformedintosystemsofdifferentialequations(Kimetal.2008a)andpolynomialdynamicalsystems(Hinkelmannetal.2011;Veliz-Cubaetal.2010),amongothers.Theimportanceofspatialheterogeneityhasbeenexaminedinspecific(Haradaetal.1995)andmoregeneral(Happe2005)cases,andpredator-preyABMshavebeenanalyzedusingstatisticalmethods(Wilsonetal.1993;Wilsonetal.1995).

Aframeworkforsolvingoptimizationproblems

1.7 TheframeworkissummarizedinFigure1;subsequentsectionsmotivateandexplaintheprocessindetail.

http://jasss.soc.surrey.ac.uk/17/2/6.html 1 16/10/2015

Figure1.Anoverviewoftheframeworkpresentedinthiswork.Thethreeshadingsrepresentthephasesofanalysis,scaling,andoptimization.

DataReliability

1.8 Akeyfactorinanalysisofagent-basedmodelsisstochasticity.Theapproachsuggestedhereistoexaminehowdataaverageschangeasthenumberofsimulations(runs)increases.Inmanycases,thedatawillsettleinonsomeaveragethatisnotimproveduponbyincreasingthenumberofruns.Determiningasufficientnumberofrunsisthefirststepinobtainingreliableresults.Theemphasisonsolvingoptimizationproblemsnecessitatesthisprocess:whilesomeofthestochasticityinherenttoanindividualrunislostwhenaveragingoverrepeatedruns,itisnecessaryinordertodeterminethegeneralefficacyofonecontrolversusanother.

1.9 Agent-basedmodelsareoftenimplementedonagrid,representingthe'space'ofthemodel(oftentimes,thegridindeedrepresentssomephysicalspace).Treatingtheoriginalsizeandscopeofthemodelastrue,thegoalofscalingistodeterminetheextenttowhichamodelcanbereducedwithoutalteringpertinentdynamics.Themodelsexaminedherecontainphysicalagentstraversingphysicallandscapes.Inthissetting,thestrategyistograduallyscaledownthemodeluntilthedynamicsnolongerfaithfullyrepresenttheoriginalmodel.Whenapplicable,thisstrategyresultsinreducedruntime–inmanycasessubstantiallyso–reducingthecomputationalrequirementsforthesolvingofoptimizationproblemsandallowingaccesstoawiderrangeofanalyticaltools.

1.10 Determiningtowhatdegreeareducedmodelisafaithfulrepresentationoftheoriginalisanimportantquestion.Intermsofoptimization,itisnecessarytodeterminetheextenttowhichmodelscanbereducedforthepurposeofoptimalcontrol.Inordertoaccomplishthis,asampleofthecontrolspaceisimplementedinboththeoriginalmodelandreducedversions.Foreachreducedversion,thecontrolsarerankedaccordingtotheireffectivenessinregardstotheoptimizationorcontrolobjective.Theaimistouseareducedmodelasaproxyfortheoriginal;thus,therankingofthecontrolsonthereducedmodelmustbecomparedtotherankingofthesamecontrolsappliedtotheoriginal.

1.11 WeproposeCohen'sweightedκ(Cohen1968)asameasureofconcordanceofrankingsfordifferentmodelsizes.Letpobsbetheobservedproportionofagreementinthetwolistsandletpexpbetheproportionofagreementexpectedbyrandomchance.Thenκ=(pobs−pexp)/(1−pexp).Henceifthelistsareinperfectagreement,κ=1;ifthelistsarenomoresimilarthanwhatisexpectedpurelybychance,κ=0.Thissimilaritymetricforrankedlistsdeterminespenaltiesbasedonthemagnitudeofdisagreement.Fordetailsofhowtocalculatepobs,pexp,andweightedpenalties,seeCohen(1968).

1.12 Forexamplesoftheuseofthisstatisticasameasureofagreement,seeFleiss(1971)andEugenio(2000).Cohen'sweightedκischosenbecauseofitswidedocumentationandimplementationinavarietyofstudies;assuch,thereisprecedentforthismeasure.Thereisnoobjectivewaytodetermineabenchmarkvalueforκ.Severalstudiesproposeaκvaluegreaterthan0.75asbeingverygood(Altman1991;Fleiss1981),whileothersrecommendavalueof0.8orhigher(Landis&Koch1977;Krippendorff1980).Inthisstudynobenchmarkisset;rather,κvaluesareassessedaposteriori.Formoredetailsonsettingabenchmarkforκ,seeSimandWright(2005),andElEmam(1999).

1.13 ItisofcoursenotguaranteedthatallABMswillbeamenabletothestrategiespresentedhere(fordiscussiononthisissueseeDurrettandLevin(1994)).Infact,modelsmayexistforwhichnoreductionispossible–nevertheless,reductionstrategiesarefrequentlyusefulandinvariablyinformative.Inparticular,theinvestigationofdifferencesinqualitativebehaviorcanbeservedbythese(andother)methodsofmodelreduction.ForexamplesofmodelreductionstrategiesappliedtoABMs,seeZouetal.(2012),Roosetal.(1991),andYesilyurtandPatera(1995).Itisalsoworthnotingthat'modelreduction'isaphrasewhosemeaningmaybediscipline-dependent:theextenttowhichamodelcanbereducedisdependentonwhichmeaningistakenandwhichmodeldetailsonewishestopreserve.

ParetoOptimization

2.1 Onceasuitablereductionhasbeenmade,anoptimizationproblemcanbesolvedusingthereducedmodelasasurrogatefortheoriginal.PerhapsthemostexploredmethodforoptimalcontrolofABMshascomeintheformofheuristicalgorithms.Giventhatenumerationofthesolutionspaceisofteninfeasible,heuristicalgorithmsareusedtoconductaguidedsearchofthesolutionspaceinordertodeterminelocallyoptimalcontrols.

2.2 SeveralheuristicalgorithmshavebeenutilizedinsolvingoptimizationproblemsforABMs.Examplesincludesimulatedannealing(Pennisietal.2008),tabusearch(Wang&Zhang2009),andsqueakywheeloptimization(Lietal.2011).Inthisstudy,attentionisfocusedonacertaintypeofgeneticalgorithm(GA).Thesealgorithms,firstbroughttogeneralattentionin1989(Goldberg1989),aredesignedtomimicevolution:solutionsthataremorefitareusedto'breed'newsolutions.GAshavebeenusedinconjunctionwithABMstofindoptimalvaccinationschedulesforinfluenza(Pateletal.2005),cancer(Lollinietal.2006),andindeterminingoptimalanti-retroviralschedulesforHIVtreatment(Castiglioneetal.2007).Vaccinationscheduleoptimizationresultsobtainedfromsimulatedannealingandgeneticalgorithmshaveevenbeencomparedandcontrasted(Pappalardoetal.2010).AstheprimaryfocusofthispaperistointroduceageneralframeworkforsolvingoptimizationproblemsforABMs,acomparisonofvariousheuristicmethodsisoutsidethescopeofthisstudy.ForamorecomprehensivelookatheuristiccontrolofABMs,seeOremland(2011).

2.3 Thecontrolproblemsdescribedherehavemultipleobjectives–thisnecessitatesassigningweightstoeachobjective.Determinationofweightsinmulti-objectiveoptimizationproblemscanbeproblematicbecauseapriori,theappropriateweightsmaybeunknown–inparticular,theassignmentisatthediscretionoftheinvestigator.Whiletherehavebeenvariousproposalsfortheseassignments(foranexample,seeGennertandYuille(1988)),anymethodwhichdoesnotrequireweightshasparticularappeal.

2.4 Paretooptimizationisjustsuchaheuristicmethod:insteadofafocusingonasinglesolution,thealgorithmreturnsasuiteofsolutions.SolutionsontheParetofrontierrepresentthosethatcannotbeimproveduponintermsofoneobjectivewithoutsomesacrificeinanother.Inthissense,eachsolutionontheParetofrontierisoptimalwithrespecttosomechoiceofweights.Thus,the'managerial'decisionofhowtoassignweightscanbesettledafterthesearchhasconcluded.

2.5 Anextensivelistofreferencesonmulti-objectiveoptimizationtechniquescanbefoundinCoello(2013).ParetooptimizationhasbeenselectedforthisstudyasitisnovelinitsapplicationtoABMs.ThealgorithmadoptedhereisaminorvariantofthatdescribedinHornetal.(1994):itisaheuristicalgorithmthatsearchesthecontrolspaceinanattempttofindtheParetofrontier.PseudocodeforthealgorithmispresentedinAlgorithm1.

http://jasss.soc.surrey.ac.uk/17/2/6.html 2 16/10/2015

Algorithm1.PseudocodefortheParetooptimizationalgorithm.

Software

3.1 Theproposedframeworkrequirestwotypesofsoftware:modeling,andstatisticalanalysis.Agent-basedmodelscanbeimplementedinavarietyofsoftwarepackages.Someofthese,suchasNetLogo(Wilensky2009),Repast(Northetal.2006),andMASON(Lukeetal.2005)havebeendesignedforgeneralagent-basedmodeling.Othersoftwarehasbeendevelopedforagent-basedmodelinginspecificfields–theseincludeC-ImmSim,VaccImm,andSIMMUNEforthehumanimmunesystem(Castiglione1995;Woelkeetal.2011;Meier-Schellersheim&Mack1999),FluTEforinfluenzaepidemiology(Chaoetal.2010),andSnAPforpublichealthstudies(Buckeridgeetal.2011).Whileonecanalwaysimplementone'sowntoolkitforexaminingagent-basedmodels,theuseofestablishedsoftwarecanreduceboththevariabilitybetweenresearchers'implementationsandthelearningcurveforconductingresearchinthisfield.NetLogowaschosenasthemodelingplatforminthisstudy,thoughthereisnoreasonwhythestudycouldnothavebeenundertakenusinganumberofdifferentsoftwarepackages.AstandardNetLogoinstallationcontainsanextensivelibraryofmodelsfromavarietyoffields;themodelsdiscussedhereareadaptationsofpopularmodelsfromthisbuilt-inlibrary.Statisticalanalysiscanbeperformedbyvirtuallyanystatisticalsoftwarepackage;inthisstudy,MicrosoftExcelwasused.DuetothefactthatsimulationdatawasneededinordertoperformParetooptimization,thisprocesswasimplementedinNetLogoaswell.Ingeneral,thetechniquesdescribedherearesufficientlystraightforwardthathighlyspecializedsoftwareisnotneeded,andtheframeworkisnotlimitedtoanyparticularsoftwarechoice.

TwoModels

RabbitsandGrass

4.1 ThefirstmodeltobeexaminedisbasedonasamplemodelfromtheNetLogolibrary(Wilensky2009)involvingrabbitsinafield.Ateachtimestep,eachrabbitmoves,eatsgrass(ifthereisgrassatitslocation),andthenpossiblyreproducesordies,basedonitsenergylevel.Thereareseveralcompellingreasonsfortheuseofthismodelasatestcasefortheproposedframework.Oneisthatthemodelissufficientlysimpletodescribe,soresultscanbeobtained,interpreted,andunderstoodwithminimaloverhead.Amoreimportantreasonisthatthismodelrepresentsthecategoryofgeneralpredator-preysystems(withgrassfunctioningasprey).Suchmodelsarecommonlyusedinecologyandhavebeenwidelystudied.Thus,theframeworkcanbepresentedthroughanexamplethatappealstoabroadcommunityofresearchers.Indeed,thismodelillustratesmanyconceptscommoninABMscontaininginteractingspecies.AdetaileddescriptionofthemodelandalistofparametervaluesareprovidedinAppendixA.

4.2 Controlconsistsofdeciding(eachday)whetherornottoapplypoisontothegrid(i.e.,uniformlytoallgridcells).Specifically,thecontrolobjectiveistodetermineapoisonscheduleuthatminimizesthenumberofrabbitsalivethroughoutthecourseofasimulationwhilealsominimizingthenumberofdaysonwhichpoisonisused.Notethatitisunlikelythatonecontrolschedulewillminimizebothobjectivessimultaneously:forexample,thecontrolwhereinnopoisonisusedcertainlyminimizesthesecondobjective,butnotthefirst.Thus,thisproblemisagoodcandidateforParetooptimization:asuiteofsolutionscanbefound,eachmemberofwhichisoptimaldependingontheweightsassignedtothetwoobjectives.

Scalingresults

4.3 Oneofthecontrolobjectivesconcernstheaveragenumberofrabbitsaliveoverthecourseofasimulation;thus,thisisthepertinentmetricintermsofmodelreduction(giventhattheothercontrolobjective–minimizingdaysonwhichcontrolisused–isentirelypreservedatanymodelsizeandforanynumberofruns).

4.4 AsnotedinDataReliability,thefirstconsiderationwhenattemptingtoscalethemodelisdeterminingthenumberofrunsnecessaryinordertoachievereliableresults.Tothisend,severalcontrolscheduleswereselectedatrandom.Eachwasappliedtotheoriginal50×50model,andresultsweretalliedupto100runs.PopulationdynamicsforthreerandomlyselectedcontrolschedulesarepresentedinFigure2:plotsshowhowtheaveragenumberofrabbitsaliveoverthecourseoftheschedulechangeasthenumberofrunsincreases,witherrorbarsrepresentingonestandarddeviation.

http://jasss.soc.surrey.ac.uk/17/2/6.html 3 16/10/2015

(a)10controldays

(b)30controldays

http://jasss.soc.surrey.ac.uk/17/2/6.html 4 16/10/2015

(c)50controldaysFigure2.Averagepopulationvaluessettleintoconsistentvaluesaround50runs.

4.5 Thesethreeschedulesrepresentthreedistinctregionsofthecontrolspace,inthateachcontainsadifferentnumberofcontroldays.Notethatinallcases,thereislittlechangeinthemeanorthestandarddeviationbeyond50runs–thissuggeststhatthereisnoadvantageinaveragingovermorethan50runs.Itisimportanttonotethatifthecontrolobjectiveswerealtered(forexample,ifgrasslevelswereofinterestratherthanrabbitlevels)thenthisconclusionmaynothold.Inparticular,thenecessarynumberofrunsdependsuponthemodeldynamicsofinterest–inthiscase,theaveragenumberofrabbits.

4.6 Onceabenchmarkforreliableresultshasbeenestablished,variousmodelreductionscanbeinvestigatedwithrespecttocontrol.Intheoriginalmodel,thereare50×50=2500patchesand

150rabbitsinitially.AmodelsizeofMmeansthattheworldwidthandheightarebothM.Hence,whenreducingthemodeltosizeM,theinitialnumberofrabbitsoughttobe150(M2/2500)inordertomaintainthesameproportionofrabbitstomodelsize.Allotherstatevariablevaluesremainthesame.

4.7 Foreachofasetofcontrolsappliedtotheoriginalmodel,averagerabbitnumbersareobtainedviasimulation;thecontrolscanthenberankedbythesenumbers.Thesesamecontrolscanbeappliedtoareducedmodel,resultingina(potentially)differentrankedlist.Theκstatistic(seeDataReliability)measuresthesimilaritybetweenthetworankings,therebyservingasameasureoftheextenttowhichthereducedmodelservesasasubstitutefortheoriginal.Itisimportantthattheserankingsaremaintainedoverawiderangeofcontrolschedules,sincesolvingtheoptimalcontrolproblemwillinvolvethepotentialexaminationoftheentiresolutionspace.

4.8 Generatingasetofcontrolsrandomlyresultsinanormaldistributioncenteredonsolutionswithfiftyzerosandfiftyones(onesindicatingtimestepsonwhichpoisonisused).Toavoidfocusingontoonarrowaportionofthecontrolspace,astratifiedrandomsamplewastaken:24valuesN1,…,N24werechosenasfrequencynumbers,representingthenumberofonesintheschedule.Thesevalueswerechosenatrandomwithinthefollowingscheme:threevalueswerechosenbetween1and10,threebetween10and20,andsoon,withthefinalthreechosenbetween70and80.Fourcontrolscheduleswerethenrandomlygenerated,eachcontainingNtonesand(100−Nt)zeros(distributedrandomlythroughouttheschedule),fort∈{1,…,24}.Thus,foreachtrial,atotalof96scheduleswereevaluated,chosenasrepresentativesofthesolutionspace.Notethatscheduleswithmorethan80non-zeroentrieswerenotconsidered,aspreliminaryinvestigationshowedthatsuchscheduleswerequicklyeliminatedfromanyheuristicoptimalcontrolsearch.

4.9 Onetrialisdefinedasfollows:96controlschedules(chosenaccordingtotheabovedescription)wererunusingtheoriginalM=50model,andthenagainonthemodelateachofthefollowingmodelsizes:50,40,30,20,10,5,and3.NotethattheschedulesareruntwiceontheoriginalM=50model:thisisdoneinordertoestablishhowconsistenttherankingsarewhenevaluatedtwiceonthesame-sizedmodel.Insomesense,thisservesasvalidationofthechoiceof50simulationsasbeingsufficientforreliableresults,andalsoprovidesinsightintotheanalysisofanappropriatebenchmarkforκ,aswillbeseenbelow.

4.10 Evaluationof150schedules(eachaveragedover50simulations)atmodelsizeM=50requiresapproximately3seconds.Table1givesthenumberofsimulationsthatcanberunforthereducedmodelsinapproximately3seconds.Giventhattheprimaryadvantageforscalingmodelsistoreduceruntime,itismoreappropriatetocomparedatabasedonequivalentruntimeratherthanusingafixednumberofsimulationsforeachsize.Thisdataaidsinscalinganalysis:ifonewishestoreducetheruntimeby50%,thenumberofrunsthatcanbeperformediseasilycalculated.

Table1:Numberofsimulationsinequivalentruntime.

Worldsize Simulations Avg.runtime(sec.)50 50 3.0440 75 3.0030 135 3.0520 290 3.0310 1100 3.035 3500 3.013 5700 3.03

4.11 Figure3summarizesκvaluesforvariousworldsizesandruntimes.Eachdatapointrepresentsthemeantakenovertentrials,witherrorbarsrepresentingonestandarddeviation.Abenchmarkvalueofκ=0.8isplottedaswell–itispresentedtoserveasapreliminarygaugeofhowwellthereducedmodelscapturethedynamicsoftheoriginal.Eachlineonthegraphconnectsdatapointsofequivalentruntime.Figure3helpsidentifyunviablereductions:acceptingabenchmarkofκ=0.8,worldsizesbelow20arenotsufficientlyaccuraterepresentativesoftheoriginalmodel(andthesize20modelisonlysufficientat100%runtime).Thedataalsoshowthatifoneinsistsonusingthesize3model,thebenchmarkforκwillhavetobelowered.Itfurthershowsthatifonewishestousethesize3modelandinsistsonaκvaluehigherthan0.8,itwillcertainlyrequireanincreaseintheruntimeofthemodel(andeventhen,maynotbepossible).

4.12 Severalimportantconclusionscanbedrawnfromthisdata:oneisthatifthepriorityisachievingthehighestpossiblevalueforκ,thentheoriginalsize50modelisalwaysthebestchoiceforanyfixedruntime.Thisisperhapsunsurprising,asonecanonlyexpecttolosesomeaccuracyasmodelsizedecreases.

http://jasss.soc.surrey.ac.uk/17/2/6.html 5 16/10/2015

Figure3.Cohen'sweightedkforvariousworldsizesandruntimes.

4.13 Anotherimportantconclusionisthatiftheonlypriorityisdecreasedruntime,itisalwaysbettertousefewerrunsofthesize50modelratherthanmorerunsofasmallermodel.Thisfollowsbecauseeachlinerepresentsafixedruntime,andforanyfixedruntime,thesize50modelresultsinthehighestvalueforκ.Afixedbenchmarkforκfurtherinformsaresearcherwithapriorityofreducedruntime:asthedatashow,ifonewishestokeepκabove0.8,thenitispossibletoreducetheruntimeby90%,butnotfurther(asindicatedbythedataforthesize50modelat0.3seconds).Thus,notonlycanonedeterminewhichworldsizeshouldbeusedinordertoobtainminimumruntime,butalsotheminimumruntimethatcanbeachievedinordertomaintainapre-setbenchmarkforκ.

4.14 Theremaybecaseswhereareducedmodelisofparticularinterest–forexample,Hinkelmannetal.(2011)describesmethodsfortranslatingABMsintopolynomialdynamicalsystems,offeringadvantagessuchassteadystateandbifurcationanalysis.Thenumberofrequiredequationsmaybetoolargeforthesize50model,butnotsoforthesize10model.Asimilarconcernappliestodifferentialequationsapproximations.ExamplesoftheseconsiderationsarediscussedinKimetal.(2008a)andKimetal.(2008b).Hence,dependingonthepriorityofthemodeler,thedatahereshowwhichworldsizesmaybeusedandwhatκvaluescanbeexpectedwhendoingso.Furthermore,notethatforsmallerworldsizestherangeofκvaluesisdecreased.Inparticular,inordertoachieve1%runtimeonthe50×50model,κdecreasesfrom0.90to0.68.However,inordertoachievea1%runtimeonthe3×3model,κdecreasesfrom0.38to0.33.Thus,forsmallermodelsκmaybelessaffectedbyadecreaseinruntime.

4.15 Inadditiontotheaboveconclusions,thedatainformsthestudyofpointsatwhichthedynamicsofthemodelundergoaqualitativechange.Thereisalargerchangeinreducingfromworldsize10to5thanthereisingoingfromsize20to10;thisindicatesthedynamicsaremorerapidlychangingbetweenworldsizes10and5.Inparticular,thedataseemtosuggestthatthepertinentdynamicsarenotdrasticallyalteredbetweentheoriginalsize50modelandthesize20model,butchangeratherquicklyatsmallersizes.Thisisofparticularinterestinlightofthefactthattheoriginalworldsizewaschosenmoreorlessarbitrarily.Ifonebeganwithasize10model,itmaynotbepossibletoreduceittothesameextentthatonecanreduceasize50model.

4.16 AsmentionedinDataReliability,severalstudiessuggestaκvalueof0.8asabenchmarkforsufficientsimilarity.Whilelargelycitedandused,theapplicabilityofthisvalueoughttobeexaminedinlightoftheresultsobtainedbyheuristicalgorithms.Inparticular,foreachmodelsizeanappropriateκvaluecanbedeterminedaposterioribasedonsaidresults.Thegoalofthismodelreductionanalysisisnottoprescribewhichmodelsizeone'should'use;rather,giventhattheprocessdependsonthepriorityofthemodeler,thegoalistopresentκvaluesandruntimesonecanexpectwhenusingaparticularreducedmodel.

ResultsfromParetoOptimization

4.17 AsdiscussedinParetooptimization,thegoalofaParetooptimizationalgorithmistoreturnasuiteofsolutions,eachofwhichisoptimalforaparticularchoiceofobjectiveweights.Forthismodel,theobjectivesaretominimizethenumberofdaysonwhichcontrolisusedandtominimizethenumberofrabbitsaliveduringthecourseofasimulation.Recallthatacontrolisavectoroflength100withentriesin{0,1}.Figure4showsanexampleoftheParetofrontier.Eachdotcorrespondstoonecontrol,plottedaccordingtothevaluesontheaxes.The×'smakeuptheParetofrontierofthisdataset:foreverypointonthisfrontier,oneoftheobjectivescannotbeimproveduponwithoutsomesacrificeintheother.Ontheotherhand,foreachpointnotonthefrontier,thereexistssomepointinthesetthatimprovesuponbothobjectives:inparticular,foreverysquare(i.e.,non-Paretofrontier)datapoint,thereexistsatleastoneotherpointwithfewercontroldaysandaloweraveragenumberofrabbits.ThegoaloftheheuristicParetooptimizationalgorithmistodetermine,asnearaspossible,thetrueParetofrontierofthecontrolspace.Thus,remainingfiguresconsistofParetofrontiersonlyandnottheentiredatasets.InordertoinvestigateavarietyofκvaluesasdeterminedinScalingresults,severalrepresentativemodelsizesandruntimeswerechosen.ForeachrepresentativemodeltheParetooptimizationalgorithmwasrunandaParetofrontierobtained.IfareducedmodelisasuitablesubstitutefortheoriginalthentheParetofrontierforthereducedmodeloughttobethesameastheParetofrontieroftheoriginalmodel.Foreachreducedmodel,thecontrolsmakingupthefrontierareimplementedintheoriginalmodelinordertodetermineiftheyareactuallyParetooptimal(asthereducedmodelresultshassuggested).NotethatParetooptimizationhasbeenperformedontheoriginalmodelaswellinordertoserveasabasisforcomparison.

4.18 Figure5summarizesresultsforrepresentativemodelswithlowerκvalues.Eachshapecorrespondstoonerepresentativemodel,withresultscomingfromtheimplementationofthesecontrolsintheoriginalmodel.Theseresultssuggestthatmodelswithκvaluesbelow0.5arenotverygoodsurrogatesfortheoriginalmodel.Inparticular,therearefewerdatapoints,andtheytendtoclusternearcertainregionsofthefrontier.Inshort,veryfewofthecontrolsdeterminedtobeParetooptimalbytheserepresentativemodelsareinfactParetooptimalintheoriginalmodel.Figure6showssimilarresultsformodelswithhigherκ

Figure4.AnexampleParetofrontierfortheRabbitsandGrassmodel.Frontierpointsaremarkedwithanxandnon-frontierpointswithasquare.

values.Therepresentativemodelwithaκscoreof0.89producesaParetofrontierveryneartothefrontieroftheoriginalmodel,suggestingthataκvalueof0.89issufficientlyhigh–hence,areducedmodelwithaκvalueof0.89canlikelybeusedasasurrogatefortheoriginalmodel.Thedataforthemodelwithaκvalueof0.76isalsoneartotheParetofrontieroftheoriginalmodel,thoughnottothesameextent.Forthemodelwithκ=0.65,therearefewerdatapoints,andtheseareabitfurtherfromthetruefrontier.

http://jasss.soc.surrey.ac.uk/17/2/6.html 6 16/10/2015

Figure5.Paretofrontiersformodelswithlowerkvalues.

4.19 Finally,Figure7suggeststhatthereisamildlylinearrelationshipbetweentheκvalueofareducedmodelandthenumberofpointsontheParetofrontier.ThesamesettingswereusedforeachParetooptimizationalgorithm;yet,ingeneral,thisdatashowsthatmodelswithlowerκvaluestendtoproducesmallerParetofrontiers(evenwithinthereducedmodelitself).Onepossibleexplanationforthisisthatforareducedmodel,thereisanarrowerrangeinthepossibledynamicsofamodel,andthusthetrueParetofrontierforareducedmodelmayindeedbesmaller.Thus,again,κvaluesindicatetheextenttowhich

Figure6.Paretofrontiersformodelswithhigherkvalues.

modeldynamicsarepreserved.Aqualitativeexaminationofthedatapresentedheresuggeststhataκbenchmarkintheregionof0.75–0.80isinfactagoodbenchmarkforthisexample.Onceagain,thefinaldecisionrestswiththeresearcherandisultimatelydeterminedbythelevelofdesiredaccuracy.

http://jasss.soc.surrey.ac.uk/17/2/6.html 7 16/10/2015

>

Figure7.Plotofkvaluevs.sizeoffrontier,withlineofbestfit(Pearson'sr2=0.66).

SugarScapeModel

4.20 ThesecondmodelisamodifiedversionofSugarScape(Epstein1996),inwhichapopulationofagentstraversealandscapeinsearchofsugar.Thismodelwaschosenforseveralreasons:first,itisspatiallyheterogeneous.ThisisacommonfeatureofmanyABMsandthusitisimportanttodemonstratehowtheframeworkpresentedherecanbeappliedtomodelswhereinspaceisanissue.Second,themodelhasbeenexaminedbyresearchersinavarietyoffields–studiesbasedonSugarScapehavefocusedonmigrationandculture(Deanetal.2000),distributionofwealth(Rahmanetal.2007),andtrade(Dascluetal.1998).Assuch,itisofbroadgeneralinterestasatestcase.Thus,futureworkmaybuildontheframeworkpresentedhereasameansofconductingresearchinareasasdiverseassocialscience,biology,andeconomics.

4.21 ThebasisofthemodelusedhereisincludedwiththestandardNetLogodistribution(Wilensky2009).Thelandscapeconsistsoffixedregionscontainingdifferentamountsofsugar;assuch,thismodelcontainsaspatialheterogeneitynotpresentintherabbitsandgrassmodel.TheoriginallandscapeispresentedinFigure8;darkerregionsrepresentareaswithmoresugar.Periodically,antsaretaxedbasedontheirvision,metabolism,andlocation(e.g.,high-visionantsinsugar-richregionsmaybetaxedathigherratesthanlow-visionantsinregionswithlesssugar).Theoptimizationproblemistodeterminethetaxschedulethatmaximizesthetotaltaxincomecollectedwhileminimizingthenumberofdeaths.Fullmodeldetails,includingthosepertainingtotaxation,areprovidedinAppendixB.Notethatcertainparametervaluesarealteredwhenconsideringreducedmodels;parametervaluesinAppendixBrefertothe50×50model.

Figure8.Landscapeofthe50x50SugarScapemodel.Darkerregionscontainmoresugar.Eachregionislabeledwithanumber;Table8providesmaximumsugarlevelsbyregion.

ScalingResults

4.22 Atotalof100controlsweregenerated,consistingofthreedifferentaveragetaxrates.Thenumberofdeathsandtaxincomeforeachwascollectedoveratotalof100runs.RepresentativedataispresentedinFigure9,witherrorbarsrepresentingonestandarddeviation.Asseeninthefigure,thereisverylittlechangeinthemeanandstandarddeviationofthedatabeyond50runs;hence,thereisnobenefittoaveragingovermorethan50simulations.

4.23 GiventheimportanceofthespatiallayoutoftheSugarScapemodel,itisnecessarytopreservethislayoutasnearlyaspossibleinanyreducedversion.Landscapereductionwasdeterminedbythenearest-neighboralgorithm,ameansofre-samplingtheoriginallandscapeinordertodeterminethelayoutofareducedversion.

4.24 Inadditiontoscalingthemap,thenumberofagentswasalsoscaled.Whilelowvisionisdefinedtobe1atanymodelsize,highvisiondependsonthesizeofthegrid:anagentwithvisionvona50×50gridhasvisionvn=v(n/50)onann×ngrid.Forgridsizes10and5,thiswouldresultinhighvisionbeingequivalenttolow;thusinthesetwocaseshighvisionwasdefinedtobe2.Themetabolismofeachagentisnotscaled:ateachmodelsize,itwasrandomlysetbetween1and4(inclusive).

4.25 Torunthesimulation50timesatmodelsizeM=50(meaninga50×50grid)takesapproximately8.5seconds.Table2showsthenumberofsimulationsthatcanberuninequivalentruntimeforreducedmodelsizes.

(a)Avg.taxrateof0.125

http://jasss.soc.surrey.ac.uk/17/2/6.html 8 16/10/2015

(b)Avg.taxrateof0.25

(c)Avg.taxrateof0.375Figure9.Averagevaluesfordeaths(solid)andtaxincome(dashed)againsettleintoconsistentvaluesby50runs.Taxincomevalueshavebeenscaledlinearlytofitonthe

plots.

4.26 Figure10showsκvaluesforvariousmodelreductions.Sincetherearetwovariablesinthiscase(deathsandtaxincome),controlscanberankedaccordingtoeither,resultingintwodifferentκvalues.Notethatwhenrankedaccordingtothenumberofdeaths,κvaluesareextremelylow–infact,closetobeingcompletelyrandom.Whiletherankingsaccordingtotaxincomeresultinhigherκvalues,theystillfallshortoftheproposedminimumbenchmarkof0.8.

Table2:Numberofsimulationsinequivalentruntime.

Worldsize Simulations Avg.runtime(sec.)50 50 8.5140 88 8.4830 173 8.4920 427 8.5210 1375 8.535 3900 8.48

http://jasss.soc.surrey.ac.uk/17/2/6.html 9 16/10/2015

Figure10.Cohen'sweightedkwithcontrolsrankedbydeaths(left)andbytaxincome(right).

4.27 Aninterestingfeatureofthisdataisthatthereappearstobenogreatdifferenceintheκvaluesobtainedfrommodelsrunat2%oftheoriginalruntimeversusthoseobtainedfrom100%runtime.Thismayindicatethatthenumberofrunsusedforreliabledatawasoriginallysettoohigh,oritmayindicatethatκvaluesatorbelow0.45areequallyunreliable.Nevertheless,thereisacleartrendshowingthatasthemodelsizedecreasestheκvaluesdecreaseaswell.Asinthepreviousexample,thismaybeanindicationofqualitativechangesinmodeldynamicsasthemodelsizeisreduced.

ResultsfromParetoOptimization

4.28 Althoughκvaluesappearlowerinthiscase,itisnecessarytoagainexaminetheperformanceofreducedmodelswithrespecttocontrol.Whilethesuggestedbenchmarkof0.75–0.8provedfittingforthepreviousmodel,itmaybethatalowerκbenchmarkisacceptableinthiscase.ResultsarepresentedinFigure11.

http://jasss.soc.surrey.ac.uk/17/2/6.html 10 16/10/2015

Figure11.ParetooptimizationresultsforSugarScape.4.29 Paretofrontiersfromthreemodelsarepresentedhere:thosewithκvaluesof0.43,0.27,and0.15(withrespecttotaxincome).Theshapesofthedatafromthethreemodelsfollowthesame

basicshapeofthetrueParetofrontier(labeled'Master'inthefigure),butthereisadistinctdifferenceinperformance.Inparticular,noneofthecontrolsfoundbyanyofthesemodelsareontheParetofrontieroftheoriginal.Furthermore,theredoesnotappeartobeanysignificantdifferencebetweensolutionsfoundusingthemodelwithκ=0.43andthosefoundusingthemodelwithκ=0.15.Thisindicatesthatnotonlyarenoneofthesemodelsappropriateasreplacementsfortheoriginal,butthattheremayinfactbeaminimumκvalue,belowwhichallmodelsareunsuitable.Inotherwords,thedownwardtrendindicatedinFigure10maybemisleading:whileitseemstosuggestthatasmodelsizedecreases,themodelsbecomelessrepresentativeofthedynamicsoftheoriginal,theresultsinFigure11suggestthatthisisn'tactuallythecase.Onthecontrary,modelswithaκvalueof0.15maybenoworsethanthosewithκvaluesof0.43.Giventhatnomodelsattainedaκvaluehigherthan0.45,itisimpossibletojudgethebenchmarkof0.75asappropriatelyhigh.Ontheotherhand,itispossibletoconcludethatκvaluesatorbelow0.45arecertainlytoolow.

Conclusions

5.1 Thegoalofthispaperistointroduceaframeworkforoptimizationofagent-basedmodels.Oncereliabledataisobtained,reducedmodelscanbecomparedtotheoriginal.SimilaritycanbemeasuredusingCohen'sweightedκ.Paretooptimizationwasimplementedinordertosolvecontrolproblemsinbothcases,allowingforaposteriorianalysisoftheκbenchmark.Resultspresentedhereshowthatinoneexample,theestablishedbenchmarkintherangeof0.75–0.8wasindeedsufficientformodelreduction,whilethesecondexampleshowedthatvaluesbelow0.45weretoolow.Theseresultssuggestκcanbeameaningfulmeasureformodelreduction.

5.2 Themodelspresentedherewereselectedfortheiruniversalityandpopularity–assuch,theyactasstandardmodelstowhichanyattemptatanalysisshouldbeapplied.InprinciplethereisnoreasonwhythemethodologywouldnotapplytoextensionsofthesemodelsortootherABMs.Inthefuturethiscollectionofmodelsshouldbeexpandedtoincludeawidervarietyofmodelsofincreasingcomplexity.Thismethodologyhasbeenappliedheretomodelswhereinspaceandagentlocationarekeyfeatures;itmayrequiresomemodificationinordertogeneralizetonon-spatialmodels.Inaddition,othermethodsforanalysisofagent-basedmodelsincludetransformationtoequationmodels.Suchwork(usingthesamemodelspresentedhere)isunderway.

5.3 AsABMsareusedmoreandmoretoinvestigatereal-worldsystems,optimizationandoptimalcontrolproblemswillnaturallyariseinthecontextofABMs.Heuristicmethodshaveseveraladvantages:theyareeasytoimplementonacomputerandtheycanbeappliedtovirtuallyanyABM.Thisisparticularlyimportantformodelsthataretoocomplexforconversiontoothermathematicalforms,e.g.,incaseswheredifferentialequationsareinsufficient.Theuserhasdirectcontroloverhoweachalgorithmruns,andcanfine-tuneparametersandsettingstobettersuitthemodel.However,therearedrawbackstothesemethods.Forthoseinterestedinthecertaintyoffindinggloballyoptimalsolutions,heuristicmethodsarelacking.Ontheotherhand,onemayobtainsufficientcontrolsusingthesemethods,andthatisastepintherightdirectionforcontrolofABMs–inparticularwhenone'sgoalistoobtaincontrolsthatareeithersufficientorsimplybetterthananypreviouslyknown.

5.4 Itispossiblethatthecomplexityofagent-basedmodelswillmakeformulaictranslationtorigorousmathematicalmodelsintractable–inthatcase,heuristicmethodsprovidetheonlymeansforoptimizationandoptimalcontrolofagent-basedmodels.Coupledwiththemodelreductiontechniquesandanalysisintroducedhere,thistechniqueprovidesvaluablemethodologyforsolvingcontrolproblemswithagent-basedmodels.

Acknowledgements

FundingforthisworkwasprovidedthroughU.S.ArmyResearchOfficeGrantNr.W911NF-09-1-0538,andsomeideasweredevelopedbytheOptimalControlforAgent-BasedModelsWorkingGroupatNIMBioS,UniversityofTennessee.Additionally,theauthorsaregratefultoreviewersfortheircarefulconsiderationofthemanuscript.Manyhelpfulsuggestionswereimplementedduetoreviewerfeedback.

Appendices

A:Overview,Designconcepts,andDetails(ODD)protocolforRabbitsandGrassThemodeluponwhichthisversionisbasedisincludedinthesamplelibraryofNetLogo(Wilensky2009),apopularagent-basedmodelingplatform.Thedescriptionhereiswarrantedasitincludesthemechanicsofanoptimizationproblem,thedetailsofwhicharenotavailableelsewhere.

Purpose

Thepurposeofthismodelistoexaminepopulationdynamicsofasimpleenvironmentalsystem.Inparticular,itisamodelofrabbitseatinggrassinafield.Oneachdayofthesimulation,poisoncanbeplacedonthefieldinordertokilltherabbits.Thisversionofthemodelisanattempttoanswerthefollowingoptimizationquestion:whatisthebestwayofcontrolling(i.e.,minimizing)therabbitpopulationwhilealsominimizingtheamountofpoisonused?

Entities,statevariables,andscales

Thissectioncontainsadescriptionofthegridcells,spatialandtemporalscales,andtherabbits.Italsocontainsadescriptionoftheformatofapoisonschedule,theinvestigationofwhichisthekeyfeatureofthemodel.

Gridcells,spatialscale,andtemporalscale.Theworldisasquaregridofdiscretecells,representingafield.Thegridistoroidal:edgeswraparoundbothinthehorizontalandverticaldirections.Thedistancefromthecenterofacelltoaneighboringhorizontalorverticalcellis1unit(thusthedistancebetweentwodiagonalcellsissqrt(2)).Unitsareabstractspatialmeasurements.Timestepsarealsoabstractdiscreteunits.Asimulationconsistsofafinitenumberoftimesteps.Theonlystatevariableforeachcellindicateswhetherornotthecellcurrentlycontainsgrass.Whengrassiseatenonagridcellthereisacertainprobabilitythatitwillgrowbackateachtimestep.Thisgrowthhappensspontaneously.

Table3:Gridcellstatevariables.

Statevariable Name ValueSidelengthoffield s 50gridcellsTotalgridcells N 2500Presenceofgrass grass? 0=nograss,1=grassGrassgrowthprobability γ/0.02Simulationtime total_sim_time 100timesteps

Rabbits.Eachtimestep,rabbitsmove,eatgrass(ornot),andreproduce(ornot).Reproductionisasexualandbasedonenergylevel,whichisraisedwhenarabbiteats.Rabbitsloseenergybothbymovingandbyspawningnewrabbits.Ifarabbit'senergyleveldropsto0orlowertherabbitdies.

Table4:Rabbitstatevariables.

Statevariable Name ValueMovementcost move_cost 0.5Energyfromfood food_energy 3Birththreshold birth_threshold 8Currentenergylevel energy varies

http://jasss.soc.surrey.ac.uk/17/2/6.html 11 16/10/2015

Poisonschedule.Apoisonscheduleuisavectoroflengthtotal_sim_timewitheachentryeither0or1.Eachentrycorrespondstoonetimestepinthesimulation;0meansthatpoisonisnotusedand1meansthatpoisonisused.Thus,thereareatotalof2^(total_sim_time)possiblepoisonschedules.Thepoisonhasamaximumefficacythatdegradesovertimewithrepeateduse.Ifthepoisonisnotusedtheefficacyincreasesagain,uptothemaximum.

Table5:Poisonscheduledetails.

Statevariable Name ValueMaximumefficacy p_max 0.3Degradationrate p_deg 5Currentefficacy p_eff Variesin(0,p_max]

Processoverviewandscheduling

Inordertominimizeambiguity,detailsofmodelexecutionarepresentedaspseudocode;seeAlgorithm2.

Designconcepts

IntheupdatedODDprotocoldescription(Grimmetal.2010)thereareelevendesignconcepts.Thosethatdonotapplyhavebeenomitted.

Basicprinciples.Inessence,thismodelisapredator-preysystemwhereintherabbitsarepredatorsandthegrassisprey.Introductionofpoisonintothemodel,andhavingthatpoisonmodeledasadirectexternalinfluenceonpopulationlevels,createsanaturalsettingforanoptimizationproblem.Onecanstudytheeffectofvariouspoisonstrategiesonpopulationlevels–intermsofminimizingtherabbitpopulationitcanbethoughtofasaharvestingproblem,butintermsofminimizingpoisonitcanbethoughtofasresourceallocation.

http://jasss.soc.surrey.ac.uk/17/2/6.html 12 16/10/2015

Algorithm2.PseudocodefortheRabbitsandGrassprocessandscheduling.

Emergence.Rabbitpopulationandgrasslevelstendtooscillateasthesimulationprogresses.Thefrequencyandamplitudeoftheseoscillationscanbeaffectedbyparametersettingsandinitialvaluesandhencemaybedescribedasemergentmodeldynamics.

Interaction.Agentinteractionisindirect:sincerabbitmovementisexecutedserially,itispossiblethatotherrabbitsdepleteallofthegrassinaparticularrabbit'spotentialfieldofmovement,therebyreducingoreliminatingthechanceforthatrabbittogainenergy.

Stochasticity.Rabbitmovementistotallyrandominthattheycannotsensewhetherneighboringgridcellscontaingrassornot.Whethergrassgrowsbackonanemptygridcellisalsorandom,andagridcellthathasbeenemptyforseveraltimestepsisnomorelikelytogrowgrassthanacellthathasonlyjustbecomeempty.

Observation.Rabbitpopulationandgrasscountsarerecordedateachtimestep.Thetotalnumberofrabbitsaliveduringthecourseofasimulationservesasameasureoffitnessofthepoisonschedule.

Initialization

Atinitialization,20%ofthegridcellscontaingrass;thesearechosenatrandom.Thereare150rabbitsplacedatrandomlocationsthroughoutthegrid.Eachbeginswitharandomamountofenergybetween0and9inclusive(rabbitswith0energymaysurvivethefirsttimestepbyeatinggrass).Totalsimulationtimeis100timestepsandeachsimulationcontainsapoisonscheduleu,describedinPoisonschedule.

Inputdata

Thereisnoinputdatatothemodel.

Submodels

Themodelcontainsnosubmodels.

Optimization

Sincethemulti-objectiveoptimizationproblemisthekeyfeatureofthemodelaspresentedhere,afewclarifyingdetailsareinorder.Theobjectivesoftheoptimizationproblemaretodetermine,fortheparametervaluesprovided,asetofParetooptimalpoisonschedulesthatminimizethenumberofrabbitswhilealsominimizingtheamountofpoisonused.Thenumberofrabbitsreferstothetotalnumberofrabbitsaliveduringthecourseofasimulation–notjustthosealiveattheendofthefinaltimestep.Sinceapoisonscheduleisabinaryvectoroflengthtotal_sim_time,theamountofpoisonusedisrepresentedbythesumoftheentriesofthatvector.

B:Overview,Designconcepts,andDetailsprotocolforSugarScapewithtaxationPurpose

TheversionofSugarScapepresentedhereisamodifiedversionoftheoriginalSugarScape(Epstein1996),amodelinwhichabstractentitiesroamalandscapemadeofsugar.Theseagentsareperiodicallytaxedfortheirsugarstores–thetaxrateisconstantbutthefrequencydiffersfromregiontoregion.ThepurposeofthisversionofSugarScapeistoinvestigatetheeffectsofvarioustaxationpoliciesontaxincomeandagentpopulation.Inparticular,themodelisusedtoinvestigatethefollowingquestion:whatistheoptimaltaxationpolicyformaximizingcollectedincomewhileminimizingdeaths?

Entities,statevariables,andscales

Ants.Eachanthasafixedvisionandmetabolismlevelforthedurationofthesimulation;theselevelsvaryfromanttoant.Antscanseeinthefourprincipaldirectionsup,down,left,andright,butcannotseeanyothergridcells.Metabolismdetermineshowmuchsugaranantloses('burns')eachtimestep.Movementisgovernedbyvision:anantmovestothenearestgridcellwithinitsvisionwiththemaximumamountofsugar.Onlyoneantmayoccupyagridcellatanygiventime.Antsdieiftheirsugarlevelreacheszero.Thereisnoupperlimittohowmuchsugaranantmayaccumulate.Lowvisionisdefinedas1,2,or3andlowmetabolismisdefinedas1or2.Thus,eachantbelongstooneofthefollowingfourcategories:lowvision/lowmetabolism(LL),lowvision/highmetabolism(LH),highvision/lowmetabolism(HL),andhighvision/highmetabolism(HH).

Table6:Antstatevariables.

Statevariable Name ValueLocation(currentregion) reg {0,1,…,8}Vision vis randomin{1,2,…,5}Metabolism met randomin{1,2,3,4}Sugar sug variesin{1,2,…}

Gridcells.Whenantsconsumethesugarfromagridcell,thesugargrowsbackatafixedrateoversubsequenttimesteps,uptoapre-determinedmaximumbasedonthelayoutofthelandscape.

Table7:Gridcellstatevariables.

Statevariable Name ValueMaximumsugar s_max oneof{0,1,2,3,4}Sugar s_here {0,1,2,3,4}Growbackrate α/1

Spatialandtemporalscales.ThelandscapeforSugarScapeispresentedinFigure8.ThemaximumsugaramountsforeachregionaregiveninTable8.Therearefiveregiontypes:thosewhosemaximumsugaris0,1,2,3,or4.Eachantoccupiesexactlyonegridcell,andthemapistoroidal–edgeswrapinboththehorizontalandverticaldirections.Thelandscapeisa50×50gridofcells.Giventhefairlyabstractnatureofthemodel,timeandspaceareunitless.Asimulationconsistsofafinitenumberoftimesteps.

Table8:Maximumsugarandgridcellcountsforeachregion.

Region 0 1 2 3 4 5 6 7 8Max.sugar 0 0 1 1 2 3 3 4 4

Taxesarecollectedatregularintervals.Thetaxrateforagivenantdependsonitscategoryandcurrentregion(forexample,anagentwithhighvisionandlowmetabolismmaybetaxedat

http://jasss.soc.surrey.ac.uk/17/2/6.html 13 16/10/2015

rate0.75inahigh-sugarregionbutonlyat0.25inalow-sugarregion).Taxamountsarealwaysroundeduptothenearestinteger–thisensuresthatanynon-zerotaxratealwayscollectsatleast1unitofsugar.Taxesarecollectedonceeverysubsequent5timestepsforatotaloftentaxcycles.Thechoicetotaxevery5ticksismotivatedbythedesiretonotletthedynamicsstabilize–withfrequenttaxationthedynamicsaremoreimmediatelyaffectedbyprevioustaxrates.

Table9:Taxationandtemporalvariables.

Variable Name Value UnitsSimulationduration total_sim_time 50 timestepsPermissibletaxrates tax {0,0.25,0.5,0.75} N/ATaxinterval tax_interval 5 timesteps

Foreachofthefourantcategoriestherearefivepossibletaxratesdependingontheircurrentregionandeachoftheseratesmaybedifferentforeachofthetentaxcycles.Thus,ataxscheduleisavectoroflength5·4·10=200witheachentryin{0,0.25,0.5,0.75}–thismeansthereareatotalof4^(200)differenttaxschedules.Theoptimizationproblemistodeterminethetaxschedulesthatmaximizethetotaltaxincomecollectedwhileminimizingthenumberofdeaths.

Processoverviewandscheduling

TheABMprocessispresentedinAlgorithm3aspseudocode.Theantandtaxroutinesareexecutedfullybyoneant,thenfullybyanother–i.e.,serially.Hencestatevariablesareupdatedasynchronously.Timestepsarediscreteunits,asismovement:antsjumpdirectlyfromthecenterofonegridcelltothecenterofanother.

Algorithm3.PseudocodeforSugarScapeprocessandscheduling.

Designconcepts

Basicprinciples.ThisversionofSugarScapebuildsontheoriginalbyincorporatingtaxation.Ingeneral,thebasicquestionunderinvestigationishowspatially-dependentlocalinputsaffectglobaldynamics.Specifically,themodelinvestigateshowlocaltaxratesaffecttaxincomeandregionalpopulationdistribution,aquestionwhichholdsinterestinavarietyofreal-worldsettings.

Emergence.Spatialpopulationdynamicsoughttobeanemergentpropertyofthemodel:forexample,hightaxratesinhigh-sugarregionsmightsubstantiallyalterregionalpopulationdynamics.Theprecisemechanismdrivingsuchchangesisnotbuiltintothemodelinanydirectsense.

Objectives.Theobjectiveofeachantistomovetoacellwithinitsvisionwiththemaximumamountofsugar.Thereisnootherconsideration,andantsdonothaveknowledgeofpastorfuturetaxratesatanylocation.

http://jasss.soc.surrey.ac.uk/17/2/6.html 14 16/10/2015

Sensing.Antsareawareofthesugarlevelandoccupancyofeachgridcellwithintheirvision.Theyarenotawareofanypropertiesofanyotherants,eventhosewithintheirvision.

Interaction.Antsinteractwithoneanotherindirectlyinthesensethatonlyoneantmayoccupyagridcellatanygiventime.Thusiftwoantshavethesamehigh-sugargridcellwithintheirvision,whicheverantisrandomlyselectedtomovefirstwilloccupythatcell.Thismayverywellalterthemovementoftheotherant.Inthisway,serialexecutionandasynchronousupdatearekeyfeaturesofagentinteraction.Ifantorderwasnotrandom(i.e.,thesameantwasallowedtomovefirsteachtimestep),populationdynamicsmightbefundamentallyaltered.

Stochasticity.Antmovementispartiallystochastic:ifanantseesfourunoccupiedgridcellswith2sugarandthreeunoccupiedgridcellswith3sugar,theantwillchoosethenearestcellwith3sugar.This'minimumdistance'policytendstoleadtoantsclusteringonregionalboundaries,astheyhavelimitedincentivetomovetotheinteriorofaregion.ThismovementfeatureisdiscussedinEpsteinandAxtell(1996).

Collectives.Inasense,antsformcollectivesthataffectindividualsinsideandoutsideofthecollective.Thisarisesbecauseonlyoneantmayoccupyagridcellatanytime.Inhigh-sugarregions,antsinthemiddleoftheregiontendtobecometrappedbecauseallavailablespacesareoccupied.Atthesametime,anindividualontheborderofsucharegionisfrequentlyunabletoenterduetothehighpopulationdensitywithintheregion.Thesecollectivesformentirelyasaresultoflocalinteractions.

Observation.Eachsimulationconsistsofafinitenumberoftimesteps.Ateachtimestep,thefollowinginformationiscollected:thetotalamountoftaxcollected,andthenumberofdeathsthatoccur.Foreachsimulation,thetaxpolicyisrecordedaswell.Attheendofeachsimulationthesedataarewrittentoacommaseparatedvalue(.csv)file,auniversalformatforspreadsheetapplications.

Initialization

Themodelisinitializedwith200ants;eachisplacedatarandomunoccupiedlocationonthelandscape.Antsbeginwitharandomamountofsugarbetween5and25(inclusive);thisvalueisdifferentforeachantandchosenfromauniformdistribution.Antsareinitializedwithvisionchosenatrandombetween1and5(inclusive)andmetabolismbetween1and4(inclusive).Visionandmetabolismofagivenantdonotchangeoverthecourseofasimulation.

Inputdata

Thelandscapeisreadinfroma.txtfile;thishelpswithimplementationandmakesiteasiertomakechanges.Ataxpolicycaneitherbechosendirectlyviacodemanipulationorchosenatrandom.Thepolicymustbechosenpriortosimulation.Assuch,thetaxpolicymaybethoughtofasinputtothemodel.

Submodels

TherearenosubmodelsforthisversionofSugarScape.

References

ALTMAN,D.(1991).PracticalStatisticsforMedicalResearch.London:ChapmanandHall.

BERNASCHI,M.&Castiglione,F.(2001).Designandimplementationofanimmunesystemsimulator.Comput.Biol.Med.31,303–331.[doi:10.1016/S0010-4825(01)00011-7]

BUCKERIDGE,D.L.,Jauvin,C.,Okhmatovskaia,A.&Verma,A.D.(2011).SimulationAnalysisPlatform(SnAP):aToolforEvaluationofPublicHealthSurveillanceandDiseaseControlStrategies.AMIAAnnuSympProc2011,161–170.

CASTIGLIONE,F.(1995).C-ImmSimSimulator.InstituteforComputingApplications,NationalResearchCouncil(CNR)ofItaly.<http://www.iac.cnr.it/~filippo/C-ImmSim.html>.

CASTIGLIONE,F.,Pappalardo,F.,Bernaschi,M.&Motta,S.(2007).OptimizationofHAARTwithgeneticalgorithmsandagent-basedmodelsofHIVinfection.Bioinformatics23(24),3350–3355.[doi:10.1093/bioinformatics/btm408]

CHAO,D.L.,Halloran,M.E.,Obenchain,V.J.&Longini,I.M.(2010).FluTE,apubliclyavailablestochasticinfluenzaepidemicsimulationmodel.PLoSComput.Biol.6,e1000656.[doi:10.1371/journal.pcbi.1000656]

COELLO,C.(2013).Listofreferencesonevolutionarymultiobjectiveoptimization.http://www.lania.mx/~ccoello/EMOO/EMOObib.html.Archivedat:<http://www.webcitation.org/6HhFo4K5H>.

COHEN,J.(1968).Weightedkappa:nominalscaleagreementwithprovisionforscaleddisagreementorpartialcredit. PsycholBull70(4),213–220.[doi:10.1037/h0026256]

COOLEY,P.,Brown,S.,Cajka,J.,Chasteen,B.,Ganapathi,L.,Grefenstette,J.,Hollingsworth,C.R.,Lee,B.Y.,Levine,B.,Wheaton,W.D.&Wagener,D.K.(2011).Theroleofsubwaytravelinaninfluenzaepidemic:aNewYorkCitysimulation.JUrbanHealth88,982–995.[doi:10.1007/s11524-011-9603-4]

DASCLU,M.,Franti,E.&Stefan,G.(1998).Modelingproductionwithartificialsocieties:theemergenceofsocialstructure.In:CellularAutomata:ResearchTowardsIndustry(Bandini,S.,Serra,R.&Liverani,F.,eds.).SpringerLondon,pp.218–229.

DEAN,J.S.,Gumerman,G.J.,Epstein,J.M.,Axtell,R.L.,Swedlund,A.C.,Parker,M.T.&McCarroll,S.(2000).UnderstandingAnasaziculturechangethroughagent-basedmodeling.Dynamicsinhumanandprimatesocieties.OxfordUniversityPress,Oxford,179–206.

DURRETT,R.&Levin,S.(1994).Theimportanceofbeingdiscrete(andspatial).TheoreticalPopulationBiology46(3),363–394.[doi:10.1006/tpbi.1994.1032]

ELEMAM,K.(1999).Benchmarkingkappa:interrateragreementinsoftwareprocessassessments.EmpiricalSoftwareEngineering4(2),113–133.[doi:10.1023/A:1009820201126]

EPSTEIN,J.M.&Axtell,R.(1996).Growingartificialsocieties:socialsciencefromthebottomup.Washington,DC,USA:TheBrookingsInstitution.

EUGENIO,B.D.(2000).Ontheusageofkappatoevaluateagreementoncodingtasks.In:InProceedingsoftheSecondInternationalConferenceonLanguageResourcesandEvaluation.

FERRER,J.,Prats,C.,Lopez,D.,Valls,J.&Gargallo,D.(2010).Contributionofindividual-basedmodelsinmalariaeliminationstrategydesign.MalariaJournal9(Suppl2),P9.[doi:10.1186/1475-2875-9-S2-P9]

FLEISS,J.(1981).StatisticalMethodsforRatesandProportions.Hoboken,NJ:JohnWileyAndSons.

FLEISS,J.L.(1971).Measuringnominalscaleagreementamongmanyraters.PsychologicalBulletin76(5),378–382.[doi:10.1037/h0031619]

GENNERT,M.A.&Yuille,A.(1988).Determiningtheoptimalweightsinmultipleobjectivefunctionoptimization.In:ComputerVision,SecondInternationalConferenceon.[doi:10.1109/ccv.1988.589974]

GOLDBERG,D.(1989).Geneticalgorithmsinsearch,optimization,andmachinelearning.Reading,MA:Addison-Wesley.

GRIMM,V.,Berger,U.,Bastiansen,F.,Eliassen,S.,Ginot,V.,Giske,J.,Goss-Custard,J.,Grand,T.,Heinz,S.K.,Huse,G.,Huth,A.,Jepsen,J.U.,Jrgensen,C.,Mooij,W.M.,Mller,B.,Pe'er,G.,Piou,C.,Railsback,S.F.,Robbins,A.M.,Robbins,M.M.,Rossmanith,E.,Rger,N.,Strand,E.,Souissi,S.,Stillman,R.A.,Vab,R.,Visser,U.&DeAngelis,D.L.(2006).Astandardprotocolfordescribingindividual-basedandagent-basedmodels.EcologicalModelling198(1–2),115–126.[doi:10.1016/j.ecolmodel.2006.04.023]

GRIMM,V.,Berger,U.,DeAngelis,D.L.,Polhill,J.G.,Giske,J.&Railsback,S.F.(2010).TheODDprotocol:Areviewandfirstupdate.EcologicalModelling221(23),2760–2768.[doi:10.1016/j.ecolmodel.2010.08.019]

HAPPE,K.(2005).Agent-basedmodellingandsensitivityanalysisbyexperimentaldesignandmetamodelling:Anapplicationtomodellingregionalstructuralchange.2005InternationalCongress,August23-27,2005,Copenhagen,Denmark24464,EuropeanAssociationofAgriculturalEconomists.

HARADA,Y.,Ezoe,H.,Iwasa,Y.,Matsuda,H.&Sato,K.(1995).Populationpersistenceandspatiallylimitedsocialinteraction.TheoreticalPopulationBiology48(1),65–91.[doi:10.1006/tpbi.1995.1022]

HINKELMANN,F.,Murrugarra,D.,Jarrah,A.&Laubenbacher,R.(2011).Amathematicalframeworkforagentbasedmodelsofcomplexbiologicalnetworks.BullMathBiol73(7),1583–1602.[doi:10.1007/s11538-010-9582-8]

http://jasss.soc.surrey.ac.uk/17/2/6.html 15 16/10/2015

HORN,J.,Nafpliotis,N.&Goldberg,D.(1994).Anichedparetogeneticalgorithmformultiobjectiveoptimization.In:EvolutionaryComputation,1994.IEEEWorldCongressonComputationalIntelligence.ProceedingsoftheFirstIEEEConferenceonComputationalIntelligence[doi:10.1109/icec.1994.350037]

KASAIE,P.,Kelton,W.,Vaghefi,A.&Naini,S.(2010).Towardoptimalresource-allocationforcontrolofepidemics:Anagent-based-simulationapproach.In:WinterSimulationConference(WSC),Proceedingsofthe2010.

KIM,P.S.,Lee,P.P.&Levy,D.(2008a).APDEmodelforimatinib-treatedchronicmyelogenousleukemia.Bull.Math.Biol.70,1994–2016.[doi:10.1007/s11538-008-9336-z]

KIM,P.S.,Lee,P.P.&Levy,D.(2008b).Modelingimatinib-treatedchronicmyelogenousleukemia:reducingthecomplexityofagent-basedmodels.Bull.Math.Biol.70,728–744.[doi:10.1007/s11538-007-9276-z]

KLANN,M.T.,Lapin,A.&Reuss,M.(2011).Agent-basedsimulationofreactionsinthecrowdedandstructuredintracellularenvironment:Influenceofmobilityandlocationofthereactants.BMCSystBiol5,71.[doi:10.1186/1752-0509-5-71]

KRIPPENDORFF,K.(1980).ContentAnalysis:anIntroductiontoitsMethodology.BeverlyHills,CA:SagePublications.

LANDIS,J.R.&Koch,G.G.(1977).Themeasurementofobserveragreementforcategoricaldata.Biometrics33(1),159–174.[doi:10.2307/2529310]

LI,J.,Parkes,A.J.&Burke,E.K.(2011).Evolutionarysqueakywheeloptimization:Anewanalysisframework.EvolutionaryComputation.<http://www.mitpressjournals.org/doi/abs/10.1162/EVCO_a_00033>.[doi:10.1162/EVCO_a_00033]

LOLLINI,P.,Castiglione,F.&Motta,S.(1998).Modellingandsimulationofcancerimmunopreventionvaccine.IntJCancer77,937–941.[doi:10.1002/(SICI)1097-0215(19980911)77:6<937::AID-IJC24>3.0.CO;2-X]

LOLLINI,P.L.,Motta,S.&Pappalardo,F.(2006).Discoveryofcancervaccinationprotocolswithageneticalgorithmdrivinganagentbasedsimulator.BMCBioinformatics7,352.[doi:10.1186/1471-2105-7-352]

LUKE,S.,Cioffi-Revilla,C.,Panait,L.,Sullivan,K.&Balan,G.(2005).MASON:Amulti-agentsimulationenvironment.Simulation:TransactionsofthesocietyforModelingandSimulationInternational82(7),517–527.[doi:10.1177/0037549705058073]

MAO,L.(2011).Agent-basedsimulationforweekend-extensionstrategiestomitigateinfluenzaoutbreaks.BMCPublicHealth11,522.[doi:10.1186/1471-2458-11-522]

MEIER-SCHELLERSHEIM,M.&Mack,G.(1999).Simmune,atoolforsimulatingandanalyzingimmunesystembehavior.CoRRcs.MA/9903017.

NORTH,M.J.,Collier,N.T.&Vos,J.R.(2006).Experiencescreatingthreeimplementationsoftherepastagentmodelingtoolkit.ACMTrans.Model.Comput.Simul.16(1),1–25.[doi:10.1145/1122012.1122013]

OREMLAND,M.(2011).OptimizationandOptimalControlofAgent-BasedModels.Master'sthesis,VirginiaPolytechnicInstituteandStateUniversity.

PAPPALARDO,F.,Pennisi,M.,Castiglione,F.&Motta,S.(2010).Vaccineprotocolsoptimization:insilicoexperiences.Biotechnol.Adv.28,82–93.[doi:10.1016/j.biotechadv.2009.10.001]

PATEL,R.,Longini,I.M.,Jr.&Halloran,M.E.(2005).Findingoptimalvaccinationstrategiesforpandemicinfluenzausinggeneticalgorithms.J.Theoret.Biol.234(2),201–212.[doi:10.1016/j.jtbi.2004.11.032]

PENNISI,M.,Catanuto,R.,Pappalardo,F.&Motta,S.(2008).Optimalvaccinationschedulesusingsimulatedannealing.Bioinformatics24,1740–1742.[doi:10.1093/bioinformatics/btn260]

RAHMAN,A.,Setayeshi,S.&Shamsaei,M.(2007).Ananalysistowealthdistributionbasedonsugarscapemodelinanartificialsociety.InternationalJournalofEngineering20(3),211–224.

RAPIN,N.,Lund,O.,Bernaschi,M.&Castiglione,F.(2010).Computationalimmunologymeetsbioinformatics:theuseofpredictiontoolsformolecularbindinginthesimulationoftheimmunesystem.PLoSONE5,e9862.[doi:10.1371/journal.pone.0009862]

ROOS,A.M.D.,Mccauley,E.&Wilson,W.G.(1991).Mobilityversusdensity-limitedpredator–preydynamicsondifferentspatialscales.Proceedings:BiologicalSciences246(1316),pp.117–122.[doi:10.1098/rspb.1991.0132]

SIM,J.&Wright,C.C.(2005).Thekappastatisticinreliabilitystudies:use,interpretation,andsamplesizerequirements.PhysTher85(3),257–268.

SWIERNIAK,A.,Kimmel,M.&Smieja,J.(2009).Mathematicalmodelingasatoolforplanninganticancertherapy.Eur.J.Pharmacol.625(1-3),108–121.[doi:10.1016/j.ejphar.2009.08.041]

VELIZ-CUBA,A.,Jarrah,A.S.&Laubenbacher,R.(2010).Polynomialalgebraofdiscretemodelsinsystemsbiology.Bioinformatics26(13),1637–1643.[doi:10.1093/bioinformatics/btq240]

WANG,T.&Zhang,X.(2009).3Dproteinstructurepredictionwithgenetictabusearchalgorithminoff-latticeABmodel.In:Proceedingsofthe2009SecondInternationalSymposiumonKnowledgeAcquisitionandModeling-Volume01,KAM'09.Washington,DC,USA:IEEEComputerSociety.[doi:10.1109/KAM.2009.2]

WANG,Z.,Bordas,V.&Deisboeck,T.(2011).IdentificationofCriticalMolecularComponentsinaMultiscaleCancerModelBasedontheIntegrationofMonteCarlo,Resampling,andANOVA.FrontiersinPhysiology2(0).[doi:10.3389/fphys.2011.00035]

WILENSKY,U.(2009).Netlogo.CenterforConnectedLearningandComputer-BasedModeling,NorthwesternUniversity,Evanston,IL.http://ccl.northwestern.edu/netlogo/.

WILSON,W.,Deroos,A.&Mccauley,E.(1993).Spatialinstabilitieswithinthediffusivelotka-volterrasystem:Individual-basedsimulationresults.TheoreticalPopulationBiology43(1),91–127.[doi:10.1006/tpbi.1993.1005]

WILSON,W.,McCauley,E.&Roos,A.(1995).Effectofdimensionalityonlotka-volterrapredator-preydynamics:Individualbasedsimulationresults.BulletinofMathematicalBiology57,507–526.[doi:10.1007/BF02460780]

WOELKE,A.L.,vonEichborn,J.,Murgueitio,M.S.,Worth,C.L.,Castiglione,F.&Preissner,R.(2011).Developmentofimmune-specificinteractionpotentialsandtheirapplicationinthemultiagent-systemVaccImm.PLoSONE6,e23257.[doi:10.1371/journal.pone.0023257]

YANG,Y.,Atkinson,P.M.&Ettema,D.(2011).AnalysisofCDCsocialcontrolmeasuresusinganagent-basedsimulationofaninfluenzaepidemicinacity.BMCInfect.Dis.11,199.[doi:10.1186/1471-2334-11-199]

YESILYURT,S.&Patera,A.T.(1995).Surrogatesfornumericalsimulations;optimizationofeddy-promoterheatexchangers.ComputerMethodsinAppliedMechanicsandEngineering121(14),231–257.[doi:10.1016/0045-7825(94)00684-F]

ZOU,Y.,Fonoberov,V.A.,Fonoberova,M.,Mezic,I.&Kevrekidis,I.G.(2012).Modelreductionforagent-basedsocialsimulation:coarse-grainingacivilviolencemodel.PhysRevEStatNonlinSoftMatterPhys85(6Pt2),066106.[doi:10.1103/PhysRevE.85.066106]

http://jasss.soc.surrey.ac.uk/17/2/6.html 16 16/10/2015