Upload
turcan-ciprian-sebastian
View
223
Download
0
Embed Size (px)
Citation preview
8/2/2019 Trinity09 Bossaerts Paper
1/28
WhatDecisionNeuroscienceTeachesUsAboutHuman
DecisionMakingUnderUncertainty
PeterBossaerts
CaliforniaInstituteofTechnology,Pasadena,CAand
EcolePolytechniqueFdraleLausanne,Switzerland
Abstract:
Humandecisionmakingunderuncertaintyistheoutcomeofcomplexneuro
physiologicalprocessesinvolving,amongothers,constantreevaluationofthestatisticsoftheproblemathand,balancingofthevariousemotionalaspects,andcomputationoftheveryvaluesignalsthatareatthecoreofmoderneconomicthinking.Theevidencesuggeststhatemotionsplayacrucialsupportingroleinthemathematicalcomputationsneededforreasonedchoice,ratherthaninterferingwithit,evenifemotions(andtheirmathematicalcounterparts)maynotalwaysbebalancedappropriately.Decisionneurosciencecanbeexpectedinthenearfuturetoprovideanumberofeffectivetoolsforimprovedfinancialdecisionmaking.
1.Introduction
Financestudiesdecisionmakingunderuncertainty.Foralongtime,thisstudyhasbeennormative,addressingthequestion:howshouldonechoosewhenfacedwithuncertainoutcomes?Thishasledtothedevelopmentofutilitytheory,andmorespecifically,expectedutilitytheoryandmeanvarianceanalysis.Thelatterhasproventobemosthelpfulwhenmultipleoptionscanbechosenatonetime(portfolioanalysis).
Normativedecisionanalysisstilldominatesassetpricingtheory,basedontheideathatthereisamarginalinvestor(oranaggregateinvestor)andthat
somehowshemustbedoingthingsright.Fromexperiments,however,weknowthatthisisfarfromaforegoneconclusion.Forinstance,insituationsofambiguity(someoftheoutcomeprobabilitiesareunknown),theirrationalinvestormaydeterminepricesevenifheisclearlynotthemarginalinvestor(Bossaertsetal2008a).
Attheindividuallevel,normativeanalysishaslongbeenknowntomakeinvalidpredictions.Eversince(Allais1953),choiceshavebeenconsistentlyfoundtoviolatethetwofoundationalaxiomsofexpectedutility,namely,theindependenceaxiomandthesurethingprinciple.Thishasledtotheemergenceofbehavioralfinance.
8/2/2019 Trinity09 Bossaerts Paper
2/28
Behavioralfinancealsostartsfromchoice,butrejectstheideathatitreflectsmaximizationofarational(expected)utilityfunction.Nevertheless,ithasoneimportantfeatureincommonwithnormativefinance:choicesstillareinterpretedasiftheymaximizesomeutility.Eventhechoiceofutilityprofileissuggestiveofnostalgiatowardexpectedutility:thereareunderlyingstatesof
nature,andutilityisobtainedasthesummationoverthesestates,ofstateprobabilities(perhapsweighted)andutilitiesofstatespecificoutcomes.
Indeed,prospecttheory(Kahneman&Tversky1979)canbeviewedasanattempttocharacterizeactualhumanchoice[andrecentlyalsononhumanprimatechoice(Chenetal2006)]withinthestraightjacketofnormativetheory.Areferencepointneededtobeintroduced,probabilitiesneededtobebiasedinspecificways,andtheideaofdecreasingmarginalutilityhadtobeabandonedforthelossdomain.
Butisitreallytruethatchoiceistheoutcomeofautilitymaximizationprocess?
Takeriskseekingforlosses,forinstance.Whatifthisisabiasthatcomesaboutbecauseofthewaydecisionsaremade,andnotreallytheresultofincreasingmarginal(hedonic)utilityforoutcomesbelowthereferencepoint?
Ostensiblyinnocuousconsequencesfollowfrompracticalimplementationoftheideathatchoiceunderuncertaintyreflectsmaximizationof(prospecttheory)utility.E.g.,wealthmanagersshouldencouragetheirclientstobuystructuredproductsthatimplementshortpositionsinputoptions.
Buthowcouldweknowwherechoicereallycomesfrom?Itisherethatneurosciencecomesin.Neuroscienceallowsustostudytheentireprocessof
decisionmaking,frominitialperceptionofastimulus(whichconveysnewinformationand/ornewinvestmentoptions),tovaluationandmotivation,andtheveryactofchoosing.
Whenlookingforinspirationinneuroscience,itisimportantfromtheoutsettorealizethatneuroscienceisasubfieldofbiology,andhence,thatitusesanoverarchingprincipletounderstandneurobiologicalphenomena(analogoustoabsenceofarbitrageortheefficientmarketshypothesisofneoclassicalfinance),namely,evolutionaryfitness.Specifically,thebrainisviewedasanorganthatisoptimized,throughevolutionarypressure,todothingsrightinitsnaturalenvironment.
Thereisampleevidenceforthisinsensoryprocessinginthebrain.E.g.,auditorysignalsareoptimallymixedwithvisualsignalstogenerateaccurateguessesofthelocalizationofthesourceofthesignals(Shamsetal2005).Optimalmeans:inaccordancewithBayeslaw.
Whenchoiceunderuncertaintyisconcerned,however,thetypesofrisksthatthebrainwouldbeoptimizedformaybeverydifferentfromrisksroutinelyencounteredinthefinancialsphere.Indeed,whilemanyecologicallyrelevantrisks(e.g.,temperaturechanges)canreadilybedescribedasGaussianandpredictable,returnsgeneratedthroughorganizedfinancialmarketsare
distinctlyleptokurtoticandunpredictable.So,itmakessensetoaskwhetherthereisamismatchbetweenthebrainandfinancialmarkets.Butwecannottell
8/2/2019 Trinity09 Bossaerts Paper
3/28
aslongaswedontknowhowdecisionscomeaboutinthebrain.Thatisoneofthetasksofanareaofneurosciencecalleddecisionneuroscience.
Whenexploringdecisionneuroscience,oneisimmediatelyconfrontedwithaninspiringdifferentiation.Indecisiontheory,allrisksarethesamethereisno
distinctionbysource.Yetthebrainevidentlydistinguishesbetween(atleast)twotypes:uncertaintygeneratedbyanunintentionalsource(nature;arandomnumbergenerator;afinancialmarketwithoutinsiders),andthatgeneratedbyanintentionalsource(astrategicopponent;afinancialmarketwithinsiders).Thedifferentiationmanifestsitselfintheengagementofdifferentfunctionalregionsofthebrain,thebehavioraleffectswhenlesionsselectivelyaffecttheseregions,anddifferencesacrosshumansinattitudestowardseithersource.Still,theseparationisnotextreme;commonregionsdogetinvolved,asweshallseelater.Wewillmostlydealherewithuncertaintyfromanunintentionalsource,leavingintentionaluncertaintyforalatersection.
Decisionneurosciencehasprogresseddramaticallyoverthelasttenyears,followingthediscoveryoftheroleofdopamineneuronsinpredictionunderuncertainty(Schultzetal1997).Thespeedofdiscoveryhasbeengrowingoverthelastfiveyears,sothatsurveyingtheaccomplishmentsbecomeslikedescribingamovingtarget.Bythetimethisreviewispublished,someclaimsmadeherewillbeoutdated,andquestionsraisedmayhavebeenanswered.
Thepickupinspeedisinlargepartduetotheintroductionofconcepts,theoriesandresultsfromeconomictheoryandexperimentaleconomics.Hencesomewouldrefertodecisionneuroscienceasneuroeconomics,or,whenitconcernsriskanduncertainty,neurofinance.Iwouldliketorefrainfromusingthese
termshere,primarilytoemphasizethatneurosciencehadmuchtosayaboutdecisionmakingevenbeforetheadventofneuroeconomics.
Onthematterofneuroeconomics,anumberofreviewshaverecentlyappeared.Alltoooftenthesereviewswerenotreallysurveysbutcriticalassessmentsfrombothproponentsandopponentsofthefieldbasedononlyafewarticleswithadistincteconomicsangle,effectivelybypassingthevastandsophisticatedliteratureindecisionneuroscience.Thegoalinthisreview,incontrast,isnotonlytolettheeconomisttalk,butalsothedecisionneuroscientist.
2.ExperimentalFinance
Virtuallyallempiricalworkinfinanceuseshistoricaldatafromthefield,i.e.,dataastheyaregeneratedbynaturallyoccurring(financial)markets.Thisisincontrastwithdecisionneuroscience,wherecontrolledexperimentationisthenorm.Whilethissometimesinvolvesfieldexperiments(Coates&Herbert2008,Lo&Repin2002),itmostlymeanslaboratoryexperiments.Iffinanceistobeenlightenedbydecisionneuroscience,itneedstoappreciatethemeritsofcontrolledexperimentation.Thisisfarfromaforegoneconclusion,soitispertinenttoelaborate.
8/2/2019 Trinity09 Bossaerts Paper
4/28
Theuseofexperimentsisnotonlydictatedbypracticalconsiderations(itwouldbehardtohaveatraderperformhisorherdaytodaydutieswhileinanfMRIscanner),butforemostbecauseofthenecessitytocontrolparameters.Indeed,naturerarelycaresaboutourinterests.WemaywanttostudythebasicprinciplesunderlyingCAPM(Bossaerts&Plott2004),yetnaturemayhave
decidedtoprovideuswithanenvironment(multiperiodsecuritieswithskewedpayoffs,delegatedportfoliomanagement,)leastpropitiousforCAPMtoemerge.Togeneratefinancialuncertaintycausedbyanintentionalsource,onemaywanttocontrolthenumberofinsiders(Bruguieretal2008),yetthenumberofinsidersisgenerallyunknowninfieldmarkets,eveniftheirpresencemaybesuspectedattimes[andexploited:see(Bossaerts&Hillion1991)].
Lackofexternalvalidityisperhapsthemostimportantreservationonecanhaveaboutlaboratoryexperimentation.Inexperimentalfinance,thismostlyconcernsthesizeofpaymentsandrisks,whicharegenerallythoughttobeinadequatefortheresultstohaveanyrelevancefortherealworld(asifalaboratoryisnot
real,butthatisnotthepoint).
Interestingly,decisionneurosciencegivesanindicationofhowtothinkaboutthisissue.Indeed,oneofthekeybrainregionsthattracksrewardpredictionerrorshasrecentlybeendemonstratedtoencoderewardsadaptively:whenthehighrewardofabinarygambleisdelivered,firingofdopamineneuronsinthebrainstemisindependentofthesizeofthereward;evenwhenincreasingtherewardtenfold,activationremainsthesame.Converselyfordeliveryofthelowrewardofsuchgambles.SeeFig.1.Fromthepointofviewoftherewardpredictionsysteminthebrain,workingwithsmallgamblesorwithlargegamblesisthesame;everythingisjustscaled.
Thereisofcourseasimpleexplanationforadaptiveencoding(orscaling):thereisaphysicalconstraintontheamountoffiringagiventypeofneuroncanachieve.Sincetherangeofstimulithattheneuronsarepossiblyconfrontedwithisfarbeyondthatoftheirfiring,thebrainhasoptedforscaling.
Adaptiveencodingofrewardscanalsobeobservedelsewhere,suchasintheorbitofrontalcortex(theregionrightabovetheeyes),whichhasbeenshowntoencodeutility(moreonthatlater).Whilethiswasfirstdemonstratedforthemonkeybrain(Tremblay&Schultz1999),thishasrecentlybeenconfirmedforthehumanbrain(Elliottetal2008).
Adaptiveencodingmaynotbesufficienttoestablishexternalvalidityoflaboratoryexperiments.Indeed,whileadaptiveencodinghasbeenobservedfortherangesofoutcomesandrisksinthelaboratory,thereisnoguaranteethatitextrapolateswithoutlimit.Specifically,beyondacertainrange,brainregionsmaybecomeinvolvedthataretypicallynotactivatedinthelaboratory.
Butthebeliefthatdecisionmakingunderuncertaintyintherealworldisfundamentallydifferentrequiresproofthatthebrainregionsthatareactivatedinthelaboratorybecomedisengaged.Thisishighlyunlikely,becausetheregionsthatareactiveinsidethelaboratoryhappentobepowerfulandpervasive,aswe
shallsee,involvingcrucialneurotransmitters(suchasdopamine)andtriggering
8/2/2019 Trinity09 Bossaerts Paper
5/28
strongemotionalreactions(arousal,fear,)thatwouldneedtobeoffsetorneutralizedsomehowforthemtobecomeirrelevant.
Infact,alltheevidencepointstothecontrary.Decisionmakingbyprofessionaltradershasbeenshowntobehighlyemotional(Lo&Repin2002),andit
actuateshormonessuchastestosteroneandcortisol(Coates&Herbert2008)thathaveadirectinfluenceontheverybrainregionsweseeatworkinthelaboratory.Likewise,theallegedwidespreaduseofpharmacologicalenhancerssuchascocainesuggestsattemptsbyprofessionaltraderstodirectlyimpactthedopamineneuronsreferredtoabove.Indeed,cocaineisknowntoaffectthesynapticstrengthofdopamineneurons,causingprojectionneuronstobecomehypersensitive,specificallytopositiverewardpredictionerrors(Saaletal2003).
Manyfinancialdecisionproblems,suchasretirementportfolioallocation,haveasignificanttimecomponent.Itisaseriouschallengetoemulatesuchproblemsinthelab.Thetimehorizoninrecentexperimentshasbeenextendedtoseveral
weeks(Kable&Glimcher2007,McClureetal2004),butnotyears.Nevertheless,cleverchoiceofexperimentaldesigndoesallowexperimenterstoexposesubjectstomultiyearhorizons,evenifonlyindirectly.Forinstance,whensubjectsarefacedwithnutritiousfoodandtastyfood,choiceeffectivelyimpliesasubstantialtimedimension,becauseofconsequencesforhealthfarintothefuture(Hareetal2008a).
Ofcourse,onecouldalwaysresorttothoughtexperiments,byaskingsubjectsforhypotheticalchoiceinimaginedsituations[asintheoriginalexperimentsin(Kahneman&Tversky1979)].Buttheproblemisthatthesethoughtexperimentsmaynotengagethesamebrainprocessesaswhenexperiencingthe
outcomes.Emotions,forinstance,areanintegralpartofreasoneddecisionmakingunderuncertainty,asweshallsee.Ifthehypotheticalquestionscannotinducethesameemotionalreactionaswhentherisksathandareexperiencedfirsthand,predictedeffectsmaynotemerge.Thiscouldexplainwhyexperimentaleconomistshavenoticedthattheyobtainbetterresultswhentheypayforperformance.Fielddataalsotellusthatoutcomesmayneedtobeexperiencedbeforetheysignificantlyinfluencechoice(Feng&Seasholes2005).
3.PerceptionOfRiskAndReward
Oneofthemostfascinatingfindingsofthelasttwoorthreeyearsisthatthebrainseparatelyencodestheriskandtherewardwhenconfrontedwitha
gamble.Thissuggeststhatthebrainengagesinmeanvarianceanalysis,wherebygamblesarerepresentedintermsoftheirstatisticalproperties(moments)ratherthanasarandomvariablelivingonanabstractstatespace.Thesuggestionisallthemorecrediblebecausethemetricofriskencodedinthebrainappearstobevariance.(Ongoingworkextendsthistohighermoments,suchasskewnessandkurtosis.)
Howdoweknowthis?Fromexperimentslikethefollowing.Subjectsarepresentedwithadeckoftenrandomlyshuffledcards,numbered1to10.Twocardswillbedrawnfromthedeck,withoutreplacement.Subjectsareaskedtobetwhetherthesecondcardisgoingtobelowerthanthefirst.Oncethesubjectpostsherbet(for$1),thetwocardsaredisplayed,oneaftertheother.Weareinterestedinbrainactivationwhenthecardsarerevealed,becausethatiswhen
8/2/2019 Trinity09 Bossaerts Paper
6/28
thebrainshouldencodethecorrespondingchangesinexpectedrewardandrisk(Fig.2A).
Noticethatthisisaverysimplegame,involvingcards,whichmostsubjectsshouldbeintimatelyfamiliarwith.Probabilitiesarenotmentionedexplicitly,to
shortcutthemathematicalregionofthebrainthatwouldbesummonedtotranslatetheabstractnumericaldataintosomethingtherewardandriskprocessingregionscandealwith.Indeed,explicitrepresentationofprobabilitieswouldaddalayerofcomplexity,whichwouldbeinteresting,butwhichwemaynotbereadytoventureintoyet.
Also,noticethatsubjectsreallydonothavetomakeanydecisionatthepointsintimethatinterestus(revelationofthetwocards).Again,thisisdeliberatelydone.Thereisalongpathinthebrainfromperceptiontochoice,andweareatfirstinterestedinperception.Doesthatmeanthattheresultingactivationsareirrelevantforchoice?No:bynowitisknownthattheseactivationspredict
choice(Christopoulosetal2008,Kuhnen&Knutson2005).Asamatteroffact,activationsinapurelyperceptualtasklikeourshavebeenfoundtobetterpredictchoicethan(past)choiceitself,especiallywhenoneisclosetoindifference(Bernsetal2008).Thatis,perceptualneuralsignalsofvaluationparametersprovidebetterforecaststhanpreferencesrevealedthrough(past)choice.Thisisnottosaythattheperceptualactivationsdeterminechoice.Insituationsoffreechoice,additionalengagementisneeded,bothsubcortical(O'Dohertyetal2004)andcortical(Hamptonetal2006).
Inourcardgame,expectedreward(mathematicalexpectationofpayoff)andrewardrisk(measuredasvariance)changeeachintheirownwayasafunction
ofprobabilityofreward.Specifically,expectedrewardincreaseslinearlyandriskdisplaysasymmetric,invertedUshapedpattern,withpeakatprobability0.5(Fig.2B).Ifexpectedrewardandriskareencodedsomewhereinthebrain,activationoughttoexhibitthesamepatterns.
Fig.3AdisplaysacrosssectionofthehumanbrainhighlightingareasthatwereidentifiedbecauseactivationinafMRI(functionalmagneticresonanceimaging)scannerincreasedlinearlyinrewardprobability.Rewardprobabilitywascomputedfromthebetthatthesubjectchoseandthenumberonthefirstcard.Activationismeasuredinthefirstsecondafterthedisplayofcard1(modulothehemodynamicresponsedelay).Fig.3Bshowsmeanactivationsacross19
subjectsand95%confidenceintervals,stratifiedbylevelofrewardprobability.Thisverifiesthattheactivationpatternsareindeedincreasingandlinear,whichisrequirediftheactivationsaretobeassociatedwithexpectedreward.
Withtheexceptionofasmallactivationintheupperleftpartofthebrain,theregionshighlightedinFig.3Aaresubcorticalprojectionareasofthedopamineneuronsinthebrainstem,inparticular,theventralstriatum.Wealreadydiscussedtheimportanceofthedopamineneuronsintrackingrewards.Traditionally,however,theywerethoughttocausearousal.Thelinkbetweenthisemotionalreactionandpredictionerrorsisintriguing,andweshallelaboratelater.Fig.3Csituatestherewardtrackingdopamineneuronsintheventraltegmentalareaofthebrainstem,andtheyprojecttotheventralstriatum.Therearealsodopamineneuronsinthesubstantianigra,buttheseareprimarily
8/2/2019 Trinity09 Bossaerts Paper
7/28
engagedinothertasks(anddamagetotheseneuronsleadstoParkinsonsdisease).
TheresultsofFig.3havebeenreplicatedmanytimes.Infact,wenowknowthatthisactivationreflectsrewardpredictionerrors,i.e.,thedifferencebetweenwhat
wasexpectedbeforethedisplayofthestimulus(card1)andtheupdatedexpectationoroutcome(McClureetal2003,O'Dohertyetal2003).Assuch,theactivationisbackwardlooking,and,consistentwiththis,itisphasic(meaning:ofshortduration)andtimelockedtothestimulusdisplay.Thattheactivationsrepresentrewardpredictionerrorshasrecentlybeenconfirmedinananalysisthatrequiredminimalassumptionsaboutthepriorexpectation(Caplinetal2008,Nivetal2008).
Fig.4Ashowsacrosssectionofthehumanbrainwithregionswhereactivationchangesquadraticallywithrewardprobability.Fig.4Bdisplaysthepatternofactivationinoneofthoseregions,namely,rightinsula.Asrequiredifthis
activationistoreflectrewardvariance,meanactivationislowestforzeroandunitrewardprobability,highestatabout0.5,andsymmetricaround0.5.TheregionsindicatedinFig.4A,leftandrightinsula,andthethalamus,areinvolvedinrelayinginformationaboutbodilystateemotionsandsensoryinput(e.g.,visualcues)tocorticalareas.Theresultshavebeenreplicatedmanytimes[e.g.,(Dreheretal2005)].Activationinatleasttwoadditionalcorticalregionscorrelateswithrisk(Huetteletal2005).Oneofthem,inferiorfrontalgyrus,willbebrieflymentionedlater:itsshallowlocationallowsfordirectelectricalormagneticmanipulation,withinterestingbehavioraleffects.
ThetimingandmodulationofthesignalsdisplayedinFig.4aresignificantly
differentfromthoseinFig3.Specifically,thesignalsemergelater,aresustained,andseemtobetimelockedtothedisplayofcard2.Thisisconsistentwiththeinterpretationthattheyreflectexpectationofriskthatistoberesolveduponrevelationofcard2.Assuch,itisforwardlooking,andthereforenotapredictionerror,butanticipation.Asweshallsee,thebraindoesalsoencodeacorrespondingpredictionerror,whichinthiscaseshouldbereferredtoastheriskpredictionerror.
Inourcardtask,probabilitiesareknown,inthatsubjectsshouldbefamiliarwithbasicfeaturesofcardgames.Thecontrastofactivationwhenprobabilitiesareknown(whicheconomistswouldrefertoaspurerisk)againstwhenthese
arenotknown(ambiguity)showsthatthedopamineprojectionareassuchasventralstriatumarelessactivatedunderambiguity(Hsuetal2005).Interestingly,anotheractivationemerges,namely,inamygdala(locatedapproximatelywherethenumber3isdisplayedinFig.3C,butmorelateral).
Amygdalaformspartofacircuit(whichalsoincludesthedopaminesystem)thathasbeenassociatedwithgoaldirectedlearning(Balleineetal2003).Amygdalaappearstosignalaneedtostartlearning.Econometricianswouldrefertothissignalasparameterestimationuncertainty(oritsinverse,precision),tobedistinguishedfromtheirreducibleriskthatremainsinanystochasticenvironmentevenwhenprobabilitiesareperfectlyknown.Itisinterestingtoobservethatthebrainlikewisedifferentiatesbetweenthesetwoaspectsofuncertainty.Itconfirmsbehavioralevidencethathumanstreatpureriskand
8/2/2019 Trinity09 Bossaerts Paper
8/28
ambiguitydifferently,butthelinkwithfundamentalconceptsfromeconometricssuggeststhatthedifferentiationisnotnecessarilytobeinterpretednegatively,incontrastwithtradition(Ellsberg1961).Indeed,arecentexperimentdemonstratesthathumanmanagetoexploitthedifferentiationtooptimizelearningunderambiguity(Payzan&Bossaerts2008).
Amygdalahasbeenassociatedwithlearningonlyrecently.Amygdalahadalwaysbeenviewedasthefearcenterofthebrain(Adolphsetal1995,LeDouxetal1990,Morrisetal1996).Thetwoviewsarenotmutuallyexclusive.Fearmaybetheemotionalexpressionofestimationuncertainty,justlikearousalaccompaniespositiverewardpredictionerrors(andthelattertwoarebothrelatedtoactivationofthedopaminesystem).
Asmentionedbefore,thedopaminesystemencodesarewardpredictionerror,andtherefore,playsacrucialroleinlearningexpectedrewards.Signalscorrelatingwithriskpredictionerrorinourcardgamehavealsobeenlocated,in
insula(Preuschoffetal2008),overlappingwiththeareaswhereriskpredictionhadbeenfoundinpriorstudies.SeeFig.5.Mathematically,theriskpredictionerrorissimplythedifferencebetweenthesquaredrewardpredictionerroranditsanticipation,rewardvariance.EconometricianswouldrecognizethisasthedrivingtermofanARCHprocess(Engle2002).Riskpredictionerrorscanalsobeobservedintheothercorticalstructureswhererisksignalshavebeenfound(d'Acremontetal2008).
Oneofthestrikingaspectsaboutbrainactivationinourcardgameisthatsignalsofexpectedrewardandrisk(andcorrespondingerrors)appeareverytimethegameisplayed.Evidently,thebraininsistsonreevaluationofbasicstatistical
featuresratherthansimplyretrievingsomevaluation(utility)numberfrommemory.Ifthisisageneralproperty,thebrainisbestviewedasacomputerwithincredibleprocessingcapacitybutasmallharddisk.Thisshouldinspirenewmodelingofboundedrationality.
Butconstantreevaluationofthestatisticsofagambletakestime.Onethereforewonderswhetherthishasaneffectonchoicewhensubjectsareforcedtomakedecisionsbeforeallneuralcomputationscanbecompleted(whichwouldbewithin23secondsafterstimulusdisplay).Revealedpreferencesareindeedsignificantlydifferentwhensubjectshavetoindicatetheirchoiceinthe2ndsecond,comparedtochoiceinthe4thor6thsecond(Bollardetal2008).This
obviouslyhasimportantimplicationsfordesignofcontinuoustradingmechanisms,whereprofitopportunitiescansometimesonlybeachievedifactionistakeninasplitsecond.
Sofar,wehavediscussedencodingofstatisticalfeaturesofagambleinthehumanbrain.Infinancialdecisiontheory,thesearemerelytheinputstovaluation,i.e.,toexpectedutility.Where,then,arethevaluesignalsinthebrain?
PerhapstheactivationdisplayedinFig.3actuallydoesnotreflectexpectedrewardprediction(errors),butexpectedutility(errors).Wedontseeanynonlinearity,however(concavityforriskaverseandconvexityforriskseeking
subjects).Thenagain,oneshouldnotoverinterprettheevidence,asitisnotentirelyclearwhetherthefMRIsignalislinearinfiringofindividualneurons.As
8/2/2019 Trinity09 Bossaerts Paper
9/28
amatteroffact,evenpatternsinneuronalfiringshouldbeinterpretedwithcare.Indeed,sincesignalprecisiondecreaseswithfiringrate,onecouldexpecttoobserveaconvexrelationshipbetweenrewardprobabilityandfiring,eveniftheneuronathandmerelyencodesexpectedreward.
Also,inourcardgame,onlyprobabilitieswerevaried.Significantly,wekept(reward)magnitudethesame:betswerealwaysfor$1.Onerecentstudydidmanipulatemagnitudes.Thisrevealedstrictconcavityintherewardpredictionerrorsignalsinventralstriatum.Moreover,thedegreeofconcavityincreasedwithriskaversion(Tometal2007).
Butifthismeansthatthesignalsinventralstriatumreflectexpectedutility,andnotexpectedreward,whataretherisksignalsininsulaorinferiorfrontalgyrusabout?Dothesereflectvarianceofutility?Atthismoment,wedontknow.Preliminaryevidence(Christopoulosetal2008)suggeststhattheriskinducedactivationsininsulaandinferiorfrontalgyruscorrelatewithriskaversion,and
hence,doreflectsubjectivevariance.Whiletheideaofseparatelyencodingutilityvarianceandexpectedutilitymayseemstrangeatfirst,itshouldberecalledthatitwasAllaiswhofirstsuggesteditasanexplanationfortheviolationsofexpectedutilityaxiomsthatheobservedinhumanchoice(Allais1953).
Tocomplicatematters,inventralstriatum,signalsencodingprobabilityofrewardandmagnitudeofrewardappeartolivealongsidesignalsthatcorrelatewithexpectedreward(Tobleretal2007,Yacubianetal2006).Probabilityandmagnitudeareofcoursethecrucialingredientsofexpectedutility.Theprobabilitysignalinventralstriatumevenshowsbiasesthatarereminiscentof
probabilityweightinginprospecttheory(Bernsetal2008,Hsuetal2008b).
Whyhasthebrainchosentoencodeallfourfeaturesofagamble:expectedreward,rewardvariance,rewardprobabilityandrewardmagnitude?Afterall,(expected)utilitycanbecomputedeitherbysummingmagnitudesaftermultiplicationwithprobabilities,orbycombiningexpectedrewardandrisk.Perhapsthesetwoapproachesarenotequivalent,incontrastwithreceivedwisdom(Krolletal1984)?Potentialfundamentaldifferenceshaverecentlybeenexploredboththeoreticallyandempirically(d'Acremont&Bossaerts2008).
Separateencodingofthekeydecisiontheoreticfeaturesofaproblemisnot
limitedtogambles.Thehumanbrainlikewiseseparatelytracksequityandefficiencyinproblemsofmoraljudgment(Hsuetal2008a).Interestingly,equitycorrelateswithactivationinregionsengagedinriskencoding,suchasinsula,whileefficiencyactivatesventralstriatum,involvedinexpectedrewardencoding.Thisfindingisunlikelytobecoincidental.Itcertainlylendssupporttomodelsofmoraldecisionmakingbasedonatradeoffofequityagainstefficiency,justliketheevidenceinFigs.3and4supportsmeanvarianceanalysisasamodelofhumanfinancialvaluation.
4.Emotions
Theregionsofthehumanbrainwhereweseeclearriskandrewardsignals
includethedopaminesystem,insula,andamygdala.Theseregionshaveincommononeimportantfeatureofthehumancondition,namely,emotions.We
8/2/2019 Trinity09 Bossaerts Paper
10/28
alreadymentionedarousalinthecontextofdopamine(sometimesinflatedartificially,throughcocaineforinstance).Amygdalahasbeenassociatedwithfear,andinsula(oratleasttheanteriorpart,whereweobserverisksignals),isknowntobeacrucialrelayofemotionstothecortex.
Atthesametime,theevidencesurveyedintheprevioussectionsuggeststhatactivationinthesesamebrainregionsreflectsencodingofcoolmathematicalquantitiessuchasexpectedreward,risk(measuredasvariance),rewardpredictionerrors,ARCHlikeriskpredictionerrors,estimationuncertainty,etc.Hasthebraindecidedtorelatethesemathematicalquantities,usuallyassociatedwithcoolheadedness,toemotions?
Inthesamevein,recentstudieshaveshownthatstrongemotionssuchasdisappointment,elation,regret,relief,envy,gloatingareassociatedwithmathematicalerrorsignalssuchas(inthesameorder):(negativeandpositive)rewardpredictionerrors,fictiveplaypredictionerrors,andpredictionerrors
relativetootherpeoplesrewards(Baultetal2008,Coricellietal2005).Allthesecanbefoundinthebrainstructuresthatwealreadydiscussed.
Inthecontextofinsula,weshouldmentionastudywheretheabilitytosenseonesownheartbeatwasfoundtocorrelatewiththesizeoftheinsula(Critchleyetal2004).Togetherwiththefindingsthat(i)theheartbeatofprofessionaltraderscorrelateswithrisk(Lo&Repin2002)andthat(ii)insulaencodesrisk(Fig.4),onemayconjecturethatthehumanbodyhaschosentoemployasomaticmarker(heartbeat)toencoderisk.
Whenthatsomaticmarkerisabsent,theconsequencesmaybeharmful.
(Becharaetal1997)showthatsubjectswhofailtoexpressemotionalanticipationwhentakingriskydecisions(becauseofspecificbrainlesions)consistentlygoforinferiorfinancialchoices.Thisledtheauthorstopositthatemotionalprocessesguidereasoneddecisionmaking,whichtheyreferredtoastheSomaticMarkerHypothesis(Bechara&Damasio2005).
Becauseofthemathematicalaccuracyofthesignalsencodedintheemotionalregionsofthehumanbrain,anupdatedinterpretationofthefindingswouldbethatemotionsaresomehowsubsumedinthemathematicsofoptimaldecisionmaking.Thisofcoursewouldexplainwhy(emotional)experienceisimportantforrationalchoice(Feng&Seasholes2005).
Anotherwaytoputthisisthatreasonedchoiceistheresultofthebalancingofemotionalinputs.Humandecisionmakingunderuncertaintythustakesonfeaturesofthecelebratedmultiattributechoicetheory(Lancaster1966),albeitthatcharacteristicsneedtobereplacedbytheemotionalaspectsofagamble.Emotionsarenotbadperse,andirrationalregretavoidance,ambiguityaversion,etc.aretobeviewedastheresultofmaladaptedweighingofthedifferentemotionalaspectsofagamble.
Itisimportanttorealizethatemotionbaseddecisionmakingmaybeentirelysubconscious.Asamatteroffact,itsspeedmayprecludeconsciousbrain
processesfrominterfering.Attimes,suchdecisionmakingmaybeinconflictwithchoicebasedonexplicitreasoning,andthefactthathumansoftenshow
8/2/2019 Trinity09 Bossaerts Paper
11/28
signsofbothfast,emotionbaseddecisionmakingaswellasslow,logicallyreasonedchoicehasledmanytoproposedual-systemtheory(Evans2003,Kahneman&Frederick2002).Accordingtothistheory,humansarecapableofopportunisticallyswitchingbackandforthbetweenthesetwowaysofgeneratingdecisions.Prefrontalandlateralorbitofrontalcorticalregions,aswell
astheadjacentanteriorcingulatecortexarethoughttobeinvolvedinexplicit,consciousdecisionmaking,butthepreciseneurobiologyremainspoorlyunderstood.
Arecentstudyonframing(DeMartinoetal2006)doesillustratehowthismaywork.Whenprospectswereframedintermsoflosses,subjectsappearedtotakeonmorerisk,andthisbehaviorwasassociatedwithincreasedactivationofamygdala(thefearcenterofthehumanbrain,asmentionedbefore).Nevertheless,subjectswhoshowedleast(orno)impactfromframingalsosimultaneouslydisplayedincreasedprefrontalactivation,suggestingattemptsoftheconscioussystemtocontrolquickemotionalreactions(Bossaertsetal
2008b,Kahneman&Frederick2007).
Howdoweknowthatthesecondaryactivationisreallyconscious?Wedont(yet).However,becauseitallowsforbettertimeresolution,EEGmayprovideinsight.Inarecentstudyontheimpactofdisappointmentonsubsequentdecisionmaking,asecondaryEEGsignalemergedinfrontalregionsafter375millisecondsbutonlyinindividualswhosuccessfullyovercamethedetrimentaleffectofreflexivereactiontodisappointment(Tzieropoulosetal2008).Whilethearrivalofthissecondarysignalissurprisinglyquick,itisnotoofastforconsciousexperience.
Itshouldberepeatedthatemotionbaseddecisionmakingneednotbebadorunsophisticated.ThemathematicalprecisionwithwhichactivationininsulacorrelateswithanARCHlikeriskpredictionerror(Fig.5)underscoresthis.Asdotherewardpredictionerrorsthatdopamineneuronsencode.Indeed,thelatterrevealalearningalgorithmthatisexquisitelyadaptedtoenvironmentswithmultiplestimuli,actionpossibilities,andrewardopportunities(Montagueetal1996,Schultzetal1997).Thisverysamealgorithm,calledtemporaldifference(TD)learning,isusedinmachinelearningtosolvecomplexdynamicprogrammingproblems.
5.ValueSignals
Standardeconomicthinking(includingbehavioralfinance)startsfromthepremisethatchoiceresultsfrommaximizationofvalue.Partlyundertheinfluenceofeconomists,oneofthemainissuesthatdecisionneuroscientistshavebeeninterestediniswhethertherearevaluesignalsinthebrain.Thefirsthandbookinneuroeconomicsismostlydevotedtothisissue(Glimcheretal2009).
Atonelevel,theexistenceofavaluesignalislikelytobeconfirmed.Indeed,ifactsfollowtheinstructionsofmotororpremotorneuronsthatfiremostintensely,thisfiringmustbeinaccordancewithmaximizationofsomeutilityfunctioniftheresultingactssatisfytheaxiomsofrevealedpreference.Consistentwiththisconjecture,(Platt&Glimcher1999)foundthatfiringofneuronsinthe
8/2/2019 Trinity09 Bossaerts Paper
12/28
intraparietalsulcus(afoldinthelateralcortexbeyondthebrainmidline)oftheMacaquebraincorrelatedwithexpectedutility.
Butwhatabouttheexecutiveregionsinthefrontofthebrain,farremovedfromthemotorcortex?Bynow,theevidenceinfavorofsignalsthateconomistswould
recognizeasvalueiswellestablished.ValuationsignalshaveindeedbeendiscoveredintheorbitofrontalcortexoftheMacaquemonkey(PadoaSchioppa&Assad2006,Tremblay&Schultz1999).Thishasrecentlybeenconfirmedforthehumanbrain(Plassmannetal2007),andextended(Hareetal2008b):activationinmedialprefrontalcortexcorrelateswithwillingnesstopay,andmorelateralactivation(andcorrespondingtotheactivationidentifiedintheMacaquebrain)correlateswitheconomicsurplus(i.e.,utilitynetofcost),whileactivationinventralstriatumrepresentsvaluepredictionerrors(asalreadydiscussed).
Inthecontextofgambles,thesignalsinlateralorbitofrontalcortexexhibitthe
characteristicsonewouldexpectiftheyaretoconveyexpectedutility:theyincreasewithexpectedreward,yetdecreasewithrisk(variance),forriskaversesubjects,whiletheyincreasewithriskforriskseekingsubjects(Fig.6).
Thepresenceofvaluesignalsishandyinmanyrespects.Foronething,evolutionofthissignalovertimeteachesusmanythingsaboutlearning.E.g.,(Hamptonetal2006)havebeenabletoshowthatvaluesignalsinarandomlyrevertingtwoarmedbandittaskreveallearningthatisdistinctlyBayesian.ItistruethatthebehavioraldataalreadymarginallyfavoredBayesianupdating(oversimplereinforcementlearning),butthevaluesignalsshowtherightpropertiesattherighttimesothateventhemostskepticalmindwouldbeconvincedofthe
hypothesisofBayesianlearning.
Valuesignalshavealsoshedlightontheendowmenteffect.Specifically,thiscognitivebiasisnotdirectlyreflectedintheabovevaluesignals,butmaybetheresultofreferencebasedencodinginventralstriatumandinsula(DeMartinoetal2009).Inthecontextofgambles,activationininsulasuggeststhattheendowmenteffectistheconsequenceofthelevelofriskthatthesubjectperceives.Specifically,thewillingnesstopay(WTP)islowerbecausesheperceiveslessriskinherstartingposition(cashendowment).Thewillingnesstoask(WTA)ishigher,becausesheisnowendowedwiththegambleandisfacedwithexchangingthisforanotherriskyposition.Thelattermayseemstrange,
becausethegambleistobesoldforcash(i.e.,ariskfreeposition).ButthisignoresthefactthatWTPisgenerallyelicitedusingtheBeckerDeGrootMarshak(BDM)mechanism,whichinducesrisk.Thefindingsuggestsnovelhypotheses,suchasthedependenceofthedifferencebetweenWTPandWTAonriskaversion.ItalsodemonstratesthattheendowmenteffectinthiscaseresultsfromafundamentalmisconceptionoftheBDMmechanism(oraviolationoftheindependenceaxiom).Hence,itshouldbeavoidablethroughappropriatetraining.Whichispreciselywhathasbeenshownrecently(Plott&Zeiler2005).
8/2/2019 Trinity09 Bossaerts Paper
13/28
6.UncertaintyCreatedByAnIntentionalSource
Sofar,wehaveconsidereduncertaintythatisobjective,inthesensethatitisgeneratedbyasourcewithoutgoaldirectnessorintentionbehindit.Opposedtothisaresituationswhereastrategicagentcausestheuncertainty(ananimal;ahuman;asophisticatedcomputer;orperhapsamarketwithinsiders?).Game
theoryhaslonghadtocopewiththistypeofuncertainty,buthassettledonanalysesthatdonotdistinguishitfromobjectiveuncertainty.Valuationisperformedusingstandardexpectedutility,andupdatingfollowsBayesianprinciples.TheresultingequilibriumconceptsarestraightforwardBayesianextensionsofNashequilibrium.
Butneurosciencehassuggestedthattherearefundamentaldifferencesinthewayhumansapproachuncertaintyfromanintentionalchoice.Forinstance,activationinasimplegameofimperfectinformationismorelikethatinsituationsofambiguity,notpurerisk(Hsuetal2005).Likewise,psychologyhaslongdifferentiateddealingwithanintentionalopponent.Assessingtheriskin
suchasituationhasbecomeknownasTheoryofMindormentalizingandspecificregionsofthebrainseemtobededicatedtoit,mostimportantlyparacingulatecortex(theregionofthefrontalcortexabovetheorbitofrontalcortex),cytoarchitecturallyarelativelyrecentbrainregion(Gallagher&Frith2003).
Paracingulatecortexisindeedpreferentiallyactivatedduringplayofstrategicgameswithotherhumans,moresothanwithcomputersprogrammedwithasimplerule(Gallagheretal2002,McCabeetal2001).(Hamptonetal2008)haveexploredthemathematicalcomputationsinthisregionduringgameplayanddiscoveredthatitencodedaninfluenceupdate,namely,theerrorfrompredictinghowonesopponentwouldshiftstrategyasaresultofonesownmoves.
Assuch,thissignaleffectivelypusheshumangameplaybeyondsimplereinforcementlearning,fictitiousplayorsomemixture[experienceweightedattractionlearning(Camerer&Ho1999)].Mostsignificantly,itdemonstratesthathumansrecognizetheintentionalityofthetarget(opponent)theyarelearningabout,apossibilitythathasbeenraisedonlyoccasionallyinthegametheorylearningliterature(Camereretal2002,Stahl2000).
Exploitingthesimpleformalstructureofthebeautycontest,thisfindinghas
recentlybeenconfirmed(Coricelli&Nagel2008).Significantly,inneitherstudy,regionstraditionallyassociatedwithformalmathematicalandlogicalthinkingwereengaged,andindeed,(Coricelli&Nagel2008)showhowskillinthebeautycontestandmathematicalproficiencywereunrelated,despitethefactthatthemathematicstestsrequiredtheverycalculationsthatareimplicitinsuccessfulgameplay.Thisdiscoverystillobtainswhenthegameisfarmorecomplex.Infinancialmarketswithinsiders,skillinpredictingfuturepricemovementsdoesnotcorrelatewithmathematicalproficiency(Bruguieretal2008).
8/2/2019 Trinity09 Bossaerts Paper
14/28
7.Discussion
Whatshouldafinancescholarmakeofallthisneurobiology?Inthefirstplace,thatitprovidesplausibilityanddisciplinetomodelingofhumandecisionmaking.Forinstance,itsuggestswhereoneshouldplaceemotions.Andconsciouscontrol.Itteachesusthatvaluesarenotmerelyretrievedfrom
memory,butrecomputedeverytime,withtheriskofoccasionalmisestimation.Thatis,choiceistheoutcomeofaprocess,fromperceptiontoaction,notmaximizationofutility.Likewise,itsuggestswhyhumanscanbegoodat,say,playingstrategicgamesevenif,whenaskedexplicitly,theydonotseemtobegoodatthemathematicalcomputationsimplicitintheirplay.
Neurobiologydoesaskustoevaluateeverythingintermsofasinglecriterion,namely,evolutionaryplausibility.Allegedcognitivebiasesremainsuspectuntiltheymakesenseinanecologicallyrelevantsetting.Suchacriterionshouldbefamiliartofinancescholars,whohavebeenapplyingtheirown,toevaluatepricingpatternsinfinancialmarkets,namely,absenceofarbitrageopportunities,
or,althoughonlyforneoclassicists,theefficientmarketshypothesis.
Novelconjectureswillbethebyproductofanyinsistencethattheoriesofhumandecisionmakingbevalidatedneurobiologically.Forinstance,itiswellknownthathumansenvyandgloat,andthattheseemotionalreactionsappeartoaffectutility,andhence,futurechoice(Baultetal2008).Whywouldenvyandgloataffectutility?Itcanbeconjecturedthatthisleadstherewardpredictionerror(differencebetweenexperiencedutilityandexpectedutility)toreact,notonlytowhatanindividualdidherself(asintraditionalreinforcementlearning)orwhatshefailedtodo[fictivelearninglikein(Lohrenzetal2007)],butalsotowhatotherschosetodo.Assuch,therewardpredictionerrorautomaticallyincludesoptionschosenbyothers.Learningofoptimalstrategiesistherebyenhanceddramatically.Thisshouldmatterespeciallyinsituationswheretheconsequencesofunchosenactionsarehardtoassess,sothatfictivelearningisineffective.Thealternativeforthebrainwouldhavebeentosetupacostlyparallellearningsystemthatsimulatestheutilityofothers,withoutreferencetoonesownutility.Oritcouldhavechosenamirrorsystem(Rizzolatti&Craighero2004)thatstoresactionsofotherswithoutrecordingtheirvalue,buttheblindnessofsuchasystemrendersitscognitiveusequestionable.
Choicebiasescanbeexpectedtoarisewhenthebrainfacesanewadversaryor
anunknownenvironment.Financialmarketsmaybeaprimeexampleofthebrainchartingnewterritory.Nevertheless,eveninthiscontext,humansoftenappeartodoremarkablywell.Forinstance,thesuccessofthecontinuousdoubleauctioningeneratingefficientoutcomesdespiteitsstrategiccomplexity(Friedman&Rust1993)andthefacilitywithwhichhumanscandetectandexploitinsiderinformationinit(Bruguieretal2008)suggestthatcertaindesignsexploitthecomputationalcapabilitiesofthehumanbrainbetterthanothers.
Butthedevelopmentofthedoubleauctionwasaccidental,inthesensethatitdidnotemergefromathoroughstudyofthehumanbrainanditscapacity.Itis
hoped,however,thatfuturediscoveriesindecisionneurosciencewillfacilitatetargeteddevelopmentofnewtoolsforimprovedfinancialdecisionmakingand
8/2/2019 Trinity09 Bossaerts Paper
15/28
exchange.Forinstance,furtherstudyofhowlongittakesthebraintoreevaluategambleseachtimetheyarepresentedshouldinspiretimingrestrictionsontradingplatformswiththeaimofavoidingchoicebiasesinducedbytimepressure.
Amongsuchtools,onecanofcoursealsoenvisagesimpleaffectcontrol(Kuhnen&Knutson2008).Pharmacologicalremedyisalsowithinreach[e.g.,glutocorticoidreceptoragonistsblockthesynapticeffectsofstressondopamineneuronsintheventraltegmentalarea(Saaletal2003)].Andinvestorsshouldbewarnedforthesideeffectsofmedication[suchasLDopa(Gschwandtneretal2001)]thatinterfereswiththebrainsystemsinvolvedindecisionmakingunderuncertainty.Directcontrolofriskattitudesisnowpossible,throughelectrical(Fecteauetal2007)ormagnetic(Knochetal2006)stimulationoftheinferiorfrontalgyrus,where,aswediscussedbefore,arisksignalresides.Butsuchcontrolraisesethicalissues.
Onealsowonderswhetheritispossibletoteachhumanstobecomebetterfinancialdecisionmakers,withoutresortingtotoolssuchasdoubleauctions,timingrestrictions,pharmacologicalremedies,etc.?Totheextentthatdecisionmakingunderuncertaintyengagespowerfulemotionalprocessesandpervasiveneurotransmitterrelease,thiswillbedifficult.Still,wecanteachtricks,suchaslookingforGARCHfeaturestodetectthepresenceofinsiders(Bruguieretal2008),justlikeautisticpatientscannowbeinstructedtolookintheeyesofotherstoovercometheirhandicapindiscerningtheintentionofothers(Spezioetal2007).Suchtricksareuseful,buttheirscopeislimited,becausewhatweareaskingforistheequivalentofcontrollingavisualbias.Avisualbiasisharmlessifoneknowswhenitoccurs,butitreallydoesnotgoaway,andthebrainwillbefooledanytimeitisinattentive.
Theforgoingalsoanswersacommonlyraisedquestion:shouldcomputersreplacehumansinfinancialmarkets?Theanswerisno.Humansaremuchbetteratcertaintasksthancomputers,buthumanshaveclearlimitationswhenaskedtosolveproblemsforwhichtheirbrainsaremaladapted.Thegoalofdecisionneuroscienceistoidentifythelimitationsandtointroducecomputersonlywheninneedtoovercomethem.Justasthehorizonwasinventedasatooltohelphumanpilots(whoincidentallyhavestillnotbeenreplacedwithcomputers)Thehorizonhelpsbecauseorientationfailswhenflyingthroughclouds.This
failureshouldnotbesurprising,becausethehumanbrainanditsevolutionaryantecedentshaveneverhadtofly.Nordidtheyhavetobeconcernedwithfinanceuntilrecently.
8/2/2019 Trinity09 Bossaerts Paper
16/28
FigureCaptions
Figure1MedianfiringofdopamineneuronsinthebrainsoftwoMacaquemonkeysafterreceivingnoorlittlerewardvs.varioushigherrewards(juice).Rewardmagnitudeschangedfromonetrialtoanother,butwereperfectly
anticipated,andthemonkeyhadlearnedthatthehigherrewardwouldbedeliveredwith50%chanceinallcases.Firingshowsadaptationtosituation:reactiontodeliveryof0.05mlisnodifferentfromthatwhententimestheamountisdelivered(0.50ml;AnimalAandB).Likewise,deliveryofalowrewardof0.15mlcausesthesamefiringaswhenalowrewardof0mlisanticipated(AnimalB).Source:(Tobleretal2005b).
Figure2A.Timelineofcardgame.Ofinterestarebrainactivationsresultingfromdisplayofcard1(andcard2),whenbeliefs(expectedreward,rewardvariance)areupdatedinfunctionofthenumberonthecardandthebet.B.Expectedrewardandrewardvarianceasafunctionofprobabilityofreward.
Brainsignalsthatencodethesestatisticalfeaturesshouldshowsimilarrelationships.Source:(Preuschoffetal2006).
Figure3A.Locationof(someofthe)activationsthatincreaselinearlyinrewardprobability:LeftandRightventralstriatum(vst).B.MeanactivationinLeftventralstriatum(vst)across19subjects(diamonds)stratifiedbyrewardprobability;verticallinesindicate95%confidenceintervals.C.Crosssectionofthebrainfromfront(Right)toback,showingkeyprojectionareasofdopamineneuronsinthemidbrain.Dopamineneuronsencodingrewardpredictionerrorscanbefoundintheventraltegmentalarea.Theyproject,amongothers,totheventralstriatum(regionactivatedinA).Source:(Preuschoffetal2006)and
openlearn.ac.uk
Figure4A.Locationof(someofthe)activationsthatarequadraticinrewardprobability:LeftandRightanteriorinsula(ins)andmediodorsalnucleusofthethalamus(md)(frontofthebrainisontop).B.MeanactivationinRightanteriorinsula(ins)across19subjects(diamonds)asafunctionofrewardprobabilityshowstheinvertedUshapedpatternneededforittoencoderewardvariance;linesegmentsindicate95%confidenceintervals.Source:(Preuschoffetal2006).
Figure5A.Locationofactivationthatcorrelateswithriskpredictionerrorupondisplayofcard1:LeftandRightanteriorinsula.B.MeanactivationinRight
anteriorinsulaacross19subjects(diamonds)asafunctionofriskpredictionerrorupondisplayofcard1(red)andcard2(blue).Outlieratzeroriskpredictionerrorreferstotrialswhennoriskremainedaftercard1.Linesegmentsindicate95%confidenceintervals.Source:(Preuschoffetal2008).
Figure6A.Locationofactivationcorrelatingwithexpectedutilityofgambles:prefrontalcortex,slightlytotherightofthemedialline(eyeballisvisibleinbottomrightcorner).B.Activationinthislocationasafunctionofexpectedrewardandrewardvariance,persubject,stratifiedbyrevealedriskattitude.Source:(Tobleretal2007).
8/2/2019 Trinity09 Bossaerts Paper
17/28
SummaryPoints
1. Choiceistheoutcomeofalongneurophysiologicalprocess,fromperceptiontoaction.Inthecontextofgambles,thisprocessincludesconstantreevaluationofstatisticalfeatures.
2. Fromthestatisticsofgambles,thebrainconstructstheveryvaluations(utilities)thatareatthecoreofmoderneconomicthinking.Thesevaluationsignalscanbefoundinexecutiveareasofthehumanbrainthatareremotefromthemotorcortex,andhence,thatarenotdirectlyinvolvedintheactsthatimplementachoice,butintheevaluationoftheoptionsathand.
3. Brainregionsthatencodethemathematicalfeaturesofadecisionproblemareoftenalsoinvolvedinthecorrespondingemotionalreactions,suggestingthatemotionsmaylargelybesubsumedinthemathematicsofdecisionmodels.
4. Emotionsareanintegralpartofreasoneddecisionmaking.Choicecanbeviewed,toalargeextent,astheresultofadelicatebalancingoftheemotionalfeaturesofadecisionproblem,suchasdisappointment,elation,regret,relief,envyandgloating.
5. Theclaimthatlaboratoryexperimentsinfinancedonothaveexternalvaliditycanonlybeupheldifitisdemonstratedthatthebraincircuitryengagedinthelaboratorysomehowbecomesneutralized.Thisishighlyunlikely,becausethatcircuitryisincontrolofpervasiveneurotransmittersandofstrongemotionalresponses.
6. Cognitivebiasesneedtobeevaluatedagainstevolutionaryfitness,justlikefinancialmarketphenomenaarejudgedagainstabsenceofrbitrage(ortheefficientmarketshypothesis).Cognitivebiasesneedtobeunderstoodasamismatchbetweenanovelenvironmentandaneurobiologicalsystemthatisoptimizedforadifferent,ecologicallymorerelevant,problem.
7. Whilethetheoryofgamesofimperfectinformationborrowslargelyfromfinancialdecisiontheory,thehumanbraindealsdifferentlywithuncertaintygeneratedbyanintentionalsourcethanwithobjective
uncertainty.
8. Animprovedunderstandingoftheneurobiologicalfoundationsofhumandecisionmakingunderuncertaintyshouldleadtothedevelopmentofeffectivetoolsofassistance,suchasappropriateemotionalcontrol,tradinginterfacesthatexploitnaturalskills,andevenpharmacologicaldecisionsupport.
8/2/2019 Trinity09 Bossaerts Paper
18/28
Sidebar
TrackingNeuronalFiring
Firingofindividualneuronscanbetrackedbyinsertinganelectrodeinthebrain,aratherinvasiveproceduredoneinhumansonlyifthereisamedicalneedand
eventhenthelocationoftheelectrodeisdeterminedbymedicalconcernandnotscientificvalue.Recently,anumberofimagingtechniqueshavebecomeavailablethatallowonetononinvasivelybutindirectlymeasurefiringofclustersofneuronsor,moreaccurately,activationofprojectionareastriggeredbyfiringofupstreamneurons(Goense&Logothetis2008).Functionalmagneticresonanceimaging(fMRI)isoneofthem.Basically,fMRIlocalizesthepresenceofoxygenrichblood.Whileoxygenrichbloodwillconcentratewherethereisactivation,theeffectcomeswithsomedelay(about4s),referredtoashemodynamicresponsedelay.ThefunctionalinfMRIstandsforthefactthatalargepartofthebrainisscannedinitsentiretyaboutevery2seconds,sothatchangesinactivationcanbepickedupatdifferentstagesofatask.PicturesofbrainactivationlikeFig.3Aareeffectivelymapsoftstatisticsofthetreatmenteffects,wherebyonlyactivationsbeyondacutoffplevelareretained(generally0.001orless).
8/2/2019 Trinity09 Bossaerts Paper
19/28
MiniGlossary
NeuronalFiring
Electricalimpulsesthataresentfromtheneuroncellbodyoutwardthroughitsaxon(anextensionofvaryinglength).
Neurotransmitter
Chemicalreleasedfromtheaxonfollowingfiringthatactivatesadendriteoftheprojectionneuronacrossthesynapticcleft.
Dopamine
Onetypeofneurotransmitter(othersincludeseretonin,acetylcholine,norepinephrine,glutamate).
PrimaryReinforcers
Physicalrewardssuchasjuice
SecondaryReinforcers
Indirectrewardssuchasmoneyorsocialreputation.Engagevaluationcircuitryasiftheywereprimary(Izumaetal2008).
EEG(Electroencephalogram)
Recordingonthescalpofelectricalactivitygeneratedbythebrain,withexcellenttimeresolution,butdifficultsourcelocalization.
8/2/2019 Trinity09 Bossaerts Paper
20/28
Keywords
Neuroeconomics,ExperimentalFinance,Risk,Emotions
Acknowledgment
TheauthorthankstheSwissFinanceInstituteandtheU.S.NationalScienceFoundation(GrantSES0527491)forfinancialsupport.
8/2/2019 Trinity09 Bossaerts Paper
21/28
References
AdolphsR,TranelD,DamasioH,DamasioAR.1995.Fearandthehumanamygdala.J.Neurosci.15:587991
AllaisM.1953.LeComportementdel'HommeRationneldevantleRisque:
CritiquedesPostulatsetAxiomesdel'EcoleAmericaine.Econometrica21:50346
BalleineBW,KillcrossAS,DickinsonA.2003.TheEffectofLesionsoftheBasolateralAmygdalaonInstrumentalConditioning.J.Neurosci.23:66675
BaultNg,CoricelliG,RustichiniA.2008.InterdependentUtilities:HowSocialRankingAffectsChoiceBehavior.PloSone3:e3477
BecharaA,DamasioAR.2005.Thesomaticmarkerhypothesis:Aneuraltheoryofeconomicdecision.GamesandEconomicBehavior52:33672
BecharaA,DamasioH,TranelD,DamasioAR.1997.DecidingAdvantageouslyBeforeKnowingtheAdvantageousStrategy.Science275:12935
BernsGS,CapraM,ChappelowJ,MooreS,C.N.2008.NonlinearNeurobiologicalProbabilityWeightingFunctionsForAversiveOutcomes.NeuroImage
BollardA,LiuR,NursimuluAD,RangelA,BossaertsP.2008.RiskandRewardPreferencesUnderTimePressure.UnderReview
BossaertsP,GhirardatoP,GuarnaschelliS,ZameWR.2008a.Ambiguityinassetmarkets:theoryandexperiment.UnderReview
BossaertsP,HillionP.1991.MarketMicrostructureEffectsofGovernmentInterventionintheForeignExchangeMarket.TheReviewofFinancialStudies4:51341
BossaertsP,PlottC.2004.BasicPrinciplesofAssetPricingTheory:Evidence
fromLargeScaleExperimentalFinancialMarkets.ReviewofFinance8:13569
BossaertsP,PreuschoffK,HsuM.2008b.Theneurobiologicalfoundationsofvaluationinhumandecisionmakingunderuncertainty.InNeuroeconomics:DecisionMakingandtheBrain,ed.PWGlimcher,CFCamerer,EFehr,RAPoldrack.London:AcademicPress
BruguierA,QuartzS,BossaertsP.2008.ExploringtheNatureofTradingIntuition".UnderReview
CamererC,HoTH.1999.ExperienceweightedAttractionLearninginNormalFormGames.Econometrica67:82774
CamererCF,HoTH,ChongJK.2002.SophisticatedExperienceWeightedAttractionLearningandStrategicTeachinginRepeatedGames.JournalofEconomicTheory104:13788
CaplinA,DeanM,GlimcherPW,RutledgeRB.2008.Measuringbeliefsandrewards:Aneuroeconomicapproach.UnderReview
ChenMK,LakshminarayananV,SantosLR.2006.HowBasicAreBehavioralBiases?EvidencefromCapuchinMonkeyTradingBehavior.JournalofPoliticalEconomy114:51737
ChristopoulosG,ToblerP,BossaertsP,DolanR,SchultzW.2008.Neuralcorrelatesofexpectedvalue,risk,andriskaversioncontributingtodecisionmakingunderrisk.UnderReview
8/2/2019 Trinity09 Bossaerts Paper
22/28
CoatesJM,HerbertJ.2008.EndogenoussteroidsandfinancialrisktakingonaLondontradingfloor.ProceedingsoftheNationalAcademyofSciences105:616772
CoricelliG,CritchleyHD,JoffilyM,O'DohertyJP,SiriguA,DolanRJ.2005.Regretanditsavoidance:aneuroimagingstudyofchoicebehavior.NatNeurosci
8:125562CoricelliG,NagelR.2008.Beautycontestinthebrain;Aneuralbasisofstrategic
thinking.UnderReviewCritchleyHD,WiensS,RotshteinP,OhmanA,DolanRJ.2004.Neuralsystems
supportinginteroceptiveawareness.NatNeurosci7:18995d'AcremontM,BossaertsP.2008.Neurobiologicalstudiesofriskassessment:A
comparisonofexpectedutilityandmeanvarianceapproaches.Cognitive,Affective,&BehavioralNeuroscience8:36374
d'AcremontM,LuZL,LiX,VanderLindenM,BecharaA.2008.Neuralcorrelatesofriskpredictionerrorsduringreinforcementlearninginhumans.UnderReview
DeMartinoB,KumaranD,HoltB,DolanRJ.2009.Theneurobiologyofreferencedependentvaluecomputation.JournalofNeuroscienceInPrint
DeMartinoB,KumaranD,SeymourB,DolanRJ.2006.Frames,Biases,andRationalDecisionMakingintheHumanBrain.Science313:6847
DreherJC,KohnP,BermanKF.2005.NeuralCodingofDistinctStatisticalPropertiesofRewardInformationinHumans.CerebCortex
ElliottR,AgnewZ,DeakinJFW.2008.Medialorbitofrontalcortexcodesrelativeratherthanabsolutevalueoffinancialrewardsinhumans.EuropeanJournalofNeuroscience27:22138
EllsbergD.1961.Risk,Ambiguity,andtheSavageAxioms.TheQuarterlyJournal
ofEconomics75:64369EngleRF.2002.NewfrontiersforARCHmodels.JournalofAppliedEconometrics
17:42546EvansJSBT.2003.Intwominds:dualprocessaccountsofreasoning.Trendsin
CognitiveSciences7:4549FecteauS,KnochD,FregniF,SultaniN,BoggioP,PascualLeoneA.2007.
DiminishingRiskTakingBehaviorbyModulatingActivityinthePrefrontalCortex:ADirectCurrentStimulationStudy.J.Neurosci.27:125005
FengL,SeasholesMS.2005.DoInvestorSophisticationandTradingExperienceEliminateBehavioralBiasesinFinancialMarkets?ReviewofFinance9:30551
FriedmanD,RustJ,eds.1993.TheDoubleAuctionMarket:Institutions,Theories,andEvidence:WestviewPress.429pp.
GallagherHL,FrithCD.2003.Functionalimagingof'theoryofmind'.TrendsinCognitiveSciences7:7783
GallagherHL,JackA,RoepstorffA,FrithC.2002.Imagingtheintentionalstanceinacompetitivegame.NeuroImage16:81421
GlimcherPW,CamererCF,FehrE,PoldrackRA,eds.2009.Neuroeconomics.DecisionMakingandTheBrain.London,UK:AcademicPress
GschwandtnerU,AstonJ,RenaudS,FuhrP.2001.PathologicGamblingin
PatientswithParkinson'sDisease.Clinicalneuropharmacology24:170
8/2/2019 Trinity09 Bossaerts Paper
23/28
HamptonAN,BossaertsP,O'DohertyJP.2006.TheRoleoftheVentromedialPrefrontalCortexinAbstractStateBasedInferenceduringDecisionMakinginHumans.J.Neurosci.26:83607
HamptonAN,BossaertsP,O'DohertyJP.2008.Neuralcorrelatesofmentalizingrelatedcomputationsduringstrategicinteractionsinhumans.
ProceedingsoftheNationalAcademyofSciences105:67416HareTA,CamererCF,RangelA.2008a.Selfcontrolindecisionmakinginvolves
modulationofthevmPFCvaluationsystem.UnderReviewHareTA,O'DohertyJ,CamererCF,SchultzW,RangelA.2008b.Dissociatingthe
RoleoftheOrbitofrontalCortexandtheStriatumintheComputationofGoalValuesandPredictionErrors.J.Neurosci.28:562330
HsuM,AnenC,QuartzSR.2008a.TheRightandtheGood:DistributiveJusticeandNeuralEncodingofEquityandEfficiency.Science320:10925
HsuM,BhattM,AdolphsR,TranelD,CamererCF.2005.Neuralsystemsrespondingtodegreesofuncertaintyinhumandecisionmaking.Science310:16803
HsuM,KrajbichI,ZhaoC,CamererCF.2008b.Neuralresponsetorewardanticipationunderriskisnonlinearinprobabilities.UnderReview
HuettelS,SongA,McCarthyG.2005.DecisionsunderUncertainty:ProbabilisticContextInfluencesActivationofPrefrontalandParietalCortices.J.Neurosci.25:330411
KableJW,GlimcherPW.2007.Theneuralcorrelatesofsubjectivevalueduringintertemporalchoice.NatNeurosci10:162533
KahnemanD,FrederickS.2002.RepresentativenessRevisited:AttributeSubstitutioninIntuitiveJudgment.InHeuristicsandBiases:ThePsychologyofIntuitiveJudgment,ed.TGilovich,DWGriffin,DKahneman,
pp.4981:CambridgeUniversityPressKahnemanD,FrederickS.2007.Framesandbrains:elicitationandcontrolof
responsetendencies.TrendsinCognitiveSciences11:456KahnemanD,TverskyA.1979.Prospecttheory:AnAnalysisofDecisionUnder
Risk.Econometrica47:26391KnochD,GianottiLRR,PascualLeoneA,TreyerV,RegardM,etal.2006.
DisruptionofRightPrefrontalCortexbyLowFrequencyRepetitiveTranscranialMagneticStimulationInducesRiskTakingBehavior.J.Neurosci.26:646972
KrollY,LevyH,MarkowitzHM.1984.MeanVarianceversusDirectUtilityMaximization.JournalofFinance39:4761
KuhnenCM,KnutsonB.2005.Theneuralbasisoffinancialrisktaking.Neuron47:76370
KuhnenCM,KnutsonB.2008.TheInfluenceofAffectonBeliefs,PreferencesandFinancialDecisions:SSRN
LancasterKJ.1966.ANewApproachtoConsumerTheory.TheJournalofPoliticalEconomy74:132
LeDouxJE,CicchettiP,XagorarisA,RomanskiLM.1990.Thelateralamygdaloidnucleus:sensoryinterfaceoftheamygdalainfearconditioning.J.Neurosci.10:10629
LoAW,RepinDV.2002.Thepsychophysiologyofrealtimefinancialrisk
processing.JCognNeurosci14:32339
8/2/2019 Trinity09 Bossaerts Paper
24/28
LohrenzT,McCabeK,CamererCF,MontaguePR.2007.Neuralsignatureoffictivelearningsignalsinasequentialinvestmenttask.ProceedingsoftheNationalAcademyofSciences104:94938
McCabeK,HouserD,RyanL,SmithV,TrouardT.2001.Afunctionalimagingstudyofcooperationintwopersonreciprocalexchange.Proceedingsof
theNationalAcademyofSciences98:118325McClureSM,BernsGS,MontaguePR.2003.Temporalpredictionerrorsina
passivelearningtaskactivatehumanstriatum.Neuron38:33946McClureSM,LaibsonDI,LoewensteinG,CohenJD.2004.SeparateNeural
SystemsValueImmediateandDelayedMonetaryRewards.Science306:5037
MontaguePR,DayanP,SejnowskiTJ.1996.AframeworkformesencephalicdopaminesystemsbasedonpredictiveHebbianlearning.JNeurosci16:193647
MorrisJS,FrithCD,PerrettDI,RowlandD,YoungAW,etal.1996.Adifferentialneuralresponseinthehumanamygdalatofearfulandhappyfacialexpressions.Nature383:8125
NivY,EdlundJA,DayanP,O'DohertyJP.2008.Abirdinthehandversustwointhetree:Neuralpredictionerrorsrevealsubjectiverisksensitivity.UnderReview
O'DohertyJ,DayanP,SchultzJ,DeichmannR,FristonK,DolanRJ.2004.DissociableRolesofVentralandDorsalStriatuminInstrumentalConditioning.Science304:4524
O'DohertyJP,DayanP,FristonK,CritchleyH,DolanRJ.2003.TemporalDifferenceModelsandRewardRelatedLearningintheHumanBrain.Neuron38:329
PadoaSchioppaC,AssadJA.2006.Neuronsintheorbitofrontalcortexencodeeconomicvalue.Nature441:2236
PayzanE,BossaertsP.2008.LearningandChoiceunderNonstationaryandUnknownProbabilities.UnderReview
PlassmannH,O'DohertyJ,RangelA.2007.OrbitofrontalCortexEncodesWillingnesstoPayinEverydayEconomicTransactions.J.Neurosci.27:99848
PlattML,GlimcherPW.1999.Neuralcorrelatesofdecisionvariablesinparietalcortex.Nature400:2338
PlottCR,ZeilerK.2005.TheWillingnesstoPayWillingnesstoAcceptGap,the"EndowmentEffect,"SubjectMisconceptions,andExperimentalProceduresforElicitingValuations.TheAmericanEconomicReview95:53045
PreuschoffK,BossaertsP,QuartzS.2006.NeuralDifferentiationofExpectedRewardandRiskinHumanSubcorticalStructures.Neuron51:38190
PreuschoffK,QuartzSR,BossaertsP.2008.HumanInsulaActivationReflectsRiskPredictionErrorsAsWellAsRisk.J.Neurosci.28:274552
RizzolattiG,CraigheroL.2004.TheMirrorNeuronSystem.AnnualReviewofNeuroscience27:16992
SaalD,DongY,BonciA,MalenkaRC.2003.DrugsofAbuseandStressTriggeraCommonSynapticAdaptationinDopamineNeurons.37:57782
SchultzW,DayanP,MontaguePR.1997.ANeuralSubstrateofPredictionandReward.Science275:15939
8/2/2019 Trinity09 Bossaerts Paper
25/28
ShamsL,MaWJ,BeierholmU.2005.Soundinducedflashillusionasanoptimalpercept.NeuroReport16:1923
SpezioM,AdolphsR,HurleyR,PivenJ.2007.AbnormalUseofFacialInformationinHighFunctioningAutism.JournalofAutismandDevelopmentalDisorders37:92939
StahlDO.2000.RuleLearninginSymmetricNormalFormGames:TheoryandEvidence.GamesandEconomicBehavior32:10538
ToblerPN,FiorilloCD,SchultzW.2005a.Adaptivecodingofrewardvaluebydopamineneurons.Science307:16425
ToblerPN,FiorilloCD,SchultzW.2005b.AdaptiveCodingofRewardValuebyDopamineNeurons.Science307:16425
ToblerPN,O'DohertyJP,DolanRJ,SchultzW.2007.Rewardvaluecodingdistinctfromriskattituderelateduncertaintycodinginhumanrewardsystems.JNeurophysiol97:162132
TomSM,FoxCR,TrepelC,PoldrackRA.2007.TheNeuralBasisofLossAversioninDecisionMakingUnderRisk.Science315:5158
TremblayL,SchultzW.1999.Relativerewardpreferenceinprimateorbitofrontalcortex.Nature398:7048
TzieropoulosH,GravedePeraltaR,BossaertsP,GonzalezS.2008.InMan,madnessistherule,rationalitytheexception.Underreview
YacubianJ,GlascherJ,SchroederK,SommerT,BrausDF,BuchelC.2006.DissociableSystemsforGainandLossRelatedValuePredictionsandErrorsofPredictionintheHumanBrain.J.Neurosci.26:95307
8/2/2019 Trinity09 Bossaerts Paper
26/28
Figure1
Figure2
8/2/2019 Trinity09 Bossaerts Paper
27/28
Figure3
C
Figure4
A B
8/2/2019 Trinity09 Bossaerts Paper
28/28
Figure5
Figure6