1

RevealedIndifference:UsingResponseTimestoInferPreferences*

ArkadyKonovalov1andIanKrajbich2

April23,2017

Abstract

Revealedpreferenceisthedominantapproachforinferringpreferences,butitrelieson

discrete,stochasticchoices.Thechoiceprocessalsoproducesresponsetimes(RTs)which

arecontinuousandcanoftenbeobservedintheabsenceofinformativechoiceoutcomes.

Moreover, there is a consistent relationship between RTs and strength-of-preference,

namelythatpeoplemakeslowerdecisionsastheyapproachindifference.Thisrelationship

arises from optimal solutions to sequential information sampling problems. Here, we

investigateseveralwaysinwhichthisrelationshipcanbeusedtoinferpreferenceswhen

choiceoutcomesareuninformativeorunavailable. WeshowthatRTsfromasingletwo-

alternativechoiceproblemcanbeenoughtousefullyrankpeopleaccordingtotheirdegree

oflossaversion.Usingalargenumberofchoiceproblems,wearefurtherabletorecover

individual utility-function parameters from RTs alone (no choice outcomes) in three

differentchoicedomains.Finally,weareabletouselongRTstopredictwhichchoicesare

inconsistentwithasubject’sutilityfunctionandlikelytolaterbereversed. Theseresults

provideaproofofconceptforanovel“methodofrevealedindifference”.JELCodes:C91;D01;D03;D87;D81;D90.

*TheauthorsthankPaulHealy,LucasCoffman,RyanWebb,DanLevin,JohnKagel,KirbyNielsen,JeevantRampal,andPujaBhattacharyafortheirhelpfulcommentsandconversationsandYosukeMorishima,ErnstFehr,ToddHare,ShabnamHakimi,andAntonioRangelforsharingtheirdata.1DepartmentofEconomics,TheOhioStateUniversity;konovalov.1@osu.edu.2DepartmentofPsychology,DepartmentofEconomics,TheOhioStateUniversitykrajbich.1@osu.edu.

2

1Introduction

When inferring a person’s preferences, businesses,managers, economists, and policy

makerstypicallyrelyontheirchoices. Thisisthestandardrevealedpreferenceapproach

(Samuelson, 1938). In this paperwe challenge this fundamental notion, by arguing that

individualpreferencescanbeinferredwithoutobservingchoiceoutcomes.

Choice itself is not the only output of the choice process. We are also often able to

observeotherfeaturessuchasresponsetimes(RT).Moreover,RTsarecontinuousandso

may carry more information than discrete stochastic choice outcomes (Loomes, 2005;

Webb,2013)3. Thetrickliesindiscoveringhowtheinteractionbetweenpreferencesand

choiceoptionsproducesRTs. Ifwecouldcharacterizeandtheninvertthisfunction,then

wecouldinferpreferencesfromRTs.

Considerthefollowingexample.Supposeweareattemptingtodeterminewhichoftwo

agentsismoreimpatient.Weaskeachofthemthesamequestion:“wouldyouratherhave

$25 todayor$40 in twoweeks?” Suppose thatbothagents take the$40. With just this

informationthereisnowaytodistinguishbetweentheagentswithoutfurtherquestioning.

Now suppose Agent A made his choice in 5 seconds, while Agent B made hers in 10

seconds.Whoislikelymorepatient?

Weargue,usingdatafromseveralchoiceexperimentsindifferentpreferencedomains,

thattheanswerisAgentA.Whilestandardeconomictheoryissilentinthissituation,there

arerelevanttheoreticalframeworksforansweringthisquestion.Onerelevantframework

of particular interest, due to its success in cognitivepsychology andneuroscience, is the

3ForareviewontheuseofRTsineconomics,seeClithero(2016a)andSpiliopoulosandOrtmann(2014).

3

class of sequential sampling models (SSM) (also known as drift-diffusion or evidence-

accumulationmodels). Thatframeworkviewsdecisionslikethisoneasamentaltug-of-

war between the options. For options that are similar in strength (utility) it takesmore

timetodeterminethewinner,andinmanycasestheweakersidemayprevail.

Toput itanotherway,rather thanbasingtheirdecisionsondeterministicutilities for

eachoption,agentsinsteadbasetheirdecisionsonstochasticutilities,likeinrandomutility

theory(RUT). AswithRUT,SSMsareagnosticaboutwheretheaverageutilitiesornoise

originate,theyaresimplytheresultofsomeevaluationprocess4.WhereSSMsdepartfrom

standardRUTisbyassumingthatagentsrepeatedlyestimateor“sample”thesestochastic

utilities, averaging out the noise across samples until one option emerges as the clear

winner. This evaluationprocess takes an amount of time that dependson thedifference

betweentheaverageutilities,i.e.thestrength-of-preference.

As a result of this process, a relationship emerges between strength-of-preference,

choiceprobability, andRT. Returning toourexample,weknow that sinceAgentAwas

relativelyfasterthanAgentB,itismorelikelythatAgentAfacedaneasierdecision.Since

itwas easier for Agent A to choose the later option,we can infer that he is likelymore

patient.

Of course, other independent factors may influence RT, but if there is a statistically

reliable relationshipbetweendecision speedandproximity to indifference, this couldbe

used toeconomists’andpolicymakers’advantage inanumberofways. First,onecould

design simpler and shorter preference-elicitation mechanisms such as the single-trial

4Theassumptionisthatthisevaluationprocess,followingthenotionofutilitytheoryasan“asif”model,doesnottypicallyinvolveexplicitutilitycalculation.OtherwisetheutilitieswouldbecomedeterministicandRTswouldnotdependonutilitydifference,astheyoftendoempirically.

4

binary-choiceexampleabove. Second,onecouldrecoverutility functions incaseswhere

agents’ preferences fall outside of the range of a given elicitation procedure,where it is

currentlyonlypossible toput a lowerorupperboundonpreferences. Third, one could

estimateutility-functionparametersinthecompleteabsenceofchoiceoutcomesorwhen

theseoutcomesarenot informative(i.e. thesame). Byaskingavarietyofquestions,one

couldseewhichoneselicittheslowestresponsesandinferthatthosequestionsmadethe

person roughly indifferent. This last point is to highlight that it is still possible to do

preference analysis in the absence of informative choice outcomes. This suggests, for

example, that in order to protect their private information, agents must consider

observabilityofboththeirchoicesandtheirRTs.

Here, we demonstrate that inferring preferences from RTs is indeed a promising

approach,usingexperimentaldata from threeprominent choicedomains: risk, time, and

social preferences. In Section 5.3 we show that single-trial RTs can be used to rank

subjects according to their degree of loss aversion. In Section5.4we show thatRTs on

“extreme” trials (wheremost subjects choose the sameoption) can alsobeused to rank

subjectsaccordingtotheirloss,time,andsocialpreferences.FinallyinSections5.5and5.6

we examine RTs from the full datasets and use each subject’s RT data to estimate their

utility-function parameters. In every case these rankings/parameters significantly align

with those estimated from subjects’ choices over the full datasets. Taken together, these

results serve as a proof of concept that individual preferences can be inferred fromRTs

alone.

2Model

5

2.1.Background

Aseconomists,weoftendefendexpectedutilitytheorybyarguingthatpeoplemerely

behave “as if” they are calculating andmaximizing utility. But if people do not explicitly

calculateutilities,howdotheymakedecisions? Thishasbeenatopicof intensestudyin

thefieldsofjudgmentanddecisionmaking,andmorerecently,neuroeconomics.Fromthis

research there is emerging consensus that people make decisions by accumulating and

comparing evidence for each of the alternatives, until the evidence for one option

sufficiently outweighs the other. In some cases this process may involve explicitly

collecting information from the environment, e.g. reading reviewsbeforedecidingwhich

movietowatch. Inothercases, itmay involve introspection:collecting internalevidence

basedonpastexperiencesandone’scurrentstate,e.g.decidingwhichofyourtwofavorite

movies to re-watch. Economists have traditionally treated these two types of decisions

separately,referringtotheformeras“perceptibility”andthelatteras“desirability”(Block

&Marschak,1960).

Whilethesemayseemlikeverydifferenttypesofdecisions,thereisnowoverwhelming

empiricalevidencethattheycanbeexplainedbyacommonmathematicalframework.The

precisemechanismbywhichthisevidenceaccumulationandcomparisonoccursisstillan

activetopicofstudyinneuroscience5,butfornowwetreatSSMsinthesamewaythatwe

treatexpectedutility: as “as if”models. That is,wedon’t yetknowexactlywhatorhow

evidence is being accumulated and compared in the brain, but thismodeling framework

5Theexactmechanismbehindnoisysampling(e.g.,samplingfrommemory,shiftsinattention,ornoiseinneuralnetworks)remainsasubjectofongoingdebateintheliterature(Shadlen&Shohamy,2016)andisoutsideofthescopeofourstudy.

6

allowsustoaccountforchoiceandRTdata(aswellaseye-trackingandneuraldata)that

othermodelscan’t.

TheideaofapplyingSSMstodecisionsthatarebasedsolelyoninternalevidenceiswell

establishedincognitivepsychology6.RogerRatcliff’s(1978)seminalworkonthediffusion

modelwasasatheoryofmemoryretrieval. Subjectswouldinitiallystudyalistofwords

andthenlaterbepresentedwitha“probe”word. Theirtaskwastoindicatewhetherthe

probewasawordthathadappearedonthestudylist.Evidenceinthismodelwassimply

aninternalcomparisonbetweentheprobewordandthesubject’simperfectmemoryofthe

study words. Similarly, economic choice relies on a comparison between imperfect

memoriesand/orrepresentationsofthechoicealternatives.

TheideaofapplyingSSMstoeconomicchoicewasfirstintroducedbyBusemeyerand

colleagues in the 1980s (Busemeyer, 1985; Busemeyer&Diederich, 2002; Busemeyer&

Rapoport, 1988;Busemeyer&Townsend, 1993; Johnson&Busemeyer, 2005;Rieskamp,

2008;Roe,Busemeyer,&Townsend,2001). Recentyearshaveseenrenewedinterestin

thisworkduetotheabilityofthesemodelstosimultaneouslyaccountforchoices,RTs,eye

movements, and brain activity in many individual preference domains such as risk and

uncertainty(Busemeyer,1985;Busemeyer&Townsend,1993;Fiedler&Glöckner,2012;

Hunt et al., 2012; Stewart, Hermens, & Matthews, 2015), intertemporal choice (Dai &

Busemeyer,2014;Rodriguez,Turner,&McClure,2014a),socialpreferences(Hutchersonet

al.2015;Krajbichetal.2015a;Krajbichetal.2015b),aswellasfoodandconsumerchoice

(DeMartino,Fleming,Garret,&Dolan,2013;Krajbich,Armel,&Rangel,2010;Krajbich,Lu,

6Forover50years,cognitivepsychologistshavebeenusingSSMstoexplainindividualbehaviorinsimpleperceptionandmemorytasks(BrownandHeathcote2008;Laming1979;Link1975;Ratcliff1978;RatcliffandMcKoon2008;Stone1960;UsherandMcClelland2001).

7

Camerer,&Rangel,2012;Krajbich&Rangel,2011;Milosavljevic,Malmaud,Huth,Koch,&

Rangel, 2010; Philiastides & Ratcliff, 2013; Towal, Mormann, & Koch, 2013), and more

complex decision problems (Caplin & Martin, 2016). Our approach builds on this

literature,withoneimportantchange:insteadoffittingthemodeltochoicesandRTs,wefit

themodeltoonlytheRTs.Thisletsustesthowusefulthemodelcanbewithoutobserving

choiceoutcomes.

Beyondtheirdescriptivepower,SSMscanalsobenormative. Wheneverydecision is

equallydifficultandequallyvaluable,theconstant-thresholddriftdiffusionmodel(DDM)is

optimal(Wald,1945), in thesense that foradesiredaccuracy, theDDMminimizesmean

RT. Whendecisions vary in difficulty or value, the optimal SSMs aremore complex and

involve collapsing thresholds (Busemeyer & Rapoport, 1988; Frazier & Yu, 2007;

Fudenberg,Strack,&Strzalecki,2015;Tajima,Drugowitsch,&Pouget,2016;Webb,2013b;

Woodford,2014).Itisworthnotingthatrecentworkindecisiontheoryhasfocusedonthe

problemofoptimal samplingand stochastic choice (Che&Mierendorff, 2016;Matejka&

McKay,2014;Sims,2003;Steiner,Stewart,&Matějka,2017;Woodford,2016).

In thispaperwedeliberatelyavoid the issueofwhetheragentsareusing theoptimal

SSM. Insteadwe rely on the simplest, most robust SSM variant, which is the DDMwith

constantthresholds.Therearemanypapersthathaveempiricallyinvestigatedoptimality,

withlittleevidenceforcollapsingthresholds(Hawkins,Forstmann,Wagenmakers,Ratcliff,

&Brown,2015;Oudetal.,2016),andourstudyisnotdesignedtoaddressit(suchstudies

typicallyinvolvehundredsoreventhousandsofchoicetrials).Admittedlythismayhinder

our ability to infer preferences, but the simple DDM provides a proof-of-concept and a

8

lowerboundontheusefulnessofRTsineconomicanalysis. Laterinthetextweexamine

othermodelsandstatisticaltechniquesforinferringpreferencesfromRTs.

Onekeythingtonoticewiththesemodels is thattheagentchoosesherstoppingrule

(thresholds)priortothesamplingprocessandsowhatdeterminesheractualchoiceand

RT, conditional on that stopping rule, is the sequenceof evidence that she accumulates7.

That sequence is stochastic, but does depend on the agent’s subjective evaluation of the

alternatives. More specifically, following previous literature, we assume that the net

evidence is the difference between the average utilities of the two options (from a RUT

perspective)8. Therefore the speed and accuracy of the decision are monotonically

increasinginthisdifference.Thisquantityisoftenreferredtoasthe“driftrate”,andinour

economicsettingitrepresentsstrength-of-preference,i.e.thedifferenceincardinalutility.

Thus,whenapersonisclosetoindifference,thedriftrateapproacheszero,causingadelay

inthechoice(Chabris,Morris,Taubinsky,Laibson,&Schuldt,2009;Dickhaut,Rustichini,&

Smith,2009;Moffatt,2005;Mosteller&Nogee,1951).

2.2.Model

Hereweuseastandarddrift-diffusionmodel(DDM)(Ratcliff1978)toestablishthelink

betweenRTsandtheunderlyingutilitiesinsimplebinarychoicetasks.Asnotedabove,this

model is a standard tool in cognitive psychology for simultaneously capturing choice

probabilitiesandRTdistributions. Ithasseenmostofitsuseinperceptionandmemory,7Thisisastandardassumption,withveryfewexceptions.Thisrelatesbacktothenormativeissuesdiscussedaboveandisdiscussedatlengthinthosearticles.8Again,weassumethatsubjectsarenotexplicitlycalculatingexpectedvaluesorotherutilities.Ifthatwasthecase,themaindeterminantofdifficultywouldbethecomplexityofthecalculations.Instead,weassumethatsubjectsestimatehowgoodeachalternativeis,anditistherelativeattractivenessofthetwooptionsthatdeterminesthedifficulty.

9

but it has alsomore recentlybeenapplied to economic choice, e.g. intertemporal choice,

performingbetterthancompetingmodelssuchaslogisticchoice(Dai&Busemeyer,2014).

Letusassumethatanagentobservesasetofalternatives𝑗 ∈ {1,2}.Thechoiceprocess

involves two components: a constant boundary threshold𝑏 and a decision variable𝑦(𝑡)

thatevolvesovertimeaccordingtothefollowingdifferentialequation:

𝑑𝑦 𝑡 = 𝑣 ∙ 𝑑𝑡 + 𝜎 ∙ 𝑑𝑊 (1)

where𝑦(𝑡) isaccumulatedevidencetowardsoption1(with𝑦(0) = 0),𝑣 is thedriftrate,

whichisassumedtobealinearfunctionoftheutilitydifference:

𝑣 ≡ 𝑧 ⋅ (𝑢9 𝜃 − 𝑢< 𝜃 ), (2)

where𝑧 ∈ 𝑅>isascalingparameter,𝑢?(⋅)istheutilityofthegivenalternative,and𝜃isthe

agent-specificutilityfunctionparameter.Finally,𝜎 ∙ 𝑑𝑊isaWienerprocess(i.e.Brownian

motion)thatrepresentsGaussianwhitenoisewithvariance𝜎<.Withoutlossofgenerality,

wenormalize𝜎 = 1asitcanonlybeidentifieduptoscale(duetothearbitraryunitsony).

We define the response time 𝑅𝑇 as the first time that the absolute value of the

decisionvariablereachesaboundary𝑏 ∈ 𝑅>,plusanon-stochasticcomponentknownas

non-decisiontime(𝜏 ∈ 𝑅>,typicallyinterpretedasthetimethatasubjectneedstoprocess

theinformationonthescreen):

𝑅𝑇 = min 𝑡: 𝑦 𝑡 ≥ 𝑏 + 𝜏. (3)

Thechoiceoutcome𝑎 ∈ {1,2}isdefinedasfollows:

𝑎 = 1𝑖𝑓𝑦 𝑅𝑇 = 𝑏2𝑖𝑓𝑦 𝑅𝑇 = −𝑏 (4)

10

Now, assuming without loss of generality 𝑣 ≥ 0, we can calculate the choice

probabilities𝑝(𝑎 = 𝑗)9,theexpectedRT,andanapproximatebivariateprobabilitydensity

function (PDF) for theRTs (minus 𝜏) as follows (Blurton, Kesselmeier,&Gondan, 2012;

Navarro&Fuss,2009;Ratcliff,1978;Srivastava,Feng,Cohen,Leonard,&Shenhav,2015;

Wabersich&Vandekerckhove,2014):

𝑝(𝑎 = 1) =NOPQR9

NOPQRNSOPQ𝑖𝑓𝑣 > 0

9<𝑖𝑓𝑣 = 0

(5)

𝐸 𝑅𝑇 =VW1 − <

NOPQRNSOPQ+ 𝜏𝑖𝑓𝑣 > 0

𝑏< + 𝜏𝑖𝑓𝑣 = 0 (6)

𝑓(𝑡) = XYVO

𝑒RPO[O −1 \R9𝑘 ⋅ 𝑒R

^O_O[`QOa

\b9 sin \X<

(𝑒WV + 𝑒RWV) (7)

Typically, if choice data is available, one can use equation (5) to estimate

parameterspurelyfromchoices,maximizinglog-likelihood:

𝐿𝐿 = (1 𝑎e = 1 ⋅ log 𝑝 𝑎e = 1 𝑏, 𝑣e +e (1 𝑎e = 2 ⋅ log 1 − 𝑝 𝑎e = 1 𝑏, 𝑣e ,(8)

where𝑛denotestrialnumber,and1(⋅)istheindicatorfunction.

Here,unlikepreviousstudies,wewillassumethatonlyRTdataisavailableandwilluse

theRTprobabilitydensitiestoestimatetheutilityparameterforeachsubject𝑖(𝜃j)given

theempiricaldistributionofRTsbymaximizingthefollowinglikelihoodfunction:

𝐿𝐿 = (log(𝑓 𝑅𝑇e, 𝑎e = 1 𝑏, 𝜏, 𝑣e ) +e log(𝑓 𝑅𝑇e, 𝑎e = 2 𝑏, 𝜏, 𝑣e )). (9)

9Previousworkhasshownthatthestandardrandomutilitymodelthatimpliesalogitchoicerulecanbederivedfromthedrift-diffusionmodel(Webb,2013b).

11

Figure 1. Example simulation of the drift-diffusion model (DDM). Response times(RTs)asafunctionofthedifferenceinutilitiesbetweentwooptionsin900simulatedtrials.Thegraydotsshowindividualtrials, theblackcirclesdenoteaverageswithbinsofwidth10.Theparametersusedforthesimulationcorrespondtotheparametersestimatedatthegroup level in the time discounting experiment (𝑏 = 1.33, 𝑧 = 0.09, 𝜏 = 0.11). Utilitydifferencesaresampledfromauniformdistributionbetween-20and20.

2.3.Modelpredictions

There are several important qualitative predictions of themodel that allow usmake

inferencesbeyondstructuralestimation.

First, the expected RTs decrease as the utility difference between the two options

becomes larger (lm(no)lW

< 0, see Figure 1 for a simulation example). If the drift scaling

parameter and non-decision time are the same (or similar) across subjects, we should

observe a group-level correlation between the RT and the utility function parameter of

interest, conditionalon thesubjectsmaking thesamechoice. Thusweshouldbeable to

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●●

●

●

●● ●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●● ●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

● ●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●● ●●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

● ●

●

●

●●

● ●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

● ●

●

●● ●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

● ●

● ●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●●●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●●

−20 −10 0 10 20

01

23

45

utility difference

RT

[s]

● ●

●

●●

●●

● ●

12

rank subjects according to their individual preferences using RTs from a single decision

problem.WeexplorethispredictioninSection5.3.

More generally, this relationship can be used to rank subjects using sets of choice

problemswheretheyallmadethesamechoice,andsochoicesareuninformative.Consider

anintertemporalchoiceexamplewithtwoindividuals:apatientpersonwhoisindifferent

between$10todayand$12tomorrow,andanimpatientpersonwhoisindifferentbetween

$10todayand$19tomorrow. Thepatientindividualwill likelytakesometimetodecide

between$10todayand$11tomorrow,butchooseveryquicklybetween$10todayand$20

tomorrow.Ontheotherhand,theimpatientindividualwillstruggletochoosebetween$10

todayand$20tomorrow,butchooseveryquicklybetween$10todayand$11tomorrow.

Notice that in thisexampleboth individualswouldmost likelychoose$10todayover

$11tomorrowand$20tomorrowover$10today,basedontheirindifferencepoints.Thus

arevealedpreferenceapproachwouldbeunlikelytomakeadistinctionbetweenthesetwo

individuals based only on these choices. In contrast, the revealed indifference approach

would tell us that a slow choice between $10 today and $11 tomorrow is indicative of

patience, while a slow choice between $10 today and $20 tomorrow is indicative of

impatience.WeconsiderthecaseofuninformativechoiceoutcomesinSection5.4.

Second, long RTs are considerably more informative than short RTs. Sequential

samplingmodelscorrectlypredictthatshortRTscanoccuratanylevelofchoicedifficulty

(i.e.strength-of-preference),butlongRTsalmostexclusivelyoccurfordifficultchoices(i.e.

nearindifference;again,seeFigure1foranexamplesimulation).Theintuitionhereisthat

there is a lotofnoise in thedecisionprocess and so thequickest choices are thosewith

noise pointed in the same direction (Ratcliff, Philiastides, & Sajda, 2009; Ratcliff &

13

Tuerlinckx,2002). Because these sequencesof alignednoiseoccur independentlyof the

driftrate,thereistypicallyno(oraveryweak)correlationbetweentheshortestRTsand

difficulty (Ratcliff & McKoon 2008). However, the slowest choices are ones where the

combinationofdriftplusnoiseleadstoroughlyzeronetevidenceaccumulation.Giventhat

thesamplednoiseismean-zero,net-zeroevidenceaccumulationisfarmorelikelytooccur

withsmalldrift(difficultproblems)thanwithlargedrift(easyproblems).Thisproducesa

correlationbetweenthelongestRTsanddifficulty.

By taking advantage of this knowledge, we can improve the RT-based estimation of

utility-function parameters (relative to structural estimation, Section 5.5), either by

focusingonthelongestRTsorbyestimatingthepeakinRTsusingthefullsubjectdataset.

ThesemethodsareexploredinSection5.6.

Finally, themodelpredicts thatmost “errors”, or choices inconsistentwith theutility

function,shouldhappenwhenthesubject iscloseto indifferenceandthustendstomake

slowchoices.WetestthishypothesisinSection5.7.

2.4.Modellimitations

WithRTs,“slow” isarelativeconceptwhereaschoice isabsolute. Inordertousethe

revealed indifference approach, we must therefore develop methods to identify slow

decisions for a given individual. Another potential issue is that even though RTs are

continuous,theycanbeinfluencedbyotherfactorsasidefromstrength-of-preferenceand

thusmayappeartobequitenoisy(Krajbichetal.2015a).Forinstance,othershaveargued

thatlongRTsreflectmoreeffortordeliberativethinkingonthepartofthesubject(Gabaix,

14

Laibson, Moloche, & Weinberg, 2006; Hey, 1995; Rubinstein, 2007; Wilcox, 1993). This

couldinterferewithourabilitytouseRTstoinferpreferences.

There are certain characteristics of the choice environment that we believe may

facilitateestimatingpreferencesfromRTs.First,RTsshouldberecordedwithmillisecond

precisionsincemanysimplebinarychoicestakeonlyafewseconds.Second,choicesshould

bemadeusingakeyboardbuttonpressratherthanusingamousetoselectanoption.This

issimplytominimizenoise inRTsduetohandmovements. Third,we limitourselves to

binary decisions since indifference is not straightforward with multiple alternatives.

Fourth,forthelastexercise,whereweusetherevealedindifferenceapproachandallthe

trials to estimateutility functions, the rangeof indifferencepoints impliedby the choice

problemsintheexperimentshouldcovertherangeofparametervaluesinthepopulation.

It shouldbementioned thatSSMshavebeensuccessfullyapplied todatasets thatviolate

someoftheserequirements(e.g.Krajbichetal.2015b;KrajbichandRangel2011),butsuch

datasetsarenotidealforourcurrentgoals.

Inadditiontotheserequirements,thereareotherfactorsthatmayaffectourabilityto

successfully infer indifferencepoints fromRTs. Onequestion iswhetherthereshouldbe

constraints on RTs. RT restrictions are common in binary choice tasks in order to keep

subjects focused, but overly restrictive cutoffs may attenuate the effect of strength-of-

preferenceonRTs(Fudenbergetal.,2015).Hereweexaminedatasetswithvaryingtime

constraints(3s,10s,andunlimited)inordertoexploretherangeofsituationsinwhichwe

canemploythemethodofrevealedindifference.Giventheveryfewmissedtrials,andthe

robustness of our results across the datasets, it is unlikely that the constraints were

binding.Anotherquestion iswhether themethodof revealed indifferencecanbeused in

15

situations where the utility function contains multiple parameters. This will obviously

complicatematters,butifoneparameterisrelativelymoreimportantthantheothers,the

methodmaystillwork,aswedemonstrateintheriskychoicetask.

3Relevantliterature

It is important toacknowledge thatwearenot the firsteconomists touseRTdata to

predictbehavior. Inadditiontotheworkmentionedabove,SchotterandTrevino(2014)

and Chabris et al. (2009) have used RTs to predict strategic behavior and aggregate

intertemporalpreferences,respectively.

Schotter and Trevino (2014) use the longest (and second longest) RTs to estimate

threshold strategies in a global game; theRT-based estimates are able to explainout-of-

samplechoicebetterthantheequilibriumprediction.Thispaperissimilarinspirittoours,

but studies strategic situationswhilewe focus on individual preference elicitation. The

authorsarguethatlongRTsintheirsettingreflecteitherdiscoveringtheoptimalthreshold

strategyorexplicitcalculation.Theseconsiderationsandexplanationsarequiteseparate

fromtheideasandmodelsexploredinourwork.

In the preference domain, Chabris et al. (2009) and an earlier unpublished working

paperbythesameauthors(Chabris,Laibson,Morris,Schuldt,&Taubinsky,2008)useda

methodinfluencedbythedirectedcognitionmodel(Gabaix&Laibson,2005;Gabaixetal.,

2006)andfullRT-distributiondataatthegroupleveltoestimateaggregatedintertemporal

preferences using a structural estimation approach. Thismethod aims to identify utility

parameters using the fact that RTs increase with choice difficulty. Their study finds a

correlation between the RT-based estimates and choice-based predictions for three

16

separatelargegroupsofsubjects,butdoesnotattemptanyanalysisofindividualsubjects’

preferences.Intheappendix,weprovideacomparisonbetweenthismethodandourown.

Finally, Clithero (2016b) used a DDM to augment choice data with RT information,

producingbetterout-of-samplepredictionsinfoodchoice.Whilethisworkisveryrelated

toourown,weshowthatRTinformationalonecanbeusedtopredictdecisions.

Ourwork also relates to a growing literature on incomplete preferences (Agranov&

Ortoleva, 2015; Cettolin & Riedl, 2015; Danan, Ziegelmeyer, & others, 2006; Eliaz & Ok,

2006;Mandler,2005;Ok,Ortoleva,&Riella,2012). This literaturehasshownthatwhen

given the choicebetween twoalternatives, individualsmayprefera lotteryover the two

alternatives.Thisproducesprobabilisticchoiceoutcomes,justlikewithSSMs.Moreover,

DananandZiegelmeyer(2006)report thatamajorityof thesubjects in theirexperiment

postponesimplechoicesbetweenlotteries,eventhoughpostponementiscostly.Whilenot

exactlyameasureofRT,thispostponementbehavioris inlinewiththeslow-indifference

phenomenonthatwereport.

Our work contrasts with the work of Rubinstein, Wilcox, and others (Chen &

Fischbacher,2015;Hey,1995;Recalde,Riedl,&Vesterlund,2014;Rubinstein,2007,2013,

2014;Wilcox, 1993),where longRTs are associatedwithdeliberative thought and short

RTsareassociatedwith intuition. Thesepapershaveprimarilystudiedstrategicsettings

where effort is likely to vary more substantially across subjects and so dominate the

strength-of-preferenceeffectsthatdriveourresults.Whilethiscanbeaccountedforwith

varying levelsof𝑏 in theDDMthatwehavediscussed, itmayalsobethatotherdecision

processesareinvolvedinstrategicdecisions.

17

4ExperimentalDesign

We analyze four separate datasets: the last two were collected with other research

goalsinmind,butincludedprecisemeasurementsofRTs,whilethefirsttwowerecollected

specificallyforthisstudy.

4.1.Loss-aversionexperiments

These experiments were conducted at The Ohio State University. In each round,

subjects chose between a sure amount of money and a 50/50 lottery that included a

positive amount and a loss (which in some roundswas equal to $0).The set of decision

problemswasadaptedfromSokol-Hessneretal.(2009).Subjects’RTswerenotrestricted.

Intheadaptiveexperiment,eachsubject’schoicedefinedthenexttrial’soptionsusinga

Bayesian procedure (DOSE, seeWang et al. 2010) to ensure an accurate estimate of the

subject’sriskandlossaversionwithinalimitednumberofrounds.Eachsubjectcompleted

the same three unpaid practice trials followed by 30 paid trials. Importantly, every

subject’sfirstpaidtrialwasidentical.Eachsubjectreceivedtheoutcomeofonerandomly

selectedtrial.61subjectsparticipatedinthisversionoftheexperiment,earning$17-20on

average10.

In the non-adaptive experiment, each subject first completed a three-trial practice

followedby276paid trials. These trialswerepresented in twoblocks of the same138

choice problems, each presented in random order without any pause between the two

blocks. Subjects were endowed with $17 and additionally earned the outcome of one

10Inordertocoveranypotentiallossesincurredduringthistask,subjectsfirstcompletedanunrelatedtaskthatendowedthemwithenoughmoneytocoverforanypotentiallosses.

18

randomly selected trial (in case of a loss it was subtracted from the endowment). 39

subjectsparticipatedinthisexperiment,earning$18onaverage.Allsubjectsgavewritten

consent,andthestudieswereapprovedbytheOSUInstitutionalReviewBoard.

For both experiments we assumed a standard prospect theory value function

(Kahneman&Tversky,1979):

𝑈 𝑊, 𝐿, 𝑆 = 0.5𝑊t − 0.5𝜆 −𝐿 t

𝑆t,

where𝑊 is the gain in the lottery, L is the loss, S is the sure amount,𝜌 reflects risk

aversion,and𝜆captureslossaversion.Inthenon-adaptiveexperiment,theutilityfunctions

wereestimatedusingastandardMLEapproachwithalogitchoicefunction.

Similartopriorworkusingthistask,wefoundthatriskaversionplaysaminimalrolein

this task relative to loss aversion, with 𝜌estimates typically close to 1. Therefore,

acknowledging that varying levels of risk aversion could add noise to the RTs, for the

analysesbelow (both choice- andRT-based)weassumed riskneutrality (𝜌 = 1). For the

non-adaptiveexperiment,weusedonlytrialswithnon-zerolosses(specifically,112outof

138 decision problems) for both estimation and out-of-sample prediction. Two subjects

withoutlyingestimatesof𝜆(beyondthreestandarddeviationsofthemean)wereremoved

fromtheanalysis.Thesameexclusioncriterionwasusedfortheotherdatasets.

4.2.Temporaldiscountingexperiment

This experiment was conducted while subjects underwent functional magnetic

resonanceimaging(fMRI)attheCaliforniaInstituteofTechnology(Hare,Hakimi,&Rangel,

2014). Ineachround,subjectschosebetweengetting$25rightafter theexperimentora

largeramount(upto$54)atalaterdate(7to200days).Therewere108uniquedecision

19

problemsandsubjectsencounteredeachproblemtwice. All216trialswerepresentedin

randomorder. Each trial, the amountwas first presentedon the screen, followedby the

delay,andsubjectswereaskedtopressoneoftwobuttonstoacceptorrejecttheoffer.The

decisionwas followedbya feedback screen showing “Yes” (if theofferwasaccepted)or

“No”(otherwise).Thedecisiontimewaslimitedto3seconds,andifasubjectfailedtogive

a response, the feedback screen contained the text “No decision received”. These trials

(2.6%acrosssubjects)wereexcludedfromtheanalysis.Trialswereseparatedbyrandom

intervals(2-6seconds).

41subjectsparticipatedinthisexperiment,earninga$50show-upfeeandtheamount

fromone randomly selected choice. Thepaymentsweremadeusingprepaiddebit cards

thatwereactivatedatthechosendelayeddate.Allsubjectsgavewrittenconsent,andthe

experimentwasapprovedbyCaltech’sInternalReviewBoard.

For this experiment, we used a hyperbolic discounting utility function (Herrnstein,

1981;Laibson,1997):

𝑈 𝑥, 𝑡 =𝑥

1 + 𝑘𝑡,

where𝑥isthedelayedamount,𝑘isthediscountfactor(higherismoreimpatient),and𝑡

is thedelayperiod indays.One subject that chose $25nowon every trialwas removed

from the analysis. The utility functionswere estimated using a standardMLE approach

withalogitchoicefunction.Twosubjectswithoutlyingestimatesof𝑘wereremovedfrom

theanalysis.

4.3.Binarydictatorgameexperiment

20

This dataset was collected while subjects underwent fMRI at the Social and Neural

Systems laboratory, University of Zurich (Krajbich et al. 2015a). Subjects made choices

between two allocations, X and Y, which specified their own payoff and an anonymous

receiver’s payoff. The payoffs were displayed in experimental currency units, and 120

predeterminedallocationswerepresentedinrandomorder.Eachallocationhadatradeoff

betweenafairoption(moreequaldivision)andaselfishoption(withhigherpayofftothe

dictator).72outof120decisionproblemspersubjecthadhigherpayofftothedictatorin

bothoptionsXandY(to identifyadvantageous inequalityaversion),whiletherestofthe

problems (48/120) had higher payoffs to the receiver in both options (to identify

disadvantageous inequalityaversion). Ineachround,subjectsobservedadecisionscreen

that included the twooptions,andhad tomakeachoicewitha two-buttonbox.Subjects

were required tomake their decisionswithin 10 seconds; if a subject failed to respond

under this time limit, that trialwas excluded from the analysis (4 trialswere excluded).

Intertrialintervalswererandomizeduniformlyfrom3to7seconds.

Subjects read written instructions before the experiment, and were tested for

comprehension with a control questionnaire. All subjects passed the questionnaire and

understoodtheanonymousnatureofthegame.Intotal,30subjectswererecruitedforthe

experiment. They received a show-up fee of 25 CHF and a payment from 6 randomly

chosenrounds,averagingatabout65CHF.Allsubjectsprovidedwrittenconsent,andthe

ethicscommitteeoftheCantonofZurichapprovedthestudy.

To fit choices in this experiment, we used a standard Fehr-Schmidt other-regarding

preferencemodel(Fehr&Schmidt,1999):

𝑈 𝑥j, 𝑥? = 𝑥j − 𝛼 ⋅ max 𝑥? − 𝑥j, 0 − 𝛽 ⋅ max 𝑥j − 𝑥?, 0 ,

21

where𝑥j isthedictator’spayoff,𝑥? isthereceiver’spayoff,𝛼reflectsdisadvantageous

inequalityaversion,and𝛽 reflectsadvantageous inequalityaversion.Asdescribedabove,

wewereabletoseparatetrialsthatidentified𝛼and𝛽,sothesechoicesweretreatedastwo

separate datasets. The utility functions were estimated using a standard MLE approach

withalogitchoicefunction.Onesubjectwithanoutlyingestimateof𝛼wasremovedfrom

theanalysis.

5Results

5.1.Choice-basedestimations

Thethreeutilityfunctionsweselectedtomodelsubjects’choicesperformedwellabove

chance. To examine the number of choices that were consistent with the estimated

parameter values, we used standard MLE estimates of logit choice functions (for the

estimation procedure details see Appendix A) to identify the “preferred” alternatives in

every trial and compared those to the actual choice outcomes. More specifically, we

calculated utilities using parameters estimated purely from choices, and in every trial

predicted that the alternativewith the higher utilitywouldbe chosenwith certainty.All

subjectswereveryconsistentintheirchoiceseveninthedatasetswithalargenumberof

trials:socialchoice𝛼:94%,socialchoice𝛽:93%,intertemporalchoice:83%,non-adaptive

riskychoice:89%(seeAppendixBforthesubject-levelchoicepredictions).

5.2.Responsetimesreflectchoicedifficulty

ThemethodofrevealedindifferenceispredicatedontheDDM-basedideathatasubject

willtakemoretimetodecideasthedecisionbecomesmoredifficult.Forthefollowing

22

Figure2.RTspeakatindifference.MeanRTinsecondsasafunctionofthedistancebetweentheindividualsubject’sutilityfunctionparameterandtheindifferencepointonaparticular trial; data are aggregated into bins ofwidth 0.02 (top row), 0.01 (bottom leftpanel), and 1 (bottom right panel), which are truncated and centered for illustrationpurposes.Binswith fewer than10 subjects arenot shown.Barsdenote standard errors,clusteredatthesubjectlevel.

analysis, our measure of difficulty was the difference between the subject’s utility

functionparameter(estimatedpurelyfromthesubject’schoices)andtheparametervalue

thatwouldmakeapersonindifferentbetweenthetwoalternativesinthattrial(wereferto

this as the “indifference point”). When a subject’s parameter value is equal to the

indifferencepointofa trial,wesay that thesubject is indifferenton that trialandso the

choiceismaximallydifficult.

●

●

●

●

●

●

●

−0.6 −0.4 −0.2 0.0 0.2 0.4 0.6

1.5

2.0

2.5

3.0

3.5

Social choice (α)

subject α − indifference α

RT [s

]

●

●

●

●

●

●

●●

●●

●

●

−1.0 −0.5 0.0 0.5 1.0

1.0

1.5

2.0

2.5

3.0

3.5

Social choice (β)

subject β − indifference β

RT [s

]●

●

●●

●

●

●

−0.03 −0.02 −0.01 0.00 0.01 0.02 0.03

0.6

0.8

1.0

1.2

1.4

Intertemporal choice

subject k − indifference k

RT [s

]

●● ●

●

●

●

●

●

●

−4 −2 0 2 4

1.0

1.5

2.0

2.5

Risk choice

subject λ − indifference λ

RT [s

]

23

Letusillustratethisconceptwithasimpleexample.Supposethatintheintertemporal-

choicetaskasubjecthastochoosebetween$25todayand$40in30days.Ifthesubjecthas

ahyperbolicdiscountingutilityfunction(asweassume),theywouldbeindifferentwithan

individualdiscountrate𝑘 that is thesolutionto theequation25 = 40/(1 − 30𝑘),ork=

0.0125. Thiswouldbetheindifferencepointofthisparticulartrial. Asubjectwiththisk

valuewouldbe indifferentonthis trial,asubjectwitha lowerkwould favorthedelayed

option,andasubjectwithahigherkwouldfavortheimmediateoption.

The bigger the absolute difference between the subject’s parameter and the trial’s

indifferencepoint, the easier thedecision, and the shorter the averageRT. Indeed, this

effectisobservedinallofourdatasets,withRTspeakingatindifference(Figure2).Mixed-

effects regressionmodels (treating subjects as random effects) show strong, statistically

significant effects of the absolute distance between the indifference point and subjects’

individualutilityparameterson log(RTs)11 forall thedatasets(fixedeffectofdistanceon

RT:dictatorgame𝛼:t=-7.5,p<0.001;dictatorgame𝛽:t=-9.1,p<0.001;intertemporal

choice𝑘:t=-9.9,p<0.001;non-adaptiveriskychoice𝜆:t=-9.6,p<0.001,adaptiverisky

choice𝜆:t=-4.4,p<0.001).

5.3.One-trialpreferenceranking

Intheadaptiveriskychoiceexperiment(Section4.1),allsubjectsfacedthesamechoice

problem in the first trial. They had to choose between a 50/50 lottery with a positive

amount($12)andaloss($7.5),andasureamount($0).Assumingriskneutrality,asubject

11 A key feature of RTs generated by sequential sampling models is that they are roughly log-normally distributed. Thus it is typical to transform RTs with a natural logarithm beforeperformingstatisticalanalyses.

24

Figure3.Preferencerankcanbeinferredfromasingledecisionproblem.RTsinthefirst roundof the adaptive risk experiment as a function of the individual subject’s loss-aversioncoefficient fromthewholeexperiment;Spearmancorrelationsdisplayed. In thisround,eachsubjectwaspresentedwithabinarychoicebetweenalotterythatincludeda50%chanceofwinning$12andlosing$7.5,andasureoptionof$0.Theleftpaneldisplayssubjectswhochosethesafeoption,andtherightpanelshowsthosewhochosetheriskyoption.ThesolidblacklinesareOLSfits.

with a loss aversion coefficient of 𝜆 = 1.6 should be indifferent between these two

options,withmore loss-averse subjects picking the safe option, and the rest picking the

riskyoption.

Because themean loss aversion in our samplewas 2.5 (median = 2.46),most of the

subjects(44outof61)pickedthesafeoptioninthisfirsttrial.Now,ifwehadtorestrictour

experimenttojustthisonetrial,theonlywaywecouldclassifysubjects’preferenceswould

betodividethemintotwogroups:thosewith𝜆 ≥ 1.6andthosewith𝜆 ≤ 1.6.Withineach

groupwewouldnotbeabletosayanythingabouteachindividual’slossaversion.

ByobservingRTswecanestablisharankingofthesubjectsineachgroup.Specifically,

theDDMpredicts that subjectswith longerRTswouldexhibit lossaversioncloser to1.6

(assuming that the threshold, drift, andnon-decision timeparameters are similar across

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

● ●

●●

●

●

●

● ●● ●

●

●

●

●

●

●

●

●

●

●

1.5 2.0 2.5 3.0 3.5 4.0 4.5

24

68

1012

Safe option chosen

λ estimated from choices

RT [s

]

r = −0.43p = 0.004

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

0.0 0.5 1.0 1.5 2.0 2.5 3.0

24

68

1012

Risky option chosen

λ estimated from choices

RT [s

]

r = 0.41

p < 0.1

25

subjects). We ranked subjects in each group according to theirRTs and then compared

thoserankingstothe“true”lossaversionparametersestimatedfromthe30choicesinthe

fulldataset(seeFigure3).

In linewith the results of the previous section, RTs peaked around the indifference

point(𝜆 = 1.6). Therewasasignificantrank-based(Spearman)correlationbetweenRTs

andlossseekinginthe“safeoption”group(r=0.43,p=0.004)andbetweenRTsandloss

aversion in the “risky option” group (r = 0.41, p = 0.1). Thus the single-trial RT-based

rankingsalignedquitewellwiththe30-trialchoice-basedrankings.

As mentioned in the introduction, this approach could have great appeal since

economistsandpractitionersareoftenconstrainedwhencollectingdata,hencetheappeal

of adaptive experiments (Blais &Weber, 2006; Cavagnaro, Myung, Pitt, & Kujala, 2010;

Kim,Pitt,Lu,Steyvers,&Myung,2014;Rodriguez,Turner,&McClure,2014b;Wangetal.,

2010,andothers).Ourresultssuggestthat in factveryfewtrialsmaybeneededtogeta

reliableestimateofsubjects’preferences12.

5.4.Uninformativechoices

Another possible use of RT-based inference is the case where an experiment (or

questionnaire) is flawed in a such way that most subjects give the same answer to the

choice problems (or onemight consider a situationwherepeople feel social pressure to

give a certain answer, even if it contradicts their true preference). This is similar to the

situationfromthelastsection,exceptthatsubjects’choicesmaybeevenlessinformative,

andwithmultipletrials,theremaybeevenmoretogainfromexaminingtheRTs.12PredictivepowermaybefurtherimprovedbyaccountingforotherpotentialcontributorstoRTs,suchasdemographics,baselinemotorresponsetime,etc.

26

To model this situation, for each dataset (non-adaptive risk choice, intertemporal

choice, and social choice) we isolated trials with the most extreme indifference point,

wheremostsubjectschosethesameoption(e.g.,thelotteries),andlimitedouranalysisto

those subjects who picked this most popular option. In some instances, this involved

several trials since some of the choice problems were repeated or shared the same

indifferencepoint.

We found that the RTs on these trials were strongly correlated with subjects’

preferenceparametersinallfourdomains(riskychoice:r=0.55,p<0.001;intertemporal

choice:r=0.43,p=0.007;socialchoice𝛼:r=0.4,p=0.03;socialchoice𝛽:r=0.68,p<

0.001,Spearmancorrelations).Againweseethatitispossibletosomewhatreliablyrank

subjectsaccordingtotheirpreferencesintheabsenceofdistinguishingchoicedata.Similar

to theprevioussection, thismethodcouldbeused tobolsterdatasets thatare limited in

scopeandsounabletorecoverallsubjects’preferences.

5.5.DDM-basedutilityestimationfromRTs

TheresultsdescribedintheprevioussectionsdemonstratethatwecanuseRTstorank

subjectsaccordingtotheirpreferenceswithoutinformativechoicedata.Inthissection,we

exploreways to estimate individual subjects’ utility-function parameters from their RTs

acrossmultiplechoiceproblems.

The DDM predicts more than just a simple linear relationship between strength-of-

preferenceandmeanRT;itpredictsentireRTdistributions.AsdescribedinSection2,the

DDMapproachassumesthatineachtrial,subjectsmakedecisionsusingaWienerrandom

processthatproducesadistributionofRTsgiventhreefreeparameters,whichcanbe

27

Figure4.TheDDMestimatesofsubjects’utilityfunctionparameters,estimatingDDMparametersat thegroup level.Subject-levelcorrelation(Pearson)betweenparametersestimated fromchoicedataandRTdatausing thedrift-diffusionmodel (DDM).Thesolidlinesare45degreelines.

estimated for each individual subject (Wabersich and Vandekerckhove 2014, see

AppendixAfortheestimationdetails).Inparticular,thedriftrateinthemodelisalinear

functionoftheutilitydifferenceandsobyestimatingdriftrateswecanidentifythelatent

utility-functionparameters.

●● ●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

−1.0 −0.5 0.0 0.5 1.0

−1.0

−0.5

0.0

0.5

1.0

Social choice (α)

α estimated from choices

α e

stim

ated

from

RTs r = 0.39

p = 0.04●●

●●

●

●●

●

●

●

●

●●

●● ●

●

●

●

●

●

●●●

●

●●● ●

●

−1.0 −0.5 0.0 0.5 1.0 1.5

−1.0

−0.5

0.0

0.5

1.0

1.5

Social choice (β)

β estimated from choices

β es

timat

ed fr

om R

Ts

r = 0.52p = 0.003

●

●●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

0.00 0.01 0.02 0.03 0.04 0.05

0.00

0.01

0.02

0.03

0.04

0.05

Intertemporal choice

k estimated from choices

k es

timat

ed fr

om R

Ts

r = 0.57

p < 0.000

●●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

0 2 4 6 8

02

46

8

Risk choice

λ estimated from choices

λ es

timat

ed fr

om R

Tsr = 0.36

p = 0.03

28

5.5.1.Group-levelDDMestimation

First,we estimated theDDM, assuming that the boundary parameter𝑏, non-decision

time𝜏, anddrift rate𝑧 are fixedacross subjects (seeSection2.2andAppendixA for the

modelandestimationdetails).Thisisacommonlyusedtechniquetohelpwithparameter

identificationwithlimiteddatasets,thoughinSection5.5.2.werelaxthisassumption.The

identificationproblemtypicallyoccurswhenRTspeakattheboundaryofanexperiment’s

indifference points. In these cases, it is impossible to say whether the RTs would have

continuedtorisebeyondtheboundaryandsowecannotknowthetruelocationoftheRT

peak. By fixing theotherparameters in themodelwe constrain thedistributionofRTs,

allowingus to estimate the expectedRTat indifference. Take for example the risk task,

wheremeanRTspeakatapproximately2.1sandthemaximumindifferencelis5.33.Ifwe

observedasubject’sRTsincreasingwiththeindifferencelbutonlyaveraging1.5satl=

5.33,wemightinferthatthesubject’slisnot5.33,butrather7.33(giventherelationship

inFig.2d).

Ineachdataset13,wefoundthat individualutility-functionparametersestimatedfrom

RTsalone(usingthestructuralDDMmodel,withoutchoicedata)werecorrelatedwiththe

sameparametersestimatedfromthechoicedata(socialchoice𝛼:r=0.39,p=0.04;social

choice𝛽:r=0.52,p=0.003;intertemporalchoice𝑘:r=0.57,p<0.001;riskychoice𝜆:r=

0.36, p = 0.03; Pearson correlations; Figure 4,). These results showed an improvement

overthedirectedcognitionmodelsuggestedbyChabrisetal.(2009)(seeAppendixAfor

estimationdetailsandAppendixEforthemodelcomparisonresults).

13Wedonotusetheadaptiveriskdatasetinthissectionsincetheadaptivenatureofthetaskcreatesautocorrelationsinthedatathatwilllikelyinterferewithourestimationprocedure.

29

Anotherway to assess the goodness-of-fit is to examine the number of choices that are

consistentwith theestimatedparametervalues,as inSection5.1. Todoso,weusedthe

RT-estimated parameters to identify the “preferred” alternatives in every trial and

compared those to the actual choice outcomes. RT-estimated parameters were able to

explainahighproportionofchoicesinthedatasets(socialchoice𝛼:79%respectively(p<

0.001); social choice𝛽: 80% (p < 0.001); intertemporal choice: 76% (p < 0.001); risky

choice:77%(p<0.001);p-valuesdenotetwo-sidedWilcoxonsignedranktestsignificance

at the subject level, comparing these proportions to chance). For a stricter test, we

calculated an average of all indifference points for each experiment, which roughly

corresponds to the mean of the experimenter’s prior parameter distribution, andmade

choice predictions for each subject using this single value. TheDDMaccuracy rates beat

thisbaselineintwooutoffourcases(fortheintertemporalchoiceandsocialchoice𝛽,p<

0.05,two-sidedWilcoxonsignedranktest).

5.5.2.Subject-levelDDMestimation

In addition, we estimated the DDM for each subject separately, assuming individual

variabilityintheboundaryparameter𝑏,non-decisiontime𝜏,anddriftrate𝑧.Asdescribed

intheprevioussection,thiscausesidentificationproblemsforcertainsubjects.Therefore,

inthissectionweexcludesubjectswhoseutility-functionparameterestimatesfelloutside

the range of indifference points employed in each experiment. After excluding these

subjects (2/30 and 2/30 in the social choice dataset, 16/39 in the intertemporal choice

dataset,and7/37intheriskchoicedataset)weobtainedcorrelationsthatshowed

30

Figure 5. The DDM estimates of subjects’ utility function parameters, fitting themodel to each subject individually. Subject-level correlation (Pearson) betweenparametersestimatedfromchoicedataandRTdatausingthedrift-diffusionmodel(DDM),including only the subjects with parameters estimated within the range of indifferencepointsintheexperiment(reddottedlines).Thesolidlinesare45degreelines.

improvementoverthepreviousDDMestimationinthreeoutof4cases,withcorrelations

ashighas0.74(Figure5,p<0.01).

Forthe𝛼parameterinthesocialchoicetask,thecorrelationwasinsignificant(Figure5,

p = 0.65), most likely due to the small number of trials (only 48 trials to estimate 4

parameters)andthetightdistributionofsubjects’indifferencepointsinthattask.

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

−1.0 −0.5 0.0 0.5 1.0 1.5

−1.0

−0.5

0.0

0.5

1.0

1.5

Social choice (α)

α estimated from choices

α e

stim

ated

from

RTs

r = 0.09p = 0.65

●●

●

●

●

●

●●

●

●

●

●●

●● ●

●

●

●

●

●

●●●

●

●●●●

●

−1.0 −0.5 0.0 0.5 1.0 1.5

−1.0

−0.5

0.0

0.5

1.0

1.5

Social choice (β)

β estimated from choices

β es

timat

ed fr

om R

Ts r = 0.74p < 0.001

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

● ●

0.00 0.01 0.02 0.03 0.04 0.05 0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Intertemporal choice

k estimated from choices

k es

timat

ed fr

om R

Ts

r = 0.63p < 0.000

●

●

●

●

●

●●●

●

●

● ●

●●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●●●●

●

●

●

●●

0 2 4 6 8

02

46

8

Risk choice

λ estimated from choices

λ es

timat

ed fr

om R

Ts

r = 0.53

p = 0.002

31

Theseresultshighlighttheimportanceoftaskdesign. Ifthetask’s indifferencepoints

span the range of indifference points in the population then the DDMmethod can yield

quiteaccurateutility-functionparameters.

5.6.AlternativeapproachestoutilityestimationfromRTs

The DDM may seem optimal for parameter recovery if that is indeed the data

generatingprocess.However,severalfactorslikelylimititsusefulnessinthissetting.The

DDMhasseveral freeparameters thatare identifiedusing featuresof choice-conditioned

RTdistributions. Identification thus typically reliesonmany trials andobserving choice

outcomes. Withoutmeeting these tworequirements, theDDMapproachmaystruggle to

identifyparametersaccurately.Belowweexplorealternativeapproachestoanalyzingthe

RTs.

5.6.1.TopRTdecilemethod

One alternative approach is to focus on the longest RTs: for instance, Schotter and

Trevino (2014) successfully use the longest RT over a number of trials to identify a

decisionthresholdinasimpleglobalgame.AslongRTsindicateindifference,theycouldbe

usedtoidentifytrialswherethesubjectiveutilitiesofthetwooptionswereequal,andthus

obtainanestimateof theutility functionparameter fromthe indifferencepoints in those

trials.Hereweexplorewhetherthismethodworksinourdatasets.

32

Figure6.Single longestRTsareanoisypredictorof indifference.RTinsecondsasafunctionofthedistancebetweentheindividualsubjectutilityfunctionparameterandtheindifference point on a particular trial; gray dots denote individual trials. Red trianglesdenotetrialswiththehighestRTforeachindividualsubject.

Figure 6 plots RTs as a function of strength-of-preference for every trial in the

experiments. It is easy to see thateven though formanysubjects the single slowest trial

providesagoodsignalof“true”preference(asdefinedbythechoice-basedestimation),for

others the longestRT is far from indifference.This is especially true for the risky choice

data (possibly due to the two-parameter utility function or the unrestricted RTs). This

suggestedtousthatitmaybebettertousemorethanthesingleslowesttrial.

●

●

●

● ●●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●● ●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

● ●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●●

● ● ●●

● ●

● ●

●

●

●

●

●● ●

●●

●

● ●●

●

●

●

●●● ●

●●●

●

●●

●

●

●

●

●

●

●●

●● ●

●

●

●● ●

●

● ●●● ●

●

●●

●

● ●●

● ●● ●● ●

●●

●●●

●

●

●

●● ●●

●

●

●

●

●

●

●●

●● ●

●

●●

●

●

●●

●●●

●

●

●

●●

●

●

●●●

●

●

●

●

●●

●●

●

●

●●

●● ● ●

●●

●

●

●

●

●● ●

●

●

●●

●

● ● ●●

●●

●

●

●

●● ●

●

●●

●● ●●

●

●

●

●

● ●●● ● ●●

● ●

●●● ●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●● ●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●●

●

●

●

●●

●●●

●●

●●

●●●

● ●●●

●●

●

●

●

●●●

● ●●

●●

●

●● ●

●● ● ●

●

● ●

●●● ●

●

●

●●

●

●

●●●

●

●

●●

●

●●

●

●

●●

●

●●

●

●●

●

●

●

●●●

●

●

●

●

●●

●●

●

●●

●

●

●●

●

●

●

●

●

●●

●

● ●

●

●● ● ●

●

●●

●

●●

●●

●● ●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

● ●

●●

●

●

● ●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

● ●●●

●

●

●

●

● ●

●

●●

●

●●

●

●● ●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●● ●●

● ●

●

●

●●●

●

●

●●

●●

●● ●

●●

●● ●●

●

●

●●

●

●●

●● ●●

●●

●

● ●

●

●●●

●

●

●● ●

●

●

●

●

●

● ●●●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●● ●

●●

●

●

●

●

●

●● ●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●●

●●● ●

● ●●

●

●

●●

●●

●●

●

●

●●

●●●

●

● ●●

●●

●

●

●

●

●

● ●● ●●

●●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●●

●●●

●●

●

●

●

●

●

●

●

●●

●

●●●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●●

●

●

●●

●

●●

●

●

●

●●

●

●

●●●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●● ●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

24

68

10

Social choice (α)

subject α − indifference α

RT [s

]

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

● ●

●●

●

●●

●

●

●●

●●

●●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●●

●

●●

●

●

●

● ●

●

●

●

●●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●●

●

●●

●

●

●

●

●

● ●●

●

●●

●●

●

●

●

●●

●●

●● ●

●

●

●

●

●

●

●●

●

●●●

●●

●

●●●

●

●

●●

●

●●●● ●

●

●

● ●

● ●●

●

● ●

●●

●

●●

●

●

●

●

●

●

●●●

●●●

● ●

●

●

●

●●

●

●

●●●

●● ●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●● ●

●●

●

●

●

●● ●

● ● ●●

●

●●●

●●

●

● ●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●●●

●●●

●

● ●

●

●●

● ●

●

●

●

●

●

●

●●●

●

●

●

●●

●●

●

●

● ●● ●

●

●

●

●●

●

●

●

●

●

●● ●●

●

●● ●●

●

●

●

●●

●●

●

● ● ●●

●●

●● ●●

● ● ●

●●

●●●

●●

●

●●

●● ●●●

● ●●

●

●●

● ●●

● ●●●

●●●

●●●

● ●● ●

●

●

●

●●

●

●●●●

●

●

●

●

●●●

●

●●

●

●

● ●●

●

● ●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●● ●

●

●

●

●

● ●●

●

●

●

● ●●●

●●

●

●

●

●

●

●

●

●

●●

●●

● ●●

●

●

●

●

●●

●

● ●●

●

●●

●

●

●●

●●●

●

●

●●

●

●

●

●

●●●

●

●

●

●●●

●●

●●● ●

●

●

● ●

●

●

● ●●

●●

●

●

●●

●

●

●●

● ●●

●

● ●

●●

●

●

●

●●

●● ●

●●

●

●

●

●●

●

●

●●

●

●

●●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●●

●●

●●

●

●

●

●●

●

●

●●

●

●

●

● ●●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

● ●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

● ●

● ●

●

●●

●

●

●

●●

●

●●

●

●

●●

●

●

●●

●●

●●●

●

● ●●

●

●

●

●●

●●

●

● ●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

● ●

●

●●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●● ●

●

● ●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

● ●

●●

●

●

●●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

● ●

●

● ●

● ●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

● ●

●●

●●●

●●

●

●●

●

●

●●

●●●

●

●

●

●

●

●● ●

●

●

●

●●

●

● ●

●●●

●

●●

●●

●

●●

● ●● ●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●●

●●

●●

●●

●●

●●●

●

●

●

●●

●

●●

●

●

● ●

●

● ●

●

●

●●

●

●●●

●

●

●

● ●●

●●

●●

●

●● ●● ●

● ●

●

●●

●●

●●

●

●●●

● ●● ●

● ●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●●●

●

●●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●● ●●

●

●

●

●

●

●

●

●

●

●●

●● ●

●● ●

●

●●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

● ●

●

●●

●●●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●● ●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

● ●●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●●

●

●

●

●●

●

●

●

●●●

●●

●●

●

●●

●

●

●● ●

●●

●●

●

●

●

●●

●

●●

●

● ●●

●●●

●

●●

●

●

●●

●

● ●●●

●●●

●

●

●●●

●

●

●

●

●●

●●●●

●

●●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●●

● ●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●●

●

●

●

●

●

●

●

●

●

● ●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●● ●

●

●

●

●

●●

●

●

● ●●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●●

●●

● ●

●

●

●●

●

●

● ●

●

●

●

●●

● ●●

●

●

●

●

●

●

●

●●

●● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●●

●●

●

●

●

●●●

●●

●

●●

●

●

●

●●

●●

●● ●

●●

●●

●

●●

● ●

●

●●

●

●

●●

●●●

●

●

●

●

●●●

●●

● ●

●

●

●●●● ●

●●

●●

●

●●

●

−2 −1 0 1 2

24

68

10

Social choice (β)

subject β − indifference β

RT [s

]●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●●

●

●

●

●

●● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

● ●

●

●

●

●

●

●

●

●

●● ●

● ●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

● ●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●●

●

●

●●●

●

●●

●●

●

●

●

●

●

●●

●●●

● ●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

● ●

● ●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●●

●● ●

●

●●●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●●

●●

●

● ●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●●

● ●

● ●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●●●

●

● ●

●●

●

●●

●

●

●

●● ●

●● ●

●

●

●●

● ●

●●

●

●

● ●●

●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●●

● ●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

● ●

●

●

●

●

●

●

●● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●●

●

●

●●

●●

●

●

●

●

●

● ●

● ● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●●●● ●

●

●

●

●●●

●

●●

●

●

●

●●

●●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●● ●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●●

●●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●

●

● ●

●

●●

●● ●

●

●

●

● ●

●

●

●●

●

●

●

●●

●● ●●

●●

●

●●

●●●

●

●

●

●

●●

●● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ● ●

●●

●

●

●

●●

●

●●

●

●●

● ●●●

● ●●

●

●

●●

●

●

●●

●

●● ●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●●●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

● ●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●● ●

●●

●

●

●

●

●●

●● ●

● ●●

●

●

●

●

●

●●

● ●●

●

●●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●● ●

●

● ●

●

●●●

●

●

●●

●

●

●

●●

●●

●

●●

●

●●

●

●

● ●

●●

●●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

● ● ●●

●

●

●

●

●

●

●

●●

●●

●●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●●●

●

●

●●

●●●

●●

●●

● ● ●

●●

●

●

●●

●

●

●●

●

●

●

●●

●

● ●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

● ●●●

●●

●

●●

●

●●

● ●●

●● ●● ●

● ●●● ●●● ●

●●

●●●● ●

●

●

●●

●

●●●●

●●

●●

●●

●●●

●

●●

●●

●

●

●

● ●

●●

●

●

●

●

●

● ●

● ●●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●●

●●●●

●●

●

●●●

●

●

●●

●●●

●

●●

●

● ●

●

●●●

●●

●

● ●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●●●

●●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●● ●

●

●

●

●

●

●●

● ●●

● ●

● ●●

●

●●

●●

●●●

●

●●

●

●

●

●

●●

●

●●

●

● ●●

●●

●

●

●

●●

●●

●

●●

● ●● ●

●

●

●

●

●

●●

●

●

●● ●●

●

●●

●●

● ●

●●

●

●

●

●

●

●

●

●

●●●

●●●

● ●

●

●●

●●●

●

●●

●● ●●

●●●

●

●

●●

●●

●

● ●●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●●

●

●●

●

● ●

●

●●

●

●

●

●

●

●● ●●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●●

●●

●

●

●

● ●●

●

●●●

●●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

● ●

●

●

●●

●

●

●●

●●

●

●●●

●●

●

●

●● ●● ●

●

●

●

●

●

●

●●●

●

●

●●

●●

●

●●

●●

●

●●

●

●●●

●

●

●

●

●●●

●

●

●●

●●

●

●●

●●

●

●

●

●

●●

●

●●●●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

● ●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●●

● ●

●

●

●

●●

● ●

●●

●● ●●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●● ●

●

●●

●

●●

●

●●●

●

●

●●●

● ● ●●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●●

●●●

●

●

●

●

●●

● ●

●

●

●●

●●

●●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●●●

●

●●

●● ●

●●

●

●

●●

●

●●●

● ●

●●

●

●

●

●

● ●

●

●

● ●●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●● ●

●●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●●●

●

●

●

●

●

●●

● ●

●

●●

● ●

●

●

●

●

●●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

● ●

●●

●

●

●

●

● ●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●●

●●

●●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●●

●

● ●

●

●

●

●

●

●

●●

● ●

●

● ●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●● ●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●●

●

●

● ●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

● ●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●

●

● ●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●

●

●

●●

●

●

●●

● ● ●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

● ●

●●

●

●

●

●●

●

●

●●

●

●

●

●●●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

● ●●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●●

●

●

●●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●●

●

●●

●

●●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

● ●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

● ●

●●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●●

●

●●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●●

●

● ●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●●

●●

●

●

●

●

●

●

●

●●●●

●

●●●●

● ●●

●

●

●

●

●

●

●

●

●● ●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●● ●

● ●

●●

●

●●

●

●●

●

●

●●

●

●

●

● ●

●● ●

●●

●

●

● ●● ●

●

●

●

●●

●

●●

●

●●●

●

●

●

●●●

●

● ●

●

●●

●

●

●

●

●

●●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●●●

●

●

●

●

● ●

●

●●●

●

●

●

●●●

●

●●

●

●

●● ●●

●●● ●

●●●

●●

●●

●

●

●

● ●●

●

●

●●●

●

●●

●

●

●

●

●●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●

●

●●

●●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●

●●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●● ●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

● ●

●

● ●

●

●●

●

●

● ●

●

●

●●●

●

●●

●

●●

●

●

●

●

●●

●

●

● ●●

●

●

●

●

●

● ●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

● ●

●

●

●●

● ●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●● ●

●

● ●

●

●●

●

●

●

●●

●●

●

●●

●

●

● ●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●●

● ●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●●

●●

●

●

●

●●

●

● ●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●●

●

● ● ●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●● ●

●

●●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●

● ●●

●

●●

●●●●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●●

●

●

●

●

● ●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●●●

●

●

●

●

● ●●

●●●

●

●

●

●

●

●

●

●●

●● ●● ●●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

● ●●

●

●● ●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●● ●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●●

●

●

●

●

●

●

●

●

●●

●●

●

● ●●

●

●

●

●●

●●

●

●

●●●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

● ●

●

●

●

●

●●

●

●

●

● ●

●

● ●

● ●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●● ●

●

●

●

●●

●●

●

●●

●

●●

●

●

●●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●●

●

●

●

●

●

●●

●

●

●

● ●

●●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●●

●

●

●●●

●

●●

●

●

●●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

● ●

●●

● ●

●

●

●

●

●

●●●

●

●

●

●

●

●●●●

● ●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

−0.04 −0.02 0.00 0.02 0.04

0.5

1.0

1.5

2.0

2.5

3.0

Intertemporal choice

subject k − indifference k

RT [s

]

●

●

●●

●●●

●●

●● ●

●●

●

●● ● ●

●

●● ● ●● ● ●●

● ● ●● ●●

● ●●● ●●●●●

●

●

●● ● ●● ●●●●●●●

●● ● ●● ●●●●●●

●

● ●●

●

●

●

●● ●

● ●●

●

●●

●● ●

●

●●

●

●● ●●● ●● ●●

●

●

●

● ●●●● ●●

●● ●●

●

●●● ●●●●● ●● ●●●

● ●●●

● ● ●●● ●●● ●● ●● ●●

●● ●●● ●●● ●●● ●● ●● ● ●● ●● ● ●●●● ●● ●●● ●●● ●●●●● ● ●●●●● ●●●●●● ● ●●

●● ●●● ●● ●

●●●● ● ● ●●● ● ●● ● ● ●●●

●●

●●●

●●

●

●

●●

●●●

● ●●●●● ●●●

●

●

●

●●●

●●●

●

●

●

● ●●

●●●●

●● ● ●●

● ●●●

●●●

●● ● ●● ●

●● ●

●●

●

●●●●●

●● ●●●

●●● ●

●

● ●● ●●● ●●●●●● ●

●

●●● ●●●●

●

● ●●●

●

● ●● ●●● ● ●●●

●●

●

●

●

●●

●

● ●● ●●●●●

●●●● ●●●● ●●●●

●

●

●●

●

●

●

●● ●●●

●● ● ●●

● ●●

●

●

●● ● ●● ●●●● ●● ● ●● ●●● ●●● ●●●

● ●● ●● ●●●

●● ● ●●●●

●

● ●●

●

●●● ●● ●●

●●●

●

● ●

●●

●

●

●

●

●● ●

●

●

●●

●

●●

●

●●

●

●

●● ●● ●

●

● ●

●

●●● ●● ●● ●

●●

●●●

●

●

●

●● ● ●

●

●●●

●●●

●●● ●●

● ●

●

● ●

●

●●●

●●●

●

● ● ●

● ●

● ● ●●

●●●

● ●●

●

●

●● ●

●●●●

●●●

●

● ●●●

●●

●●

●●● ●● ●●

●

● ●●

●

●

●●

● ●●

●

●●

●

●

●

●●●● ●●

●

●●

●

●

●●●●

●

● ●●● ●

●

●

●●●

● ●●

●

●

●

●● ●● ●

●

●●●

●●●

● ●●

●

●●●● ● ●

● ● ●●●

●

●●●

●

●● ●● ●●

●

●● ●●● ●●●● ●●● ●● ● ●● ●

●

●●●● ●●●

●● ● ●

●

●●

● ●●●

●

●●

●

●● ●●● ●● ●● ●●

● ●●

●

● ● ●●●●

●● ●● ●● ●●

● ●●●●● ● ●

●●●● ●●●●

●● ●●●●● ●● ●● ●

●● ●●● ●●

●●

●● ●●● ●● ●●●

●●● ●●

●●

● ●●●● ● ●●●● ●●

●●● ●●●

●● ●●

●● ●● ● ●● ●● ● ●●●● ●

●●●●● ●●●

●●● ●● ●●● ●●● ●●● ●

●●●●

●● ●●●● ●●●●● ●

●●● ●● ●● ●

●●●● ●●

●● ●

●● ●●● ●●

●● ● ●●● ●●● ● ●●●

●●●●●

●

●●

●

● ●

●

●● ●●●

●●●●

●●● ●●

●●● ●● ●●●●●● ●● ●●● ●

●●●●

●● ●● ● ●● ●

●

●●●

● ● ●●●● ●● ●● ● ●●

●●● ●● ●

●●

●

● ●● ● ●●●● ●●● ●

●

●●● ●●

● ●●●

● ●●

●

●●

●●● ● ●● ●●

● ●● ●● ●

●

●● ●●

●●

● ●● ●

●●

●● ● ●●●●● ●●●● ●●● ●

●●

●●●●●● ●●

●

●● ●●●●●●● ●●

●●●● ●● ● ●●● ●●

●●●

●●

●●●●● ●●

● ●●●

●

● ●●● ●● ●● ● ●● ● ●● ●●●● ●●●● ●●● ●

●

●

●

●

●

●

●●●

● ●●

●

●

●●●

●●●

●●●

●

●●●

●

●●

● ●

●

●●●●●●

●● ●●●

● ●●●●

●

●●● ●● ●●●●

●●

●●

●

●

●

●

●●● ● ●● ● ●

●

●

●

●

●●

●● ●

●

●

●

● ●●

●

●

●● ●●● ● ●●●

●

● ●●

●

●

●●

●● ●●

●

●

●●●

●● ● ● ●● ●●●

●

●

●●●● ●● ● ●● ●●

● ●●●●● ● ●

●● ●●

● ●● ●

●●● ●●●●

●

●● ●

●●

● ●● ●● ●

●●●● ●●

●

●●● ● ●●● ●●● ●● ●

● ●●● ●● ● ● ●●● ● ●● ● ●●● ●●● ● ● ●●●●●

● ●

●

●●

●

●

●●●

● ●●

●

●●●●

●

●

●

●

●

● ●● ●

●

●●

●●

●

●

●

●

●

●●● ●●●

●

●●

● ●

●

●●

●

●● ●●

●●

●● ●● ● ●●

●

●●●

●●

●

●

●●●

●● ●

●●

●

●●

●

●

●● ●● ●●● ●●●

●

● ●

●

●● ●

●

●

●● ●●

●

● ●

●●●

●●●●●

●●●●●

●●

●

●●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

● ● ●●● ●●●● ●● ●●

●

●

●

●

●●

● ●●

●

●● ●● ●●●

●

●

●●

●

●● ● ● ●

●● ●● ●

●

●

● ●●● ●

●●●

● ●● ● ●●●● ● ●

●

●● ● ●

●●●

●

●●●●●

●

●

●

●

●

●

● ●

●

●

●●

●● ●

●●

●

●

●●

●●

●

●

●●

●

● ● ●●●

●

●

●

●●

●

●● ●●● ●●● ●●

●

●

●●●●

● ● ●●● ●

●

●●●●●● ●●● ●●

●

●

●

● ●

●

●●●●● ● ●●

●●

●●

●●

●

●● ●

●

●

●● ●● ●● ●●

●●●

● ●

●

●●

●

●●● ●●● ●

● ●

● ●

●

●

●

●●

●

●

●●

●●

●●● ●

●

●

●

●●

●

●●

●

● ●● ●●●● ●●

●●

●●

● ● ●●

●

●

●

●●

●●●●

●●● ●●

●●

●● ●●●

●

●

●●●

●

●●● ● ●

●

●

● ●● ●●

●● ●

●

●

●●

●●● ● ●● ●

●

●

●

●

●

●●

●

●

● ●● ●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●● ●●

● ●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●●

●

● ●

●

●●●

●

●

●●

● ● ●

●

●●●

●

●

●

●●●

●●●

●●

●

●●●●●●● ● ● ●

●●● ●

●

●●● ●

●

● ●● ●●●●

●● ●

● ●●

● ● ●

●●●

●

●● ●●● ●● ● ● ●●●●● ●●●

●

●

●

●

●● ●● ● ●●●●●●

●

●●●

●●●● ●● ●● ●● ●●●● ●

●●

●

●

●●●

●

●

● ●● ●●

● ●

●

● ●● ●●●●

● ●●● ● ●●●● ●● ●●

●

●●● ●● ●●●● ● ●●●●● ●●● ●

●●● ●● ●

●

●●

● ●●●

●●●

● ●● ●● ●●● ●

●●● ●●●●●●●

●●●● ●● ●●●●●●●●● ●

●

●

●●●●●

●●●

● ● ●● ●●●● ●● ●●

● ●● ●●● ●●

● ●●● ●● ●●●

●● ●●●● ● ●

●●●● ●● ●● ●●● ●●● ● ●●●●

●●●●●

● ●● ●●●● ●●● ●

●●

●● ●● ●● ●●

●●●●● ●●

●●● ●●●●● ●●●● ●

●●

●

●●●● ●●●● ●

●●●●● ●●

●● ●●● ● ●● ●●

●●●● ●

● ● ●●

●●●●

●●●●●● ●●●●●

●

●

●●

●

●

●

●●

●

●

●●●

●

●

●

●● ● ●

●

● ●●

●

●

●

●

●

● ●●

●

●● ●● ●●●●●

●●

●●

●

● ●●

●●

●

●

●

● ●

●●

●● ●● ●

●●● ●● ●

●

●

●●

●

●

●●●●

●

●

●●●

●●

●

●●

●●

●

●

●

●

●

● ●●●

●

●●

●

●●

●

● ●● ●●● ●

● ●●

●

●

●●●

●● ●

●● ● ●●

●

●● ●

●

●

●

●

● ● ● ●●● ●

●

●●●

●

●

●●● ●

●

●

●

●●

●

●●

●●

● ●●●

● ●

●

●● ●● ● ●●

●●●●

●

●

●

●●●

●●

●●

●

●

● ●

●

●● ●● ●

●●

●

● ● ●●

●● ●●

●

● ●

●●●

●

●●●

●●

●

●●

●●

●●● ●

● ●●●●●

● ●●

●● ●●● ●

●

●●

●

●

●●●

●

●

● ●●

●●

●

●●●

●

●●

●

●

●●●

●●

●

●● ●●● ●

●

●● ● ●

●

● ●●●●

●

● ●

●

●●

● ●

●

●●●

●

●

●

●

●

●

●●●● ●

● ●● ●

●

●●

●

●

●

●

●●

●

● ●●●

● ●

●

●● ●●

●●●

●●

●

●

●

● ●●● ●

●●

●

●●

●

●

● ●

●●

●●●●●●●

● ●

●

●●

●● ●

●

●

●●●

●● ●●● ●●●●●

●● ●

●

● ●●

●

●●

●●

●● ●●

●●

●

●● ●●

●●

●● ● ●

●●●●●

●

●●

●● ●● ●● ●

●●

●●

●

●●

●

●

●

●

●

●

●●

●

●●●

●

●

● ●●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●● ●●

●●

●

● ●● ●● ●

●

●

●

●

●

●

●

●

●● ● ●●●

●

●●●● ●●

●

● ● ●

●

●● ●

●

● ●●

●●

●●

●● ●● ●

●

●

●

● ●●●

●

●●● ● ●●

● ●

●

●●

●

●

●●●

●

●

●●● ●●●

●

●●

●●

● ●

●

●● ●●

●

●

●●● ●●

●

●●

●

●

●

●

●●●

●●●

●

●

●

●●●

●●

●

●●●●

● ●●●

● ● ● ●●

● ●● ●

●

● ● ●

●

● ●

●

●

●

● ●●●● ●

●

●

●●

●●

●

● ● ●●

●

●●

●●●

●

●

●

●

●

●

●●●●

●●

●●

●

●

●●●●

● ●●

●

● ●

● ●

●

●●

●

●

●

●

●

●●● ● ●

●

● ●

●●

● ●● ●

●

●

●●

●● ●

●●

●● ●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●●● ● ●

●●

●●

●●

● ●●

●●

●●

● ●●

●

●●●

●

●●

●

●

●●

●

●●● ●

●

●● ●

●

●

●●

●

● ●

●

●●

●●

●

●● ●

●

●

●

●

●

●

●

●●● ●● ●

●

●●● ● ●

●●

●

●● ●●

●●

●●●●

●

●●● ● ●

●

● ●

●

●●

●●

●● ●●● ●●●●● ●●

●

● ●●●● ●●● ●●●● ●● ●●

● ●●● ●●● ●●● ●●●●●● ●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●●●

●●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●●

●●

●

● ●

●

●●

●●

●

●

●

●●

●

●

●

●●●●

●

● ●

●

●

●●

● ●

●●●

●●

●●

●●●

●●

● ●●

●●

●● ●●●

●●

●

●

● ●● ●● ●●

●

●

●●●●

●●●

●●

●

●●

●

● ●

●

●●

● ●● ●●

●●● ●●

●● ●●●

●●

●

● ●●●●● ●● ●●● ●● ●● ●● ● ●● ●●●● ●●●● ●●●

● ● ●●● ● ●● ●● ● ●● ●●● ●● ●

●●●● ●● ● ●● ●●●● ●● ●

●

●●

●

●●

●

●

●●

●

●●

●

●

●

●●●● ●● ●●● ●●

●

● ●●●●

● ●●

●

●●●●●● ●

●●●● ●●

●●● ●●● ●● ●

●●

●

●●●

●●

●● ●●● ●●●

●● ●●

●

●●●●●

●● ● ● ●●● ●● ●●

●●● ● ●●●● ●● ●●

● ●●

●●●

●

● ●●●

●

●● ●

● ●

●

●

●

●●

● ●●●

●●●●● ●●●

●●● ●●

●

●●● ●●●● ●●●● ●● ●●●

●● ●● ● ●●●●

●●●●●● ●● ●● ●● ●● ●●●● ●●●● ●● ●●

●

●● ● ● ●●●● ●●● ●●● ● ●●●●

●

●●● ●

●

●

●● ●●

●

●●●

●●●●

●● ●● ●

●●●

●

●

●

●

●

●

●

●● ●

●●

●

●

●

●●●●

●

●●

●

●● ●●● ● ●

● ●●● ●● ●●

●

●

●●

● ●●● ●●●

●●●

●

●●● ●

●

●

● ●●

● ●● ● ● ●●

●● ●

●

●● ●● ●●●

●

●● ●

●

● ●●●●● ●●

●● ●● ●● ●●

●

●● ● ● ●●

● ●

●●

●●●●● ●

●

●● ●●●●●

●

●● ●● ●● ●●● ●● ●●● ●●

●

●●●● ●

●

●● ●●●

● ●

●●

● ●● ●● ●●●

●●

●●

●●

●● ●●

●●● ●

●●●● ●●

● ● ●● ● ●●● ●●●

●●

●●

●

●

●

●

●●

● ●

●

●

●

●

●● ●●

●

●

●

●● ●

●●●

●

● ●

●

●

● ● ● ●●

●

● ●

●

●

●

● ●●●

●●

●

●●

●●

●●

●●

●●●

● ●

●

●

●

● ●●●●●

●●

●●

●●●

●● ●

●

●●

●●

●

●●

●

●

● ●●●

●

●●

●

●●

●

●

●●

●

●●●

● ●●

●

●●

●●

●●●●● ●

●

●● ●

●

●

● ●●

●

●

●

●● ●● ●●

●

●●

●

●●●

●

●● ● ●●

●●●

●

●●●

●

●

●●●

●

● ●

● ●

●

●

●●

●● ●●●● ●●

●●

●

●● ●●● ●

●●

●

●●

● ●●

● ●●●

● ● ●● ●

●

● ●●

●●

●●

●●●●●

●●●

●●

●●●

●●

●●●

●

●●

●●●

●●

●

●●

●● ●● ● ●●●

●● ●● ●

●● ●

●

●● ●● ●●

● ● ●●

●●

●

●● ● ●●

●

●● ●● ● ●●●

●● ●●

●

●●

●●●●● ● ●

●

●● ●● ●● ●●

●● ●● ●● ●●

●

●

●

●

● ●

●

● ●

●●

●●

●

● ●● ● ● ●●● ●●●● ● ●

●● ●●

●

●● ●● ●●●

●● ●●●●●●

●● ●●

●●●

● ●

● ●●● ●● ●

●●

●

●● ●● ●●● ●

●●

●●●

●

● ●●●●

●●

●

● ●●● ●●● ●●● ● ●

●● ● ●● ●●

●● ●●● ● ●● ● ●●● ● ●●● ● ●● ●

●●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●● ●

●●

●

● ●

●

●

●

●

● ●●

●

●

●●

●

●

●●

●●

●

●

●●●

●

●●

● ●

●

●

●●

●●

●

●

● ●

●

●

●

● ●● ●●●●

●

●●

●

● ●●

●

●

●

●

●●●● ●

●

●●●

●

●●●●●●●

●●

● ●●● ●

●●●

●

●●●

●●● ●

●●

●

● ● ●● ●●●

● ●●●●

●●●●●

●

●

●●

●

●● ●● ●●●● ●● ● ●●●● ●● ●●

●●

● ●● ●●

●

●● ●● ●● ● ●●●● ●

●

●

● ● ●● ●●

● ●●● ●●●●

● ● ●● ● ●● ●●● ●●●●● ● ●● ●● ● ●●●●● ● ●●●● ●● ● ●●

●

●

●

●●● ● ●●●●● ● ●● ●● ●● ●●● ●

●● ●● ●●●

●● ●●● ● ●● ●●

●

● ●●●●● ●● ●●●●● ● ●

●

● ●●

●

●●● ● ●●●●● ●● ●●●

●●

●● ●●●● ●●● ●●●●

● ●

●

● ●●●● ●●●● ●

●

●● ●●● ●●

●●● ●●●● ●●

●

●● ● ●● ●

●●

●

●

●

●

●

●

●●● ● ●●●

●● ● ●

●●

●

●

●

● ●●

●●●

●●

● ●●

●●

●

●

● ●

●

●●

●

●●

●

●●

● ●

●

●●●

●●

●

●

●

●

●

●

●●

●

●

●

● ●●●

●

●●

●

●● ●

●●● ●●●

●

●●

●

●

●

●● ●●●

●●

●● ●

●●

● ●● ●●● ●

●●

● ●

●●

● ●

●●●● ●●

●●

● ●●●

●●

● ●

●●● ●

●

●

●●

●●

●● ● ●

●●● ●

●

●●● ●

●

● ●

●

●● ●

●●

●● ●

●●

●●

●

●

●

●●● ● ●●●

●●●

●●

●●●●

●

●●

●

● ●●●

● ●● ●●

●

●

●●

●●

● ●●● ● ● ●●

●

●●●●

●

●●

●

● ● ●●●

●

●

●● ●●●●●● ●●

●

●● ●●● ●● ● ●●

●

● ●●●●

●

●● ●●●

●

●

● ● ●● ●●●●

● ●●

●

●● ●● ●●

●● ●●

●

●

●●● ●●● ●●● ●●● ●●

●●● ●

●● ● ●● ●●● ● ●

● ●

●

● ●● ●

●●●● ●●●●● ● ●●● ●●● ●● ●● ●● ●● ● ●●●●

● ●●● ●● ●● ●● ●●

● ●●● ●● ●●● ●● ●●●

● ●●● ●●

●

●

●

● ● ●●● ●●● ●● ●●

●●

●

●

●●●

●

●●

●

●●● ●●● ●

●●●● ●●

●● ●

●● ● ●

●●●● ●● ●●● ●●

●

●

●

●

●●

●●●●

●

●

●●

●●

●

● ●

●

●●● ●

●● ●

●

●

●●

●●

●

●●

●

●

●●●

●

●●● ●

●

●

●

●

●

●

●

●● ●●

●

●●●

●●

●

●●

●

●●

●●

●●

● ●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●●●

●

●●

●

●

●

●●● ●

●

●● ●●

● ●●● ●

●

● ●●

●

●●

●●● ●

● ●● ●● ●●●

●●

● ●●●● ●●●

●

●

●

●●

●

● ●●

● ●

●

●● ●● ● ●

●

●● ●

●

● ●●●

●

●●

● ● ●●

●

●

●●●

● ●

●●

●●

●

●

●

●●

● ●●

●

●

●●

●

●

● ●●●● ●

●●

●

● ● ●●● ●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●●

●

●

●●

●●●

●●

●

●●

●

●●

●

●

●

●

●●

●

● ●

●

● ●

●

●

●

●

●●

●●

●●

●●●●

●

●

●

●

●

● ●●

●

●

●●

●

●●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

● ●● ●●

● ●●

●

●

●

● ●● ●●

●

●

●●

●●

●● ● ●● ●●

●

●● ●●

● ●●● ●●● ●● ●

●

● ●●●

●●●

●●

● ● ●●

●●

●

●● ● ●● ●●

● ● ●●●●● ●●●● ●●●

●

●

●●● ● ●

●●

●●

●

●●

●●● ●

●

● ●●

●

●

●●

●

● ●● ●●

●

●●

●●

●●●

●●

●●

●●

●● ●

●

●●●

●●

●●

●● ●●● ●●●● ●

●

●● ●●

●

●

●

●●●

●

●

●

●

●●

●● ● ●

●● ●●●●● ● ●● ●

● ●●●● ●

●●

●●●

●

●●

●●●● ●

●

●

●●●

●

●●●●●●● ●●

●●●●●● ●●●

● ●●● ●●● ● ● ●●● ●● ●● ● ●● ●● ●

●

● ●●●

●

●●● ● ●●●● ●

●●●●●● ● ●

●●

●

●

●●● ●●●●

● ●

●

●●●●●●

●● ●● ● ●● ●● ●●

●●●

●

● ●●●

●●● ●●● ● ●● ● ●● ●●

●

●

●● ●

●●●

●

●●

●●

●

●● ●●●

● ● ●● ●●● ● ●●● ●●

●

●

●●

●

● ●●●● ● ●●●

●●

●

●● ● ●

●●● ●●●

● ●●

● ●● ●

●

● ●● ●● ●●●

●

● ●

●

● ● ●●●

●

● ●●●●●●● ●●

●

● ●●

●

● ●●●● ●● ●

● ●● ● ●

●

●● ●●

●

● ●●

●● ●●

●

●●● ●●●● ● ●● ●●

●

●● ●● ●●● ●

●

●● ●● ●●

● ●

●

●

●

●●●●● ● ● ●●●●● ●●● ●●

●

●

●●

●●● ●●● ●●●●

● ●

●

●● ● ●●

●

●● ●●

●

●●

●● ●●●

●●●

● ●●

● ● ●● ●●●

●

●● ●● ●

●

●●● ● ●

●●

●●●

●●

●● ●●●● ●

●

● ●●

●

●●●● ●

●

●

●●●

●●

●●●● ●

● ●●

●

● ●● ● ●●

●

●●

● ●●● ●●● ●

●

●●●

● ●●●●

●● ●●●●●●

●●● ● ●

●

●

●●●●●

●

●

●● ● ●●●

●

●

●

● ●●

●● ●

●

●

●●● ●●● ●●● ●

●●●

●● ●● ●●● ● ●

●●●

● ●●

●●● ●

●

●

●

●●

●●

●

●

●

●● ●● ●●

●

●

●●●●

●●● ●●

●●

●●● ●

●●

●●● ●● ●● ● ●● ●●

●●●●● ●● ●●● ●

●●

●●●●● ● ●●

●

●●●●●

●

●● ● ●●●●●●● ●● ●

●●

●●

●●

●●

●●

●● ● ● ●

●●

●●

●

●

●●

●

●

●●

●●● ● ●

●

●●

●

●

● ●

●● ●●●

● ●●

● ● ●●● ●

●●●

●●●●●●●

●

●● ●

●●●● ●●●● ●● ●● ●●● ●● ●●

●● ●● ●●●

●●●

●

●●

●● ●●●●

●● ●● ● ●● ●● ●● ●● ●●● ● ● ●● ●● ● ●●● ●●● ●●● ● ●●●● ●● ● ●●● ●● ●

●●●● ●● ●●● ●● ●● ●● ●●

●●●● ●● ● ●●● ●●● ●●●●●●● ●● ●● ●● ● ●●●● ● ●●●● ● ●●● ●●● ● ●●● ●●● ●●●● ●●● ● ●

● ●●● ● ●●●● ●●●● ●

●

●

●●

●●

●

●

●●

●

●●

●

●●●●

●

●● ●●

●

●●

●

●● ●● ●●●●

●●●

● ● ●

●

●●

●

●●●

●

●●●●

● ●● ● ●

●

● ●● ●●

● ●

●●

● ●● ●

●

● ●

●

●

●

●

●

●

●

●●

●

●●●

●●

●●

● ●

●

●●

●●

●

●

●●

●

● ● ● ●

●●

●●

●●

●

●●

●●

●

●

●●●● ●

●●●●

●

●●

●●●●● ●

●

●

●

● ●

●

●●●●

●

● ● ●●● ●● ●● ● ●●●●●●

●

●●●

●●●● ●●● ●●●●● ● ●●●● ● ●●● ●● ●● ●●●

●

●

●● ●●●●

● ●●● ●●●

●●● ●

●●● ●

●●●●

●

●

●

●

●●

● ●●●

●

●

● ●

●

●●

●

●●●●● ●●

●

●●● ●

●

●●

●●● ●

●

●● ●●●●

●

●

●● ●●●

●

● ● ●

●

● ●●●●

●●● ●

●

●●●

●● ● ●

●●●●

●

●●

●●●

● ●

●

●●●

●

●

●●

●●

●●●

●

●

●● ●

●

● ●●●●●

●●●● ●●●● ●●● ●●

●●●

● ●

●

●● ●●●●●

●●

●● ● ● ●●

●● ● ●● ●●●

●

● ● ●● ●● ●● ●●●● ●

●● ●

●● ●●●●●● ● ●●● ●● ●● ● ●●● ● ●

●●

●●

●

●

●

● ●●● ●

●●

●

●●

●

●

●● ●● ●● ● ●●● ● ●●

●

●

●●

●●

●

●

●●●

●● ●●

●

●

●●

●●

●●

●●

●

●●

●

●●

●

●●

●

● ● ●

●●● ●

●

●●

●

●

●● ●●● ● ●●● ●●

●● ●

●●● ●● ●

●

●

● ● ●● ●● ●●●

●●●● ●● ● ●● ●● ●●●● ● ●●

● ●●

● ●● ●●

●

●● ●

● ●●

●●●●● ●● ●●● ● ●● ●● ●●

● ● ●●●● ● ● ●● ●●

●●

● ●●● ●● ●●●● ●●●● ●

●●● ● ●● ●● ●●●

●●●●

● ● ●●

●● ● ● ●

●● ●● ●●● ●●● ● ●●● ● ●●● ●

●●● ●●●

●● ●● ●●●● ●●●● ●● ●● ●●

●●

●

●●●

●● ●

●

●

●●●

●

●

●

●

●

●

●●

●●●

●

●

●

●● ●●

●

●●

●●

●

● ● ●

●

●● ●● ●● ● ●

●

●●

●●

●

●●●●●● ● ●●

●

●●

●●●

●

●

●

●● ●

●

●

●●

●

●●

●●

●

●

●

●●

● ●● ●

●

●● ● ●●●

● ● ●●

●

●●

●

●

●

●

●

●●

●

●●●

●●

●

●●

●● ●● ●●●

●●● ●●●●

●●

●

●●

●

● ●●●●

●

● ●●●

●

●● ●

●

●

●

●●

●

●

●●

● ●● ●● ●●

●

●

●●● ●

●●●●●

● ● ●

●

● ●●●● ●●●

●

●● ●● ●●

● ●● ●●● ●●

●

● ●●● ●● ● ●● ●

●

●

●●

●

●●●●

●●●●

●

●

●● ●

●

●● ●●●●

●

●

●

● ●●●●

●●

●●

●● ●●

●

●

●

●

●●●● ●

●

●●●

●●

●

●

●

●●●

●

●●

●

●●

●

●●●

●

● ●

●

● ●

●

●●

●●●

●

● ●

●

●●

●●

●

●

●

●●●

●

●

●●

●●●

●

●

●

●

● ●●

●

●

●

●●

●

●●●

●

●●● ●●

●●● ● ●● ●● ●

●

●

●

●●●● ●●●●

●

●●● ●●

●

●

●

●●●●●● ●● ●

●

●

●●●

●

●●

●

● ●● ●●

●● ●●● ●●

●●

●

●●● ●● ●●

●● ●●

●●

●● ● ●

●

●

●

●●●● ●●● ● ● ●

●●●●●

●

●

●●● ●●

● ●●●

●

●

● ●●● ●

●●●

●●●●

●

● ●

●● ●

●

●● ●●● ●● ●●

●

●●●

●

● ●● ●●● ●● ●

●●● ●● ●● ●● ● ●●

● ●

●

●

● ●

●

●●

●● ●●● ●● ●●●

● ●● ●● ●●

●

●●●● ● ●

●●● ●● ● ●●●●●

●

●●● ● ●●● ● ●● ●● ●●

●

●

●

●●

●●●●● ●● ● ●●

● ● ●●●● ● ●● ●●● ●● ● ●● ●● ●● ●●

●● ●● ● ●●● ●●●

●●● ● ●●

●●● ●

●● ●●●

●●● ●●● ● ●●● ● ●● ●●●

● ●●● ● ●●●● ●● ● ●

●

●● ●●

●

●●

●

●●

●

● ●

●●

●●

●

●

●

●

● ● ●●

● ●● ●●●

●

●

●

●●

●

●

●●●

●●●

●

●●●

●

●●

●

●●

●

●

●

●

●

●●

●●

●● ●

●

●● ●●●● ●

●●

●

● ● ●

●

●●

●

●●

●

●

●

●

●

●

● ●●

●●

● ●

●

● ●

●

●

●

● ●●●●

●

●●

●● ●

●●

● ●●●●

●

●

●●●●● ●

●

●

●

●● ● ●

●●

●

●●

●

●●

●

● ●●●●

●●●● ●

●

●●

●●

●

●●

●●●● ●

●

●●

●

●● ●●● ●● ●●

● ●●●

●

●

●●●● ●

●

● ●

●

●●●●●

●

●●●●●●● ●●

●● ●

●

●

●

● ●

●●●●●●

● ●

●

●

●●

●

●

● ●

●●

●●●

●

●

●

●●●●

●

● ●●

●

●

●

●

● ●●● ●● ●●

●

● ●●

●

●●

●● ●

● ●

●●

●

●●●●

●

●●

●

●● ●●

●

● ●● ●●●

●

●●●

●● ●● ●

●

● ●●●●● ● ●●●● ●●●

●●● ● ●

●

●

●

● ●●●●● ●

●● ●

● ●● ●

●●

●

●● ●

●

●●● ●●●

●● ● ●● ●●●● ●●

●●●

●●

●

●●●●

● ●●● ●●●

●●

● ● ●●●●●●● ●●●● ●●● ●●

●

● ●●● ●●●● ● ●●●

●●●● ●●●●

●● ●● ●●● ●

●●●● ●●●● ●●

●●● ● ●●

●

−6 −4 −2 0 2 4 6

010

2030

Risk choice

subject λ − indifference λ

RT [s

]

33

Figure7.Exampleofanindividualsubject’sRT-basedparameterestimation.Theplotshows RTs in all trials as a function of the indifference parameter value on that trial.ObservationsinthetopRTdecileareshowninred.TheredtriangleshowsthelongestRTfor the subject. The solid vertical red line shows the subject’s choice-based parameterestimate.ThedottedverticalredlineshowstheaverageindifferencevalueforthetopRTdecileapproach.Thedottedgrey lineshows the local regression fit (LOWESS, smoothingparameter=0.5).

Withthese facts inmind,wesetaboutconstructingamethodforusingRTsto infera

subject’sindifferencepoint.Clearly,focusingontheslowesttrialswouldyieldlessbiased

estimates of subjects’ indifference points. However, using too few slow trials would

increase the variance of those estimates. We settled on a simplemethod that uses the

slowest10%ofasubject’schoices,thoughwealsoexploredothercutoffs(AppendixD).

Inshort,ourestimationalgorithmforanindividualsubjectincludesthefollowingsteps:

(1) identify trialswith RTs in the upper 10% (the slowest decile); (2) for each of these

trials, calculate the value of the utility-function parameter that would make the subject

indifferentbetweenthetwoalternatives;(3)averagethesevaluestogettheestimateofthe

subject’s utility-function parameter (see Figure 7 for an example and Appendix A for

estimation details). It is important to note that this method puts bounds on possible

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

−1.0 −0.5 0.0 0.5 1.0

1.0

1.5

2.0

indifference β

RT

[s]

●

●

●

●

●

●

●●

34

parameterestimates:theaverageofthehighest10%ofallpossible indifferencevaluesis

the upper bound, while the average of the lowest 10% of the indifference values is the

lower bound. Thus it is crucial to choose choice problems with enough range in their

indifferencevaluestomorethanspantherangeofvaluesinthepopulation.

Again, the parameters estimated using this method were correlated with the same

parameters estimated purely from the choice data, providing a better fit than the DDM

approach (social choice 𝛼: r = 0.44, p = 0.02; social choice 𝛽: r = 0.56, p = 0.001;

intertemporal choice𝑘: r = 0.71, p < 0.001; risky choice𝜆: r = 0.64, p < 0.001; Pearson

correlations; Figure 8, see Appendix A for estimation details for both methods).

Furthermore, these parameters provided prediction accuracy that was better than an

informedbaseline(seeSection5.5)inthreeoutoffourcases(excludingsocialchoice𝛼).In

all cases, a random10%sampleof trialsproducedestimates thatwerenotameaningful

predictorof thechoice-basedparametervalues (since theseestimatesare justameanof

10% random indifference points).14 As noted previously, the RT-based estimations have

upperandlowerboundsduetoaveragingovera10-percentsampleoftrialsandthusare

notabletocapturesomeoutliers(e.g.seeFigure8,bottomleftpanel).Furthermore, the

numberof“extreme”indifferencepointsinthedatasetsthatweconsideredislow,biasing

theRT-basedestimatestowardsthemiddle.

14Wedrewarandom10%sample1000times foreach individual ineachparameterdatasetandestimated the correlation between the average indifference point and the true parameter value(socialchoice𝛼:meanPearsonr=-0.01;socialchoice𝛽:r=0.01;intertemporalchoice𝑘:r=-0.02;riskychoice𝜆:r=-0.003).

35

Figure 8. The top RT decile estimates of subjects’ utility function parameters..Subject-level correlation (Pearson) between parameters estimated from choice data andresponse time (RT) data using trials with RTs in the top decile. The solid lines are 45degrees.ThedottedredlinesshowtheboundsonRTparameterestimations.

5.6.2.Localregressionmethod

Asthetop10%approachonlyutilizespartofthedata,wedevelopedanotherapproach

basedon thepeaks inRTs, this timeusingall theavailableRTdata.The local regression

(LOWESS) method also uses the “revealed indifference” approach: for each individual

subject,werunalocalpolynomialregressionofRTsontheindifferenceparametervalues

and use that regression to identify the indifference value that produces the highest

predictedRT(seeFigure7foranexampleandAppendixAfordetails).Asthepeakofthis

●●

●

●●

●●●

●●●●

●

●

●●

●

●●

●●●

●

●●

●

●

●

●

−1.0 −0.5 0.0 0.5 1.0

−1.0

−0.5

0.0

0.5

1.0

Social choice (α)

α estimated from choices

α e

stim

ated

from

RTs

r = 0.44

p = 0.02 ●●

●

●

●●

●●

●

●

●

●●

●●

●

●

●●

●

●

●●●

●

●●

● ●

●

−1.0 −0.5 0.0 0.5 1.0

−1.0

−0.5

0.0

0.5

1.0

Social choice (β)

β estimated from choices

β es

timat

ed fr

om R

Ts

r = 0.56p = 0.001

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●●

●

●●

●

●

●

●●●

●

●

●

●

●●

●

0.00 0.01 0.02 0.03 0.04

0.00

0.01

0.02

0.03

0.04

Intertemporal choice

k estimated from choices

k es

timat

ed fr

om R

Ts

r = 0.71

p < 0.000

●

●●

●

●● ●●

●

●● ●●

●

●

●

●●

● ●●

●●

●

●

● ●●

●●●●

●●

● ●

0 1 2 3 4 5 6

01

23

45

6

Risk choice

λ estimated from choices

λ es

timat

ed fr

om R

Ts

r = 0.64p < 0.000

36

line is typically close to the choice-basedestimateand theobservationswith thehighest

RTs, this method is quite similar in its predictions to the top-RT-decile method. This

methodrequirestheresearchertochoosethelocalregressionsmoothingparameter.Here

weuseavalueof0.5asitproducesthebestresultsacrossallofthedatasets(thoughother

valuescanworkbetterforspecificdatasets;seeAppendixC).

Althoughthisapproachgenerallyproducesresultssimilartothetopdecileapproach,it

can be affected by outliers (e.g. sparse data and unusually high RTs around extreme

indifference points) and thus sometimes misestimates individual subject parameters,

producing correlations that are in some casesworse than thoseproducedby the topRT

decile approach: social choice𝛼: r = 0.12, p = 0.52; social choice𝛽: r = 0.6, p = 0.001;

intertemporal choice 𝑘: r = 0.44, p = 0.01; risky choice 𝜆: r = 0.88, p < 0.001; Pearson

correlations.Thiscanbemitigatedbyusingaspecificsmoothingparameterforeachdata

set,butourgoalwastoidentifyamethodthatworkswellacrossallthedatasets.

5.7.Choicereversals

Finally, we wish to explore one additional set of predictions from the revealed

indifference approach. We know that when subjects are closer to indifference, their

choicesbecomelesspredictable.Wealsoknowthatsubjectsgenerallychoosemoreslowly

theclosertheyaretoindifference.Therefore,wehypothesizedthatslowerchoicesareless

predictableandthereforealsolesslikelytoberepeated.

Inallthreedatasets,thechoice-estimatedutilitymodelwassignificantlylessconsistent

withlong-RTchoicesthanwithshort-RTchoices(basedonamediansplitwithinsubject):

80%vs 89% (p <0.001) in the risky choice experiment, 71%vs 79% (p < 0.001) in the

37

intertemporalchoiceexperiment,88%vs94%(p=0.008)and90%vs96%(p<0.001)in

thedictatorgameexperiment;p-valuesdenoteWilcoxonsignedrank test significanceon

thesubjectlevel.

Asecond,morenuancedfeatureofSSMsisthatthey(mostly)predictslowerrors,even

conditioningondifficulty.Whiletherearesomeexceptions(seeRatcliff&McKoon2008),

the“slowerror”phenomenoniscommonlyobservedinexperimentsandaccountingforit

wasabreakthroughintheliterature(Ratcliff1978).Totestforslowerrorsinourdata,we

ran mixed-effects regressions of choice consistency on the RTs and the absolute utility

differencebetween the twooptions. In all caseswe founda strongnegative relationship

betweentheRTsandthechoiceconsistency(slowerchoice=lessconsistent)(fixedeffects

of RTs: social choice 𝛼: z = -2.62, p = 0.009; social choice 𝛽: z = -3.3, p < 0.001,

intertemporalchoice:z=-5.28,p<0.001,riskchoice:z=-5.35,p<0.001).Thusweindeed

observeevidenceforslowerrorsinallofourtasks.

In two of the datasets (intertemporal choice and non-adaptive risk choice) subjects

faced thesamesetofdecisionproblems twice.Thisallowedus toperformamoredirect

testof theslowerrorhypothesisbyseeingwhetherslowdecisions in the firstencounter

weremorelikelytobereversedonthesecondencounter.

In the intertemporal choice experiment, themedian RT for a later-reversed decision

was1.36 s, compared to1.17 s for a later-repeateddecision. Amixed-effects regression

effect of first-choiceRTon choice reversal, controlling for the absoluteutility difference,

washighly significant (z=4.04,p<0.001).Thedifferencewaseven stronger in the risk

choice experiment: subsequently reversed choices took 2.36 s versus only 1.4 s for

subsequently repeated choices.Again, amixed-effects regression revealed thatRTwas a

38

significant predictor of subsequent choice reversals (controlling for absolute utility

difference,z=5.2,p<0.001).

6Discussion

Herewehavedemonstratedaproof-of-conceptforthemethodofrevealedindifference.

The method of revealed indifference contrasts with the standard method of revealed

preference,byusingresponse times (RTs) rather thanchoices to inferpreferences. This

newmethodreliesonthefactthatpeoplegenerallytakelongertodecideastheyapproach

indifference. Using datasets from three different choice domains (risk, temporal, and

social)weestablishedthatpreferencesarehighlypredictablefromRTsalone.Finally,we

also found that longRTs are predictive of choicemistakes, as capturedby inconsistency

withtheestimatedutilityfunctionandlaterpreferencereversals.

Throughout the paper we have highlighted ways in which we think RTs may be

important foreconomists. First,usingRTsmayallowus toestimateagents’preferences

usingvery shortandsimpledecision tasks, evena singlebinary-choiceproblem(Section

5.3). For example, if youwant to knowwhether peoplewill buy your product for $50,

knowingthattheywouldbuyitfor$30isnotveryusefulinformation.However,ifyoualso

knowthat theywereveryquick tosayyesat$30,youmight reasonably infer thatmany

wouldstillpurchasetheproductfor$50.

Second,usingRTscanhelpustorecoverpreferenceswhen,evenwithmultiplechoice

problems,someagentsalwaysgive thesameresponse,meaning thatwecouldotherwise

onlyputboundsontheirpreferences(Section5.4).

39

Third,thefactthatRTscanbeusedtoinferpreferenceswhenchoicesareunobservable

oruninformative (Sections5.5&5.6) isan importantpoint for thosewhoareconcerned

about private information, mechanism design, etc. For instance, while voters are very

concernedabouttheconfidentialityoftheirchoices,theymaynotbethinkingaboutwhat

their time in thevotingboothmight conveyabout them. Inanelectionwheremostofa

community’s voters strongly favor one candidate, a long stop in the voting booth may

signal dissent. Another famous example from outside of economics is the implicit

associationtest(IAT),wheresubjects’RTsareusedtoinferpersonalitytraits(e.g.racism)

thatthesubjectswouldotherwiseneveradmittoorevenbeawareof(Greenwald,McGhee,

&Schwartz,1998).Thusprotectingprivacymayinvolvemorethansimplymaskingchoice

outcomes.

Fourth,ourworkhighlightsamethodfordetectingchoiceerrors. Whilethestandard

revealed preference approach must equate preferences and choices, the revealed

indifferenceapproachallowsustoidentifychoicesthataremorelikelytohavebeenmade

bymistake,orattheveryleast,withverylowconfidence.ThusRTsmayplayanimportant

normativeroleinestablishinghowconfidentlywecansaythatagivenchoicetrulyreveals

thatagent’sunderlyingpreference.

InthispaperwehavedescribedaframeworkthatmathematicallylinksRTsandchoices

tounderlyingpreferences.Thisframeworkisnotaheuristic;infact,itarisesastheoptimal

solutiontoasequentialsamplingproblem,wherewhatissampledarestochasticsignalsof

the underlying true preference. In addition to their normative appeal, these SSMs have

enjoyedmuchempirical success incapturingchoiceprobabilitiesandRTdistributions in

many domains, including economic choice (Fudenberg et al. 2015; Krajbich et al. 2010,

40

2014; Ratcliff and McKoon 2008; Webb 2013; Woodford 2014). Moreover, they are

appealing from a neuroscience perspective, as SSMs are biologically plausible and align

wellwithneuralrecordingsinbothhumansandotheranimals(Bogaczetal.2009;Polania

etal.2014).However,itisimportanttonotethatSSMsarenotuniqueintheirprediction

ofslowdecisionscorrespondingwithindifference.Forexample,thisisalsoafeatureofthe

directedcognitionmodel(Gabaix&Laibson,2005),thoughthatmodelperformedslightly

worseinourmodelcomparison.

OnequestionthatarisesfromourresultsiswhatistheoptimalwaytomakeuseofRTs

inordertoinfersubjects’preferences?Ofthemethodswetested,thetop10%ruleseems

toworkverywellacrossourdatasetsandisaneasymethodtouseinpractice.However,

this cutoffwill generally depend on the number of trials in a particular experiment: the

fewerthetrials,thelargerthetoppercentileneedstobe.Foreachofthethreedatasets,we

calculatedtheoptimalpercentilecutoffandfoundthatthetop10-20%RTsgeneratedthe

bestpredictions.Astheindividual-trialRTinformationisnoisy(fasttrialscanbebotheasy

anddifficult,butslowtrialsarealmostalwaysdifficult),usingfewertrialsproduceshigher

variancepredictions,whilemoretrialsmayintroducebias.

ThereareofcourselimitationstousingRTstoinferindifference.Itimportanttokeep

inmindthatotherfactorsmayinfluenceRTsinadditiontostrength-of-preference,suchas

complexity,stakesize,andtrialnumber(Moffatt2005;Krajbichetal.2015b).Aswithany

analysis, it is important to control or account for these factors in order tomaximize the

chanceofsuccess.

A second potential criticism of these findings is that we have focused on repeated

decisions which are made quite quickly (1-3 seconds on average) and so may not be

41

representativeof“realworld”decisionsormaybebeingmadeusingsimpleheuristics.We

haveseveralresponsestothiscriticism.First,theseareallmulti-attributechoiceproblems

and so it is unclear what simple heuristics subjects could be using. Second, the use of

simpleheuristicswouldonlyimpairourabilitytoestimatepreferencesfromRTs,sincein

thosecasesthereshouldbenorelationshipbetweenstrength-of-preferenceandRT. The

less people can rely on heuristics and instead have to evaluate the alternatives to

determinewhichisthebest,themoreeffectivethemethodofrevealedindifferenceshould

be.Third,wewouldarguethatmany,ifnotmost,realworlddecisionsareminorvariants

of otherdecisions thatwemake repeatedly over the courseof our lives. Sowhile these

tasksmaynot,forexample,fullycapturetheprocessofbuyingahouse,theymaybevery

representativeofroutineeconomicdecisions.Finally,whatresearchhasbeendoneonRTs

inone-shot,slowdecisions,issofarconsistentwiththeSSMpredictions.

For example,Krajbich et al. 2015b studya voluntary contributionpublic goods game

experimentwheresubjectsmadeonlythreedecisionsandtookonaverage43.5stodecide

eachtime.Inthatexperiment,slowdecisionstendedtofavorthelessattractivealternative,

consistentwithbeingclosertoindifference. Inparticular,withalow-benefitpublicgood,

slow contributions tended to be higher, while with a high-benefit public good, slow

contributionstendedtobelower.

More careful analysis is required to distinguish between the SSM and alternative

interpretationsbyRubinsteinandothers(Chen&Fischbacher,2015;Hey,1995;Recaldeet

al.,2014;Rubinstein,2007,2013,2014,2016),where longRTsareassociatedwithmore

carefulordeliberativethoughtandshortRTsareassociatedwithintuition. Itmayinfact

bethecasethat insomeinstancespeopledousea logic-basedapproach, inwhichcasea

42

longRTmaybemoreindicativeofcarefulthought,whileinotherinstancestheyrelyona

SSM approach, inwhich case a long RT likely indicates indifference. This could lead to

contradictoryconclusions fromthesameRTdata; forexampleoneresearchermayseea

longRTandassumethesubject isverywell informed,whileanotherresearchermaysee

thatsameRTandassumethesubjecthasnoevidenceonewayortheother.Moreresearch

isrequiredtotestwhetherSSMs,whicharedesignedtoteaseapartsuchexplanations,can

besuccessfullyappliedincomplexeconomicdecisions.

43

References

1. Agranov, M., & Ortoleva, P. (2015). Stochastic choice and preferences for

randomization.AvailableatSSRN2644784.

2. Blais,A.-R.,&Weber, E.U. (2006).Adomain-specific risk-taking (DOSPERT) scale

foradultpopulations.JudgmentandDecisionMaking,1(1).

3. Blurton,S.P.,Kesselmeier,M.,&Gondan,M.(2012).Fastandaccuratecalculations

forcumulative first-passage timedistributions inWienerdiffusionmodels. Journal

ofMathematicalPsychology,56(6),470–475.

4. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice

responsetime:Linearballisticaccumulation.CognitivePsychology,57(3),153–178.

https://doi.org/10.1016/j.cogpsych.2007.12.002

5. Busemeyer,J.R.(1985).Decisionmakingunderuncertainty:Acomparisonofsimple

scalability, fixed-sample, and sequential-samplingmodels. Journal of Experimental

Psychology: Learning, Memory, and Cognition, 11(3), 538–564.

https://doi.org/10.1037/0278-7393.11.3.538

6. Busemeyer, J. R., & Diederich, A. (2002). Survey of decision field theory.

MathematicalSocialSciences,43,345–370.

7. Busemeyer, J.R.,&Rapoport,A. (1988).Psychologicalmodelsofdeferreddecision

making.JournalofMathematicalPsychology,32,91–134.

8. Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-

cognitiveapproach todecisionmaking inanuncertainenvironment.Psychological

Review,100(3),432–459.

44

9. Caplin, A., & Martin, D. (2016). THE DUAL-PROCESS DRIFT DIFFUSION MODEL:

EVIDENCE FROM RESPONSE TIMES. Economic Inquiry, 54(2), 1274–1282.

https://doi.org/10.1111/ecin.12294

10. Cavagnaro, D. R., Myung, J. I., Pitt, M. A., & Kujala, J. V. (2010). Adaptive design

optimization: A mutual information-based approach to model discrimination in

cognitivescience.NeuralComputation,22(4),887–905.

11. Cettolin,E.,&Riedl,A.(2015).Revealedincompletepreferencesunderuncertainty.

12. Chabris, C. F., Laibson, D., Morris, C. L., Schuldt, J. P., & Taubinsky, D. (2008).

Measuring intertemporal preferences using response times. National Bureau of

EconomicResearch.

13. Chabris, C. F.,Morris, C. L., Taubinsky,D., Laibson,D., & Schuldt, J. P. (2009). The

Allocation of Time in Decision-Making. Journal of the European Economic

Association,7(2–3),628–637.

14. Che, Y.-K., & Mierendorff, K. (2016). Optimal Sequential Decision with Limited

Attention.Unpublished,ColumbiaUniversity.

15. Chen, F., & Fischbacher, U. (2015). Response Time and Click Position: Cheap

IndicatorsofPreferences.ThurgauerWirtschaftsinstitut,Universit�tKonstanz.

16. Cleveland, W. S. (1979). Robust locally weighted regression and smoothing

scatterplots.JournaloftheAmericanStatisticalAssociation,74(368),829–836.

17. Clithero,J.A.(2016a).ImprovingOut-of-SamplePredictionsUsingResponseTimes

andaModeloftheDecisionProcess.AvailableatSSRN2798459.

18. Clithero,J.A.(2016b).ResponseTimesinEconomics:LookingThroughtheLensof

SequentialSamplingModels.AvailableatSSRN.

45

19. Dai,J.,&Busemeyer,J.R.(2014).Aprobabilistic,dynamic,andattribute-wisemodel

ofintertemporalchoice.JournalofExperimentalPsychology:General,143(4),1489–

1514.https://doi.org/10.1037/a0035976

20. Danan, E., Ziegelmeyer, A., & others. (2006). Are Preferences Complete?: An

Experimental Measurement of Indecisiveness Under Risk. Max Planck Inst. of

Economics, Strategic Interaction Group. Retrieved from

https://www.researchgate.net/profile/Anthony_Ziegelmeyer/publication/5018150

_Are_preferences_complete_An_experimental_measurement_of_indecisiveness_unde

r_risk/links/0c960516d8217911fe000000.pdf

21. DeMartino,B.,Fleming,S.M.,Garret,N.,&Dolan,R.J.(2013).Confidenceinvalue-

basedchoice.NatureNeuroscience,16(1),105–110.

22. Dickhaut, J., Rustichini, A., & Smith, V. (2009). A neuroeconomic theory of the

decisionprocess.ProceedingsoftheNationalAcademyofSciences,106(52),22145–

22150.

23. Eliaz, K., & Ok, E. A. (2006). Indifference or indecisiveness? Choice-theoretic

foundationsofincompletepreferences.GamesandEconomicBehavior,56(1),61–86.

https://doi.org/10.1016/j.geb.2005.06.007

24. Fehr,E.,&Schmidt,K.M.(1999).Atheoryoffairness,competition,andcooperation.

TheQuarterlyJournalofEconomics,114(3),817–868.

25. Fiedler,S.,&Glöckner,A.(2012).TheDynamicsofDecisionMakinginRiskyChoice:

An Eye-Tracking Analysis. Frontiers in Psychology, 3.

https://doi.org/10.3389/fpsyg.2012.00335

46

26. Frazier,P.,&Yu,A.(2007).Sequentialhypothesistestingunderstochasticdeadlines

(pp.465–472).PresentedattheAdvancesinneuralinformationprocessingsystems.

27. Fudenberg, D., Strack, P., & Strzalecki, T. (2015). Stochastic Choice and Optimal

SequentialSampling.arXivPreprintarXiv:1505.03342.

28. Gabaix, X., & Laibson, D. (2005). Bounded rationality and directed cognition.

Harvard University. Retrieved from

http://pages.stern.nyu.edu/~xgabaix/papers/boundedRationality.pdf

29. Gabaix, X., Laibson, D., Moloche, G., & Weinberg, S. (2006). Costly Information

Acquisition: Experimental Analysis of a Boundedly Rational Model. American

EconomicReview,96(4),1043–1068.https://doi.org/10.1257/aer.96.4.1043

30. Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual

differencesinimplicitcognition:theimplicitassociationtest.JournalofPersonality

andSocialPsychology,74(6),1464.

31. Hare, T. A., Hakimi, S., & Rangel, A. (2014). Activity in dlPFC and its effective

connectivity to vmPFC are associated with temporal discounting. Frontiers in

Neuroscience,8.https://doi.org/10.3389/fnins.2014.00050

32. Hawkins, G. E., Forstmann, B. U., Wagenmakers, E.-J., Ratcliff, R., & Brown, S. D.

(2015). Revisiting the Evidence for Collapsing Boundaries andUrgency Signals in

Perceptual Decision-Making. The Journal of Neuroscience, 35(6), 2476–2484.

https://doi.org/10.1523/JNEUROSCI.2410-14.2015

33. Herrnstein,R. J.(1981).Self-controlasresponsestrength.QuantificationofSteady-

StateOperantBehavior,3–20.

47

34. Hey, J. D. (1995). Experimental investigations of errors in decisionmaking under

risk.EuropeanEconomicReview,39(3),633–640.

35. Hunt,L.T.,Kolling,N.,Soltani,A.,Woolrich,M.W.,Rushworth,M.F.S.,&Behrens,T.

E. (2012). Mechanisms underlying cortical activity during value-guided choice.

NatureNeuroscience,15,470–476.

36. Hutcherson,C.A.,Bushong,B.,&Rangel,A.(2015).ANeurocomputationalModelof

AltruisticChoiceandItsImplications.Neuron,87(2),451–462.

37. Johnson,J.G.,&Busemeyer,J.R.(2005).Adynamic,stochastic,computationalmodel

ofpreferencereversalphenomena.PsychologicalReview,112(4),841–861.

38. Kahneman,D.,&Tversky,A.(1979).Prospecttheory:ananalysisofdecisionunder

risk.Econometrica,4,263–291.

39. Kim, W., Pitt, M. A., Lu, Z.-L., Steyvers, M., & Myung, J. I. (2014). A hierarchical

adaptiveapproachtooptimalexperimentaldesign.NeuralComputation.

40. Krajbich, I.,Armel,K.C.,&Rangel,A. (2010).Visual fixationsand thecomputation

andcomparisonofvalueinsimplechoice.NatureNeuroscience,13(10),1292–1298.

41. Krajbich,I.,Bartling,B.,Hare,T.,&Fehr,E.(2015).Rethinkingfastandslowbased

on a critique of reaction-time reverse inference.Nature Communications,6, 7455.

https://doi.org/10.1038/ncomms8455

42. Krajbich, I., Hare, T., Bartling, B., Morishima, Y., & Fehr, E. (2015). A Common

MechanismUnderlyingFoodChoiceandSocialDecisions.PLoSComputBiol,11(10),

e1004371.

48

43. Krajbich,I.,Lu,D.,Camerer,C.,&Rangel,A.(2012).TheAttentionalDrift-Diffusion

Model Extends to Simple Purchasing Decisions. Frontiers in Psychology, 3.

https://doi.org/10.3389/fpsyg.2012.00193

44. Krajbich, I., Oud, B., & Fehr, E. (2014). Benefits ofNeuroeconomicModeling:New

Policy Interventions and Predictors of Preference †. American Economic Review,

104(5),501–506.https://doi.org/10.1257/aer.104.5.501

45. Krajbich, I.,&Rangel,A.(2011).Multialternativedrift-diffusionmodelpredictsthe

relationship between visual fixations and choice in value-based decisions.

ProceedingsoftheNationalAcademyofSciences,108(33),13852–13857.

46. Laibson,D.(1997).Goldeneggsandhyperbolicdiscounting.TheQuarterlyJournalof

Economics,443–477.

47. Laming, D. (1979). A critical comparison of two random-walk models for choice

reactiontime.ActaPsychologica,43,431–453.

48. Link, S. W. (1975). The relative judgment theory of two choice response time.

JournalofMathematicalPsychology,12(1),114–135.

49. Loomes, G. (2005). Modelling the stochastic component of behaviour in

experiments:Someissuesfortheinterpretationofdata.ExperimentalEconomics,8,

301–323.

50. Mandler, M. (2005). Incomplete preferences and rational intransitivity of choice.

Games and Economic Behavior, 50(2), 255–277.

https://doi.org/10.1016/j.geb.2004.02.007

49

51. Matejka, F., & McKay, A. (2014). Rational inattention to discrete choices: A new

foundationforthemultinomiallogitmodel.TheAmericanEconomicReview,105(1),

272–298.

52. Milosavljevic, M., Malmaud, J., Huth, A., Koch, C., & Rangel, A. (2010). The drift

diffusion model can account for the accuracy and reaction time of value-based

choicesunderhighandlowtimepressure.JudgmentandDecisionMaking,5(6),437–

449.

53. Moffatt, P. G. (2005). Stochastic choice and the allocation of cognitive effort.

ExperimentalEconomics,8(4),369–388.

54. Mosteller,F.,&Nogee,P.(1951).AnExperimentalMeasurementofUtility.Journalof

PoliticalEconomy,59(5),371–404.

55. Navarro,D. J.,&Fuss, I.G. (2009). Fast andaccurate calculations for first-passage

times inWienerdiffusionmodels. JournalofMathematicalPsychology,53(4),222–

230.

56. Ok,E.A.,Ortoleva,P.,&Riella,G.(2012).Incompletepreferencesunderuncertainty:

Indecisivenessinbeliefsversustastes.Econometrica,80(4),1791–1808.

57. Oud, B., Krajbich, I., Miller, K., Cheong, J. H., Botvinick, M., & Fehr, E. (2016).

Irrational time allocation in decision-making. Proceedings of the Royal Society B:

BiologicalSciences,283(1822),20151439.https://doi.org/10.1098/rspb.2015.1439

58. Philiastides,M.G.,&Ratcliff,R. (2013). Influenceofbrandingonpreference-based

decisionmaking.PsychologicalScience,24(7),1208–1215.

59. Ratcliff,R.(1978).Atheoryofmemoryretrieval.PsychologicalReview,85,59–108.

50

60. Ratcliff,R.,&McKoon,G.(2008).Thediffusiondecisionmodel:theoryanddatafor

two-choicedecisiontasks.NeuralComputation,20(4),873–922.

61. Ratcliff,R.,Philiastides,M.G.,&Sajda,P.(2009).Qualityofevidenceforperceptual

decisionmakingisindexedbytrial-to-trialvariabilityoftheEEG.Proceedingsofthe

NationalAcademyofSciences,106(16),6539–6544.

62. Ratcliff,R.,&Tuerlinckx, F. (2002).Estimatingparametersof thediffusionmodel:

Approaches todealingwithcontaminantreaction timesandparametervariability.

PsychonomicBulletin&Review,9(3),438–481.

63. Recalde, M. P., Riedl, A., & Vesterlund, L. (2014). Error prone inference from

response time: The case of intuitive generosity. Retrieved from

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2507723

64. Rieskamp, J. (2008). The probabilistic nature of preferential choice. Journal of

ExperimentalPsychology:Learning,Memory,andCognition,34(6),1446.

65. Rodriguez, C. A., Turner, B.M., &McClure, S.M. (2014a). Intertemporal Choice as

DiscountedValueAccumulation.PloSOne,9(2),e90138.

66. Rodriguez, C. A., Turner, B.M., &McClure, S.M. (2014b). Intertemporal choice as

discountedvalueaccumulation.PLoS.ONE,9,e90138.

67. Roe,R.M.,Busemeyer,J.R.,&Townsend,J.T.(2001).Multialternativedecisionfield

theory: A dynamic connectionist model of decision making. Psychological Review,

108(2),370–392.

68. Rubinstein, A. (2007). Instinctive and Cognitive Reasoning: A Study of Response

Times*.TheEconomicJournal,117(523),1243–1259.

51

69. Rubinstein,A.(2013).Responsetimeanddecisionmaking:Anexperimentalstudy.

JudgmentandDecisionMaking,8(5),540–551.

70. Rubinstein, A. (2014). A Typology of Players: between Instinctive and

Contemplative.Retrievedfromhttp://arielrubinstein.tau.ac.il/papers/Typology.pdf

71. Samuelson, P. A. (1938). A note on the pure theory of consumer’s behaviour.

Economica,5(17),61–71.

72. Schotter, A., & Trevino, I. (2014). Is response time predictive of choice? An

experimental study of threshold strategies. WZB Discussion Paper. Retrieved from

http://www.econstor.eu/handle/10419/98843

73. Shadlen, M. N., & Shohamy, D. (2016). Decision Making and Sequential Sampling

from Memory. Neuron, 90(5), 927–939.

https://doi.org/10.1016/j.neuron.2016.04.036

74. Sims, C. A. (2003). Implications of rational inattention. Journal of Monetary

Economics,50(3),665–690.

75. Sokol-Hessner,P.,Hsu,M.,Curley,N.G.,Delgado,M.R.,Camerer,C.F.,&Phelps,E.A.

(2009). Thinking like a trader selectively reduces individuals’ loss aversion.

ProceedingsoftheNationalAcademyofSciences,106(13),5035–5040.

76. Spiliopoulos, L., & Ortmann, A. (2014). The BCD of response time analysis in

experimental economics. Available at SSRN 2401325. Retrieved from

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2401325

77. Srivastava, V., Feng, S. F., Cohen, J. D., Leonard, N. E., & Shenhav, A. (2015). First

PassageTimePropertiesforTime-varyingDiffusionModels:AMartingaleApproach.

arXivPreprintarXiv:1508.03373.

52

78. Steiner, J.,Stewart,C.,&Matějka,F. (2017).Rational InattentionDynamics: Inertia

andDelayinDecision-Making.Econometrica,85(2),521–553.

79. Stewart,N.,Hermens,F.,&Matthews,W.J.(2015).EyeMovementsinRiskyChoice:

Eye Movements in Risky Choice. Journal of Behavioral Decision Making, n/a-n/a.

https://doi.org/10.1002/bdm.1854

80. Stone,M.(1960).Modelsforchoice-reactiontime.Psychometrika,25(3),251–260.

81. Tajima, S., Drugowitsch, J., & Pouget, A. (2016). Optimal policy for value-based

decision-making. Nature Communications, 7, 12400.

https://doi.org/10.1038/ncomms12400

82. Towal, R. B., Mormann, M., & Koch, C. (2013). Simultaneous modeling of visual

saliency and value computation improves predictions of economic choice.

ProceedingsoftheNationalAcademyofSciences,110(40),E3858–E3867.

83. Usher,M.,&McClelland, J.(2001).Thetimecourseofperceptualchoice:theleaky,

competingaccumulatormodel.PsychologicalReview,108(3),550–592.

84. Wabersich, D., & Vandekerckhove, J. (2014). TheRWiener package: AnR package

providingdistributionfunctionsfortheWienerdiffusionmodel.RJournal,6(1).

85. Wald, A. (1945). Sequential tests of statistical hypotheses. The Annals of

MathematicalStatistics,16(2),117–186.

86. Wang, S.W., Filiba,M.,&Camerer, C. F. (2010).Dynamicallyoptimized sequential

experimentation (DOSE) for estimating economic preference parameters.

ManuscriptSubmittedforPublication.

87. Webb,R.(2013a).DynamicConstraintsontheDistributionofStochasticChoice:Drift

DiffusionImpliesRandomUtility(SSRNScholarlyPaperNo.ID2226018).Rochester,

53

NY: Social Science Research Network. Retrieved from

http://papers.ssrn.com/abstract=2226018

88. Webb,R.(2013b).Dynamicconstraintsonthedistributionofstochasticchoice:Drift

Diffusion implies Random Utility. Available at SSRN 2226018. Retrieved from

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2226018

89. Wilcox,N.T. (1993).Lotterychoice: Incentives, complexityanddecision time.The

EconomicJournal,1397–1417.

90. Woodford, M. (2014). Stochastic Choice: An Optimizing Neuroeconomic Model.

American Economic Review, 104(5), 495–500.

https://doi.org/10.1257/aer.104.5.495

91. Woodford,M.(2016).OptimalEvidenceAccumulationandStochasticChoice.

54

ONLINEAPPENDIX

A. Parameterestimationmethodology

Choice-basedmethod

Weestimateeachindividualutilityfunction𝑢(⋅ |𝜃),where𝜃isasubject-specificparameter

described in Section 3, in the standardway as follows.We assume that for each pair of

options and choice 𝑎 = 1,2 the error terms in utilities follow the type I extreme value

distribution,sotheprobabilityofchoosingoption1isalogisticfunction

𝑝(𝑎j = 1) =1

1 + 𝑒R� ��(⋅|�)R�O(⋅|� ),

where 𝜇 and 𝜃 are free parameters that can be estimated for each subject individually

maximizingalikelihoodfunction

𝐿𝐿 = (log 𝑝(𝑎e = 1) ⋅ 1 𝑎� = 1 + log 1 − 𝑝 𝑎e = 2 ⋅ 1(𝑎� = 2)),e

where𝑛 isthetrialnumber,𝑎e isthechoicemadebythesubjectonthattrial,and1(⋅) is

theindicatorfunction.

Appendix B shows the subject level correlations between the predicted and the actual

choices.

Top10%responsetimemethod

Foreachdecisionproblemoneach trial𝑛,wecalculate the indifferenceparametervalue

𝜃eje� asasolutiontotheequation

𝑢9 ⋅ 𝜃e = 𝑢< ⋅ 𝜃e .

55

ThenweaveragetheindifferencevaluesonthetrialsinthetopRTdeciletoobtainthefinal

parameterestimate:

𝜃 =(𝜃eje�e ⋅ 1(𝐹(𝑅𝑇e) ≥ 0.9))

1(𝐹(𝑅𝑇e) ≥ 0.9)e,

where𝑅𝑇e is the response time on trial𝑛,𝐹(⋅) is the empirical RT distribution for the

specificsubject,and1 ⋅ istheindicatorfunction.

Localregression(LOWESS)method

As in the previous method, for each decision problem on each trial 𝑛, we estimate the

indifferenceparametervalue𝜃eje� solvingtheequation

𝑢9 ⋅ 𝜃e = 𝑢< ⋅ 𝜃e .

For each individual subject, we regress response time log(𝑅𝑇) in every trial 𝑛 on the

corresponding indifference parameter value 𝜃eje� using a local polynomial regression

(LOWESS,Cleveland1979)intheRpackagestats:

𝑅𝑇 = 𝑓 𝜃eje� + 𝜀e.

We set the smoothing parameter to 0.5 as it has provided the best prediction accuracy

acrossallfourdatasets(seeAppendixC).

Then we obtain the parameter estimate 𝜃 by inverting the fitted regression line at the

maximumpredictedresponsetime𝑅𝑇:

𝜃 = 𝑓R9 max 𝑅𝑇 .

56

Drift-diffusionmodel(DDM)method

In theDDM (see Section2) a latent decision variable evolves over timewith an average

drift rate plus Gaussian noise (the Wiener diffusion) until it reaches one of two pre-

determinedboundaries,whichcorrespondtothetwochoiceoptions..Giventheboundary

separation parameter, the drift rate, the non-decision time (the component of RT not

attributable to thedecisionprocess itself, e.g.movingone’s hand to indicate the choice),

andthevarianceof theGaussiannoise, it ispossible tocalculatechoiceprobabilitiesand

choice-contingentRTdistributions.

Inourparticularcaseweassumethatchoicesareunknown,andsowecanonlyuse

thecombined(summed)RTdistributiontoestimatethefreeparametersofthemodel.We

assumethatallsubjectssharethesameconstantboundaryparameter𝑏,non-decisiontime

𝜏,anddriftrateparameter𝑧,whichmultipliestheutilitydifferenceoneverysingletrial:

𝑣 ≡ 𝑧 ⋅ 𝑢9 ⋅ 𝜃 − 𝑢< ⋅ 𝜃 .

We use a density function of the Wiener distribution from the RWiener R package

(WabersichandVandekerckhove2014)toestimatethelikelihood(9)fortheobservedRT

on every given trial assuming a set of parameters (𝑏, 𝜏, 𝑧, 𝜽), where 𝜽 is a vector of

individualsubjects’parameters.Essentially,theidentificationoftheindividualparameters

ispossibleduetothefact thatRTsarepredictedtovaryastheutilitydifference𝑣varies

acrosstrialsandsubjects.

Chabrisetal.(2009)method

Here we follow the method suggested by Chabris et al. (2009), which uses the full RT

distributiontoestimatetheutilityfunctionparameters.

57

Letthedifferenceinthetwoutilitiesbe

𝛥� ≡ 𝑢9� ⋅ 𝜃 − 𝑢<� ⋅ 𝜃 .

Assumethatthedecisiondifficultyisaconvexanddecreasingfunctionofthisdifference:

𝛤 𝛥� ≡2

1 + 𝑒��[,

where𝜔isafreeparameter.

The response times are then modeled as a function of the trial number and the choice

difficulty:

𝑅𝑇� = 𝛽� + 𝛽9𝑡 + 𝛽<𝛤 𝛥� + 𝜀�.

To estimate the set of parameters (𝜃, 𝜔, 𝛽�, 𝛽9, 𝛽<) we follow the original paper and

minimizetheerrorfunction

(𝑅𝑇� − 𝛽� − 𝛽9𝑡 − 𝛽<𝛤 𝛥� ).�

UnlikeChabrisetal.(2009),weestimatethismodelforeveryindividualsubject.

58

B. Choice-basedfits

FigureB1.Subject-levelcorrelation(Pearson)betweenchoiceproportionsinthedataand

aspredictedby choice-estimatedutility functions (seeAppendixA fordetails). The solid

linesare45degrees.

●

●

● ●●● ●●

●

●●●

●

●●●

●

●

●

●

●

●

●

●●

●●

●

●

0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Social choice (α)

P(fair) in data

P(fa

ir) e

stim

ated

from

cho

ices

r = 0.91

p < 0.001

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

0.0 0.2 0.4 0.6 0.8 1.00.

00.

20.

40.

60.

81.

0

Social choice (β)

P(fair) in data

P(fa

ir) e

stim

ated

from

cho

ices

r = 0.99

p < 0.000

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●●

●

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Intertemporal choice

P(later) in data

P(la

ter)

estim

ated

from

cho

ices

r = 0.97

p < 0.000

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Risk choice

P(safe) in data

P(sa

fe) e

stim

ated

from

cho

ices

r = 0.99

p < 0.000

59

C. Local regression (LOWESS) method choice prediction accuracy for various

levelsofthesmoothingparameter

Figure C1. Choice prediction accuracy as a function of the smoothing parameter of the

LOWESSregressionmodel.Thesolidblack linesdenotemeanpredictionaccuracyacross

subjects,theshadedareasshowstandarderrorsatthesubjectlevel.

0.0 0.2 0.4 0.6 0.8 1.0

0.80

0.85

0.90

0.95

Social choice (α)

smoothing parameter

pred

ictio

n ac

cura

cy

0.0 0.2 0.4 0.6 0.8 1.0

0.80

0.85

0.90

Social choice (β)

smoothing parameter

pred

ictio

n ac

cura

cy

0.0 0.2 0.4 0.6 0.8 1.0

0.70

0.75

0.80

0.85

Intertemporal choice

smoothing parameter

pred

ictio

n ac

cura

cy

0.0 0.2 0.4 0.6 0.8 1.0

0.75

0.80

0.85

0.90

Risk choice

smoothing parameter

pred

ictio

n ac

cura

cy

60

D. ExploringtheoptimalcutoffforthetopRTestimationmethod

Figure D1. Averaging the longest 10-20% RT trials provides the best choice

predictionaccuracy.Choicepredictionaccuracyasafunctionofthepercentageofslowest

trialsusedintheparameterestimationfrom1to100%.Thesolidblacklinesdenotemean

predictionaccuracyacrosssubjects,theshadedareasshowstandarderrorsatthesubject

level,theredlinesabovethegraphsindicatesignificantdifferencefromthebaselineatthe

p=0.05 level (Wilcoxonsignedrank test).Thebaseline is theaverageofall indifference

0 20 40 60 80 100

0.80

0.85

0.90

0.95

Social choice (α)

top percentile

pred

ictio

n ac

cura

cy

0 20 40 60 80 100

0.75

0.80

0.85

0.90

Social choice (β)

top percentilepr

edic

tion

accu

racy

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

0 20 40 60 80 100

0.65

0.70

0.75

0.80

Intertemporal choice

top percentile

pred

ictio

n ac

cura

cy

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●

0 20 40 60 80 100

0.76

0.78

0.80

0.82

0.84

Risk choice

top percentile

pred

ictio

n ac

cura

cy

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●

61

pointsacrosstrials.Itisimportanttoemphasizethatthebaselinetowhichwecomparethe

predictive power is not chance (50%) as almost any experimenter uses some prior

knowledgeoftheparameterdistributioninthepopulationtoselecttheirchoiceproblems.

For example, an experimenter studying intertemporal choicemight select a set of choice

problemssothattheaveragesubjectwouldchoosetheimmediateoptionhalfofthetime

andthedelayedoptiontheotherhalfofthetime.Soifyouweretoaveragetheindifference

pointsfromthetrialsinsuchanexperiment,youwouldbeabletopredictbehaviorquite

accurately,onaverage.Insuchanexperiment,behaviorintrialswithextremeindifference

pointswillbeverypredictable.Thatis,onatrialdesignedtomakeaverypatientsubject

indifferent, most subjects will have a strong preference for the immediate option.

Similarly,ona trialdesigned tomakean impatientperson indifferent,most subjectswill

haveastrongpreferenceforthedelayedoption.Thusbehaviorinmanyofanexperiment’s

trials isquiteeasytopredictbecausethosetrialsareonly includedto identifyparameter

values for extreme subjects. For instance, a single loss-aversion coefficient of𝜆 = 2 can

predictabout75%ofchoicesinourrisky-choicedataset.

62

E. Modelcomparisonresults

DDM Chabrisetal(2009) Top10%RT LOWESS(0.5)

Socialchoice(𝛼) 0.39 0.22 0.44 0.12

Socialchoice(𝛽) 0.52 0.53 0.56 0.6

Intertemporalchoice 0.57 0.59 0.71 0.44

Riskchoice 0.36 0.16 0.64 0.88

PearsoncorrelationbetweenparametersestimatedfromchoicesandRTs,atthesubject

level.Thebestperformingmethodforeachdatasetisshowninbold.Forestimation

methodsdetailsseeAppendixA.

63

F. Instructionsfortheriskexperiment

InstructionsThankyouforparticipatingintoday’sstudy.Pleasecarefullyreadthematerialonthefollowingpagestounderstand

• Therules• Thedecisionsyouwillbemakingtoday

Ifyouhaveanyquestionsafterreadingtheseinstructionsorduringtheexperiment,please

askthembeforetheexperimentorduringthedesignatedbreaks.Therules

• Pleasechecknowtoensurethatyourmobilephoneisonsilentmodeandputitinyourbagorpocket.

• Pleasedonottalkduringtheexperiment.ThestudyTodayyouwillbemakingaseriesofchoices,andyour finalpaymentwilldependonlyon

yourownchoicesandchance.PaymentYourpaymentwillconsistoftwoamounts:

• Afixedendowmentof34experimentalcurrencyunits(ECUs)thatyouaregivenatthebeginningofthestudy.

• Yourearnings fromone randomly selected choice round. Youmayearnadditionalmoneybeyondthe34ECUs,oryoumay losesomeofthat34ECUs,dependingonyourchoicesandonchance.Theminimumamountofmoneyyoucanearntodayis10ECUs=$5.

• All the amounts in today’s studywill be shown in ECUs andwill be converted todollarsattheendofthestudyatarateof2ECUs=$1.

64

Yourchoices

In each round of the experiment you will be asked to make a choice between one of twooptions. Option one consists of two possible amounts, each onewith a probability of 50%.Optiontwoconsistsofoneamount,withaprobabilityof100%.

Belowisanexampledecisionscreen.InthisroundOptionone(ontheleft)consistsofagain(ingreen)of30ECUsandaloss(inred)of10ECUs.Ifyoupickthisoption,thecomputerflipsafairdigital coin (chances are 50-50). In case of heads, you would earn 30 ECUs on top of yourendowment. In caseof tails, youwould lose10 ECUs,whichwould thenbe subtracted fromyourendowment.Tochoosethisoption,youwouldpress1.Optiontwo(ontheright)isasuregainof15ECUs.Ifyoupickthisoption,youwouldearn15ECUsontopofyourendowment.Tochoosethisoption,youwouldpress2.

Yourfinalearningswillonlydependononeofyourchoices:attheendofthestudy,onlyoneoftheroundswillberandomlyselectedforpayment.Theoutcomefromthisroundwillbeaddedorsubtractedfromyourinitialendowment.

Ifyouhaveanyquestions,pleaseraiseyourhandnow.

30

-10

15vs

1 2