View
39
Download
2
Category
Tags:
Preview:
DESCRIPTION
UNCLASSIFIED / FOR OFFICIAL USE ONLY. Decision Making and Stochastic Delay at Workshop on Social Computing, Behavior Modeling and Prediction 1 April 2008. Dr. Bruce J. West Chief Scientist Mathematical & Information Science Directorate Army Research Office Bruce.j.west@us.army.mil - PowerPoint PPT Presentation
Citation preview
Dr. Bruce J. West
Chief Scientist
Mathematical & Information Science Directorate
Army Research Office
Bruce.j.west@us.army.mil
919-549-4257
UNCLASSIFIED / FOR OFFICIAL USE ONLY
UNCLASSIFIED / FOR OFFICIAL USE ONLY
Decision Making and Stochastic Delay
at
Workshop on Social Computing, Behavior Modeling and
Prediction
1 April 2008
• Discounted Utility Model & intertemporal choice• Anomalies from discounted utility theory
– irrationality– hyperbolic discounting
• Objective and subjective time– entropy and the direction of time
– time as a stochastic variable
• Individuality and paternalism– some experiments
– fit of theory to data
• Conclusions
Outline of talk
Decision Making and Delay
Discounted Utility Model (DUM)
uuuuUT
tTtt
t
11 ,...,,
• Discount factor δ compresses many mechanisms• mortality, uncertainty, time compression,…
• Accepted as both normative (how things should be) and descriptive (how things are)…..but was initially arbitrary Samuelson (1937).
• Exponential form implies time consistency (rationality)
Decision Making and Delay
Anomalies from DUM
• Time inconsistency– empirical discount factor is not constant
• over time
• across type of intertemporal choices
• Delay effect (hyperbolic discounting)• Interval effect (non-stationarity)• Sign effect (gains vs. loses)• Magnitude effect (small vs. large)• Direction effect• Sequence effects (ordered set vs. single)
Decision Making and Delay
Model comparison
• Exponential delay model– monotonic decrease in valuewith objective time– constant rate results in timeconsistency – rationality
• Hyperbolic delay model– decreasing rate results in time inconsistency – irrationality (preference reversal)
hyperbolic
exponential
exponential
hyperbolic
hyperbolic
exponential
Decision Making and Delay
Objective vs. subjective time
• Hyperbolic models– Objective time
• clockwork universe
• entropy and the direction of time
– Subjective time• unidirectional
• probability and statistics
• Motivate decision-making using
ensemble distributions– subjective time– stochastic delays
Decision Making and Delay
Delay and uncertainty• Decision-making models of intertemporal choice can be extended to incorporate probabilistic choice where
p is the probability of reward at time t and F is an unspecified function.
discrete continuous
• No reward before delay time t
• Delay-time probability density
)F(p)U(U(t) 0
dττ ψp
t)ΨU(t)=U(dtt
0
dt
tdt
Decision Making and Delay
F
Fr
'• Deterministic discount rate is replaced with a conditional
probability per unit time
• The ratio of the delay time distribution function to the survival probability density, integrates to
• The utility function in terms of subjective time is therefore
tt
ttttobt
trt
Pr1
lim)(0
tdttrt
0''exp
tdttrUtUtU
0''exp00
Stochastic rate
Decision Making and Delay
Example rate
• Rate of reward production suggested by hyperbolic distribution
• Probability of no reward before time t is
so that the utility function is inverse power law
• T measures response time and α measures irrationality
tr
rtr
1
0
1)(
tT
T
trtr
dtrt
r
r
t
1
0
1
01
0
1
1
'1
'exp
Trr
TtT
TUtU 0
1
;1
;)0()( = =
Decision Making and Delay
Experimental data
• Students (20) asked to make decisions about hypothetical
money to be received immediately or at a later time, concerning
the subjects themselves or another person not known to them. Takahashi, Physica A (2007).
self other
T=31α=0.28
T=1.85α=0.11
Decision Making and Delay
Implications from experiments
• The response times could describe paternalistic policy making government officials, where irrationality is enhanced.
• Irrationality is nowhere more significant than in the military where choices may determine whether others live or die.
Decision Making and Delay
Decision Making and Delay
• Nonlinear dynamic equation solved on the interval (0,1).
• define a delay-time distribution density
• assume a uniform distribution of initial conditions to obtain
• delay-time distribution density is non-Poisson, renewal and non-ergodic
dq
dtaq z
zazq 1
1
0 11
ddqqp 00 )(
1
;11
T
T
Measured discount rates
• Higher discount rates compared with controls* ( smaller T and α in stochastic intertemporal model)– smoking– excessive alcohol consumption– illicit drug use (cocaine, crack-cocaine and heroin)– pathological gambling– age– cognitive ability (negative correlation with intellectual
achievement)
• Consistent with neuroeconomic hypothesis that prefrontal cortex is essential for patient (forward looking) decision making.
* Chabris, Laibson & Schuldt, The New Palgrave Dictionary of Economics (2007).
Decision Making and Delay
Brain Activity
Sanfey, Loewenstein, McClure and Cohen, TRENDS in Cognitive Science (2006).
β network”: midbrain dopamine network; reward processing (ventral striatum V.Str. and medial prefrontal cortex mPFC.)
δ network: cognition; dorsolateral prefrontal cortex (diPFC) and right posterior parletal cortex (R.Par.)
Decision Making and Delay
More brain activity
15.0 5.0
5.0• Two discounting slopes
• < one year
• > one year
• Different parts of the brain light
up under fMRI• short-term
• long-term
Wittmann & Paulus (2007)
5.0
Decision Making and Delay
Decision Making and Delay
Conclusions and Speculations
• decision-making is not always rational
• irrationality in intertemporal choice models take a hyperbolic form
• inverse power laws or hyperbolic utility functions can be generated by stochastic delay times
• different parts of the brain control decisions associated with long and short delay times
• the complexity of the brain produces the subjective nature of biological time
Recommended