Upload
frank-burns
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
ECONOMIC EXPERIMENTS IN LOW IMMERSIVE VIRTUAL ENVIRONMENTS. A METHODOLOGICAL
ASSESSMENT
ALESSANDRO INNOCENTI(University of Siena)
Siena Workshop Rutgers University Center for Cognitive Sciences
May 21, 2013
To propose Low immersive virtual experiments (LIVE) as
tools for experimental and cognitive economics where
the laboratory approach has important limitations
To explore the potentiality of LIVE by providing the
preliminary findings of specifically designed experimental
studies investigating risk perception in workplaces and
risk attitude under social exposure
Talk Purpose
Failures of Lab Experiments
The Context-Free Bias
Virtual Experiments
The ALBO Project
Preliminary Results
Conclusions
Talk Outline
a) situations are not really presented, but only
described through language
b) choices and decisions are only to be evoked, not to
be really performed
c) there is lack in the normal cascade of events as
actions and reactions
d) temporal frame is generally compressed
e) irrelevance of the context
Failures of Lab Experiments
Many experimental economists seem to view their
enterprise as akin to silicon chip production. Subjects are
removed from all familiar contextual cues. Like the
characters 'thing one' and 'thing two' in Dr. Suess' Cat in
the Hat, buyers and sellers become 'persons A and B', and
all other information that might make the situation familiar
and provide a clue about how to behave is removed.
George Loewenstein (1999)
Lab as silicon chip production
The context-free experiment is an elusive goal
A major tenet of cognitive psychology is how all forms of
thinking and problem solving are context-dependent
The laboratory is not a socially neutral context, but is
itself an institution with its own formal or informal, explicit
or tacit, rules
Games in the laboratory are usually played without labels
but subjects inevitably apply their own labels
The context-free bias
Labels increase experiments’ external validity with a
minimal sacrifice of internal validity
In particular, to test learning and cognitive models, it is
necessary to remind and to evoke contexts which may
activate emotions, association, similarities in the
laboratory
Labels can make subjects more or less rational in
relation to the evoked context
The power of labels
Jones and Sugden Theory and Decision (2001)
Positive confirmation bias: tendency, when testing an existing belief, to search for evidence which could confirm that belief, rather than disconfirming it
The original Wason’s selection task was formulated in highly abstract terms
Correct response was facilitated by adding thematic content to the task, i.e. a cover story which accounts for the statement and gives some point to the task
Labels make subjects more rational
Jones and Sudgen’s CriticismThe original selection task was formulated in highly abstract terms
Design
Subjects have to pay a fixed cost per card turned over
After they have made this choice, the chosen cards are turned over
Then they make the judgment that the statement is ‘true’ or ‘false’
Finally, the remaining cards are turned over and they receive a fixed reward if and only if their judgment was in fact correct
Jones and Sudgen’s Story 1 Relatives . A survey is taken of 100 people in Los Angeles,
Seattle, London and Norwich who have relatives living in other cities. Each person in the survey living in Britain has relatives in Los Angeles or Seattle and each person living in America has relatives in Norwich or London. No one has relatives in more than one city. The details of the survey are written down on report cards by putting the city each person lives in on one side of the card and the city their relatives live in on the other side. A sample of four report cards is selected. Look at whichever cards you wish to test the statement:
[Standard statement] Every person in the sample who lives in London also has a relative who lives in Los Angeles.
[Contraposed statement] Every person in the sample who lives in Seattle also has a relative who lives in Norwich.
Jones and Sudgen’s Story 2Drinkers . Only people over the age of eighteen are allowed to drink alcohol in a pub in Britain. A survey is carried out of 100 people in a large public house which identifies their age and whether they are drinking alcohol or a soft drink. Each person’s details are put down on a report card with the person’s age on one side and their drinking behaviour on the other. A sample of four report cards is selected. To find out if the four people in the sample are obeying the law, look at whichever cards you wish to test the statement: [Standard statement] Every person in the sample who is drinking alcohol is also over eighteen. [Contraposed statement] Every person in the sample who is under eighteen is also drinking a soft drink
Jones and Sudgen’s Results
In favour of the confirming bias hypothesis: 62% of the choices (445/720)
Overwhelming evidence that subjects’ information-gathering decisions are systematically biased in favor of information which is potentially confirming
But behavior seems to have been closer to Bayesian rationality than in many other selection task experiments
Especially the drinkers story facilitates Bayesian rationality
What is the effect of financial incentives?
Innocenti, Pazienza and Lattarulo Transport Policy (2013)
Main finding: Subjects’ inclination to prefer cars over bus and metro tends to override the incentives’ effect
Laboratory behavior depends more on prior learning outside the laboratory than on gains in the laboratory
In the experiment, it is as if subjects take into the lab the preferences applied to real choices between car, bus and metro and stick to them with high probability
Labels give subjects clues to become less and not more rational
Labels make subjects less rational
Aim: to extend previous experimental evidence on travel mode choice by providing subject not only with information acquired through personal experience, but also with actual travel times of the alternative non chosen travel modes Key Findings:
subjects show a marked preference for cars
are inclined to confirm their first choices
exhibit a low propensity to change travel mode
Travel mode choice
Experimental literature on travel mode choice relies on studies on route choice
Common object: coordination games, i.e. the payoff each traveler can achieve is conditional on her/his ability to diverge from or to converge with other travelers’ choices
Selten et al. (2007), Ziegelmeyer et al. (2008), Razzolini-Dutta (2009) provide laboratory evidence that choices between route A and route B generate Nash equilibria
Background literature
62 undergraduate students (31 women and 31 men) from the University of Firenze
Computerized experiment
Between subject
Each session lasted approximately an hour
Average earnings 18.4 euro
The design
1) Choice between car or metroMetro travel costs are fixed, while car costs are uncertain and determined by the joint effect of casual events and traffic congestion
2) Choice between car or busCar and bus are both uncertain and determined by the combination of casual events and traffic congestion.
Travelers’ utility only depends on travel times, which are converted in monetary costs. After each choice, subjects are informed of actual times of both available modes.
The design
Metro Car treatment - the expected total costs of car and metro were equivalent if the share of car users was not greater than 55%;
Bus 1.0 Car treatment - the expected total costs of car and bus were equivalent if the share of car users was not greater than 55%;
Bus 0.8 Car treatment - the expected total cost of the bus was 20% lower than car expected total costs if the share of car users was not greater than 55%.
The design
Results – Preference for CarsTable 8 Proportion of car choices by treatment (each five periods)
Period Metro Bus 1.0 Bus 0.8 1 0.70 0.60 0.59 5 0.67 0.67 0.35
10 0.60 0.47 0.35 15 0.57 0.67 0.47 20 0.57 0.53 0.53 25 0.77 0.53 0.41 30 0.67 0.73 0.71 35 0.70 0.60 0.71 40 0.60 0.53 0.53 45 0.67 0.60 0.53 50 0.73 0.53 0.53
Total 0.68 0.58 0.50
Results – Preference for cars
Results – Preference for cars
Results – Preference for cars
Results – First Choice Effect
Travel mode choice is significantly affected by heuristics and biases that lead to robust deviations from rational behaviour
Travelers choose modes using behavioural rules that do not necessarily involve the minimization of total travel costs.
Subjects show a marked preference for cars, are inclined to confirm their first choice and exhibit a low propensity to change travel mode.
24
Conclusions 1
In repeated travel mode choice, available information is not properly processed, cognitive efforts are generally low and rational calculation play a limited role
The habit of using cars should be assumed to be relatively resistant, to the effect of economic incentives
Little progress can be expected by asking travelers to voluntarily reduce the use of a car or even by subsidizing public transport costs
25
Conclusions 2
Why People Choose Cars, Even When Mass Transit Would Serve Them Better
By Eric Jaffe
Feb 01, 2013
http://www.theatlanticcities.com/commute/2013/02/why-people-chose-cars-even-when-metro-would-be-faster/4566/
Reactions - The Atlantic Cities
WithheldName • 2 months ago
PRIVACY, CONTROL, COMFORT, FLEXIBILITY are very rational advantages to cars over buses and trains. This is the dumbest study and dumbest article I've read all week.
ChicagoStreetcarRenaissance WithheldName • 2 months ago
I look at all those hundreds of thousands of people stuck in traffic together out on the highway every morning and every evening, and here are some words that never come to mind: privacy, control, comfort, and flexibility.
WithheldName ChicagoStreetcarRenaissance • 2 months ago
When I look at city buses rolling down the street, the words that come to mind are things "passengers who have soiled themselves since their last change of clothing", "passengers who are mumbling to themselves", "passengers who are swatting at invisible flies", "passengers who are asking others for spare change", "passengers who are convicted felons", etc.
neroden • 3 months ago
FWIW, this is an Italian study and therefore only describes the biases of Italians, and in fact only of Sienna and Florence residents. There's any number of reasons why Italians might have preconceived pro-car biases, including a history of terrible urban transit. Particularly in Sienna and Florence.Try this study in London (hell to drive, everyone takes the train) and see if you get the same results. I guarantee you won't. People are using their prior biases.
Reader’s Comments
One of the basic tenets of laboratory methodology is that the use of non-professional subjects and monetary incentives allows making subjects’ innate characteristics largely irrelevant
In our experiment, it is as if subjects take into the lab the preferences applied to real choices and stick to them with high probability
This inclination to prefer cars tends to override the incentives effect
Labels give subjects clues to become less and not more rational
Back to Methodology
The use of presentations with virtual reality (VR) simulations can convey objectively this kind of context
“A Virtual Experiment is an experiment set in a controlled lab-like environment, using typical lab or field participants, that generates synthetic field cues using Virtual Reality (VR) technology.” Fiore, Harrison, Hughes and Rutström (2009) FHHR (p. 66)
Virtual experiments are not defined as just those occurring over the web (Virtual Worlds experiments as a subset of Virtual Experiments)
Virtual Experiments (VE)
Virtual Experiment combines insights from virtual reality (VR) simulations in computer science, decision making and ecological rationality from psychology, and experiments from economics
The methodological objective of Virtual Experiments is to combine the strengths of the artificial controls of lab experiments with the naturalistic domain of field experiments
Virtual Experiments (VE)
High Immersive Virtual Experiments (HIVE) utilize specialized displays such as CAVE, head-mounted displays or augmented reality, which perceptually surround subjects. The individual perceives himself to be enveloped by, included in, and interacting with an environment providing a continuous stream of stimuli.
Desktop or Low Immersive Virtual Experiments (LIVE) use computer screen based applications of virtual reality, such as “ad hoc” virtual simulations or virtual worlds (Second Life) to provide a weaker sense of presence.
High and Low Immersive VE
Cave (HIVE)
Head-Mounted Display (HIVE)
Virtual simulations (LIVE)
Bateman et al. 2009In the majority of choice experiments on gain-loss asymmetry (WTA>WTP) the attributes of non-market goods are conveyed to respondents as a table of numeric and/or categorical data.
Compared to the standard presentation, preferences elicited in the Virtual Experiment are less variable and exhibit a significant reduction in asymmetry between willingness to pay for gains and willingness to accept for corresponding losses.
Applications – Gain/Loss Asymmetry
Virtual Rural Areas (Bateman 2009)
Fiore et al. (2009)Virtual Experiment to elicit risk perception from wild fires and the opportunity cost of public funds allocated to prescribed burns
Subjects experience four dynamic visual simulations of specific wild fires, with varying weather and fuel conditions. Simulations are selected to represent high and low risk of fire damage
Participants experience a sense of presence, a psychological state of “being there and take decisions closer to real behavior” (with cognitive constraints )
Applications – Risk Perception
Wild fires renderings (Fiore 2009)
Main objectives
To demonstrate that the standard tools for detecting work-related factors of risk and stress (questionnaires and interviews) are inadequate to capture workers’ real perception
To prove that low immersive simulations of work activities provide a better awareness of psycho-social risks in workplaces
ALBO Project
40
BackgroundThe 2004 European framework agreement on work-related stress, “one of the biggest health and safety challenges that we face in Europe”
“Stress is a state, which is accompanied by physical, psychological or social complaints or dysfunctions and which results from individuals feeling unable to bridge a gap with the requirements or expectations placed on them”
It can lead to changes in behaviour and reduce effectiveness at work.
Different factors originating outside the working environment or linked with it, such as work content, organisation, environment, communication
Work-related stress is the second most reported work-related health problem, affecting 22% of workers from EU 27 (in 2005)
It cuts across men and women, technicians, managers and absenteeism or staff turnover, frequent interpersonal conflicts or complaints by workers are some of the signs
Causes
42
ALBO purposesTo customize a version of the Adventure-based Learning™ set, and implement it in a number of real business cases
To issue guidelines for the assessment of psycho-social hazards in the
To dynamically changethe current and future workplaces in a manner that is mostly driven by the real needs of the people that are populating them (Living Lab approach)
Technological frameworkAdventure-based Learning™ is an innovative training and coaching method based on simulated game scenarios, which companies use to improve the knowledge and skills of their employeesA Virtual Coach accompanies the user through the entire Adventure and facilitates the process. The Coach motivates, gives tips, and provides feedback on what the learner has done. The Gaming environment motivates people to face the challenges and find solutions to the tasks that are to be performed.
44
SaaS scheme
45
Scientific frameworkIndividuals are generally myopic in assessing risks and stressful situations [Tversky & Kahneman (1981), Slovic (1987, 2001)]
If individuals exhibit a positive attitude to a risky prospect, they overvalue the associated benefits while under-assess the associated costs. Similarly, when their attitude is negative, the related costs are systematically overvalued [Rigoni (2006)]
Reactions to risks are triggered emotionally and not determined by rational scrutiny [Damasio (1994)]
46
Scientific framework
Social Cognitive Theory (Bandura 1977) argues that a model demonstrating a given behaviour can have the effect of encouraging the same behaviour in the observer
Identification increases the likelihood of performing learned behaviours without the need of experiencing individual rewards or
Fox, Arena & Bailenson (2009) provide evidence that virtual self-models can be effective instigators of positive health behaviour change
47
Research approachPilot selectionIdentification of business process(es)Identification of working space(s) where these
processes developDefinition of a number of interactive scenarios,
within each working space
Video recording of real-life instantiations of the selected interactive scenarios, with the aim of identifying the most common “misinterpretations” and “misbehaviours” in terms of risk and stress perception
48
Research approach
Transformation of the video clips into Virtual Reality-like ambientations, in order to prevent from identification of the targeted organisation to third parties.
Embedment of the virtual simulations into the Adventure Game, as an interactive learning environment inviting people to examine, interpret and assess the various scenarios depicted guided by the Virtual Coach.
Expected results
A number of customised Adventure Games are created to explore the emergence and dynamics of psycho-social risks among employees
Employees are protected in their anonymity by the Virtual Reality representations, to ensure the formulation of a critical judgement and a more objective and contextualized assessment of the situation represented in the virtual simulation
Ritorno
Informazioni sull‘Avventura Città virtuale Uscire
Benvenuti nel nostro spazio interattivo virtuale!
X CONGRESSO NAZIONALE S.I.P.S.A. Eva Venturini, Alessandro Innocenti e Valeria Faralla
X CONGRESSO NAZIONALE S.I.P.S.A. Eva Venturini, Alessandro Innocenti e Valeria Faralla
X CONGRESSO NAZIONALE S.I.P.S.A. Eva Venturini, Alessandro Innocenti e Valeria Faralla
Come valuti l'organizzazione del lavoro rispetto a…?Riguardo alla situazione rappresentata nel video, lei percepisce
questi aspetti come soddisfacenti oppure no
X CONGRESSO NAZIONALE S.I.P.S.A. Eva Venturini, Alessandro Innocenti e Valeria Faralla
LIVExperiment 1 Workers’ awareness of biases in risk perception is
enhanced by virtual simulations of their work activities
LIVExperiment 2Individual risk attitude under social exposure in the lab is modified by the presence of a virtual coach
ALBO Research Hypotheses
Virtual Movies vs. Real Movies in the Assessment of Work Related Stress
Obj.: to verify the presence of differences in the physiological and cognitive activations while subjects watch real movies vs. virtual movies
Hp.: Vision of virtual movies is associated with a lower physiological activation, a more detailed narrative and a greater detection of ‘errors’, due to a more objective perspective.
LIVExperiment 1
Between-subject 24 postgraduate students2 Conditions: Real clip of a job stress situation + Virtual simulation of the same clipsDetection of three physiological indices:
Heart rate Electromyography (EMG), i.e. electrical impulses
of face muscles at rest and during contraction Skin Conductance Level Eye-tracking
Questionnaire Generalized Self-Efficacy and Locus of Control (beliefs about control of events)
LIVExperiment 1 - Design
LIVExperiment 1- Material
LIVExperiment 1 – HR
1 2 3 4 5 6 7 8 9 10 110
5
10
15
20
25
HR
FC Avatar FC Reale
LIVExperiment 1 – EMG
1 2 3 4 5 6 7 8 9 10 110
2
4
6
8
10
12
14
EMG
EMG Avatar EMG Reale
Test per campioni appaiati
Differenze a coppie t df Sig. (2-code)
Media Deviazione std. Errore std. Media
Intervallo di confidenza per la differenza al 95%
Inferiore Superiore
Coppia 1
-,62364 ,54294 ,16370 -,98839 -,25888 -3,810 10 ,003
Coppia 2 EMG_avatar2 - EMG_reale2
-,12143 ,34944 ,09339 -,32319 ,08033 -1,300 13 ,216
Coppia 3 EMG_avatar3 - EMG_reale3
-,71875 ,62818 ,22210 -1,24392 -,19358 -3,236 7 ,014
Coppia 4 FC_avatar1 - FC_reale1 ,97455 3,15244 ,95050 -1,14329 3,09238 1,025 10 ,329 Coppia 5 FC_avatar2 - FC_reale2 -3,39143 3,16552 ,84602 -5,21914 -1,56371 -4,009 13 ,001 Coppia 6 FC_avatar3 - FC_reale3 -4,22250 3,22786 1,14122 -6,92106 -1,52394 -3,700 7 ,008
tab 1: extracts dialogues and difference between the average of the corresponding activation
SEQUENCE TOPIC WHAT IS SHOWN:
dialogue between employee and user
statistical significance of difference in activations
“avatar group” vs “real group”
EMG BPM
Extract 1 Introduction
to the problem
E: “I’m sorry lady, but how could we help you?”
U: “We don’t have money for the rent and we must leave our
hause, we don’t know where to go!”
,003** ,329 (N.S)
Extract 2 Help denied U: “but we are poor, the City must give us a house!”
E: “lady, I understand your difficoult situation but th City must
not do nothing, a graduatory has been redacted...all
regular...and you are not found to be eligible”
U: “ I don’t understand...what do you mean?”
,216 (N.S) ,001***
Extract 3 Employee
liquidates
the problem with
strict rules
E: “Next year will be done another graduatory...you can
partecipate to that!”
U: “yes but where shall go, live in the open? And whare shall
we eat?!”
,014* ,008**
Note: *p<.05; **p<.01; ***p<.001
Physiological activations (HR and EMG) are lower
under virtual simulations
Limitations: Small sample and the use of pilot
movies
Research Implications: Results obtained through
the use of virtual reality tools allow to design stress
assessment interventions and online training
courses with virtual coaches
LIVExp 1 – Preliminary Findings
VS..
LIVExp 2- Design
To test if subjects’ behaviour in VE conforms to results generated in conventional experimentation
“Virtual experiments might be more convenient than lab experiments if he sees people behave in he same way in real-world and virtual experiments” (List 2007)
“Determining where virtual world behavior mimics real world behavior is quite important for methodological reasons. If virtual world behavior can be treated as a model of human behavior in general, this would allow a fresh approach to empirical social science” (Castronova 2008)
Main Approach to VE
More naturalistic and less simple settings than
laboratory
Cheaper to maintain virtual laboratory facilities
Easier to control decision tasks and enviroments
No involuntary non-verbal communication
Wider and unbiased population
LIVE Experiments - PROS
Virtual situations project a game-like atmosphere
Proteus effect / deindividuation (may be an asset)
it is difficult to establish subject trust in computer
software
(virtual worlds experiments) subjects’ identity is not
checked because physical presence is lacking
LIVE Experiments - CONS
The difference between virtual and laboratory experiments and between virtual and real behavior is an asset rather than a problem for experimental economics.
It can very helpful in solving some failures of lab experiments
Ir/relevance of the context Intertemporal choice – longtime experiments Heterogeneous subject pools Cross-cultural and professional comparisons
Conclusions
Induced-value theory: use of a reward medium allows to induce pre-specified characteristics in experimental subjects and to make subjects’ innate characteristics largely irrelevant (Smith 1992)
The central aspect of the VX methodology is a VR environment that makes participants experience a sense of presence, a psychological state of ‘‘being there.’’ This sense depends on the degree of involvement that participants experience as a consequence of focusing attention on the set of stimuli and activities generated by the VR simulation.
(Fiore et al. 2009)
Back to methodology