Upload
sevita
View
24
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Towards improved user-product testing with cognitively enhanced scenarios. Wilfred van der Vegte. Where I work. Delft University of Technology (2011 figures) Founded in 1842 17,250 students 12 undergraduate programmes 33 post-graduate programmes 2,540 academic staff - PowerPoint PPT Presentation
Citation preview
Towards improved user-product testing with cognitively enhanced
scenariosWilfred van der Vegte
Delft University of Technology
(2011 figures)• Founded in 1842• 17,250 students• 12 undergraduate
programmes• 33 post-graduate
programmes• 2,540 academic staff• 8 faculties (departments)
Faculty ofIndustrial Design Engineering
• Founded in 1969
• Largest university-based design* school worldwide(*product design, consumer durables)
• 1,980 students
• >4,700 alumni (MSc level)
• 3 departments,94 scientific staff
Where I work
Where I work
Section of Computer-Aided Design Engineering• ICT in product design• Historically: CAx• Currently: Cyber-physical systems, ubiquitous
computing, smart products:how to apply technology in products & in product design
• Headed by Prof. dr. mult. Imre Horváth• ~20 people, ~10 permanent scientific staff
Towards improved user-product testing with cognitively enhanced
scenariosWilfred van der Vegte
Typical engineering simulations
Involving humans in product simulations
Real humans(interactive simulation)
Virtual humans(computer models)
computer simulation
product model
computer simulation
product model
human model
human model
perception
decision-making(brain)
action (muscles)
motor control(brain, CNS)
Does not require human
subjects
?
meta
bolis
m
Simulating use with virtual humans
virtual human
perception
decision-making(brain)
action (muscles)
motor control(brain, CNS)
do we have to simulate
all of this?No ...let’s
simplify
scenario bundle(by designer)
approximate
approximate
already solvedby others
meta
bolis
m
The scenario bundle
• Designer’s conjecture of human actions /decision-making
• Graphical notation
• Possible paths through use processconnecting interactions
• Contains multiple options(courses of use process)
• Controls a physics simulation of the product
• Is specified as an automaton(←in this case a statechart)
hold retract align
reach
reorient
release
hesitate
place
clasp
move
join
unwrap
enter
wait
hesitate
switch off
switch on
reach
push
drop
lift
catch
drop
throw
guide
break
locate
grasp
reach
pull
linger
hold
close
loosenreach
carry
retrieve
wait
carrylift
mix
pull
followenter
reduce
unfold
open
insert
increase
tighten
hold
hold
push
adjust rotate
Example: snack dispenser
sim
ula
tion
scen
ari
o
bu
nd
le
pro
gra
mm
ing
of
pro
du
ct
Advantages
• Use process of the product can be tested if only a computer model is available
• No human subjects needed
• Designer can do what-if studies
• One scenario, multiple varieties of the product design
• Varieties of how the product is programmed
• Varieties in human behaviour, e.g., hesitation
• Varieties of scenario: rearrange blocks & arrows
• Building blocks of scenarios are reusable in other design projects
Key limitations
• Simulation of physics computationally demanding and not (yet) very stable
• Algorithms for realistic simulation of low-level human motor control exist, but have not been included
• Lack of realism in human information processing (thinking, reasoning, ...): only what the designer preconceives is included
But ... Are animations of physics and
human motor behavior always needed?
No! Often only the course of values of specific variables over time is important!(power consumption, temperature, speed, ...)
Demonstrative example: espresso machine with power-save function for water heating
Investigation of• boiler temperature• energy consumptionover a typical use process in which several coffee-making sessions take place, over several hours / days / weeks /...with given settings for power saving (reduce power by x% after non-use over Δt = t1 and by 100% after another Δt = t2)
Make_one_cup/ size=choosefrom(2)
Push_Buttonwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A Release_Button
entry:button=none
Wait_for_coffee
after (wait,sec)
[pump_sound== 1]
[pump_sound== 0]
[size== 1]
Remove_cup
Building a scenario bundle for using the espresso machine
Make_one_cup/ , intervention=choosefrom(3)size=choosefrom(2)
Push_Buttonwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A Release_Button
entry:button=none
Wait_to_halt_serving
Espr/desired_volume= arbitrary(55 ,1 , 200 , 17.5 )
Lungo/desired_volume= arbitrary ( 128 ,1, 250 , 30 )Wait_for_coffee
Push_Button_againwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A
Release_Button_again/button=none
after (wait,sec)
[pump_sound== 1]
[intervention== 2]
[pump_sound== 0][perceived_cup_vol>desired_volume||perceived_res_content== 0]
after (wait,sec) [pump_sound== 0]
[size== 1]
[size== 1 ]
[size== 1]
Remove_cup
Building a scenario bundle for using the espresso machine
Make_one_cup/ , intervention=choosefrom(3)size=choosefrom(2)
Keep_pressedEspr/desired_volume= arbitrary (55 ,1, 200 , 17.5 )
Lungo/desired_volume= arbitrary( 128 ,1, 250 ,30 )
Push_Buttonwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A Release_Button
entry:button=none
Wait_to_halt_serving
Espr/desired_volume= arbitrary(55 ,1 , 200 , 17.5 )
Lungo/desired_volume= arbitrary ( 128 ,1, 250 , 30 )Wait_for_coffee Release1_Button
entry:button=none
Push_Button_againwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A
Release_Button_again/button=none
after (wait,sec) [intervention== 3]
[perceived_cup_vol>desired_volume||perceived_res_content== 0]
[pump_sound== 1]
[intervention== 2]
[pump_sound== 0][perceived_cup_vol>desired_volume||perceived_res_content== 0]
[pump_sound== 0 ]
after (wait,sec) [pump_sound== 0]
[size== 1 ]
[size== 1]
[size== 1 ]
[size== 1]
Remove_cup
Building a scenario bundle for using the espresso machine
Make_one_cup/ , intervention=choosefrom(3)size=choosefrom(2)
Keep_pressedEspr/desired_volume= arbitrary (55 ,1, 200 , 17.5 )
Lungo/desired_volume= arbitrary( 128 ,1, 250 ,30 )
Push_Buttonwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A Release_Button
entry:button=none
Wait_to_halt_serving
Espr/desired_volume= arbitrary(55 ,1 , 200 , 17.5 )
Lungo/desired_volume= arbitrary ( 128 ,1, 250 , 30 )Wait_for_coffee Release1_Button
entry:button=none
Push_Button_againwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A
Release_Button_again/button=none
after (wait,sec)[pump_sound== 0 ]
{wait= 0.1 * arbitrary(2,1,4,1)}
after (wait,sec) [intervention== 3]
[perceived_cup_vol>desired_volume||perceived_res_content== 0]
after (wait,sec)
[pump_sound== 0]{wait= 0.1 * arbitrary (2 ,1 ,4 ,1 )} [pump_sound== 1]
[intervention== 2]
[perceived_res_content== 0] [pump_sound== 0]
[pump_sound== 0][perceived_cup_vol>desired_volume||perceived_res_content== 0]
[pump_sound== 0 ]
after (wait,sec) [pump_sound== 0]
[size== 1 ]
[size== 1]
[size== 1 ]
[size== 1]
Remove_cup
Building a scenario bundle for using the espresso machine
Make_n_tot_cupsentry:n_tot= choosefrom (5 ),n_cups= 0
Eject_capsulecapsule_inserted= 0Insert_capsule_and_place_cup
capsule_inserted= 1busy= arbitrary (4 ,3 ,6 ,2 )
Refill_reservoir/supply= arbitrary ( 800 , 10 , 1000 ,500 )just_filled= 1 ,busy= arbitrary (10 ,6, 20 ,6)
Make_one_cup/ , intervention=choosefrom(3)size=choosefrom(2) , just_filled=0
Keep_pressedEspr/desired_volume= arbitrary (55 ,1, 200 , 17.5 )
Lungo/desired_volume= arbitrary( 128 ,1, 250 ,30 )
Push_Buttonwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A Release_Button
entry:button=none
Wait_to_halt_serving
Espr/desired_volume= arbitrary(55 ,1 , 200 , 17.5 )
Lungo/desired_volume= arbitrary ( 128 ,1, 250 , 30 )Wait_for_coffee Release1_Button
entry:button=none
Push_Button_againwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A
Release_Button_again/button=none
Remove_cup/n_cups++
[perceived_res_content< 0.1 ]
after(5 ,sec)
after (busy, sec)[capsule_inserted== 1]
after (busy,sec)
after (wait,sec)[pump_sound== 0 ]
{wait= 0.1 * arbitrary(2,1,4,1)}
after (wait,sec) [intervention== 3]
[perceived_cup_vol>desired_volume||perceived_res_content== 0]
after (wait,sec)
[pump_sound== 0]{wait= 0.1 * arbitrary (2 ,1 ,4 ,1 )} [pump_sound== 1]
[intervention== 2]
[perceived_res_content== 0] [pump_sound== 0]
[pump_sound== 0][perceived_cup_vol>desired_volume||perceived_res_content== 0]
[pump_sound== 0 ]
after (wait,sec) [pump_sound== 0]
[size== 1 ]
[size== 1]
[size== 1 ]
[size== 1]
Remove_cup
Building a scenario bundle for using the espresso machine
User
Models_and_specifications
User User_cupboard
Espresso_machine Supplier
Espresso_machine_logic Supplier_processing
Courier_service
Espresso_machine_physics
PumpShipping
Boiler Cup_filling
Prepare_orders
Track_consumption_remotelyBoiler_thermostatCoffee_serving
Reservoir
Boiler
Detect_power_change
Compute_T_water
Compute_input_time
heating_or_coolingduring:input_time=t-t1exit:input_time=0
compute_P_change/P_change=P_heat-P_prevT_last=T_water
store_previous_value/P_prev=P_heatP_change=0
no_heating_or_cooling
Exponentialduring: T_water=T_amb+P_heat*k_1...-(T_amb+P_heat*k_1-T_last)*ml.exp(-(k_3/m_water)*input_time)
T_water = T_amb + P_heat*k1 - k2*exp[-(k3/m_water)*t];k_2 = 1-(T_amb_P_heat*k_1-T_last)
Constant_and_cold
[P_change>0]/t1=t/T_water=T_ambT_last=T_amb
[hasChanged(P_heat)==1]
[P_change<0]/t1=t
[P_change>0]/t1=t
[P_heat==0&&...T_water-T_amb<0.5]/T_water=T_amb
[P_heat>0]
Pump 2
ON/P_pump=59
OFF/discharge=0,P_pump=0,empty_sound=0
Run 2
detect_capsule 1absent/flow=5.86
present/flow=4.29
build_up_pressure/ deliver/discharge=flow
run_dry/discharge=0,empty_sound=1
[pump==1] [pump==0]
[capsule_inserted==1]
[capsule_inserted==0]
[capsule_inserted==0]
after(2.5,sec)
[reservoir_content<5]
To compensate for the missing 3D physics simulation, we have now used a hybrid automaton (discrete+continuous)
In addition, the statechart has become a timed hybrid automaton (THA) to efficiently deal with timing (latency, delays, time-outs,...)
User
Scenario_network
Go_through_m_tot_sessions_of_coffee_makingentry:m_tot= choosefrom (5 ),m_sessions= 0
Fill_reservoir/supply= arbitrary ( 900 , 10 , 1000 , 650 )just_filled= 1
Wait_for_light
Make_n_tot_cupsentry:n_tot= choosefrom (5 ),n_cups= 0
Eject_capsulecapsule_inserted= 0Insert_capsule_and_place_cup
capsule_inserted= 1busy= arbitrary (4 ,3 ,6 ,2 )
Refill_reservoir/supply= arbitrary ( 800 , 10 , 1000 ,500 )just_filled= 1 ,busy= arbitrary (10 ,6, 20 ,6)
Make_one_cup/ , intervention=choosefrom(3)size=choosefrom(2) , just_filled=0
Keep_pressedEspr/desired_volume= arbitrary (55 ,1, 200 , 17.5 )
Lungo/desired_volume= arbitrary( 128 ,1, 250 ,30 )
Push_Buttonwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A Release_Button
entry:button=none
Wait_to_halt_serving
Espr/desired_volume= arbitrary(55 ,1 ,200 , 17.5 )
Lungo/desired_volume= arbitrary ( 128 ,1, 250 , 30 )Wait_for_coffee Release1_Button
entry:button=none
Push_Button_againwait= 0.1 * arbitrary (2,1,4,1)
Lungo/button=B
Espr/button=A
Release_Button_again/button=none
Remove_cup/n_cups++
Take_short_break
Break/wait= arbitrary( 100 , 30 , 200 ,200 )go_further= choosefrom (3)
Release_Button_Aentry:button=none
Push_Button_A/button=ACheckLight
Empty_reservoir_and_take_long_break/supply= 0 ,just_filled= 0 ,switch_status= 0wait= arbitrary ( 200 ,100 , 1000 , 300 )
[m_sessions==m_tot]after (wait,sec)
after (0.1 ,sec)/switch_status= 1
[blinking== 0]
[switch_status== 1 ]
[blinking== 0]
[n_cups==n_tot]/m_sessions++
[blinking== 0]
[perceived_res_content< 0.1 ]
after(5 ,sec)
after (busy, sec)[capsule_inserted== 1]
after (busy,sec)
after (wait,sec)[pump_sound== 0 ]
{wait= 0.1 * arbitrary(2,1,4,1)}
after (wait,sec) [intervention== 3]
[perceived_cup_vol>desired_volume||perceived_res_content== 0]
after (wait,sec)
[pump_sound== 0]{wait= 0.1 * arbitrary (2 ,1 ,4 ,1 )} [pump_sound== 1]
[intervention== 2]
[perceived_res_content== 0] [pump_sound== 0]
[pump_sound== 0][perceived_cup_vol>desired_volume||perceived_res_content== 0]
[pump_sound== 0 ]
after (wait,sec) [pump_sound== 0]
[size== 1 ]
[size== 1]
[size== 1 ]
[size== 1]
after ( 0.2 ,sec)[go_further> 1]after (wait,sec) [blinking== 1]
Remove_cup
User
Energy consumption (kWh)
Reservoir content (ml)
Boiler water temperature (K)
Power- save mode
Power- save mode
Serving lungos
Serving espressos
User takes break
User takes break
Thermostatcontrols temperature
Thermostat controls temperature
User switches machine off
Time (s)
User refills reservoir
These simulation outcomes can be
generated up to 5000 faster than real time
Revisiting the limitations
• Simulation of physics has been simplified (no more 3D), and is fast and reliable
• Low-level human motor control is disregarded, still the whole use process can be simulated (in this case)
• Still lack of realism in human information processing (thinking, reasoning, ...): only what the designer preconceives is included
So ... how to increase realism in human information processing (thinking, reasoning, ...)?Aspects of human information processing to be simulated
1. Logic of decision making:under which condition what action is taken?e.g. “IF cup is full THEN retrieve cup from machine”:straightforward execution of ‘normal’ use, assuming a particular history of preceding events.
• But can a simulation predict a user acting according tothe production rule“IF cup is full THEN stick finger in it”? →unlikely!
So ... how to increase realism in human information processing (thinking, reasoning, ...)?Aspects of human information processing to be simulated
1. Logic of decision making:under which condition what action is taken?e.g. “IF cup is full THEN retrieve cup from machine”:straightforward execution of ‘normal’ use (instructions), assuming a particular history of events, including required preceding actions.
• We can howevertry to generatetypical aberrationsfrom ‘regular use’:the so-callederror phenotypes
taxonomy accordingto Hollnagel→
action in wrong place
action at wrong time
action of wrong type
action not included in current plans
repetition
reversal
omission
delay
premature action
replacement
insertion
intrusion
restart
jumping
undershoot
sidetracking
capture
branching
overshoot
error mode simple phenotype( applies to one action)
complex phenotype( applies to multiple
interconnected actions)
So ... how to increase realism in human information processing (thinking, reasoning, ...)?Aspects of human information processing to be
simulated
2. Processing time:How long does it take to accomplisha given action, taking into accountaspects such as memory retrieval,memory capacity, learning,multitasking, distraction, etc.
• These aspects can be simulatedusing cognitive architecturessuch as ACT-R
• A cognitive architecture is
• a blueprint of the human mind• based on findings from brain science• filled with psychologically validated
task models expressed as production rules
central production system
retrieval buffer
declarative module goal buffer
intentional module
external world
visualbuffer
motorbuffer
visualmodule
motormodule
(basal ganglia )(ventrolateral
prefrontal cortex )
(temporal cortex / hippocampus)
(dorsolateral prefrontal cortex)
(not identified)
(parietal cortex)
(motor cortex)
(occipital cortex)
(motor cortex / cerebel-
lum)
central production
systemretrieval buffer
declarative module goal buffer
intentional module
external world
visual
buffer
motor
buffer
visual
module
motor
module
How to realize simulations with cognitively enhanced scenarios (CES)
central
produc -
tion
system
re -
trieval
buffer
declar -
ative
module
goal
buffer
inte
nti
on
al
mo
du
le
ex
tern
al
wo
rld
visual
buffer
motor
buffer
visual
module
motor
module
inte
nti
on
al
mo
du
le
ex
tern
al
wo
rld
How to realize simulations with cognitively enhanced scenarios (CES)
central
produc -
tion
system
re -
trieval
buffer
declar -
ative
module
goal
buffer
visual
buffer
motor
buffer
visual
module
motor
module
Scenario bundle (THA)(human actions)
THA of the ‘world’(product, environment)
Cognitive Architecture (ACT-R)
action in wrong place
action at wrong time
action of wrong type
action not included in current plans
repetition
reversal
omission
delay
premature action
replacement
insertion
intrusion
restart
jumping
undershoot
sidetracking
capture
branching
overshoot
error mode simple phenotype(applies to one action)
complex phenotype(applies to multiple
interconnected actions)Human-ErrorPhenotype Generator
Operation of ship locks –our first real-life application?
usually combined with a movable bridgevarious types of boats and skippers
gates, traffic lights, leveling, etc., are allman-controlled
usually 2-3 chambers
operator works remotely, usingmultiple monitors
Operation of ship locks –our first real-life application?
The Dutch Government Agency of Public Works & Water Management is developing a new system of centralized control rooms for locks, movable bridges and other objects With CES simulations the
system can be pre-tested• without human subjects• batchwise, systematically
varying parameters (weather, traffic density, ...)
• to reveal incidents theoretically happening once in 10/100/1,000 years
• that cannot be discovered through interactive testing
Potential problems:• Multitasking, but also
boredom, can cause cognitive overload, cognitive lock-up& other error-invoking mental phenomena
• Errors can cause injury, collisions or even flooding
Envisioned setup for first application
central
produc -
tion
system
re -
trieval
buffer
declar -
ative
module
goal
buffer
visual
buffer
motor
buffer
visual
module
motor
module
Scenario bundle (THA)(human actions)
THA of the ‘world’(product, environment)
Cognitive Architecture (ACT-R)
action in wrong place
action at wrong time
action of wrong type
action not included in current plans
repetition
reversal
omission
delay
premature action
replacement
insertion
intrusion
restart
jumping
undershoot
sidetracking
capture
branching
overshoot
error mode simple phenotype(applies to one action)
complex phenotype(applies to multiple
interconnected actions)Human-ErrorPhenotype Generator
Conclusions
Simulation with cognitively enhanced scenarios (CES)
• can help evaluating human-system interactions where many variations can influence the outcomes
• based on psychologically validated knowledge on the workings of the human brain
• benefits from simplifications & shortcuts:simulation speed is determined by slowest element in simulation loop → avoid complex models & algorithms