View
216
Download
0
Category
Preview:
Citation preview
Attention detection in driver simulator
project #4
INPG : L. Bonnaud, A. CaplierUCL : D. trevisan, B. Macq
Participants : A. Benoit, G. Chanel, P. Ngo,V. Levacic, C. Thillou
Guest star : L. Lawson
04/18/23 2
Objectives
• Multimodal detection of user state– Hypo vigilance– Stress level– In the context of driving simulator
• Alarm => multimodal feedback to the user
• Test of OpenInterfaceas an integration platform
04/18/23 3
Signals used foruser state assessment
Video(camera)
Physiological (biopac)
Head Eyes Mouth ECG GSR
Hypovigilance ? ?
Stress ?
04/18/23 4
General architecture
Significative rotation ?
High frequency ?
Both eyes closed ?
Video
Mouth opening detector
ECG analysis
Head rotation estimation
Eyes blinks detector
GSR analysis
GSRECG
Yawn ?
Fusion (redundancy, equivalence)
High heart rate variation ?
High GSR ?
Local Peak ?
Fission (redundancy, equivalence)
Sonic feedback
Graphical feedback
Haptic feedback
Screen loudspeaker Vibrating wheel
Dialog controller
04/18/23 5
Face motions analysis
04/18/23 6
Physiological signals “database”
• 3 subjects : thanks to participants!• 3 situations :
– rest / relaxation– Stress stimuli while reading a newspaper
• Hand clapping• Light in eyes• Answering simple question• Telephone call, “your dead” (killer game)
– Playing with the driving simulator (difficult tracks)
04/18/23 7
GSR signal
04/18/23 8
GSR signal
04/18/23 9
GSR signal
04/18/23 10
ECG signal : preprocessing
04/18/23 11
ECG : stress assessment
04/18/23 12
Simplified architectureof 1st prototype
Significative rotation ?
High frequency ?
Both eyes closed ?
Video
Mouth opening detector
ECG analysis
Head rotation estimation
Eyes blinks detector
GSR analysis
GSRECG
Yawn ?
Fusion (redundancy, equivalence)
High heart rate variation ?
High GSR ?
Local Peak ?
Fission (redundancy, equivalence)
Sonic feedback
Graphical feedback
Haptic feedback
Screen loudspeaker Vibrating wheel
Dialog controller
Network messages (XML)
1st PC, Matlab, Linux
2nd PC, Simulator, Windows
04/18/23 13
Realization of 1st prototype
Significative rotation ?
High frequency ?
Both eyes closed ?
Video
Mouth opening detector
ECG analysis
Head rotation estimation
Eyes blinks detector
GSR analysis
GSRECG
Yawn ?
Fusion (redundancy, equivalence)
High heart rate variation ?
High GSR ?
Local Peak ?
Fission (redundancy, equivalence)
Sonic feedback
Graphical feedback
Haptic feedback
Screen loudspeaker Vibrating wheel
Dialog controller
Network messages (text)
04/18/23 14
Conclusion
• First working prototype (demo available)– Firewire camera Matlab interface under Linux– Modification of the simulator (multi threading, TCP server,
message display)– TCP client in Matlab
• First try with Open Interface integration problem with Matlab integration to be solved by Lionel next week
• Usefulness of physiological signals• Reliable online face motions analysis
Still a lot of work to do !!!
Recommended