Upload
kartheedhanya008
View
219
Download
0
Embed Size (px)
Citation preview
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
1/40
Gilbert Beyer, Gian Mario Bertolotti, Andrea Cristiani, Shadi Al Dehni
AN ADAPTIVE DRIVER ALERT
SYSTEM MAKING USE OFIMPLICIT SENSING AND
NOTIFICATION TECHNIQUES
2ND WORKSHOP ON USER CENTRIC PERVASIVEADAPTIVE SYSTEMS | 06.12.2010
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
2/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 2 of 41
Agenda
1 Adaptation Strategy
2 Sensor-Actuator Support3 Hardware and Sotware
4 Perormance Tests
5 Discussion
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
3/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 3 of 41
Driver Alertness
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
4/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 4 of 41
Causes o Driver Distraction
The driver himsel (atigue, drunkeness)
The vehicle (type, condition)The environment (trac, weather)
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
5/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 5 of 41
Long-Term Driver Distraction
Psycho-physiological state (atigue)
Secondary tasks (operating car stereo)Social activities (chatting with co-driver)
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
6/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
adapted from Riener and Ferscha (2009)
Page 6 of 41
Feedback Loops in Vehicles
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
7/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 7 of 41
Sensing Driver Distraction
Hands-o-wheel Time
Eyes-o-road TimeConversational Level
Emotional expressions
Signs o drowsiness
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
8/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 8 of 41
Notication Modalities
1 Vision, Hearing, Touch (each
having its own timely behavior)2 Multimodal inormation
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
9/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 9 of 41
Adaptation Strategy
Actuator: Visual, implicit
Sensor 1: DrowsinessSensor 2: Head-position
Sensor 3: Can Bus data
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
10/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 10 of 41
Combing Car and User Data
Eyes-o-road while...
1 driving ast, is dangerous2 standing, is sae
3 turning, is required
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
11/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
http://www.bmw.com/ (2010)
Page 11 of 41
Head-up Displays
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
12/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Karvonen et al. (2006) Dienelt, M. (2005)
Page 12 of 41
Adaptive Head-up Displays
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
13/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Blanco et al., US Patent No. 7764247 AWAKE Project, IST-2000-28062
Page 13 of 41
Adaptive Head-up Displays
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
14/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 14 of 41
Sensing Drowsiness
1 Intrusive techniques:
Electromyogram, Respiration,
GSR, Electro-oculography
2 Non-intrusive techniques:
Vision-based
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
15/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 16 of 41
Automotive Demonstrator
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
16/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 17 of 41
Sensor-Actuator Support
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
17/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 18 of 41
Simulator Setup
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
18/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 19 of 41
Implicit Sensing Sotware
1 Drowsiness Detection
2 Head Position Detection
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
19/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 20 of 41
Drowsiness Detection
Johns, M.W. (2003)
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
20/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 21 of 41
Drowsiness Detection
Johns, M.W. (2003)
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
21/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 22 of 41
Main Constraints
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
22/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 23 of 41
Main Constraints
1 Reduce computational load
2 Reduce error rate
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
23/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 24 of 41
Eye Detection Algorithm
1 Search eye using Viola-Jones
2 I eye is ound, save copy as template
3 I eye is not ound: template matching
4 Successive ailures: re-initialization
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
24/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 25 of 41
Eye Detection Algorithm
Largest rectangle: Viola Jones, Template Matching Adjusted search window
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
25/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 26 of 41
Blink Test Algorithm
1 Eye-image processed to binary picture
2 Eliminating pupil refections and noise
3 Obtain vertical projection histogram
4 Calculate eye openness level
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
26/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Greyscale image Binary image
Page 27 of 41
Blink Test Algorithm
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
27/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Open eye Closed eye
Page 28 of 41
Blink Test Algorithm
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
28/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 29 of 41
Head-Position Detection
Eye within rest area Eye outside rest area
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
29/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 30 of 41
Experiments and User Analysis
1 Perormance Tests
2 Driver Observation in the Lab
3 Road Tests
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
30/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 31 of 41
Passive Sensing Perormance Tests
Collected sets o data:
1 Recognition o slow/ast blinks
2 Tracking o slow/ast eye movements
3 Slow/ast head movements
4 Blink ail errors
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
31/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 32 of 41
Passive Sensing Perormance Tests
Kinds o error:
1 tracking miss (in successive rames)
2 detection miss (shadows or occlusions)
3 blink miss (e.g. due to rapid blinks)
4 blink ail (erroneously detected blink)
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
32/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 33 of 41
Passive Sensing Perormance Tests
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA 2010 | 06 12 2010
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
33/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 34 of 41
Passive Sensing Perormance Tests
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA 2010 | 06 12 2010
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
34/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 35 of 41
Night Vision Camera
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA 2010 | 06 12 2010
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
35/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 36 of 41
Lab Study
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA 2010 | 06 12 2010
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
36/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 37 of 41
Lab Study
1 Eyes-o-road and Car State Symbols
2 15 Participants (students, colleagues)
3 Co-Driver was investigator
4 Semi-structured Interviews and
Camera Observations
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA 2010 | 06 12 2010
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
37/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 38 of 41
Lab Study
1 80 % understood that displayed
icons related to their driving behavior
2 66 % ound notications helpul
to adapt their driving style
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA 2010 | 06.12.2010
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
38/40
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
Page 39 of 41
Onboard Testing Session
1 Displaying Car State Inormation
2 Thinking Aloud Method
3 Renement o the hardware
and sotware setup
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
39/40
Page 40 of 41
Onboard Testing Session
AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010
7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni
40/40
Page 41 of 41
Discussion
Algorithm Improvements
Sensing Techniques
Future Tests and Studies
HUD Development
System Design