Embed Size (px)
Citation preview
The ULg Multimodality Drowsiness Database (called DROZY) and Examples of Use
IEEE Winter Conference on Applications of Computer Vision (WACV 2016); Lake Placid, NY; 7-10 March 2016
DROZY is a database containing various types of drowsiness-related data (signals, face images, etc.) and intended to help researchers to carry out experiments, and to develop and evaluate systems (i.e. algorithms), in the area of drowsiness monitoring.
Description
Quentin MASSOZ, Thomas LANGOHR, Clémentine FRANÇOIS, Jacques G. VERLY INTELSIG Laboratory, Dept. of Electrical Engineering and Computer Science, University of Liège, Liège, Belgium
Contact: [email protected]
• 14 young, healthy participants (3 M, 11 F) • PVT = Psychomotor Vigilance Test [1] (duration of 10 minutes) • Protocol approved by the Ethics Committee of our university.
Data acquisition
No stimulant
No sleep7:00
10:00 11:00
12:00
3:30 4:00 12:00 12:30DAY 1 DAY 2
PVT1 PVT2 PVT3
Using 24 hour times:
We thank: • the participants for enduring the acute sleep deprivation of 28-30 hours, • David Grogna and Philippe Latour (ULg researchers) for their help in supervising the data collection, • the Belgian FRIA F.R.S-FNRS for supporting Quentin MASSOZ with a fellowship.
Acknowledgments
[1] M. Basner and D. F. Dinges. Maximizing sensitivity of the psychomotor vigilance test (PVT) to sleep loss. Sleep, 34(5):581-591, 2011. [2] T. Åkerstedt and M. Gillberg. Subjective and objective sleepiness in the active individual. International Journal of Neuroscience, 52(1-2):29-37, 1990. [3] M. Gillberg, G. Kecklund, and T. Åkerstedt. Relations between performance and subjective ratings of sleepiness during a night awake. Sleep: Journal of Sleep Research & Sleep Medicine, 1994.
References
Database content
Range images
Standard drowsiness measures
Kinect v2 data
Electroencephalogram(EEG) = brain waves
Electrooculogram(EOG) = eye movements
Electromyogram(EMG) = muscle tension
Electrocardiogram(ECG) = heart activity
Annotations www.drozy.ulg.ac.be
Karolinska Sleepiness Scale [2]
KSS1 = Extremely alert
9 = Very sleepy
2345678
Reaction times [1]
RT
68 face landmarks (2D & 3D)
Polysomnography signals [2,3]
Near-infrared intensity images
Time [ms] ×1051.62 1.63 1.64 1.65 1.66 1.67 1.68 1.69 1.7N
orm
aliz
ed e
ye o
peni
ng
0
0.5
1
Closing Opening
Blink
True reaction time [s]0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Pred
icte
d re
actio
n tim
e [s
]
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.95a) Regression: we use epsilon-SVR models (with an RBF kernel) to predict the post-stimulus 1-min mean reaction time (RT) from pre-stimulus ocular parameters. We obtain an RMSE of 105.84 ms and a Pearson’s correlation of 0.67 using leave-one-subject-out cross-validation.
Two examples of use
5b) Classification: we use SVM classifiers (with an RBF kernel) to predict the post-stimulus lapses (i.e. RT>500 ms) from pre-stimulus ocular parameters. We obtain a specificity of 86%, a sensitivity of 78%, and an accuracy of 85% using leave-one-subject-out cross-validation. Time [ms] ×105
1.45 1.5 1.55 1.6 1.65
Our
alg
orith
m [m
m]
0
5
10Baseline
Eyelids distance
4) Compute ocular parameters: • values of a 10-bin histogram • mean duration of blinks • number of microsleeps • etc.
1) Get the (global) eyelids distance from the 8 eyelids landmarks.
2) Compute the baseline, i.e. the maximum opening of the eye, with an adaptive exponential smoothing.
3) Normalize and segment the blinks.
2256 360100 348
≤.5s
≤.5s
>.5s
>.5sPredictedRT
True
RT
Confusionmatrix:
