Hips do lie! A position-aware mobile fall detection system
Christian Krupitzer, Timo Sztyler, Janick Edinger, Martin Breitbach,
Heiner Stuckenschmidt, and Christian Becker
Proposition
So be wise and keep on
Reading the signs of my body
And I'm on tonight you know
my hips don't lie.
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
SHAKIRA: “HIPS DON’T LIE” (2005)
3
Alternative Proposition
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Observation:Hips might not be the most reliable source of information
Research Question
How can a mobile fall detection
system react to changes in the sensor position?
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
?
5Hips Do Lie! A Position-Aware Mobile Fall Detection System
C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach, H. Stuckenschmidt, and C. Becker
Fall Detection in a Nutshell
Self-Adaptive Fall Detection
Implemen-tation
Case Studies
Future Work
What to Expect from this Presentation?
Fall Detection Systems
Wearables
6Hips Do Lie! A Position-Aware Mobile Fall Detection System
C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach, H. Stuckenschmidt, and C. Becker
Vision-based Ambient-based
Mubashir, Shao & Seed: A survey on fall detection: Principles and approaches. In: Neurocomputing, Volume 100, 2013, Pages 144-152.
7
Positions of Wearables for Fall Detection
Head Chest
Waist Wrist
Hip Ankle
Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt and C. Becker
Jump Sit Walk Bend Run Sit Walk Get up JumpWalk
Fall
Acc. Data Caregivers
Notification
Adaptation Logic
Monitor
Planner
Executor
Analyzer
8
Self-Adaptive Fall Detection
Data acquisition
Data preprocessing
Fall alertFall detection
FD FA
Patient Sensors
Acc.
Z
Y
X
Gyro
Magn.
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
FD FADPDPDADA
Acc. Data
Monitor
Planner
Executor
Analyzer
9
Self-Adaptive Fall Detection
Data acquisition
Data preprocessing
Fall alertFall detection
FD FA
Patient Sensors
Acc.
Z
Y
X
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
FD FADPDPDADA
FA
FD
DPDA
Adaptation Logic
Caregivers
Notification
10
Implementation: Adaptation Logic
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Planner
Executor
Acc. Data:(x,y,z)
Trigger
FallNotification
FallHandling
Notifi-cationMonitor
Analyzer
?
Fall AlertAcc. Data (x,y,z)
Window Manager
11
Implementation: Adaptation Logic
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Planner
Executor
Acc. Data:(x,y,z)
Trigger
FallNotification
FallHandling
Monitor
Analyzer
Notifi-cation
t1: (x1,y1,z1)...tn: (xn,yn,zn)...tn+x: (xn+x,yn+x,zn+x)
w1 Feature1 Feature2 ...
w2 Feature1 Feature2 ...
... ... ... ...
Window Manager
12
Implementation: Adaptation Logic
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Planner
Executor
Acc. Data:(x,y,z)
Trigger
FallNotification
FallHandling
Monitor
Analyzer
Event-based approach
Notifi-cation
Continuous approach
versus
Window Manager
13
Implementation: Adaptation Logic
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Planner
Executor
Acc. Data:(x,y,z)
Trigger
FallNotification
FallHandling
Monitor
Analyzer
Event-based approach
Notifi-cation
Continuous approach
versus
Acc. Data Caregivers
Notification
Adaptation Logic
Monitor
Planner
Executor
Analyzer
14
Self-Adaptive Fall Detection
Data acquisition
Data preprocessing
Fall alertFall detection
FD FA
Patient Sensors
Acc.
Z
Y
X
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
DPDA
DAFA
FD
DP
15
Self-Improving Fall Detection
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Monitor
Planner
Executor
Analyzer
Adaptation Logic
Subject Analyzer Algorithm Chooser
Algorithms
A1 A2 An…
Data Monitor Adaptation Executor
Position Analyzer
Self-Improvement Module
Data
Instructions
Subject Clusterer
X-Means
Age
Bo
dy
Mas
s In
dex
(B
MI)
Sztyler 2016
Algorithm Optimizer
16
Evaluation Setting
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Training of models / evaluation: Leave-one-subject-out approach
Training TestingSet Set
Propositions:1. Origin of data matters2. Position matters3. Self-Improvement matters
Algorithms:• Classification-based:
SVM, ANN, Random Forest, k-NN, J48 Decision Tree
• Threshold-based (params learned)• Supports deep learning,
but dataset is too small
Data:
MMSys UMA SisFallUniMiB
#32 #10 #30 #23
17Hips Do Lie! A Position-Aware Mobile Fall Detection System
C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach, H. Stuckenschmidt, and C. Becker
Experiment 1: Cross dataset
analysis
Experiment 2: Position-aware fall detection
Evaluation Overview
Experiment 3: Self-improving fall detection
18
Experiment 1: Cross-Datasets Fall Detection
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Trained across datasets Tested on Trained on same dataset
• Proposition 1: Origin of data matters• The results indicate issues in labeling and setup of capturing data
0,52
0,68 0,63
0,48
0,730,71
0,45
0,54
0,00
0,20
0,40
0,60
0,80
MMSys UMA UniMiB SisFall
F2-S
core
s A
NN Trained across
datasets
Trained onsame datasets
19
Experiment 2: Position-Aware Fall Detection
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Trained on one position Tested on all positions
0,82 0,80
0,610,69 0,65
0,730,80 0,76
0,57
0,00
0,20
0,40
0,60
0,80
Tested on Chest Tested on Waist Tested on Hip
F1-S
core
s
Model trainedon Chest
Model trainedon Hip
Model trainedon Waist
• Proposition 2: Position matters• The results indicate that hips performed worst
20
Experiment 3: Self-Improving Fall Detection
Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt and C. Becker
Falls
Raw Data
Position
21
Experiment 3: Self-Improving Fall Detection
Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt and C. Becker
Falls
Raw Data
Position
22
Experiment 3: Self-Improving Fall Detection
Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt and C. Becker
Falls
Raw Data
Position
23
Experiment 3: Self-Improving Fall Detection
Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt and C. Becker
Falls
Raw Data
Position
24
Experiment 3: Self-Improving Fall Detection
Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt and C. Becker
Falls
Raw Data
Position
25
Experiment 3: Self-Improving Fall Detection
Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt and C. Becker
Detected falls:ANNJ48KNNRandom ForrestALM
PositionChestHip
ChestHip
(Ground truth)
(Classified)
XYZ acceleration data
Detected falls:ANNJ48KNNRandom ForestSelf-Improving
• Recall: 0.93 - Any falls missed?
• No: We missed fall windows, but for each fall we detected at least one window!
26
Experiment 3: Self-Improving Fall Detection
Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt and C. Becker
Detected falls:ANNJ48KNNRandom ForrestALM
PositionChestHip
ChestHip
(Ground truth)
(Classified)
XYZ acceleration data
• Proposition 3: Self-improvement matters• Precision is improved decreases false positives
Detected falls:ANNJ48KNNRandom ForestSelf-Improving
27
Current State
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Acc.
Z
Y
X
28
Current State
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Fall Alert
Acc.
Z
Y
X
J48 Dec. Tree
Random Forest
SVM
kNN
ANN
Thres-hold
29
Future Work
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Acc.
Z
Y
X
Fall AlertGyro
Magn.
1. Extension:Consideringother sensors
J48 Dec. Tree
Random Forest
SVM
kNN
ANN
Thres-hold
30
Future Work
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Fall Alert
1. Extension:Considering other sensors
2. Extension:Cross-positionalsensor fusion
Acc.
Z
Y
X
Gyro
Magn.
J48 Dec. Tree
Random Forest
SVM
kNN
ANN
Thres-hold
31
Future Work
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Fall Alert
1. Extension:Consideringother sensors
NC
F I
M
L
Tn
T2
IP
KE Q
T1
3. Extension:Optimized algorithms
Acc.
Z
Y
X
Gyro
Magn.
...
J48 Dec. Tree
Random Forest
SVM
kNN
ANN
Thres-hold
2. Extension:Cross-positionalsensor fusion
32
Future Work
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Fall Alert
1. Extension:Consideringother sensors
NC
F I
M
L
Tn
T2
IP
KE Q
T1
3. Extension:Optimized algorithms
Acc.
Z
Y
X
Gyro
Magn.
4. Extension:Intelligentfall alert
...
J48 Dec. Tree
Random Forest
SVM
kNN
ANN
Thres-hold
2. Extension:Cross-positionalsensor fusion
33
Future Work
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
1. Extension:Consideringother sensors
NCM
L
IP
3. Extension:Optimized algorithms
Acc.
Z
Y
X
Gyro
Magn.
4. Extension:Intelligentfall alert
5. Extension:Outlierdetection
Fall Alert
...
J48 Dec. Tree
Random Forest
SVM
kNN
ANN
Thres-hold
2. Extension:Cross-positionalsensor fusion
34
Big Picture
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
Your project could appear here...
Fall Detection Intelligent Transportation Systems [ICAC17]
Smart Vacuum Cleaner [PerIoT17]
Self-Improvement for Pervasive Systems
35
The End
Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,
H. Stuckenschmidt, and C. Becker
• Shakira: Hips don’t lie • We: Hips do lie