Christian Krupitzer, Timo Sztyler, Janick Edinger, Martin ... · • Proposition 1: Origin of data...

Hips do lie! A position-aware mobile fall detection system

Christian Krupitzer, Timo Sztyler, Janick Edinger, Martin Breitbach,

Heiner Stuckenschmidt, and Christian Becker

Proposition

So be wise and keep on

Reading the signs of my body

And I'm on tonight you know

my hips don't lie.

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

SHAKIRA: “HIPS DON’T LIE” (2005)

3

Alternative Proposition

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Observation:Hips might not be the most reliable source of information

Research Question

How can a mobile fall detection

system react to changes in the sensor position?

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

?

5Hips Do Lie! A Position-Aware Mobile Fall Detection System

C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach, H. Stuckenschmidt, and C. Becker

Fall Detection in a Nutshell

Self-Adaptive Fall Detection

Implemen-tation

Case Studies

Future Work

What to Expect from this Presentation?

Fall Detection Systems

Wearables

6Hips Do Lie! A Position-Aware Mobile Fall Detection System

C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach, H. Stuckenschmidt, and C. Becker

Vision-based Ambient-based

Mubashir, Shao & Seed: A survey on fall detection: Principles and approaches. In: Neurocomputing, Volume 100, 2013, Pages 144-152.

7

Positions of Wearables for Fall Detection

Head Chest

Waist Wrist

Hip Ankle

Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt and C. Becker

Jump Sit Walk Bend Run Sit Walk Get up JumpWalk

Fall

Acc. Data Caregivers

Notification

Adaptation Logic

Monitor

Planner

Executor

Analyzer

8

Self-Adaptive Fall Detection

Data acquisition

Data preprocessing

Fall alertFall detection

FD FA

Patient Sensors

Acc.

Z

Y

X

Gyro

Magn.

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

FD FADPDPDADA

Acc. Data

Monitor

Planner

Executor

Analyzer

9

Self-Adaptive Fall Detection

Data acquisition

Data preprocessing

Fall alertFall detection

FD FA

Patient Sensors

Acc.

Z

Y

X

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

FD FADPDPDADA

FA

FD

DPDA

Adaptation Logic

Caregivers

Notification

10

Implementation: Adaptation Logic

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Planner

Executor

Acc. Data:(x,y,z)

Trigger

FallNotification

FallHandling

Notifi-cationMonitor

Analyzer

?

Fall AlertAcc. Data (x,y,z)

Window Manager

11

Implementation: Adaptation Logic

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Planner

Executor

Acc. Data:(x,y,z)

Trigger

FallNotification

FallHandling

Monitor

Analyzer

Notifi-cation

t1: (x1,y1,z1)...tn: (xn,yn,zn)...tn+x: (xn+x,yn+x,zn+x)

w1 Feature1 Feature2 ...

w2 Feature1 Feature2 ...

... ... ... ...

Window Manager

12

Implementation: Adaptation Logic

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Planner

Executor

Acc. Data:(x,y,z)

Trigger

FallNotification

FallHandling

Monitor

Analyzer

Event-based approach

Notifi-cation

Continuous approach

versus

Window Manager

13

Implementation: Adaptation Logic

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Planner

Executor

Acc. Data:(x,y,z)

Trigger

FallNotification

FallHandling

Monitor

Analyzer

Event-based approach

Notifi-cation

Continuous approach

versus

Acc. Data Caregivers

Notification

Adaptation Logic

Monitor

Planner

Executor

Analyzer

14

Self-Adaptive Fall Detection

Data acquisition

Data preprocessing

Fall alertFall detection

FD FA

Patient Sensors

Acc.

Z

Y

X

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

DPDA

DAFA

FD

DP

15

Self-Improving Fall Detection

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Monitor

Planner

Executor

Analyzer

Adaptation Logic

Subject Analyzer Algorithm Chooser

Algorithms

A1 A2 An…

Data Monitor Adaptation Executor

Position Analyzer

Self-Improvement Module

Data

Instructions

Subject Clusterer

X-Means

Age

Bo

dy

Mas

s In

dex

(B

MI)

Sztyler 2016

Algorithm Optimizer

16

Evaluation Setting

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Training of models / evaluation: Leave-one-subject-out approach

Training TestingSet Set

Propositions:1. Origin of data matters2. Position matters3. Self-Improvement matters

Algorithms:• Classification-based:

SVM, ANN, Random Forest, k-NN, J48 Decision Tree

• Threshold-based (params learned)• Supports deep learning,

but dataset is too small

Data:

MMSys UMA SisFallUniMiB

#32 #10 #30 #23

17Hips Do Lie! A Position-Aware Mobile Fall Detection System

C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach, H. Stuckenschmidt, and C. Becker

Experiment 1: Cross dataset

analysis

Experiment 2: Position-aware fall detection

Evaluation Overview

Experiment 3: Self-improving fall detection

18

Experiment 1: Cross-Datasets Fall Detection

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Trained across datasets Tested on Trained on same dataset

• Proposition 1: Origin of data matters• The results indicate issues in labeling and setup of capturing data

0,52

0,68 0,63

0,48

0,730,71

0,45

0,54

0,00

0,20

0,40

0,60

0,80

MMSys UMA UniMiB SisFall

F2-S

core

s A

NN Trained across

datasets

Trained onsame datasets

19

Experiment 2: Position-Aware Fall Detection

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Trained on one position Tested on all positions

0,82 0,80

0,610,69 0,65

0,730,80 0,76

0,57

0,00

0,20

0,40

0,60

0,80

Tested on Chest Tested on Waist Tested on Hip

F1-S

core

s

Model trainedon Chest

Model trainedon Hip

Model trainedon Waist

• Proposition 2: Position matters• The results indicate that hips performed worst

20

Experiment 3: Self-Improving Fall Detection

Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt and C. Becker

Falls

Raw Data

Position

21

Experiment 3: Self-Improving Fall Detection

Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt and C. Becker

Falls

Raw Data

Position

22

Experiment 3: Self-Improving Fall Detection

Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt and C. Becker

Falls

Raw Data

Position

23

Experiment 3: Self-Improving Fall Detection

Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt and C. Becker

Falls

Raw Data

Position

24

Experiment 3: Self-Improving Fall Detection

Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt and C. Becker

Falls

Raw Data

Position

25

Experiment 3: Self-Improving Fall Detection

Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt and C. Becker

Detected falls:ANNJ48KNNRandom ForrestALM

PositionChestHip

ChestHip

(Ground truth)

(Classified)

XYZ acceleration data

Detected falls:ANNJ48KNNRandom ForestSelf-Improving

• Recall: 0.93 - Any falls missed?

• No: We missed fall windows, but for each fall we detected at least one window!

26

Experiment 3: Self-Improving Fall Detection

Hips Do Lie! A position-aware mobile fall detection system C. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt and C. Becker

Detected falls:ANNJ48KNNRandom ForrestALM

PositionChestHip

ChestHip

(Ground truth)

(Classified)

XYZ acceleration data

• Proposition 3: Self-improvement matters• Precision is improved decreases false positives

Detected falls:ANNJ48KNNRandom ForestSelf-Improving

27

Current State

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Acc.

Z

Y

X

28

Current State

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Fall Alert

Acc.

Z

Y

X

J48 Dec. Tree

Random Forest

SVM

kNN

ANN

Thres-hold

29

Future Work

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Acc.

Z

Y

X

Fall AlertGyro

Magn.

1. Extension:Consideringother sensors

J48 Dec. Tree

Random Forest

SVM

kNN

ANN

Thres-hold

30

Future Work

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Fall Alert

1. Extension:Considering other sensors

2. Extension:Cross-positionalsensor fusion

Acc.

Z

Y

X

Gyro

Magn.

J48 Dec. Tree

Random Forest

SVM

kNN

ANN

Thres-hold

31

Future Work

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Fall Alert

1. Extension:Consideringother sensors

NC

F I

M

L

Tn

T2

IP

KE Q

T1

3. Extension:Optimized algorithms

Acc.

Z

Y

X

Gyro

Magn.

...

J48 Dec. Tree

Random Forest

SVM

kNN

ANN

Thres-hold

2. Extension:Cross-positionalsensor fusion

32

Future Work

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Fall Alert

1. Extension:Consideringother sensors

NC

F I

M

L

Tn

T2

IP

KE Q

T1

3. Extension:Optimized algorithms

Acc.

Z

Y

X

Gyro

Magn.

4. Extension:Intelligentfall alert

...

J48 Dec. Tree

Random Forest

SVM

kNN

ANN

Thres-hold

2. Extension:Cross-positionalsensor fusion

33

Future Work

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

1. Extension:Consideringother sensors

NCM

L

IP

3. Extension:Optimized algorithms

Acc.

Z

Y

X

Gyro

Magn.

4. Extension:Intelligentfall alert

5. Extension:Outlierdetection

Fall Alert

...

J48 Dec. Tree

Random Forest

SVM

kNN

ANN

Thres-hold

2. Extension:Cross-positionalsensor fusion

34

Big Picture

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

Your project could appear here...

Fall Detection Intelligent Transportation Systems [ICAC17]

Smart Vacuum Cleaner [PerIoT17]

Self-Improvement for Pervasive Systems

35

The End

Hips Do Lie! A Position-Aware Mobile Fall Detection SystemC. Krupitzer, T. Sztyler, J. Edinger, M. Breitbach,

H. Stuckenschmidt, and C. Becker

• Shakira: Hips don’t lie • We: Hips do lie