1 1 Date HealthCare Monitoring: GERHOME Project Monique Thonnat, Francois Bremond & Nadia Zouba PULSAR, INRIA

11

Date

HealthCare Monitoring: HealthCare Monitoring: GERHOME Project GERHOME Project

Monique Thonnat, Francois Bremond & Nadia ZoubaMonique Thonnat, Francois Bremond & Nadia Zouba

PULSAR, INRIAPULSAR, INRIA

22

Approach for video analysisApproach for video analysis

AlarmsDetectionDetection ClassificationClassificationTracking &

posture detection

Tracking & posture

detection

Activity Recognition

Activity Recognition

A person is in the kitchen

33

Gerhome laboratory (CSTB, PULSAR) Gerhome laboratory (CSTB, PULSAR)

– GERHOME (Gerontology at Home) : GERHOME (Gerontology at Home) : homecare homecare laboratory laboratory

– Experimental site in CSTB (Centre Scientifique et Experimental site in CSTB (Centre Scientifique et Technique de Bâtiment) at Sophia Antipolis Technique de Bâtiment) at Sophia Antipolis http://gerhome.cstb.frhttp://gerhome.cstb.fr

44

Position of the sensors in Gerhome laboratory Position of the sensors in Gerhome laboratory

55Sensors in Gerhome laboratorySensors in Gerhome laboratory

• Video cameras:Video cameras: installed in the kitchen and in the living-room used to detect and installed in the kitchen and in the living-room used to detect and track a track a mobile objects (mostly people) evolving in the scene.mobile objects (mostly people) evolving in the scene.

• Environmental sensors:Environmental sensors: used to used to collectcollect information about the information about the interactionsinteractions with the with the objects in the environment:objects in the environment:

• Contact sensors:Contact sensors: installed on different locations, they detect usage for doors, installed on different locations, they detect usage for doors, cupboards, drawers, fridge, … cupboards, drawers, fridge, …

• Usage sensors:Usage sensors: installed into an electrical outlets with the appliance plugging directly installed into an electrical outlets with the appliance plugging directly to the sensor, they detect whether an appliance is on or off (e.g. microwave). to the sensor, they detect whether an appliance is on or off (e.g. microwave).

• Presence sensors:Presence sensors: installed in front of sink and cooking stove to detect the installed in front of sink and cooking stove to detect the

presence of people near sink and stove.presence of people near sink and stove.

• Water sensors:Water sensors: to detect the use of a water in the kitchen and in the bathroom. to detect the use of a water in the kitchen and in the bathroom.

• Pressure sensors:Pressure sensors: to detect when a person is sitting on a chair, armchair, … to detect when a person is sitting on a chair, armchair, …

66

Dans la cuisine

Sensors installed in Gerhome laboratorySensors installed in Gerhome laboratory

Video camera in the living-room

Pressure sensor underneath the legs of armchair

Contact sensor in the windowContact sensor in the cupboard door

77

Posture recognitionPosture recognition

Posture recognition approach [Boulay, et al. PRL06]:Posture recognition approach [Boulay, et al. PRL06]: Using a set of 3D models defining the postures of interest.Using a set of 3D models defining the postures of interest.

standingstanding (a), (a), standing with arms upstanding with arms up (b), (b), standing with hands upstanding with hands up (c), (c),

bendingbending (d), (d),

sitting on a chairsitting on a chair (e), (e), sitting on the floor with outstretched legssitting on the floor with outstretched legs (f), (f), sitting on the sitting on the floor with flexed legsfloor with flexed legs (g), (g), slumpingslumping (h), (h),

lying on the side with flexed legslying on the side with flexed legs (i), (i), lying on the back with outstretched legslying on the back with outstretched legs (j). (j).

These postures are used to detect primitive states and complex events (e.g. falling).These postures are used to detect primitive states and complex events (e.g. falling).

88

PosturesPostures

99

Recognition of the “Recognition of the “Feeling faintFeeling faint” event” event

• The person is recognized with the postures "The person is recognized with the postures "standingstanding” “” “bendingbending”, " ”, " sitting on sitting on the floorthe floor with flexed legswith flexed legs”, and “”, and “sitting on the floor with outstretched legssitting on the floor with outstretched legs”.”.

Visualization of the recognition of “Feeling faint” in the Gerhome laboratoryVisualization of the recognition of “Feeling faint” in the Gerhome laboratory

1010

Recognition of the “Recognition of the “Falling downFalling down” event” event

Visualization of the recognition of “Falling down” in the Gerhome laboratoryVisualization of the recognition of “Falling down” in the Gerhome laboratory

• The person is recognized with the postures "The person is recognized with the postures "standingstanding”, " ”, " sitting on the floor with sitting on the floor with flexed legsflexed legs ”, and “ ”, and “lying with outstretched legslying with outstretched legs”.”.

1111

Recognition of the “Recognition of the “Prepare mealPrepare meal” event” event

• The person is recognized with the postures "The person is recognized with the postures "standingstanding”, “”, “bendingbending”, “”, “located in located in the kitchenthe kitchen”, “”, “using the fridgeusing the fridge”,”,, and “and “using the microwaveusing the microwave”.”.

Visualization of the recognition of “Prepare meal” in the Gerhome laboratoryVisualization of the recognition of “Prepare meal” in the Gerhome laboratory

1212

Recognition of the “Recognition of the “Resting in living-roomResting in living-room” event” event

• The person is “The person is “located in thelocated in the living-roomliving-room” and recognized with the postures “” and recognized with the postures “sittingsitting” ”

and “and “slumpingslumping”.”.

Visualization of the recognition of “Resting in living-room” in the Gerhome laboratoryVisualization of the recognition of “Resting in living-room” in the Gerhome laboratory

1313

Recognition of the “Recognition of the “Prepare mealPrepare meal” event with elderly ” event with elderly personperson

1414

Recognition of the “Recognition of the “Take mealTake meal” event with elderly ” event with elderly personperson

1515

Trajectory

Trajectory of the person in the apartment Stopping zones: person is standing for more than 20 s

Distance walked during 3 hours = 3.71 km

1616

Preliminary results

4512 :000.1620030 :000.09-Etre debout immobile

611 :001 :8348 :002 :00-Etre dans l’entrée

55 :001 :00.22 :001 :00-Entrer dans la salle de bain

723 :003 :2845 :001 :25-Entrer dans la chambre

38 :002 :66512 :002 :45-Entrer dans la cuisine

1335 :002 :682025 :001 :25-Entrer dans le salon

155 :000.20302 :000.04-Se baisser

60:060 :01000-Assis sur le fauteuil

0001273:276 :07-Assis sur la chaise2


465:1816 :194169:1242 :18-TV allumée

10 :560 :56000-Utiliser les toilettes

942:244 :422521:340 :51-Utiliser le tiroir haut

190:580 :03209:360 :28-Utiliser l’eau froide (cuisine)

223:310 :094012:400 :19-Utiliser l’eau chaude (cuisine)

000000-Utiliser le micro-onde

10227:570 :16354 :520 :08-Utiliser les plaques chauffantes

51:090 :13142 :500 :12-Utiliser le frigidaire

Nb instancesDurée totale

(min)Temps moyen

(min)Nb instances

Durée totale (min)

Temps moyen (min)

Activités / s.ous activités

Volontaire 2 ( > 80 ans)Volontaire 1 ( < 65 ans)Volontaires

1717







155 :000.20302 :000.04-Se baisser




465:1816 :194169:1242 :18-TV allumée









(min)Temps moyen

(min)Nb instances

Durée totale (min)

Temps moyen (min)



Incohérence capteurs

Preliminary results

1818







155 :000.20302 :000.04-Se baisser




465:1816 :194169:1242 :18-TV allumée









(min)Temps moyen

(min)Nb instances

Durée totale (min)

Temps moyen (min)



Déplacements + ou - rapides

Preliminary results

1919







155 :000.20302 :000.04-Se baisser




465:1816 :194169:1242 :18-TV allumée









(min)Temps moyen

(min)Nb instances

Durée totale (min)

Temps moyen (min)



Principales activités

Preliminary results

2020







155 :000.20302 :000.04-Se baisser




465:1816 :194169:1242 :18-TV allumée









(min)Temps moyen

(min)Nb instances

Durée totale (min)

Temps moyen (min)



+ ou - dynamique

Preliminary results

2121







155 :000.20302 :000.04-Se baisser




465:1816 :194169:1242 :18-TV allumée









(min)Temps moyen

(min)Nb instances

Durée totale (min)

Temps moyen (min)



Utilisation + ou – facile

Preliminary results

Documents

1 1 Date HealthCare Monitoring: GERHOME Project Monique Thonnat, Francois Bremond & Nadia Zouba PULSAR, INRIA