Scene Understanding perception, multi-sensor fusion, spatio-temporal reasoning

1

Scene Understandingperception, multi-sensor fusion, spatio-temporal reasoning

and activity recognition.

Francois BREMOND

Orion project-team,

INRIA Sophia Antipolis, FRANCE

Francois.Bremond@sophia.inria.fr

http://www-sop.inria.fr/orion/orion-eng.html

Key words: Artificial intelligence, knowledge-based systems,

cognitive vision, human behavior representation, scenario recognition

2

Alarms

access to forbidden

area

3D scene modelScenario models A priori Knowledge

A scene understanding platform: real-time interpretation of videos from pixels to events

SegmentationSegmentation ClassificationClassification TrackingTracking Scenario RecognitionScenario Recognition

3

Video Understanding Applications

metro station monitoring

building control accesstrain monitoring

airport monitoring

4

shadowsstrong perspectivetiny objects

close view

clutterlightingconditions

ETISEO: performance evaluation depends on sensors (position, type) and scenes conditions

5

ETISEO: selection of 6 sites for ETISEO video acquisition.

6

Classification into more than 8 classes (e.g. Person, Groupe, Train) based on 2D and 3D descriptors (position, 3D ratio height/width, …)

7

Vandalism in metro station

8

Fighting in metro station

9

Posture Recognition : silhouette comparison

Real world Virtual world

Generated silhouettes

10

Multi sensors information fusion: Lateral Shape Recognition

11

Bank agency monitoring

12

Parked aircraft monitoring in Toulouse:

Apron Apron

Toulouse-Blagnac Toulouse-Blagnac AirportAirport

FranceFrance

Video

Sensors

13

Parked aircraft monitoring in Toulouse: “Unloading Front Operation”

14

Scenario recognition: “Disturbing people in a train”

15

Détecteur de présence

Contacts de portes

Caméra

Capteur de chute

Capteur de pouls

Microphones

Pèse personne

Surveillance respiratoire

HealthCare Monitoring :

16

HealthCare Monitoring

17

HealthCare Monitoring: Object detection and tracking

(a) Segmentation (b) Classification (c) Tracking

18

HealthCare Monitoring: Object detection and tracking

(a) Segmentation (b) Classification (c) Tracking

19

HealthCare Monitoring: Object detection and tracking

(a) Segmentation (b) Classification (c) Tracking

20

HealthCare Monitoring: Object detection and tracking

(a) Segmentation (b) Classification (c) Tracking

21

HealthCare Monitoring: Object detection and tracking

(a) Segmentation (b) Classification (c) Tracking

22

HealthCare Monitoring: Object detection and tracking

(a) Segmentation (b) Classification (c) Tracking

23Trajectory clustering in Torino subway (45min), 2052 trajectories

24

People Classification (based on 3D parallelepiped) into 3 people classes : 1Person, 2Persons, 3Persons, Unknown

25

Standing Sitting Bending

Hierarchical representation of postures

Lying

Posture Recognition : Set of Specific Postures