VISION for Security

Monique THONNAT

ORIONINRIA Sophia Antipolis

18/03/05 CS

M. Thonnat 2

Which Security Problems? Safety and security of goods and human beings

How? Data captured by video surveillance cameras

Original video understanding approach mixing: computer vision: 4D analysis (3D + temporal analysis)

artificial intelligence: a priori knowledge (scenario, environment)

software engineering: reusable VSIP platform

Introduction

18/03/05 CS

M. Thonnat 3

Definition: real time and automated analysis of video sequences video understanding= from people detection and

tracking to behavior recognition

Recognition of complex behaviors: of individuals (fraud, graffiti, vandalism, bank attack) of small groups (fighting) of crowds (overcrowding) interactions of people and vehicles (aircraft refueling)

Video Understanding for Security

18/03/05 CS

M. Thonnat 4

Video UnderstandingVideo Understanding

4 D analysis:multi-cameras

tracking

Video understanding

People detection

and tracking

Scenario recognition

A PRIORI KNOWLEDGE:• 3d models of the environment • Camera calibration• Scenario Models

Alarms

People detection

and tracking

Interpretation of the videos from pixels to alarms

18/03/05 CS

M. Thonnat 5

Impact: Visual surveillance of metro stations, bank agencies,

trains, buildings and airports 5 European projects (PASSWORDS, AVS-PV, AVS-RTPW,

ADVISOR, AVITRACK)

4 contracts with End-users companies (metro, bank, trains)

2 transfer activities with Bull (Paris) and Vigitec (Brussels)

Cooperation over more than 11 years with partners

Creation of a start-up (spring 2005)

Video Understanding

18/03/05 CS

M. Thonnat 6

Typical problems

Metro station surveillance Surveillance inside trains

Building access control Airport monitoring

18/03/05 CS

M. Thonnat 7

Behavior recognition: approach based on a priori knowledge

model of the empty scene (3D geometry and semantics)

models of predefined scenarios

a language for representing scenarios based on

combination of states and events more than 20 states and 20 events can be used

a reasoning mechanism for real time detection of states,

events and scenarios (e.g. temporal reasoning,

constraints solving techniques)

Video Understanding

18/03/05 CS

M. Thonnat 8

Video Understanding: 3D Scene Model

3d Model of 2 bank agencies

objet du contexte

mur et portezone d’accès

salle du coffrerue

rue

salle automates

zone d’entrée de l’agence

zone des distributeurs

zone de jour/nuit

zone devant le guichet

zone derrière le guichet

zone d’accès au bureau du

directeur

zone de jour

ported’entrée

porte salleautomates

armoire

guichet

commode

Les Hauts de Lagny Villeparisis

18/03/05 CS

M. Thonnat 9

States, Events and Scenarios : State: a spatio-temporal property involving one or several actors on a time interval

Ex : « close», « walking», « seated»

Event: a significant change of states

Ex : « enters», « stands up», « leaves »

Scenario: a long term symbolic application dependent activity

Ex : « fighting», « vandalism»

Video Understanding

18/03/05 CS

M. Thonnat 10

Results for Bank Monitoring

Bank attack scenario description :

scenario Bank_attack_one_robber_one_employeephysical_objects: ((employee : Person), (robber : Person), z1: Back_Counter, z2: Entrance_Zone, z3: Front_Counter, z4: Safe, d: Safe_door)

components: (State c1 : Inside_zone(employee, z1)) (Event c2 : Changes_zone(robber, z2,z3))

(State c3 : Inside_zone(employee, z4)) (State c4 : Inside_zone(robber, z4))) constraints : ((c2 during c1) (c2 before c3) (c1 before c3) (c2 before c4) (c4 during c3) (d is open))

18/03/05 CS

M. Thonnat 11

Video Understanding for bank surveillance

Examples : Brussels and Barcelona Metros

Exit zone

Jumping over barrier

Blocking

Overcrowding

Fighting

Group

behavior

Crowd

behavior

Individual

behavior

Groupbehavior

Results in Metro Surveillance

12

18/03/05 CS

M. Thonnat 13

Video Understanding: Conclusion

Hypotheses: fixed cameras 3D model of the empty scene predefined behavior models

Results: + Behavior understanding for Individuals,

Groups of people, Crowd or Vehicles

+ an operational language for video understanding (more than 20 states and events)

+ a real-time platform (5 to 25 frames/s)

18/03/05 CS

M. Thonnat 14

Knowledge Acquisition Design of ontology driven knowledge acquisition:

video event ontology (T. Van Vu PhD) Design of learning techniques to complement a priori knowledge:

visual concept learning(Nicolas Maillot PhD) scenario model learning (A. Toshev)

Reusability is still an issue for vision programs Use of program supervision techniques: dynamic configuration of

programs and parameters (B Georis PhD)

Video event detection Finer human shape description:3D posture models (B. Boulay PhD) Video analysis robustness: Uncertainty management (M. Zuniga

PhD)

Conclusion: Where we go

18/03/05 CS

M. Thonnat 15

Computer Vision Mobile object detection (Wei Yun I2R Singapore) Tracking of people using geometric approaches (T.

Ellis et al. Kingston University UK)

Event Recognition Probalistic approaches HMM, DBN (A Bobick Georgia

Tech USA, H Buxton Univ Sussex UK)

Reusable platform Realtime video surveillance platform (Multitel, Be)

State of the Art