Prototype 3: MI prototype for video surveillance and biometry

Prototype 3: MI prototype for video surveillance and biometry

CVDSP-UJI

Computer Vision Group – UJIDigital Signal Processing Group – UV

November 2010

Review Meeting, Castellón

Outline

► Scope► overview: system architecture► main functional requirements► interactive and multimodal expectations► context constraints and feasible extensions► research groups, commitments, contributions, support

► Prototype development plan► overview► current state► work plan for the 2nd period (01/10/10-30/09/11)

► work plan for the 3rd period (01/10/11-30/09/12)

11/11/2010 CVDSP-UJI


Scope (1): overview


System architecture

Hardware: Wireless, Firewire and USB cameras

System software: camera drivers, camera management

Video analytics: detection, tracking, recognition, re-training

GUI: a control centre for interaction and management


Scope (2): main functional requirements


► Low-level requirements:► online video acquisition► detection of objects of interest (humans, faces, crowd)► object tracking► configurable recording of video segments

►Mid-level requirements:► counting people► detection of intruders in a controlled perimeter► detection of abandoned objects

► High-level requirements► a Control Centre for interaction purposes and management► near-real-time recognition tasks

► biometric identification of individuals based on face and/or gait► gender recognition based on face and/or gait


Scope (3): Interactive and Multimodal opportunities


► Interaction opportunities:► in operational conditions, an alarm could be assessed by a

human operator to either make decisions (if the alarm is true) or feedback to correct future system’s behaviour

► simple use of feedback for improving “perceptive” parameters of the vision systems, for example, to choose, by clicking, objects of interest among those that are being tracked

►Multimodality, as a fusion of different types of data, in particular, ...► gait characterization from video and face description from a

video key frame ► video data and information from temperature sensors or other

biometric systems


Scope (4): Context constraints and extensions


► Context constraints► scenes with few changes, where video analytics is more appropriate

than a human monitor (risk of inattention because it is boring)► single-camera system, but an open design to admit new cameras

►Other feasible functions► trajectory analysis► object tracking in several cameras► crowd analysis and monitoring►merging video data with temperature sensors or biometric

systems► IP camera and a web-based interface


Scope (5): research groups, commitments …


► Computer Vision Group (CV-CVC), CVC – UAB►commitments, contributions and support (tracking method),

but profitless

► Robotics and Perception Group (RP-IRI), IRI – UPC►commitments (ground-truth video sequences, benchmarking),

no contributions (so far)

► Pattern Recognition and Human Language Technology Group (PRHLT), ITI – UPV►no answers, no commitments, no contributions

► Computer Vision and Digital Signal Processing Group (CVDSP), UJI + UV►full commitment and more ...


Prototype Development Plan: overview


Iterative life-cycle process

► Prototype definition [Done, February 2010]

► Iteration 1: Architecture module [Done, July 2010]

► Iteration 2: Detection and tracking module [Done: 30/10/2010]

► Iteration 3: Analysis, recognition and Interaction modules [Deadline: 30/09/2011]

► Iteration 4: Software performance and stability testing. [Deadline: 30/09/2012]


Current state


► Linux-based prototype architecture

► USB and FireWire cameras management

► Video recording

► Integration of a tracking module provided by the CV-CVC; it only worked fine for a particular scenario (fixed parameters) [discarded]

► Integration of two other parameterizable tracking modules, one from OpenCV and a second one developed by a member of the team

► Non-integrated estimation of two context awareness factors (they may affect the relationship between gait and face in a potential fusion)► View angle (w.r.t. the camera)► Distance from the subject to the camera

► Several non-integrated methods for characterizing and recognizing faces and gait footage (under controlled conditions)


Work plan, 2nd period (01/10/10-30/09/11)


► GUI analysis and design [11/10 – 01/11][responsible: Ramón]

► Implementation of mid-level functions (counting, detection of intruders and objects) [12/10 – 02/11][r: Henry]

► GUI implementation [01/11 – 03/11][r: Vicente]

► Integration of gait-based biometric methods [02/11 – 05/11][r: Raúl]

► Integration of face-based biometric methods [02/11 – 05/11][r: Yasmina]

► Implementation of the multimodal approach for biometric solutions based on the combination of face and gait [06/11-09/11][r: Ramón]

► Evaluation and refinement of the GUI and, in particular, of the interactive functions [05/11 – 09/11][r: Vicente]

► Identification and assessment of performance evaluation strategies for interactive surveillance functions [07/11 – 09/11][r: Ramón]


Work plan, 3rd period (01/10/11-30/09/12)


► Evaluation, improvement and refinement of all surveillance functions [10/11 – 09/12] [r: Ramón]

► Development of extra and complementary functions [10/11 – 09/12] [r: Vicente]

► To evaluate the possibility of re-implementing certain functions in a parallel architecture [10/11 – 09/12][r: Vicente]

► To write a technical report [07/12 – 09/12][r: Ramón]

Documents

Prototype 3: MI prototype for video surveillance and biometry