IMPROVED FACE TRACKING THANKS TO LOCAL FEATURES CORRESPONDENCEAlberto Piacenza, Fabrizio Guerrini, RiccardoLeonardi

Department of Engineering Information – University of Brescia, Italy

Dataset of YouTubeshots with faces(average: 181 frames)

FACE TRACKING ENHANCEMENTOVERVIEW

Apply the face track enhancement stage

SOLUTION

Semantic description of thecontent in the InteractiveMovietelling system [1]

MOTIVATION

Identify the frames in whicha main character is present

SPECIFIC CHALLENGE

Use off-the-shelf tools for:1) face detection2) face recognition on the

detected faces

BASELINE SOLUTION

1) Imprecise or missed face detection

2) Face bounding box drifting

PROBLEMS

Character recognitionis unreliable

EFFECTS

Flowchart of the operationsinvolved in the creation of theenhanced face tracks.

Output tracks: the frames of a small excerpt of one shot are presented.

Blue rectangles: detected faces correctly identified in a given face track. but the face detection has failed to find the face in the in-between frames. Green rectangles: recovered faces for in-between frames thanks to the face tracks enhancement process.

• Re-extract POI in the bounding box• Use KLT tracker to the next frame• Estimate the new bounding box using RANSAC• Use backward tracking as well

: number of ground-truth objects in frame: number of detected objects in frame: -th ground-truth object: -th detected object

Frame Detection Accuracy (FDA):

Comparison with the CAMSHIFT algorithm

EXPERIMENTAL RESULTS

ADDITIONAL INFOInteractive Movietelling system:• Reference: [1] A. Piacenza, F. Guerrini, N. Adami, R. Leonardi, J. Porteous, J. Teutenberg, M. Cavazza, “Generating Story Variants with Constrained Video Recombination”,19th ACM Multimedia, pp. 223-232, 2011.• Link to example output clips:www.ing.unibs.it/alberto.piacenza/TrackWithPoints

Acknowledgements: This work has beenfunded (in part) by the EC under grantAgreement IRIS (FP7-ICT-231824).