Mining Actor Correlations with Hierarchical Concurrence Pasring

Reporters: R98922004 Yun-Nung Chen, R98922033 Yu-Cheng Liu

Reference

Ming Actor Correlations with Hierarchical Concurrence Parsing (ICASSP 2010) Kun Yuan, Hongxun Yao, Rongrong Ji,

Xiaoshuai Sun Computer Science & Technology, Harbin

Institute of Technology

Outline

Introduction Actor Indexing Mining Actor Correlations

Context-Based Actor Concurrence Graph Ranking Concurrent Shots Actor Correlation Changes Analysis

Experimental Results

Introduction

Actor correlations graph interfaces

Introduction

Actor correlations graph interfaces

Introduction

Actor correlations graph interfaces Top 20 shots

Overview

Actor Indexing

Locating Actors

Shot boundary detection (SBD) Shots and scenes

Locating actor faces & face tracking algorithm Face set: different poses from the same

actor

Clustering Actors

2D-PCA reduces dimension Features of same person may

distribute discretely in feature space Given 2 face sets Fk and Fl, 2 pose sets

If distance < T, 2 face sets belong to the same person

Clustering Actors

Mining Actor Correlations

Context-Based Actor Concurrence

A shot and its surrounding shots may present a plot between two actors in video Gaussian weight measurement

Context-Based Actor Concurrence

Hierarchical Concurrence

Scene level correlation

Video level correlation

Hierarchical Concurrence

All Actor correlations graph

Construct correlations graph from

Ranking Concurrent Shots

A single character i

Character correlations between i and j

Ranking Concurrent Shots

Given i, j, sort RankScore<i, j>(k) for all k Show top 20 shots

Actor Correlation Changes Analysis

Two actors’ correlation changes with story

Analyze the difference of concurrence R(i, j)A

correlation measure between i and j in the part A

Change ratio Hlp

Experimental Results

Experiment Setup

20 hours video of “Friends” TV series About 4000 shots Over 800 face sets

Clustering into about 60 face sets (T = 0.25)

Manual labeling to 17 actors

Evaluation

The actor concurrence precision in all ranking shots is up to 90%

The precision of each two actor’s co-occurrence in ranking top 20 is up to 98%

Thanks for your attention.