VIPP Symposium, January 12, 2010

Content-Based Video Analysis

Markus Koskela

Background

Aalto University School of Science and Technology

Department of Information and Computer Science

(http://ics.hut.fi/ )

Adaptive Informatics Research Centre, Academy CoE(http://www.cis.hut.fi/research/ )

Erkki Jorma Ville Mats He Xi JingOja Laaksonen Viitaniemi Sjoberg Zhang Chen Wu

(2/20)

Overview of my talk

• Introduction

• Concept-based video search

• Video summarization

• Video analysis in AR applications

(3/20)

Content-based analysis of video

• Visual features

• Audio features,

speech, text, . . .

(4/20)

Indexing with low-level features

(5/20)

Content-based image retrieval

(6/20)

Google Images

(7/20)

Types of video material

Shot-based video, obtained with

shot boundary detection

Continuous material, sampled

with regular interval

Online video, received one frame

at a time

(8/20)

Shot-based video indexing

Texture SOM

Temporal Color SOM

Motion SOM

Word histogram Keyword freq.Color SOM

text

video shot

launch...

...missile

keyframe audio

Audio SOMbinary keyword

inverted file

(9/20)

Concept-based video search

(10/20)

(11/20)

Concept ontologies

sports weather court office meeting studio outdoor building desert

snowvegetation mountain road sky urban crowd face

US flagperson police/ military prisoner animal computer/TV airplane car

bus truck boat/ship walking/ people explosion/

waterscape/

natural maps charts

waterfront

running

security screen

marching fire disaster

(12/20)

Concept-based video search

(13/20)

Video summarization

(14/20)

Summarization pipeline

(15/20)

Video analysis in AR applications

(16/20)

Contextual IR in AR applications

(17/20)

Devices for mobile AR

(18/20)

Video analysis for AR

(19/20)

Thank you!

(20/20)