Multimedia Database Management System

Wei Tsang OoiCS731

MMDBMS : Querying Interface,

Indexing and

Buffer Management

Why MMDBMS ?

Concurrency ControlRecoveryPrivacyQuery SupportVersion Control

Example of MMDBMS

Digital LibraryNews-On-DemandVideo-On-DemandMusic DatabaseTelemedicineGeographic Information

System

No Intergration

size fps titleid filename

000001 530M 30 soam l6.mpg

000002 450M 30 tibor l7.mpg

000003 600M 30 parag l5.mpg

000004 510M 30 wei l4.mpg

Semi-intergrated

000001b70ae9902...

BLOB

Fully Intergrated

index

buffers

storage

Nature of Multimedia Data

Large amount of dataTime sensitiveVague matching

Database Components

Query Interface

Query Processing

indexbuffermanager

storagemanager

Query Interface and

Processing

Problems

Needs support for :• temporal and spatial relations• “natural” interface• fuzzy query

SQL is not suitable.

13 Temporal Relations

Allen 83

before

meets

overlaps

finished by

contains

started by

equals

starts

finishes

during

overlapped by

met by

after

Spatio Relations

Papadias, Theodoridis 96• Topological Relations• Directional Relations

Topological Relations

disjoint

meet

overlap

covered by

inside

equal

covers

contains

Directional Relations

North

South

West East

NENW

SESW

Spatio-temporal Relations

overlap-aftermeet-during...

Querying

ImageAudio

• Music• Sound• Speech

Video

Querying Image

Common approach• allow query by sketches (color,

shape, texture) or examples.• perform matching by Feature

VectorsF = (v1, v2, ... vn)

• e.g. Color Histogram

Querying Image

Exisiting Systems :• QBIC • VisualSEEK• PhotoBook• Virage• FourEyes

Querying Music

Hawley 93• Input by MIDI Keyboard• Measure relative pitch (U, D, S)• Perform exact match with

existing database.

Querying Music

Ghias, Logan, Chamberlin & Smith 95• Input by humming• Extract relative pitch• Perform approximate matching

Querying Music

Chou, Chen, & Liu 96• Query by chord• Represents musics by chord

• C Am Em F C Am Em F ...

• Perform fuzzy match

Querying Music

Chen & Chen 98• Query by “rhythm” (tempo ?)• Represents musics by rhythm

• | ¶¶— | ¶·¶·| ¶¶¶¶ | ...

• Perform fuzzy match

Querying Sound

Wold, Blum, Keisar & Wheaton 96• Analyze audio to extract features

•loudness, pitch, brightness, bandwidth and harmonicity

• Segment the audio to pieces• Feature Vector Matchings

Querying Speech

Hauptmann & Witbrock 97Informedia

• Use speech recorgnition to convert audio to text

• segment audio using silence detection

Query by speaking keywords

Querying Video

Query by speech• Informedia

Visual approach• VideoQ

VideoQ

Chang et. al 97• User can sketch objects• Specifies

• color• texture• shape• motion• duration• camera zoom and pan.

VisualQ Example

Someone SkiingTitanic Sinking

Indexing

Indexing

Requirements• support spatio-temporal

operations• support fuzzy matches

Indexing Images

N-dimentional indices for feature vector

Well studied in DB/CG community

Two examples :• VP-tree• R-tree

VP-tree

PQ

R

ST

U

V

W

R

PQS VWTU

VP-tree

PQ

R

ST

U

V

W

R

P

S Q

U

T VW

R-tree

Indexing Audio

Audio are modeled as stringsInexact match is neededCommon indices for string

search can be used

Example• PAT-tree

PAT-tree

ab b c

abc c abc c

ababc abc babc bc c

Indexing Video

Treat time as third dimensionWe can use any

multidimension indexing structures

Buffer Management

Buffer Management

Minimizing response timeEnsuring continuity and

synchronization

Prefetching & Replacement

Glossaries

PresentationMedia StreamMedia Object

Buffer Management

Relevance BasedDistance BasedBuffer Consumption

Relevance Based

Moser, Kraiß & Klas 95 : L/MRPDefinition:

• State = (curr obj, skip)• Relevance : State Real

Relevance Based

Prefetch :• future objects with highest

relevanceReplace :

• old objects with lowest relevance

Example (5,+2)

History

ReferencedSkip

Time Scale

Relevance

Distance Based

Özden, Rastogi & Silberschatz 96

Definition• distance of a client C is the

offset between C and the immediate following client.

Distance Based

Replace :• blocks consumed by clients with

largest distance

Example

C4C3

C2

C1

Example

C4C3

C2

C1

Example

C4C3

C2

C1

Buffer Consumption

Wu & Yu 97Definition :

• Buffer consumption = amount of buffer used x total time

Buffer Consumption

Result :• Minimizing buffer consumption

increase system throughput• Increase retrieval rate of current

stream is better then prefetchingnext waiting stream

Open Problems

Querying

Query video by action ?Query music by

• relative pitch + chord + rhythm ?

Continuing quest to improve accuracy

Indexing

Well studied problemIndexing for approximate

string search ?

Buffer Management

Should take into consideration:• object dependencies (MPEG)