Slide 1

A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression Chenlei Guo Liming Zhang Image Processing 2010

Slide 2

Outline Introduction Phase Spectrum of Quaternion Fourier Transform (PQFT) Detect Proto-Objects in the Spatiotemporal Saliency Map Hierarchical Selectivity (HS) Experiment Result Applications in Image and Video Coding Conclusions and Discussions

Slide 3

Introduction Most traditional object detectors need training Graph-based visual saliency detection can be very powerful but it demands a very high computational cost Most of the models only consider static images

Slide 4

Phase Spectrum of Quaternion Fourier Transform(PQFT) (1/3) Locations with less periodicity or less homogeneity create pop out proto objects in the reconstruction of the images phase spectrum An early saliency detection model : PFT

Slide 5

Quaternion Representation (2/3) Define the input image captured at time t as F(t) r(t), g(t), b(t) are color channels of F(t)

Slide 6

Calculate the Saliency Map By PQFT (3/3) 2-D gaussian filter

Slide 7

Detect Proto-Objects (1/3)

Slide 8

Alpha (2/3)

Slide 9

Gamma (3/3)

Slide 10

How PQFT Select Visual Resolution PQFT simulates the human vision system(HVS)

Slide 11

Hierarchical Selectivity Set hierarchical level

Slide 12

Experiment Results Video Sequence Natural Images Psychological Patterns

Slide 13

Video Sequence (1/3)

Slide 14

Video Sequence (2/3)

Slide 15

Video Sequence (3/3)

Slide 16

Natural Image

Slide 17

Evaluation Method - ROC True Positive Rate(TPR), False Positive Rate(FPR) Receiver Operating Characteristic (ROC) ROC curve = TPR/FPR ROC area = area beneath ROC curve The larger ROC area is, the better the prediction power of a saliency map.

Slide 18

Psychological Patterns (1/3)

Slide 19

Psychological Patterns (2/3)

Slide 20

Psychological Patterns (3/3)

Slide 21

Applications in Image and Video Coding Multiresolution Wavelet Domain Foveation Model (MWDF) Evaluate the performance of the HS-MWDF model in Image and video compression

Slide 22

Multiresolution Wavelet Domain Foveation Model (MWDF) JPEG 2000 has included the region-of-interest(RoI) coding in drafts A better way to find RoI:use Hierarchical Selectivity

Slide 23

Multiresolution Wavelet Domain Foveation Model (MWDF)

Slide 24

The Performance of HS-MWDF in Image Compression We use HS-MWDF model as a front end before standard compression (JPEG 2000) Set n fov => we only use the first n OCAs found by PQFT Auto fov => let the program itself decide the number

Slide 25

Slide 26

The Performance of HS-MWDF in Video Compression

Slide 27

Conclusion and Discussion Extend PFT model to PQFT model PQFT model is independent of parameters and prior knowledge, and is fast enough to meet real- time requirements Develop a model called HS-MWDF as a front end before the image/video encoder Problems: Cant deal with closure patterns well Only considers bottom-up information Insert the model into the image/video encoders

Slide 28

References