Marc CHAUMONT ICIP 2003 Fully scalable object based video coder based on analysis- synthesis scheme Marc Chaumont, Nathalie Cammas 1 and Stéphane Pateux

Marc CHAUMONTICIP 2003

Fully scalable object based video coder based on analysis-

synthesis schemeMarc Chaumont, Nathalie Cammas1 and Stéphane Pateux

Temics, IRISA/INRIA, France, 1France Telecom, France


What is scalability ? ordering the coded information by decreasing significant order. different scalability : object, SNR, temporal, spatial, bitstream.

Why looking for scalability ? bitstream can be decoded at different bitrate and different qualities

Why using an object coder instead of an non object coder ? object manipulation bitrate allocation better motion estimation (limit mesh degeneracy on occlusion boundaries) good tradeoff between pixel based and model based approach.

Context


Objectives : scalability on the 3 informations : motions - textures - shapes independent coding of the 3 informations long term approach

Why those objectives : scalability : … independent coding : to allow a better rate distribution between motion -texture - shape. long term approach : to allow an efficient coding with wavelet

Our proposed solution : using wavelet (to allow scalability) decorrelation (to allow independent coding) long term motion estimation (to allow a long term approach)

Proposed approach


Proposed approachanalysis-synthesis scheme

z-ordershapes

frames

z-ordershapes

frames1 - ANALYSIS2 - CODING3 - DECODING

z-order

NOSHAPE

frames

z-order

frames

shapes

1 - ANALYSIS2 - CODING3 - DECODING

sequence

SYNTHESIS

SEGMENTATION

obj1

obj2reconstructed

sequence



z-ordershapes

frames

z-ordershapes


z-order

NOSHAPE

frames

z-order

frames

shapes


sequence

SYNTHESIS

SEGMENTATION

obj1

obj2reconstructed

sequence



z-ordershapes

frames

z-ordershapes




z-ordershapes

frames

z-ordershapes





z-ordershapes

frames

z-ordershapes

frames

GOP GOP



long termmotion

estimation(active mesh) decorrelation

thanks tomotion

and paddingz-order

shapes

frames

z-ordershapes

frames

GOP

SYNTHESIS

motions

contours

textures

motions

textures

contours

codingdecoding

codingdecoding

codingdecoding

GOP



long termmotion


thanks tomotion

and paddingz-order

shapes

frames

z-ordershapes

frames

GOP

SYNTHESIS

motions

contours

textures

motions

textures

contours

codingdecoding

codingdecoding

codingdecoding

GOP


Long term motion estimation

Motion estimation via active mesh

GOP size = 8



long termmotion


thanks tomotion

and paddingz-order

shapes

frames

z-ordershapes

frames

GOP

SYNTHESIS

motions

contours

textures

motions

textures

contours

codingdecoding

codingdecoding

codingdecoding

GOP


Decorrelation:motion projection & padding

textures projected

contours projected

contours projected

MOTIONPROJECTION PADDING

MOTIONPROJECTION PADDING

contours padded

textures paddedinitial textures

initial contours



long termmotion


thanks tomotion

and paddingz-order

shapes

frames

z-ordershapes

frames

GOP

SYNTHESIS

motions

contours

textures

motions

textures

contours

codingdecoding

codingdecoding

codingdecoding

GOP


motion - texture - contour

motion (mesh) textures projected and padded contours projected and padded

3 independent informationGOP size = 8


Decorrelation allows :

independent coding of the 3 informations : motion -texture - shape. allows independent lossy coding on each information

a better rate distribution between motion -texture - shape.

distribution example for Foreman foreground CIF 15Hz at 85 Kb/s :

texture : 76 %

motion : 17 %

shape : 7 %

long term approach efficient coding with wavelet

Example : our scheme on Erik sequence CIF 15Hz at 64Kb/s is better than H26L VM 8.4 (less than 1 dB)

scalability on the 3 information thanks to wavelet

Benefits ofthe decorrelation



long termmotion


thanks tomotion

and paddingz-order

shapes

frames

z-ordershapes

frames

GOP

SYNTHESIS

motions

contours

textures

motions

textures

contours

codingdecoding

codingdecoding

codingdecoding

GOP


Coding step

textures

contours

motions

Spatialdecomposition

(9/7 Daubechies filter)

Temporal decomposition

(5/3 lifting filter)

Temporaldecomposition

(9/7 filter)

Spatialdecomposition

(9/7 Daubechies filter)

Temporal prediction

IPB

Spatialpyramidal

decomposition

Bit planarithmetic

coder

EBCOT

Bit planarithmetic

coder

Spatio-temporal transformation


Results

Background object :

Foreground object :

Shape : 3 Kb/s

Texture : 39,7 Kb/s Motion : 7,3 Kb/s

Texture + Motion : 11,5 Kb/s

Reconstructed sequence at 62 Kb/s


With or without shape distortion

Results

Without shape distortion Reconstructed sequence at 62 Kb/s(with shape distortion)


H264/AVC versus our object scalable scheme

Results

Our scheme62 Kb/s

PSNRtexture-foreground = 29.3

H264/AVC non object62 Kb/s

PSNRforeground = 27.9. 1 B frame,. RD optimization,. CABAC.


Spatial scalability

Results

Sequence 62 Kb/s CIF 15 Hz

shape : 3 Kb/sbackground : text + mvt : 11,5 Kb/sforeground : text + mvt : 47 Kb/s

Sequence 49 Kb/s QCIF 15 Hz

shape : 1,7 Kb/sbackground : text + mvt : 9,6 Kb/sforeground : text + mvt : 37,5 Kb/s


Spatio-temporal scalability

Results

Sequence 49 Kb/s QCIF 15 Hz Sequence 36 Kb/s QCIF 7.5Hz




SNR scalability

Results

Sequence 49 Kb/s CIF 15 Hz Sequence 126 Kb/s CIF 15 Hz




Based on : analysis-synthesis scheme decorrelation of the 3 informations

(active mesh - padding - z-order) fully scalable

Benefits : fully scalable (SNR, spatial, temporal & on each information) independent coding of the 3 informations

better bitrate distribution long term approach

allow the use of longer wavelet kernels

Future work : improving texture coding improving motion estimation in occlusion part

Conclusion :A novel object based video coder

Documents

Marc CHAUMONT ICIP 2003 Fully scalable object based video coder based on analysis- synthesis scheme Marc Chaumont, Nathalie Cammas 1 and Stéphane Pateux