Generalized Lifting for Sparse Image Representation and Coding

Generalized Lifting for SparseImage Representation and Coding

Julio C. Rolón1,2, Philippe Salembier1

1Technical University of Catalonia (UPC), SpainDept. of Signal Theory and Telecommunications

2National Polytechnic Institute (IPN), MexicoCITEDI Research Center

2

Generalized Lifting for Sparse Image Representation and Coding

Outline

Introduction• Motivation• Previous work

Lifting structures• Classical lifting• Generalized lifting

Coding scheme

Experimental results• Energy and entropy• Rate-distortion• Perceptual quality

Conclusions

3


Outline



Coding scheme


Conclusions

4


Introduction: Motivation

Limitations of wavelets in image representation• Excellent for decorrelation of impulsional-like events

• Limitations on images because wavelets are not able to fully decorrelate edges

DWT

LH subband zoom

5


Introduction: Motivation

Limitations of wavelets in image representation• Excellent for decorrelation of impulsional-like events

• Limitations on images because wavelets are not able to fully decorrelate edges

DWT

LH subband zoom

Need of additional decorrelation for coding applications

6


Outline



Coding scheme


Conclusions

7


Introduction: Previous work

Main approaches• Go beyond wavelets trying to overcome their limitations• Edges become 1-dimensional localizable singularities• Increased sparsity of the coefficients

Curvelets[Candès,Donoho 1999; Candès 2006]

Contourlets[Do,Vetterli 2005]

Frequency-domain methodsBandelets

[LePennec,Mallat 2005; Peyré,Mallat 2005]

Adaptive directional lifting[Chappelier et al 2006; Ding et al 2007;

Heijmans et al 2005; Mehrseresht,Taubman 2006; Piella et al 2002,2006; Wang et al 2006;

Zhang et al 2005 ]

Spatial-domain methods

Generalized liftingMay be highly non-linear,

preserving perfect reconstruction

8


Outline



Coding scheme


Conclusions

9


Lifting structures: Classical lifting

Approximation signal: x'[n] = x[n] + U(y'[n])

s[n]

x[n]

y[n]

x[n]

y[n]

s[n]

x'[n]

y'[n]

y'[n] = y[n] – P(x[n])Detail signal:

[Sweldens 1996]

10


LAZY WAVELET TRANSFORM

s[n]

nx[n]

y[n]

x'[n]

y'[n]

x[n]

n

y[n]

n

LWT

11


x[n]

n

y[n]

n

x[n]

y[n]

x'[n]

y'[n]


12


x[n]

n

y[n]

n

x[n]

y[n]

x'[n]

y'[n]

P(x[n])

n

n

P

y'[n]


13


x[n]

n

y[n]

n

x[n]

y[n]

x'[n]

y'[n]

P(x[n])

n

n

P

y'[n]Details are the prediction error


14



Lifting structures: Classical lifting

Approximation signal: x'[n] = x[n] + U(y'[n])

s[n]

x[n]

y[n]

x[n]

y[n]

s[n]

x'[n]

y'[n]

[Sweldens 1996]

+

+

Invertibility key:+/- operations

15


Outline



Coding scheme


Conclusions

16


Lifting structures: Generalized lifting

Approximation signal: x'[n] = U(x[n],y'[n])

y'[n] = P(y[n],x[n – i])|i∈CDetail signal:

[Solé,Salembier 2004]

s[n]

x[n]

y[n]

x[n]

y[n]

s[n]

x'[n]

y'[n]

17


LAZY WAVELET TRANSFORM

s[n]

nx[n]

y[n]

x'[n]

y'[n]

x[n]

n

y[n]

n

LWT

18


x[n]

n

y[n]

n

x[n]

y[n]

0 1 2 3 4 5 6 7 8 9

P


19


x[n]

n

y[n]

n

x[n]

y[n]

0 1 2 3 4 5 6 7 8 9

Py'[n]

n0 1 2 3 4 5 6 7 8 9

y'[n]


20


x[n]

n

y[n]

n

x[n]

y[n]

0 1 2 3 4 5 6 7 8 9

Details produced by P operator

y[n] → y'[n]P

Py'[n]

n0 1 2 3 4 5 6 7 8 9

y'[n]


21


x[n]

n

y[n]

n

x[n]

y[n]

0 1 2 3 4 5 6 7 8 9

Details produced by P operator

y[n] → y'[n]P

Py'[n]

n0 1 2 3 4 5 6 7 8 9

y'[n]


x0

x2

x4

x6

x8

y1

y3

y5

y7

y9

y'3

y'5

y'1

y'7

y'9

y'x y P(y,x)

22



Approximation signal: x'[n] = U(x[n],y'[n])


[Solé,Salembier 2004]

y[n] y[n]

s[n] s[n]

x'[n]

y'[n]

x[n] x[n]

BIJECTIVITY OF P, U IS MANDATORYTO ACHIEVE PERFECT RECONSTRUCTION

23



Previous work has been done in: [Solé,Salembier 2004,2007]

• Lossless coding• 1-dimensional separable context• Applied directly in the lmage domain

24



Previous work has been done in: [Solé,Salembier 2004,2007]

• Lossless coding• 1-dimensional separable context• Applied directly in the lmage domain

Our main goal is to evaluate the potential of the GL method for lossy coding• Quantization• Comparison between 1-dimensional context and 2-dimensional

non-separable context• Applied in the wavelet domain (as in bandelets)• Assumption: the pdf of the signal is known in the decoder

(as well as in the coder)

25


Outline



Coding scheme


Conclusions

26


Coding scheme: The encoder

DWT Q

A 011101…

A 10001…

ORIGINAL

27



DWT Q

A 011101…

A 10001…

ORIGINAL

2-D WAVELETTRANSFORMED

28



DWT Q

A 011101…

A 10001…

ORIGINAL


SCALARQUANTIZED

29



DWT Q

A 011101…

A 10001…

ORIGINAL

LWT 1-D row-wisedownsampling

and 2-D quincunxdownsampling


SCALARQUANTIZED

30



DWT Q

A 011101…

A 10001…

ORIGINAL




SCALARQUANTIZED

31



DWT Q

A 011101…

A 10001…

ORIGINAL




SCALARQUANTIZED

GL – MAPPEDDETAIL

32


Coding scheme: The decoder

DWT-1Q-1

A-1011101…

A-110001…

33


-4 -3 -2 -1 0 1 2 3 4

Coding scheme: Generalized predict design

-4 -3 -2 -1 0 1 2 3 4

Input signal pdf mapped pdf

GL

Criterion: Energy minimization of the mapped details

][ny ][' ny

34


-4 -3 -2 -1 0 1 2 3 4


-4 -3 -2 -1 0 1 2 3 4 ][ny


GL

][' ny


22

11

33

44

-2-2

-3-3

-1-1

00

22

11

33

44

-2-2

-3-3

-1-1

00

][' ny][ny

35


-4 -3 -2 -1 0 1 2 3 4


-4 -3 -2 -1 0 1 2 3 4 ][ny


GL

][' ny


22

11

33

44

-2-2

-3-3

-1-1

00

22

11

33

44

-2-2

-3-3

-1-1

00

][' ny][ny

36


-4 -3 -2 -1 0 1 2 3 4


-4 -3 -2 -1 0 1 2 3 4 ][ny


GL

][' ny


22

11

33

44

-2-2

-3-3

-1-1

00

22

11

33

44

-2-2

-3-3

-1-1

00

][' ny][ny

37


-4 -3 -2 -1 0 1 2 3 4


-4 -3 -2 -1 0 1 2 3 4 ][ny


GL

][' ny


22

11

33

44

-2-2

-3-3

-1-1

00

22

11

33

44

-2-2

-3-3

-1-1

00

][' ny][ny

38


-4 -3 -2 -1 0 1 2 3 4


-4 -3 -2 -1 0 1 2 3 4 ][ny


GL

][' ny


22

11

33

44

-2-2

-3-3

-1-1

00

22

11

33

44

-2-2

-3-3

-1-1

00

][' ny][ny

39


-4 -3 -2 -1 0 1 2 3 4


-4 -3 -2 -1 0 1 2 3 4 ][ny


GL

][' ny


22

11

33

44

-2-2

-3-3

-1-1

00

22

11

33

44

-2-2

-3-3

-1-1

00

][' ny][ny

40


-4 -3 -2 -1 0 1 2 3 4


-4 -3 -2 -1 0 1 2 3 4 ][ny


GL

][' ny


22

11

33

44

-2-2

-3-3

-1-1

00

22

11

33

44

-2-2

-3-3

-1-1

00

][' ny][ny

41


-4 -3 -2 -1 0 1 2 3 4


-4 -3 -2 -1 0 1 2 3 4 ][ny


GL

][' ny


22

11

33

44

-2-2

-3-3

-1-1

00

22

11

33

44

-2-2

-3-3

-1-1

00

][' ny][ny

42


-4 -3 -2 -1 0 1 2 3 4


-4 -3 -2 -1 0 1 2 3 4 ][ny


GL

][' ny


22

11

33

44

-2-2

-3-3

-1-1

00

22

11

33

44

-2-2

-3-3

-1-1

00

][' ny][ny

ASSUMPTION: Mapping is known by the decoder, i.e. the decoder knows the pdf of the signal

43


Outline



Coding scheme


Conclusions

44


0

20

40

60

80

100

1 2 3 4 5

100% reference: DWT DWT + GL, 1-D context DWT + GL, 2-D context

energy entropy% %

DWTscale

DWTscale

Experimental results: Energy and entropy

0

20

40

60

80

100

1 2 3 4 5

45


Outline



Coding scheme


Conclusions

46


Experimental results: Rate-distortion

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.530

32

34

36

38

40

42

Bitrate (bpp)

PSN

R (

dB)

Wavelet + GLBandeletsWavelets

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 130

32

34

36

38

40

42

Bitrate (bpp)

PSN

R (

dB)

Wavelet + GLBandeletsWavelets

Generalized liftingPotential gain is ~1-3 db, for 2-D context

47


Experimental results: Rate-distortion

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 124

26

28

30

32

34

36

38

40

Bitrate (bpp)

PSN

R (

dB)

Wavelet + GLWavelets

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 124

26

28

30

32

34

36

38

40

Bitrate (bpp)

PSN

R (

dB)

Wavelet + GLWavelets

Additional comparisons against wavelets

48


Outline



Coding scheme


Conclusions

49


Experimental results: Perceptual quality @ 0.2bpp

original wavelets32.03 dB

Bandelets31.24 dB

GL33.33 dB

50


Experimental results: Perceptual quality @ 0.11bpp

original wavelets29.47 dB

Bandelets28.52 dB

GL30.63 dB

51


Outline



Coding scheme


Conclusions

52


Conclusions

The potential of GL has been evaluated for lossy coding

53


Conclusions


The use of GL approach proposed here follows the ideas of spatial-domain methods(bandelets and adaptive directional lifting)

54


Conclusions



The GL method differentiates from these methods in thatis a non-linear framework

55


Conclusions




Our predict design is based on the pdf of the image

56


Conclusions




Our predict design is based on the pdf of the image

Generalized lifting approach gives promising results atreducing the energy while improving the R-D performance in 1-3 dB

57


Conclusions: Future work

Remove the assumption that the decoder knows the pdf of the signal• This work: pdf is assumed to be known in coder and decoder• Future: pdf estimation with fixed as well as adaptive strategies

58




Multiscale GL approach• This work: one scale of GL applied for 1-D and 2-D contexts• Future: multiscale GL in 2-D context

59





Quantization• This work: scalar pre-quantization• Future: include Q in the GL mapping, post-quantization

60





Quantization• This work: scalar pre-quantization• Future: include Q in the GL mapping, post-quantization

Efficient entropy coding• This work: arithmetic encoding• Future: embedded-block coders like EBCOT, SPECK, etc.

Questions


Julio C. Rolón, Philippe SalembierTechnical University of Catalonia (UPC), SpainDept. of Signal Theory and Telecommunications

National Polytechnic Institute (IPN), MexicoCITEDI Research Center

{jcrolon,philippe}@gps.tsc.upc.edu

Thank you !!!


Julio C. Rolón, Philippe SalembierTechnical University of Catalonia (UPC), SpainDept. of Signal Theory and Telecommunications

National Polytechnic Institute (IPN), MexicoCITEDI Research Center

{jcrolon,philippe}@gps.tsc.upc.edu

Documents

Generalized Lifting for Sparse Image Representation and Coding