EE465: Introduction to Digital Image Processing 1

Data Compression TechniquesText: WinZIP, WinRAR (Lempel-Ziv

compression’1977)Image: JPEG (DCT-based), BMP (run-

length coding)Video: VCD/DVD (MPEG), Real

Player/Windows Media Player (H.263/H.264), BluRay

Audio and Speech: MP3, G.728 (CELP)

EE465: Introduction to Digital Image Processing 2

Lossless vs. Lossy Compression

One is original and the other is compressed, which one is original?

Mathematical lossy can still be visually lossless

EE465: Introduction to Digital Image Processing 3

Engineering is About Design and Implementation

cost

performance

Product A

Product B

EE465: Introduction to Digital Image Processing 4

A Tour of JPEG Coding Algorithm

Flow-chart diagram of DCT-based coding algorithm specified by Joint Photographic Expert Group (JPEG)

T Q C

EE465: Introduction to Digital Image Processing 5

First Building Block: 8-by-8 DCT

8881

1811

88

..............................

......

aa

aa

A

81,82,16

)1)(12(cos21

81,1,8

1

lkkl

lkakl

i

jDC

TAXAY X T

EE465: Introduction to Digital Image Processing 6

Transform Coding of ImagesWhy not transform the whole image

together? Require a large memory to store transform

matrix It is not a good idea for compression due to

spatially varying statistics within an imageIdea of partitioning an image into blocks

Each block is viewed as a smaller-image and processed independently

It is not a magic, but a compromise

EE465: Introduction to Digital Image Processing 7

Block Processing under MATLABType “help blkproc” to learn the usage of

this function B = BLKPROC(A,[M N],FUN) processes the

image A by applying the function FUN to each distinct M-by-N block of A, padding A with zeros if necessary.

Example I = imread('cameraman.tif'); fun = @dct2; J = blkproc(I,[8 8],fun);

EE465: Introduction to Digital Image Processing 8

Block-based DCT Example

JI

note that white lines are artificially added to the border of each 8-by-8 block to denote that each block is processed independently

EE465: Introduction to Digital Image Processing 9

Boundary Padding

padded regions

Example

1213141516171819

1213141516171819

1213141516171819

1213141516171819

When the width/height of an image is not the multiple of 8, the boundary isartificially padded with repeated columns/rows to make them multiple of 8

EE465: Introduction to Digital Image Processing 10

Work with a Toy Example

169130173129

170181170183

179181182180

179180179179

169132171130

169183164182

179180176179

180179178178

167131167131

165179170179

177179182171

177177168179

169130165132

166187163194

17611615394

153183160183

Any 8-by-8 block in an image is processed in a similar fashion

EE465: Introduction to Digital Image Processing 11

Encoding Stage I: Transform• Step 1: DC level shifting

169130173129

170181170183

179181182180

179180179179

169132171130

169183164182

179180176179

180179178178

167131167131

165179170179

177179182171

177177168179

169130165132

166187163194

17611615394

153183160183

412451

42534255

51535452

51525151

414432

41553654

51524851

52515050

393393

37514251

49515443

49494051

412374

38593566

48122534

25553655

128 (DC level)

_

EE465: Introduction to Digital Image Processing 12

• Step 2: 8-by-8 DCT

412451

42534255

51535452

51525151

414432

41553654

51524851

52515050

393393

37514251

49515443

49494051

412374

38593566

48122534

25553655

1342

1209

4021

1344

3055

4773

3046

3216

113916

109621

1793310

8101720

10242727

1326078

44131827

273856313

Encoding Step 1: Transform (Con’t)

88DCT

EE465: Introduction to Digital Image Processing 13

Second Building Block: Quantization

99103101120

100112121103

98958778

92726449

9211377103

1048110968

64555637

35242218

62805669

87515740

29222416

17141314

55606151

58264024

19141610

12121116

Q-table

8,1

ˆ:

:

1

ji

Qsxf

Qx

sf

ijijij

ij

ijij

: specifies quantization stepsize (see slide #28)

Notes: Q-table can be specified by customerQ-table is scaled up/down by a chosen quality factor Quantization stepsize Qij is dependent on the coordinates (i,j) within the 8-by-8 block Quantization stepsize Qij increases from top-left to bottom-right

EE465: Introduction to Digital Image Processing 14

Encoding Stage II: Quantization (Con’t)

1342

1209

4021

1344

3055

4773

3046

3216

113916

109621

1793310

8101720

10242727

1326078

44131827

273856313

0000

0000

0000

0000

0000

0000

0000

0000

0000

0001

1011

0111

0001

0123

2113

23520

Example

f

xijsij

EE465: Introduction to Digital Image Processing 15

The Third Building Block: Entropy Coding

Zigzag Scan

0000

0000

0000

0000

0000

0000

0000

0000

0000

0001

1011

0111

0001

0123

2113

23520

(20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB)

zigzag scan

End Of the Block:All following coefficients are zero

EE465: Introduction to Digital Image Processing 16

Run-length Coding(20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB)

DCcoefficient

ACcoefficient

- DC coefficient : DPCM coding- AC coefficient : run-length coding (run, level)

(5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB)

(0,5),(0,-3),(0,-1),(0,-2),(0,-3),(0,1),(0,1),(0,-1),(0,-1),(2,0),(0,1),(0,2),(0,3),(0,-2),(0,1),(0,1),(6,0),(0,1),(0,1),(1,0),(0,1),EOB

Huffman codingencoded bit stream

encoded bit stream

EE465: Introduction to Digital Image Processing 17

JPEG Decoding Stage I: Entropy Decoding

(20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB)

Huffman decodingencoded bit stream

AC coefficients

DC coefficient

DPCM decoding

(0,5),(0,-3),(0,-1),(0,-2),(0,-3),(0,1),(0,1),(0,-1),(0,-1),(2,0),(0,1),(0,2),(0,3),(0,-2),(0,1),(0,1),(6,0),(0,1),(0,1),(1,0),(0,1),EOB

encoded bit stream

EE465: Introduction to Digital Image Processing 18

JPEG Decoding Stage II: Inverse Quantization

(20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB)

0000

0000

0000

0000

0000

0000

0000

0000

0000

0001

1011

0111

0001

0123

2113

23520

zigzag

0000

0000

0000

0000

0000

0000

0000

0000

0000

00040

2902416

0141314

00051

0268072

38141630

243655320

f-1

EE465: Introduction to Digital Image Processing 19

JPEG Decoding Stage III: Inverse Transform

0000

0000

0000

0000

0000

0000

0000

0000

0000

00040

2902416

0141314

00051

0268072

38141630

243655320

169130185127

170181162192

179181181184

179180178183

173128161129

169191175186

174181168186

178181182172

170131177128

170185166187

171187168172

180169169174

175124170120

168193171197

158143148119

153186140195

412571

42533464

51535356

51525055

450331

41634758

46534058

50535444

423490

42573859

43594044

52414146

474438

40654369

3015209

25581267

88IDCT

128 (DC level)

+

EE465: Introduction to Digital Image Processing 20

Quantization Noise

169130173129

170181170183

179181182180

179180179179

169132171130

169183164182

179180176179

180179178178

167131167131

165179170179

177179182171

177177168179

169130165132

166187163194

17611615394

153183160183

169130185127

170181162192

179181181184

179180178183

173128161129

169191175186

174181168186

178181182172

170131177128

170185166187

171187168172

180169169174

175124170120

168193171197

158143148119

153186140195

X X^

MSE=||X-X||2^Distortion calculation:

Rate calculation: Rate=length of encoded bit stream/number of pixels (bps)

EE465: Introduction to Digital Image Processing 21

JPEG Examples

1000

90 (58k bytes)50 (21k bytes)10 (8k bytes)

best quality,

lowest compression

worst quality,

highest compression

EE465: Introduction to Digital Image Processing 22

Rate-Distortion Tradeoff

EE465: Introduction to Digital Image Processing 23

Block artifacts of JPEG when the bit rate is low (0.25bpp)

EE465: Introduction to Digital Image Processing 24

JPEG Coding Algorithm Summary

EE465: Introduction to Digital Image Processing 25

Lossy Compression of Color Images

Color-space transformation

BGR

CCY

b

r

063.0375.0438.045.03.015.01.06.03.0luminance

chrominance

b

r

CCY

BGR

0217994.03341.01

5987.10016.01

inverse transform

forward transform

Luminance channel can be treated just like a gray-scale image

EE465: Introduction to Digital Image Processing 26

Compression of Chrominance Channels

Human eye is relatively insensitive to the high-frequencycontent of the chrominance channels.

Down-sampling

2

Cr/Cb

Q-Table

99999999

99999999

99999999

99999999

99999999

99999999

99999999

99999999

99999999

99999999

99999956

66472624

99999999

99999999

66264724

21181817

EE465: Introduction to Digital Image Processing 27

Technology is About New ToolsThe same design principle but each

building block can be replaced by a more advanced technique Transform: DCT becomes Wavelet Transform Quantization: uniform quantization becomes

bitplane coding (to support scalability) Entropy coding: Huffman coding becomes

arithmetic coding (more flexible)JPEG becomes JPEG2000

EE465: Introduction to Digital Image Processing 28

A Glimpse into Wavelet Transform*Multi-Resolution Analysis in Vision

Human vision system is capable of interpretingthe visual information regardless of the resolution

EE465: Introduction to Digital Image Processing 29

Progressive Image TransmissionI hate to wait – show me a piece and see if

I like it or not

Low-resolution (LR)

High-resolution (HR)

EE465: Introduction to Digital Image Processing 30

Haar Transform – the simplest wavelet transform

212

1

,

212

1

,11

112

12121 bbbbA

TAXAY

Example

5050100100

X

0500150

Y

EE465: Introduction to Digital Image Processing 31

Reordering Coefficients

50505050

50505050

100100100100

100100100100

X

Apply Haar transformto each 2-by-2 block

000100

000100

000200

000200

Y regrouping

0000

0000

0000

100100200200

Y

HHHLLHLL

approximation horizontal details

diagonal detailsvertical details

EE465: Introduction to Digital Image Processing 32

Multi-level Wavelet Transform

50505050

50505050

100100100100

100100100100

X

Apply one-level Haar transform

0000

0000

0000

100100200200

Y

Apply Haar transformto Low-Low band only

0000

0000

0000

01000300

Z

11

1

HHHLLH

22

22

HHHLLHLL

EE465: Introduction to Digital Image Processing 33

Image Example

EE465: Introduction to Digital Image Processing 34

Wavelet Image Compression

WaveletTransform

ScalarQuantization

EntropyCodingimage

compressedbit stream

Major improvements

Wavelet transform instead of DCT

Arithmetic coding instead of Huffman coding

Modeling location instead of intensity

Those are the topics in EE565: Advanced Image Processing

EE465: Introduction to Digital Image Processing 35

FBI Wavelet Scalar Quantization (WSQ) StandardEstablished by FBI in 1990s as the

standard for fingerprint image compression

Wavelet-based: Daubechies’ wavelets were built just in time to win the race

Demonstrate better Rate-Distortion performance than DCT-based JPEG standard

EE465: Introduction to Digital Image Processing 36

JPEG2000 StandardDeveloped by ISO in late 1990s to replace

the DCT-based JPEG standardAccumulative research progress on wavelet

image compression has dramatically pushed the art of lossy image compression forward

Demonstrate much better Rate-Distortion performance than DCT-based JPEG standard for typical photographic images

EE465: Introduction to Digital Image Processing 37

Wavelet vs. DCT

JPEG (CR=64) JPEG2000 (CR=64)discrete cosine transform based wavelet transform based

EE465: Introduction to Digital Image Processing 38

BIGGER QuestionWhy haven’t we seen JPEG being

replaced by JPEG2000?Technical superiority DO NOT always

imply economical wins (think about Unix vs. Windows)

THE next image compression standard has to wait for the right timing

Anyone including you can be a part of that business