Computer graphics & visualization Image Compression

computer graphics & visualization

Image Synthesis

Image Compression


Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group

Motivation1920 x 1440 x 32 bit = 10,54 MB

High Quality JPEG 0,9 MB



Classification• Lossless– TGA– GIF– PNG– …

• Lossy– JPEG– JPEG 2000– MPEG



Lossless Compression schemes

• RLE• LZ– LZ77– LZ78 / LZW

• Huffmann• Deflate / “ZIP”

• GIF• PNG



Run length EncodingExample:WWWWWWWWWWBWWWWWWWWWWBBB 10WB10W3B

What to do if numbers appear in the code?Uses control char (X in this case) and only encode sequences of more than two chars.121211111111122222212111 1212X91X6212111

What if X appears in the code?Use XX to encode a regular X (and hope that you will not see too many Xs in you data).



LZ - Lempel-ZivLZ77 (1977)Idea: Conversion of the data in (hopefully only few) 3-tupel Implementation: sliding window + look ahead approach

Example: ANANAS

01234567 0123 Output

ANAN (0,0,A)

A NANA (0,0,N)

AN ANAS (6,2,A)

ANANA S (0,0,S)

ANANAS Done



DecodingLZ77 (1977)Idea: Conversion of the data in (hopefully only few) 3-tupel Implementation: sliding window + look ahead approach

Example: ANANAS

Input 012334567

(0,0,A) A

(0,0,N) AN

(6,2,A) ANANA

(0,0,S) ANANAS

Done



Implementation DetailsEncoding of periodic entries

Example: 0101010101010101010101… (length n)

01234567 0123 Output

0101 (0,0,’0’)

0 1010 (0,0,’1’)

01 0101 (6,n-2,’1’)



Implementation ChallengesFinding the right position

Example: 0120101601015

01234567 0123 Output

0120 (0,0,’0’)

0 1201 (0,0,’1’)

012 0101 (5,2,’0’)

012010 1601 (0,0,’1’)

0120101 6010 (0,0,’6’)

01201016 0101 (0,2,’1’) or (3,4,’5’)



ProblemsEncoding of non-periodic

Example: 0123456789ABCDE…

“Compression” actually increases data size by a factor of three Combination with Huffman encoding Deflate (later)

01234567 0123 Output

0123 (0,0,’0’)

0 1234 (0,0,’1’)

01 2345 (0,0,’2’)

012 3456 (0,0,’3’)

... .... .........



LZW - Lempel-Ziv-WelchDictionary based compression with adaptive dictionaryUsually 12 bit codebook, first 8 bits reserved for ASCII

LZWLZ7…string Entry Output New Entry

LZWLZ7… L L LZ <256>

ZWLZ7… Z Z ZW <257>

WLZ7… W W WL <258>

LZ7… LZ<256> <256> LZ7 <259>

7… 7 7 7. <260>

L Z W <256> 7…



Decoding

Current Char New Entry Output

L - - L

Z Z LZ <256> Z

W W ZW <257> W

<256> L WL <258> LZ

7 7 LZ7 <259> 7

…

LZWLZ7…

L Z W <256> 7…



Features and Issues

• „Codebook overflow/reset“• Simple deterministic encoding / decoding• Worst-Case 8 bit 12 bit (or 8 bit -> dict size)• was subject to software patent until 2002 (2006)



Huffman Encoding 1951Given:A set of Symbols and their weights (usually probabilities).

Find:A prefix-free binary code (a set of codewords) with minimum expected codeword length (equivalently, a tree with minimum weighted path length).



Variable Length Problem Example:

What is the meaning of 10010?

ACA (10 0 10) or ABC (10 01 0)?

Prefix – free codes

A B C

10 01 0



Huffman EncodingStart with as many leaves as there are Symbols. Queue all leaf nodes into the first queue (in order). While there is more than one node in the queues:

– Remove two nodes with the lowest weight (frequency of appearance) from the queues.

– Create a new internal node, with the two just-removed nodes as children (either node can be either child) and the sum of their weights as the new weight.

– Update the parent links in the two just-removed nodes to point to the just-created parent node.

– Queue the new node into the second queue.

The remaining node is the root node; the tree has now been generated.



Example

A

B

C

D

E

A B C D E

15 7 6 6 5

A B C D E

0 100 101 110 111

Codes:

Frequencies:



Properties• Huffman coding is optimal

i.e. it generates an optimal tree for a given input• Requires knowledge of frequencies– can be extended to do adaptive tree updates

• Can be combined with LZ77 a post processing step, thus compressing the lengthy tuples Deflate, used primarily in ZIP, gzip, etc.



Now back to imagesBMP, TGAstore images as an RLE array of n-bit (n usually 24/32) values

GIFUses 256 colors (8bit) and uses LZW compressiona few GIF versions exist, usually GIF89a is used today, allowing for a single transparent color and stacking multiple images together to generate an animation



PNG – Portable Network GraphicInitially developed to circumvent GIFs, patent pending LZW compression.

• uses prefilter + LZ77 + Huffman for compression• supports not only 8bit palettes but also 1,2,4,8

or 16bit per channel color information• Supports 8 or 16 bit alpha channel• Now finally supported by all major browsers• No support for animation



PNG Prefilter

ID Name Description

0 None Use original pixel values

1 Sub Use difference to left pixel

2 Up Use difference to upper pixel

3 Average Compute Average of upper and left and use difference

4 Paeth Use difference to paeth predictor

Used per line



Paeth Predictorfunction PaethPredictor (a, b, c){ a = left, b = above, c = upper left p := a + b - c // initial estimate pa := abs(p - a) // distances to a, b, c pb := abs(p - b) pc := abs(p - c) // return nearest of a,b,c, // breaking ties in order a,b,c.

if pa <= pb AND pa <= pc then return a else if pb <= pc then return b else return c}



Paeth Predictor: Idea

c b

a xGoal: Minimize Filter(x)

“Optimal way” to do this: Compute abs(a-x), abs(b-x), abs(c-x) and store minimal value

But this would require two bits to remember what difference was used.

Instead: compute a “good” estimate of x using only a,b,c (called p) and find the minimalabs(a-p), abs(b-p), abs(c-p) to select which of abs(a-x), abs(b-x), abs(c-x) is used



Lossy CompressionJPEG (Joint Photographic Experts Group)• Lossless compression not working well for photos

• Useses lossy compression withadjustable compression ration

• Can do lossless • Fast deconding• Works for all types of static images,

no restrictions on color depth

Compression ratio



Compression Overview• Color Space conversion to YUV• Block based color sub sampling• Discrete Cosine-transformation

(DCT) • Quantization of the DCT

Coefficients• Coefficient serialization

(dove tailing)• Coefficient encoding



YUVYUV

Y is LuminanceU Chrominance: Color change towards blueV Chrominance: Color change towards red

Red : Y= 77 U= -43 V= 127 Green: Y= 151 U= -84 V= –107 Blue : Y= 28 U= 127 V= -20

Example:

YUV ConversionY = 0.30 R + 0.59 G + 0.11 BU = -0.17 R - 0.33 G + 0.50 BV = 0.50 R - 0.42 G - 0.08 B

RGB ConversionR = Y + 1.40 VG = Y - 0.34 U - 0.71 VB = Y + 1.78 U



YUV Conversion Example

Y

U

V



YUVSub sampling Split YUV-ChannelsEye is more sensitive to brightness than to color changes:Color sub sampling

Default: Mode 4:2:2 Full Quality : Mode 4:4:4

• First digit: Luma horizontal sampling reference• Second and Third:

U and V (chroma) horizontal factor (relative to first digit) Except when third is zero. Zero indicates that V horizontalfactor is equal to second digit, and, in addition, both U and V are subsampled 2:1 vertically



Sub sampling4:4:4 best quality 4:2:2 best quality

4:4:4 worst quality 4:2:2 worst quality

Without Sub sampling With Sub sampling



Preparation for DCT

From now on consider YUV planes separately

Index shiftTransform [0 - 255] to [-128 - 127]

BlockingAlways consider a 8x8 data block for the DCT



DCTDCT (1D)Related to FT in that it transforms a spacial signal to the frequency domain, however the DCT avoids the imaginary sine part by representing the signal as a sum of cosine waves

1t=0

( ) 2sonst

1

C t

DCT(u): Cosine amplitude for frequency uN: number of pixels 8x8=64 for JPEGf(x): Pixel value at position xC: correction factor

1

0

2 12( ) ( ) ( ) cos

2

N

x

x uDCT u C u f x

N N



DCTDCT (2D)

1t=0

( ) 2sonst

1

C t

Large, regular areas are stored in the upper right while higher frequencies are stored towards the lower left

1 1

0 0

2 1 2 11( , ) ( ) ( ) ( , ) cos cos

2 22

N N

i j

x i y jDCT i j C i C j Pixel x y

N NN



DCT

Pixel

(IDCT)DCT



DCT

Pixel

(IDCT)DCT



DCT700 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 00 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

87 87 87 87 87 87 87 87

87 87 87 87 87 87 87 87

87 87 87 87 87 87 87 87

87 87 87 87 87 87 87 87

87 87 87 87 87 87 87 87

87 87 87 87 87 87 87 8787 87 87 87 87 87 87 87

87 87 87 87 87 87 87 87

DCT IDCT

Result



DCTDCT IDCT

Result

700 100 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 00 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

105 102 97 91 84 78 73 70

105 102 97 91 84 78 73 70

105 102 97 91 84 78 73 70

105 102 97 91 84 78 73 70

105 102 97 91 84 78 73 70

105 102 97 91 84 78 73 70105 102 97 91 84 78 73 70

105 102 97 91 84 78 73 70



DCTDCT IDCT

Result



DCTDCT IDCT

Result



DCTDCT IDCT

Result



DCTDCT IDCT

Result



DCT PropertiesA real life photo has the following properties after DCT

the coefficient DC is by far the largestthe ACs become smaller with increasing frequencymost of the ACs are close to zero

Properties of the DCT?The DCT in general is a reversible transform thus applying the DCT and back transforming is lossless (in theory)



QuantizationFor JPEG the DCT gives us 64 Values, each of them is divided bya quantization matrix entry and rounded afterwards

Quantizermatrix

BacktransformationMultiply DCT-value with Quantizermatrix:

Quality SettingsAdjust Matrix Q

Data is lost here!

( , )( , )

( , )

DCT i jDCT i j Round

Q i j

( , ) ( , ) ( , )DCT i j DCT i j Q i j



QuantizationDCT

After quantization

Matrix Q



DC EncodingStore first (upper left DC) as is, encode other values as differences from predecessor



DC EncodingCategorize values

Huffman encode category sequence and append index into category results in 2 tuple



AC Encoding

Symbol-1(Runlength, Size)

Symbol-2(Amplitude)

Runlength = Zero countSize = CategoryAmplitude = Index

First block is Huffman encoded (or table is used) and second block is appended

(2)(3), (1,2)( 1), (0,1)( 0), (0,1)( 0), (0,1)( 0), (2,1)(0), (0,0) ~ 51Bit

Runlength is 4 bit [0-15]

DC AC



JPEG 2000• does not necessarily split the image into blocks• uses wavelets instead of DCT• uses zero-tree encoder (EBCOT) instead of Huffman• embedded code allows for better preview

• better compression• better quality

JPG JPEG 2000



H.261• Divide image into 8x8 blocks• Motion compensation via cross correlation• Subtract from previous frame• Compress JPEG like 4:2:0• Uses one of 31 fixed quantizer matrices

+ one uniform matrix for DC

Documents

Computer graphics & visualization Image Compression