View
7
Download
0
Category
Preview:
Citation preview
Image Processing
9. Image Compression
Computer Engineering, Sejong University
Dongil Han
2/64
The storage and communication of video requirements are immense• storage requirement for uncompressed video
512pixel x 512 pixel x 3bytes/pixel x 30frame/sec = 23.6MBytes/sec
image compression : reducing the amount of data required to represent a digital imageex) 23.6MBytes =>187KBytes
Fundamentals of image compression• reduce or eliminate the redundant data in the image or
video
Image Compression
.
3/64
application• Digital TV broadcasting• VOD(Video on Demand)• Televideo-conferencing• Medical imaging• Facsimile transmission• or diverse Multi-media environment
=> mainly used for storages and transmissions
Evaluation of compression efficiency• compression ratio
compression ratio = original data/compressed data
• the greater the compression ratio, the smaller the final image will be
Image Compression
.
4/64
Selection criteria for image compression techniques• achievable compression ratio• compression/decompression time• complexity of algorithm• cost of computational resources• availability of computational resources• adoption of standard or not • lossy or loseless compression • the quality of decoded image when lossy compression is
used
Image Compression
.
5/64
Fundamentals
Two basic types of image compression• Lossless compression : encodes and decodes the data
perfectly. resulting image matches the original image exactly• Lossy compression : allows redundant and nonessential
information to be lost
Data compression• process of reducing the amount of data required to represent
a given quantity of information• key idea : reducing the data redundancy
Three basic data redundancies in digital images• coding redundancy• interpixel redundancy• psychovisual redundancy
.
6/64
Elimination of coding redundancy• assigning fewer bits to the more probable data• assigning longer bits to the less probable data• this process is referred to as variable length coding• reduces the total number of encoded bits
• Image data : assigning fewer bits to the gray levels with higher historam
Fundamentals
.
7/64
coding redundancy• discrete random variable rk in the interval [0,1] represents
the gray levels of an image• each rk occurs with probability pr(rk)
where L is the number of gray levels• If the number of bits used to represent each value of rk is
l(rk), then the average number of bits required to represent each pixel is
• Elimination of coding redundancy => minimize the Lavg
n
nrp kkr )(
1
0
)()(L
kkrkavg rprlL
Fundamentals
8/64
Elimination of coding redundancy
bits
rprlLk
krkavg
7.2
)02.0(6)03.0(6)06.0(5)08.0(4
)16.0(3)21.0(2)25.0(2)19.0(2
)()(7
02
code 1 code 2
bits
rprlLk
krkavg
0.3
)02.0(3)03.0(3)06.0(3)08.0(3
)16.0(3)21.0(3)25.0(3)19.0(3
)()(7
01
Fundamentals
9/64
Elimination of coding redundancy
Fundamentals
10/64
Elimination of interpixel redundancy• use similarities between adjacent pixels• use similarities between adjacent fields• Run-length encoding, DPCM, ADPCM, etc.
• Terminologies for representing interpixel redundancy- spatial redundancy- geometric redundancy- interframe redundancy
Fundamentals
.
11/64
Elimination of interpixel redundancy
Fundamentals
12/64
Elimination of interpixel redundancy
Fundamentals
13/64
Elimination of psychovisual redundancy• brightness of region depends on factors other than
simply the light reflected by the regions• certain information has less importance than other
information in normal visual processing• this information is said to be psychovisual redundant• this redundancy can be eliminated without significantly
impairing the quality of image perception=> results in a loss of quantitative information
• related to the sampling, quantization
Fundamentals
.
14/64
Elimination of psychovisual redundancy
Fundamentals
15/64
Fidelity Criteria• reproducible means of quantifying the nature and extent
of information loss– objective fidelity criteria(객관적 충실도 기준)– subjective fidelity criteria(주관적 충실도 기준)
• objective fidelity criteria offer a simple and convenient mechanism for evaluating information loss
• sometimes, subjective evaluations by a human observer is more appropriate
Fundamentals
.
16/64
Example of objective fidelity criteria• root-mean-square(rms) error between an input and output
image• Let be input and estimate of input image,
then the total error between the two images is
where the images are of size MxN• The root-mean-square erms between the two images is
),(ˆ),,( yxfyxf
1
0
1
0
)],(),(ˆ[M
x
N
y
yxfyxf
2/11
0
1
0
2)],(),(ˆ[1
M
x
N
yrms yxfyxf
MNe
Fundamentals
17/64
Example of objective fidelity criteria• mean-square signal-to-noise ratio of the estimated
image, denoted SNRms is
1
0
1
0
2
1
0
1
0
2
)],(),(ˆ[
),(ˆ
M
x
N
y
M
x
N
yms
yxfyxf
yxf
SNR
Fundamentals
18/64
Image Compression Model
Image Compression System• two distinct structural blocks : Encoder and Decoder• source encoder : removes input redundancies• channel encoder : increases the noise immunity• noise free environment, the channel encoder/decoder are
omitted
Source encoder
Channel encoder Channel Channel
decoderSource decoder
Encoder Decoder
),( yxf ),(ˆ yxf
.
19/64
Image Compression Model
Source encoder• reduces or eliminates any coding, interpixel, psychovisual
redundancies in the input image
Source encoding stages• Mapper : reversible• Quantizer : irreversible• Symbol encoder : reversible
Mapper Quantizer Symbol encoder Channel
Source Encoder
),( yxf
.
20/64
Image Compression Model
Mapper• transforms the input data into a format designed to reduce
interpixel redundancies in the input image• may or may not reduce the amount of data• example : run-length coding• transformed results : array of coefficients
Quantizer• reduces the accuracy of the mapper’s output• reduces the psychovisual redundancy• must be omitted when error-free compression is desired
Symbol Encoder• reduces the coding redundancy• creates a fixed- or variable-length code
.
21/64
Image Compression Model
Source Decoder• performs the inverse operations of the source encoder• inverse quantizer block is not included in the general
source decoder
Source Decoding stages• Symbol decoder : inverse operations of the symbol
encoder• Inverse mapper : inverse operations of the mapper
Symbol Decoder
Inverse Mapper
Channel
Source Decoder
),(ˆ yxf
.
22/64
Error-free compression
Error-free compression• means for data reduction• reduces interpixel, coding redundancy
Application• medical or business documents• satellite imagery : cost of collecting the data• digital radiography(방사선 사진) : for diagnostic accuracy• normally provide compression ratios of 2 to 10
Error-free compression techniques• Variable-length coding• Arithmetic coding• Bit-plane coding : Run-length coding• Lossless predictive coding, etc.
.
23/64
Variable-length coding • the simplest approach to error-free compression technique• reduces coding redundancy only • assigns the shortest possible code words to the most
probable gray levels• source symbol could be gray levels or pixel differences of
an image, output of run-length encoding, etc.
Types of variable-length coding • Huffman coding• Truncated Huffman coding• Shift coding• Huffman shift coding, etc.
Error-free compression
.
24/64
Huffman coding• In 1952, a paper by David Huffman was published• variable length codes can achieve a higher data density than
fixed length codes• assigns short code for the most frequently occurring data• yields the smallest possible number of code symbols per
source symbol• resulting code is optimal with constraint that the source
symbols be coded one at a time
Huffman coding process1. Ordering the probabilities of symbols2. Combining the lowest probability symbols into a single
symbol. This process is repeated until a reduced source with two symbols is reached
3. To code each reduced source starting with the smallest source4. Working back to the original source
Error-free compression
25/64
symbol reductions
Code assignment procedure
Huffman coding
26/64
전치 특성(Prefix property)• 모든 코드는 다른 코드의 prefix가 될 수 없는 성질
– 예 : 영문자 e의 코드가 “01”일 경우 “010”, “011” 혹은“0100” 등의 코드는 존재하지 않는다
– 즉 e의 코드 “01”이 다른 코드의 prefix가 될 수 없다.– 만약 “010”을 b의 코드로 정하면 “010”을 복호화 할 때다음의 모순 발생
=> “01” 을 e로 복호화 혹은=> “010”을 b로 복호화
Another property• bit operation is required
Huffman coding
27/64
Advantages• creates the optimal code for a set of symbols and
probabilities • coding/decoding is accomplished in a simple lookup table• block code : each source symbol is mapped into a fixed
sequence of code symbols• can be decoded without referencing succeeding symbols
Disadvantages• encoding requires two nontrivial passes over the data
(probability calculation, code table creation)• one corrupted bit will wipe out the rest of the data
(disadvantage of variable length coding)• when a large number of symbols is decoded, the
construction of the optimal code is an nontrivial case
Huffman coding
.
28/64
Other near optimal VLCs
29/64
Arithmetic coding• nonblock code : a sequence of source symbols is
assigned a single arithmetic code word• the code word defines an interval of real numbers
between 0 and 1• as the number of source symbols increases, the interval
used to represent it becomes smaller• any number within the subinterval can be used to
represent the source symbol• encoding requires two nontrivial passes over the data
– probability calculation– code table creation
Error-free compression
.
30/64
Arithmetic coding example
Error-free compression
31/64
LZW(Lempel-Ziv-Welch) coding• assigns fixed-length code words to variable length
sequences of source symbols• for 8-bit images
– first 256 words of the dictionary are assigned to the gray values 0, 1, …, 255
– image sequences are added to the next code word (39-39 => 256, 39-126=>257, etc.)
Properties of LZW coding• requires no a priori knowledge of the probability• codebook is created while the data are being encoded• repeated source symbols such as “the”, “. “ are
compressed effectively• has been integrated into gif, tiff, pdf file format, etc.
Error-free compression
.
32/64
LZW coding example
Error-free compression
33/64
Bit-plane coding• effective technique for reducing an image’s interpixel
redundancy• process the image’s bit planes individually• compress each binary image via well-known binary
compression methods
Error-free compression
.
34/64
Bit-Plane Coding
Bit-plane decomposition• small changes in gray levels could generate many bit
change(e.g. 127(01111111) => 128(10000000))• m-bit gray code => reduces the bit change
– successive code words differ in only one bit position– small changes in gray levels are less likely to affect all
m bit planeDecimal number BCD code gray code
0 0000 0000 1 0001 0001 2 0010 0011 3 0011 0010 4 0100 0110 5 0101 0111 6 0110 0101 7 0111 0100 8 1000 1100 9 1001 1101
.
35/64
original image
original gray coded original gray coded
Bit-Plane Coding
36/64
Run-length coding• reduces interpixel redundancy• binary image compression methods• applied each bit-plane of gray level image• describe successive runs of black and white pixels• standard compression approach in FAX coding• compression technique in bmp file• applied for 1, 2, 4-bit gray level image• additional compression can be realized by variable-length
coding the run lengths themselves
– input : AAAABBBBBCCCCCCCCDEEEE– output : 4A5B8C1D4E– compression ratio : 22/10 = 2.2
Error-free compression
.
37/64
Run-length coding
Notes for RLC• If compression ratio is less than 1, send original data
– input : MyDogHasFleas– output : 1M1y1D1o1g1H1a1s1F1l1e1a1s– compression ratio : 13/26 = 0.5
• Use special prefix character (for example: +)– input : ABCDDDDDDDDEEEEEEEEE– output : ABC+8D+9E– compression ratio : 19/9 = 2.11
• It make sense to encode only runs of 3 or longer• If the special prefix character is found in the source : use
3 byte notation(run of length 1: + => +1+)
38/64
Lossless predictive coding• reduces interpixel redundancy• does not require decomposition of an image into bit planes• extracts and codes only the new information in each pixel• new information : the difference between the actual and
predicted value of that pixel
Error-free compression
.
39/64
Lossless predictive coding
Original image
Prediction error image
Histogram
Histogram of
prediction error
Error-free compression
40/64
Lossy compression
Lossy compression• compromise the accuracy of the reconstructed image in
exchange for increased compression• reduces interpixel, coding redundancy, psychovisual
redundancy
Application• Digital TV : MPEG-2, Video conferencing : H.261• still image coding: JPEG• normally provide compression ratios of 10 to 100
Lossy compression techniques• Lossy predictive coding• Transform coding• Hierarchical coding• Hybrid coding, wavelet coding, etc.
.
41/64
Lossy Predictive Coding
Lossy predictive coding• predictive coding with quantizer• predictions generated by the encoder and decoder are
equivalent• well-known form of lossy predictive coding
- DM(Delta Modulation)- most simple
• other lossy predictive coding techniques- ADM(Adaptive Delta Modulation)- DPCM(Differential Pulse Code Modulation)- ADPCM(Adaptive Differential Pulse Code Modulation)
• optimal predictor : DPCM- minimizes the encoder’s mean-square prediction error
.
42/64
Lossy predictive coding
Lossy predictive coding model
Lossy Predictive Coding
43/64
DM(Delta Modulation)• utilize a 1-bit fixed-length code• delta modulation process
- evaluate the input and present encoded level- allocate 1 when input level exceeds present encoded
level- else allocate 0
• implementation- positive/negative constant delta value is assigned to
each code- code 1 : + delta- code 0 : – delta
• relatively smooth region : granular noise appears• rapidly changing region : slope overload appears
Lossy Predictive Coding
.
44/64
DM(Delta Modulation)
Lossy Predictive Coding
45/64
Problem of Delta Modulation :• big delta : granular noise is increased• small delta: slope overload is increased
ADM(Adaptive Delta Modulation)• adaptively adjust the delta value• relatively smooth region
- reduce the delta value- encodes the small variation of input signal- granular noise is reduced
• rapidly changing region- increase the delta value- follows rapidly the input signal- slope overload is reduced
Lossy Predictive Coding
.
46/64
DPCM(Differential Pulse Code Modulation)• minimizes the encoder’s mean-square prediction error
• optimal predictor• prediction is constrained to a linear combination of m
previous pixels
}]ˆ{[}{ 22nnn ffEeE
Lossy Predictive Coding
47/64
)1,(97.0),(ˆ yxyxf ),1(5.0
)1,(5.0),(ˆ
yxf
yxfyxf
)1,1(5.0),1(75.0
)1,(75.0),(ˆ
yxfyxf
yxfyxf
otherwiseyxf
vhifyxfyxf
),1(97.0
)1,(97.0),(ˆ
|)1,1()1,(|
|)1,1(),1(|
yxfyxfv
yxfyxfh
Linear Prediction Example
48/64
Quantization example
Lossy Predictive Coding
49/64
DPCM example
1 bits/pixel 1.25 bits/pixel
2 bits/pixel
2.125 bits/pixel
3 bits/pixel3.125
bits/pixel
50/64
1.25 bits/pixel
2.125 bits/pixel
3.125 bits/pixel
1 bits/pixel
2 bits/pixel
3 bits/pixel
DPCM error example
51/64
Compression with transform coding• Most information is located in the low frequency region• modifying the transform of an image• map the image into a set of transform coefficients, which
are then quantized and coded• significant number of the coefficients have small magnitude • and can be coarsely quantized with little image distortion
Transform coding techniques• image is divided into subimages of size n x n (8x8, 16x16)• transform each block
- pack as much information as possible into the smallest number of transform coefficients
• more coarsely quantizes the coefficients that carry the least information
• final stage : usually VLC the quantized coefficient is used
Transform Coding
.
52/64
Typical transform coding system
Transform Coding
53/64
Transform coding• transform kernel(basis function) constructs each
transform coding techniques
• Types of transform coding- KLT((Karhunen-Loeve Transform)- DFT(Discrete Fourier Transform)- DCT(Discrete Cosine Transform)- WHT(Walsh-Hadamard Transform)
),,,(),(1
),(
),,,(),(),(
1
0
1
0
1
0
1
0
vuyxhvuFN
yxf
vuyxgyxfvuF
N
u
N
v
N
x
N
y
Transform Coding
.
54/64
Types of transform coding• Discrete Fourier Transform(DFT)
• Walsh-Hadamard Transform(WHT)
• Discrete Cosine Transform(DCT)
NvyuxjNvyuxj evuyxheN
vuyxg /)(2/)(22
),,,(,1
),,,(
1
0
)()()()(
)1(1
),,,(),,,(
m
iiiii vpybupxb
Nvuyxhvuyxg
]2
)12(cos[]
2
)12(cos[)()(),,,(),,,(
N
vy
N
uxvuvuyxhvuyxg
1,...,2,1,2
0,1
)(),(
NvuforN
vuforN
vu
Transform Coding
55/64
Transform kernel example
Walsh-Hadamard basis function for N=4
Discrete-cosine basis function for N=4
Transform Coding
56/64
Approximations using
(a) Fourier
(b) Hadamard
(c) Cosine
(a)
(b)
(c)
difference
8x8 block
Comp. ratio : 2
rms error:
(a):1.28
(b):0.86
(c):0.68
57/64
RMS error comparision• Reconstruction error versus subimage size comparison
Transform Coding
58/64
Transform coding quantization• quantization of transform coefficients
),(
),(),(ˆ
vuZ
vuFroundvuF
Typical normalization matrix : Z(u,v)
Transform Coding
59/64
128136132140150157161154
132140140161154157168161
140154157150154161161164
132154161143154157161161
132147157157161154171164
154143161164164147168171
164150171157150161154171
154164168154150161161168
24232011
43311322
214743895
1591521512
66101544108
105339846
6153513112534
6111020349214
DC component
AC component
Original image
DCT
Transform coding quantization• DCT example
Transform Coding
60/64
24232011
43311322
214743895
1591521512
66101544108
105339846
6153513112534
6111020349214
00000000
00000000
00000000
00000001
00000011
00000100
00001123
000010413
Quantization
9910310011298959272
10112012110387786449
921131048164553524
771031096856372218
6280875129221714
5669574024161314
5560582619141212
6151402416101116
W
Quantization Matrix
Quantization Example
61/64
Video Compression Technique
Typical compression technique with DPCM, DCT and VLC
62/64
Zig-Zag scan Alternate scan
00000000
00000000
00000000
00000001
00000011
00000100
00001123
000010413
13 4 3 0 -2 0 1 1 0 1 -1 -1 1 1 0 0 0 … 0
Zig-Zag scan
Scan• AC values are strung together in a sequence• this irregular ordering keeps low frequency coefficient
together
Video Compression Technique
63/64
[ 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 2 ]
(5, 1) (11, 3) (0, 1) (4, 2)
A
B
C
0
10
1
ABC
0.50.30.2
01011
Symbol Probability Code
Run-Length Coding• zig-zag 시퀀스를 (앞에 있는 ‘0’의 개수, ‘0’이 아닌 상수)의형태로 표현
Huffman Coding• 출현 빈도가 높은 값에 적은 비트 할당
• 무손실 부호화
Video Compression Technique
64/64
160162161159
163160155150
156153151144
153149144139
1027
22911
361723
5121154
Original image ordifference between original & predicted images
0000
0011
0012
01010
00000001001112010
DCTQuantization
/16
Zig-Zag scan
00
12
10
10
10
21
100
Run-Length CodeHuffman Code
10011011100011...
Video Compression Process
Recommended