View
18
Download
0
Category
Preview:
Citation preview
Image and Multidimensional Signal Processing Colorado School of Mines
Colorado School of Mines
Image and Multidimensional Signal Processing
Professor William Hoff
Dept of Electrical Engineering &Computer Science
http://inside.mines.edu/~whoff/
Image and Multidimensional Signal Processing Colorado School of Mines
Image Compression - Lossy
Image and Multidimensional Signal Processing Colorado School of Mines
Lossy Compression
• Reconstructed image is different from original
• Hopefully differences are unnoticeable, or minor
• We will look at: – Block transform coding methods, using the discrete cosine transform
(such as the JPEG standard)
– Predictive coding
3
Image and Multidimensional Signal Processing Colorado School of Mines
Block Transform Coding
• Divides the image into subimages, or blocks
• Apply a transform (e.g., Fourier) to each block
• Quantize and encode the coefficients
4
Compression
Image and Multidimensional Signal Processing Colorado School of Mines
Transform Coding
• General forward transform of image g, size nxn
• Inverse transform
• Example: Fourier transform
1 1
0 0
( , ) ( , ) ( , , , )n n
x y
T u v g x y r x y u v
1 1
0 0
( , ) ( , ) ( , , , )n n
u v
g x y T u v s x y u v
2 ( )/ 2 ( )/
2
1,j ux vy n j ux vy nr e s e
n
r,s are the forward and inverse transformation kernels (also called basis functions)
5
Image and Multidimensional Signal Processing Colorado School of Mines
Example: Walsh-Hadamard Transform (WHT)
• Kernels:
• where – nxn is the size of the kernel, and n = 2^m – bk is the kth bit – Summation is done in modulo 2 arithmetic – The p’s are
p0(u) = bm-1(u) p1(u) = bm-1(u) + bm-2(u) p2(u) = bm-2(u) + bm-3(u) : pm-1(u) = b1(u) + b0(u)
1
0
( ) ( ) ( ) ( )
( , , , ) ( , , , )
11
m
i i i i
i
b x p u b y p v
r x y u v s x y u v
n
6
Image and Multidimensional Signal Processing Colorado School of Mines
Example WHT Basis Functions
7
+1 -1
Image and Multidimensional Signal Processing Colorado School of Mines
Discrete Cosine Transform (DCT)
• Kernels:
• DCT used in JPEG (wavelets are used in JPEG2000)
1,...,2,1for2
0for1
)(
where
2
)12(cos
2
)12(cos)()(
),,,(),,,(
NuN
uNu
N
vy
N
uxvu
vuyxhvuyxg
8
% Show DCT kernels
N = 32;
x=0:N-1;
y=0:N-1;
u = 1;
v = 4;
au = sqrt(2/N);
av = sqrt(2/N);
if u==1
au = sqrt(1/N);
end
if v==1
av = sqrt(1/N);
end
gx = au*cos((2*x+1)*u*pi/(2*N));
gy = av*cos((2*y+1)*v*pi/(2*N));
figure, plot(x,gx);
figure, plot(y,gy);
g = gx'*gy;
figure, surf(g);
Image and Multidimensional Signal Processing Colorado School of Mines
DCT 4x4 Basis Functions
9
Image and Multidimensional Signal Processing Colorado School of Mines
Approximation Errors
Apply a transform to each 8x8 subimage block
Keep highest 50% of coefficients in each block
Then reconstruct image (by taking the inverse transform) using the remaining coefficients
Reconstructed Error Image
10
RMS error: 2.32 RMS error: 1.78 RMS error: 1.13
Image and Multidimensional Signal Processing Colorado School of Mines
Effect of Subimage Size
Image: lena
Truncate smallest 75% coefficients in each subimage
Figure 8.26
11
Image and Multidimensional Signal Processing Colorado School of Mines
12
Image and Multidimensional Signal Processing Colorado School of Mines
Quantizing Transform Coefficients
• Methods: – Threshold coding: Within each 8x8 subimage, keep the top N% of the
coefficients; or those with magnitude greater than a threshold
• Matlab exercise with “blkproc”
– Zonal coding: Keep coefficients with maximum variance across all subimages
• Matlab’s “dctdemo”
• Then quantize to a fixed or variable number of bits
13
Image and Multidimensional Signal Processing Colorado School of Mines
Threshold coding – Matlab example
• dct2
– The function dct2 performs 2D discrete cosine transform on a matrix
B = dct2(A)
– The function idct2 performs the reverse transformation A2 = idct2(B)
• blkproc
– Use “blkproc” to apply dct2 to each 8x8 block J = blkproc(I,[8 8],@dct2);
– You could also apply your own function (eg., threshold) to each block J = blkproc(I,[8 8],@mythresh);
14
Image and Multidimensional Signal Processing Colorado School of Mines 15
• DCT is performed on each 8x8 subimage (block) >> I = imread('cameraman.tif');
>> J = blkproc(I,[8 8],@dct2);
DCT coefficients
Image and Multidimensional Signal Processing Colorado School of Mines
A note on functions in Matlab
• You can write a function and call it from your program
• Syntax: function B = myfunc(A)
% This function computes something from A and
returns B
:
B = …
• Store this in a file called myfunc.m
• Put in current working directory
16
Image and Multidimensional Signal Processing Colorado School of Mines
Truncating coefficients
• Write a function called “mytrunc” that truncates the smallest 75% coefficients in an image
• The function should – Take the absolute value of each pixel in the image
– Sort the values in ascending order
– Find the value that is 75% down the list
– Threshold the image using that value
17
function B = mytrunc(A)
% Truncate the lowest 75% of the magnitudes within A
Aabs = abs(A);
vals = sort(Aabs(:)); % Sort the values from low to high
:
B = A .* (Aabs > thresh);
Image and Multidimensional Signal Processing Colorado School of Mines
18
clear all
close all
I = double(imread('lena.tif'));
wsize = 8;
J = blkproc(I,[wsize wsize],@dct2);
imshow(J,[]), title('J');
% Truncate 75% of the values within each block
Jtrunc = blkproc(J,[wsize wsize],@mytrunc);
figure, imshow(Jtrunc,[]), title('Jtrunc');
pct = sum(sum(Jtrunc == 0))/(size(I,1)*size(I,2));
fprintf('Percentage of zero coeffs: %f\n',100*pct);
K=blkproc(Jtrunc,[wsize wsize],@idct2);
figure, imshow(K,[]), title('K');
R = I - K;
disp('RMS error:');
sqrt(mean2(R .^ 2))
• Try on different block sizes
• Do you get this result?
Image and Multidimensional Signal Processing Colorado School of Mines
Quantizing Transform Coefficients
• Methods: – Threshold coding: Within each 8x8 subimage, keep the top N% of the
coefficients; or those with magnitude greater than a threshold
• Matlab exercise with “blkproc”
– Zonal coding: Keep coefficients with maximum variance across all subimages
• Matlab’s “dctdemo”
• Then quantize to a fixed or variable number of bits
19
Image and Multidimensional Signal Processing Colorado School of Mines
Zonal coding – Matlab “dctdemo”
• Apply DCT to each 8x8 block
• Discard coefficients with the smallest variance
20
DCT coefficients
Original Saturn Image
Reconstructed Image Error Image
Image and Multidimensional Signal Processing Colorado School of Mines
Ordering sequence
• Convert a nxn matrix to a one-dimensional vector
Typical threshold mask (we keep the coefficients in the shaded positions)
Ordering sequence, to convert 8x8 array to a 64x1 vector
Resulting vector will have long runs of 0s
Different mask for each subimage
21
Image and Multidimensional Signal Processing Colorado School of Mines
Quantizing Coefficient Magnitudes
• Once we decide which coefficients to keep, we now quantize the remaining non-zero coefficients
• We divide each coefficient by a number (depending on its location) and round to integer – Smaller magnitudes can be represented by fewer bits
– Division by a large number will tend to give a zero result
22
Z(u,v)
Image and Multidimensional Signal Processing Colorado School of Mines
Variable quantization
• You can achieve more or less compression by scaling the normalization matrix Z (i.e., dividing by larger values)
• Resulting compression:
(a) 12:1 (b) 19:1 (c) 30:1 (d) 49:1 (e) 85:1 (f) 182:1
23
Image and Multidimensional Signal Processing Colorado School of Mines
JPEG Algorithm
• Divide into 8x8 subimages
• Discrete cosine transform on each
• Quantize the coefficients
– Uses threshold coding
– Order coefficients in zig-zag pattern
• Encode the 1D sequence using run-length encoding and Huffman encoding
24
Ordering of coefficients
Threshold quantization array
Image and Multidimensional Signal Processing Colorado School of Mines
Input 8x8 subimage
Subtract 128 from each value
25
Image and Multidimensional Signal Processing Colorado School of Mines
Do forward DCT
Quantize and truncate values
26
Example: round(-415/16) = -26
Image and Multidimensional Signal Processing Colorado School of Mines
JPEG Algorithm (continued)
• Re-order in zig-zag pattern
• Use variable length code words to encode non-zero values
• Use run length encoding to encode # zeros
• Results – bit count = 92
– Compression: 512/92 = 5.6:1
27
Image and Multidimensional Signal Processing Colorado School of Mines
Details of coding the coefficients
• We use a pre-computed Huffman code (Appendices A.4-A.5)
• It assumes that values are clustered around zero
• The code word consists of a “base” code (which represents the most significant bits), followed by a coding of the least significant bits
• The base code is determined by the magnitude of the coefficient
First find what range the coefficient value lies in, and the corresponding category K
Then look up the base code for that category
28
Image and Multidimensional Signal Processing Colorado School of Mines
Predictive Coding
• Takes advantage of interpixel redundancy
• Predict next pixel from previous pixel, encode only the difference from the actual and the predicted
29
Image and Multidimensional Signal Processing Colorado School of Mines
30
A simple predictor: fpred(x,y) = f(x,y-1)
Image and Multidimensional Signal Processing Colorado School of Mines
Lossy Predictive Coding
• Error values are quantized
• Predictions by encoder and decoder must be same to prevent error buildup
31
Image and Multidimensional Signal Processing Colorado School of Mines
Optimal quantization of Error Values
• Lloyd-Max interval quantizer: staircase function with L values
32
Image and Multidimensional Signal Processing Colorado School of Mines
Choosing Intervals
• Assume a Laplacian pdf
• The optimal 2-bit quantizer is
e
e eep
2
2
1)(
Quantized output
Error (e) 0 1.102 -1.102
-0.395 0.395 1.81 -1.81
33
Image and Multidimensional Signal Processing Colorado School of Mines
Lloyd-Max Quantization
Quantized output
Error 0 1.102 -1.102
-0.395 0.395 1.81 -1.81
2 bit (4 level):
34
Image and Multidimensional Signal Processing Colorado School of Mines
Compression of Image Sequences
• To compress a video we take advantage of the redundancy between successive frames
• See NASA Shuttle Movie
– 1829 color frames (~1 minute)
– Compression using Quicktime (H.264)
• Reduction from 5 GB to 45 MB (100:1)
• Predict the value of each pixel, transmit the residual error • Simplest prediction method: Prediction is the value of the pixel in the
previous image (forward prediction) • Periodically insert I-frames (“independent” frames)
– These are compressed as single images (like DCT) – Needed for initialization – Or to handle cases where there are too many changes between successive
images
• Can also base the prediction on the next frame (backward prediction)
35
Image and Multidimensional Signal Processing Colorado School of Mines
Motion Compensation
• Predict motion of small blocks (e.g, 16x16)
Encoder has to estimate motion of each macroblock … usually finds dx,dy to minimize “mean absolute distortion”, which is the average of the absolute values of the differences
36
Image and Multidimensional Signal Processing Colorado School of Mines
Example
std dev = 12.7 std dev = 5.6
37
Image and Multidimensional Signal Processing Colorado School of Mines
Example – Subpixel Motion Estimation
std dev = 12.7 std dev = 4.4
std dev = 4 std dev = 3.8
Need to interpolate values
38
Image and Multidimensional Signal Processing Colorado School of Mines
Video Compression Standards
P: prediction (forward) B: backward prediction
39
Recommended