1

University of Balamand Department

of Electrical Engineering ELEN 443

Digital Communication

Pro ject I

Sub mitted t o :

Dr. Jihad Daba

Sub mitted b y:

Houssam Nouaman

Mohammad Awad

9 D ece m b e r 2013

2

I. I nt r o duc t i o n:

In today’s digital world, when we see digital movie, listen digital music, read

digital mail, store documents digitally, making conversation digitally, we have to deal

with huge amount of digital data. So, data compression plays a very significant role to

keep the digital world realistic. If there were no data compression techniques, we would

have not been able to listen to songs over the Internet, see digital pictures or movies, or

we would have not heard about video conferencing or telemedicine. How data

compression made it possible? What are the main advantages of data compression in

digital world? There may be many answers but the three obvious reasons are the saving

of memory space for storage, channel bandwidth and the processing time for

transmission. Every one of us might have experienced that before the advent MP3, hardly

4 or 5 songs of wav file could be accommodated. And it was not possible to send a wav

file through mail because of its tremendous file size. Also, it took 5 to 10 minutes or even

more to download a song from the Internet. Now, we can easily accommodate 50 to 60

songs of MP3 in a music CD of same capacity. Because, the uncompressed audio files

can be compressed 10 to 15 times using MP3 format and we have no problem in sending

any of our favorite music to our distant friends in any corner of the world. Also, we can

download a song in MP3 in a matter of seconds. This is a simple example of significance

of data compression.

Similar compression schemes were developed for other digital data like images

and videos. Videos are nothings but the animations of frames of images in a proper

sequence at a rate of 30 frames per second or higher. A huge amount of memory is

required for storing video files. The possibility of storing 4/5 movies in DVD CD now

rather than we used 2/3 CDs for a movie file is because compression. We will consider

here mainly the image compression techniques.

II. J P EG I m a g e:

JPEG is the most common image format used by digital cameras and other

photographic image capture devices for storing and transmitting photographic images on

the World Wide Web. JPEG compression is used in a number of image file formats these

format variations are often not distinguished and are simply called JPEG. The term

3

"JPEG" is an acronym for the Joint Photographic Experts Group which created the

standard.

Image data compression is concerned with minimizing the number of bits

required to represent an image with no significant loss of information. Image

compression algorithms aim to remove redundancy present in the data (correlation of

data) in a way which makes image reconstruction possible; this is called information

preserving compression Perhaps the simplest and most dramatic form of data

compression is the sampling of band limited images, where an infinite number of pixels

per unit area are reduced to one sample without any loss of information. Consequently,

the number of samples per unit area is infinitely reduced.

Transform based methods better preserve subjective image quality, and are less

sensitive to statistical image property changes both inside a single images and between

images. Prediction methods provide higher compression ratios in a much less expensive

way. If compressed images are transmitted an important property is insensitivity to

transmission channel noise. Transform based techniques are significantly less sensitivity

to channel noise. If transform coefficients are corrupted during transmission, the resulting

image is spread homogeneously through the image or image part and is not too

disturbing.

Applications of data compression are primarily in transmission and storage of

information. Image transmission applications are in broadcast television, remote sensing

via satellite, military communication via aircraft, radar and sonar, teleconferencing, and

computer communications.

III. A b s tr a c t :

For the past few years, a joint ISO/CCITT committee known as JPEG (Joint

Photographic Experts Group) has been working to establish the first international

compression standard for continuous-tone still images, both grayscale and color. JPEG’s

proposed standard aims to be generic support a wide variety of applications for

continuous-tone images. To meet the differing needs of many applications, the JPEG

standard includes two basic compression methods, each with various modes of operation.

A DCT-based method is specified for “lossy” compression, and a predictive method for

4

“lossless” compression. JPEG features a simple lossy technique known as the Baseline

method, a subset of the other DCT-based modes of operation. The Baseline method has

been by far the most widely implemented JPEG method to date, and is sufficient in its

own right for a large number of applications. This article provides an overview of the

JPEG standard, and focuses in detail on the Baseline method.

Advances over the past decade in many aspects of digital technology especially

devices for image acquisition, data storage, and bitmapped printing and display - have

brought about many applications of digital imaging. However, these applications tend to

be specialized due to their relatively high cost. With the possible exception of facsimile,

digital images are not commonplace in general-purpose computing systems the way text

and geometric graphics are. The majority of modern business and consumer usage of

photographs and other types of images take place through more traditional analog means.

In the beginning, the image is subdivided into a block of pixels with block size of

N x N. Firstly; the block must be classified as foreground block or background block

based on a pre-processing step. The foreground blocks will be compressed via JPEG

technique but with significant quantized coefficients and the DC coefficient only from

one block in the background is used to code it. The simulation result shows that the

proposed technique provides competitive compression performance relative to the most

recent image compression techniques.

Image compression maps an original image into a bit stream suitable for storage

or transmission over suitable channel in a digital medium, such as multimedia

communications, integrated services digital networks (ISDN), storage of medical images,

archiving of finger print sand transmission of remote sensing images. The number of bits

required to represent the coded image should be smaller than that required for the original

image, so that one can use less communication time or storage space. A fundamental goal

of data compression is to reduce the volume of data for transmission or storage while

maintaining an acceptable fidelity or image quality. Consequently, pixels must not

always be reproduced exactly as the originated also, the human visual system (HVS)

should not detect the difference between original image and reproduced image.

5

The following is a glance at the steps used in the DCT based image compression:

1. The image is broken into 8*8 blocks of pixels.

2. Working from left to right, top to bottom, DCT is applied to each block.

3. Each block is compressed through quantization.

4. RLC Based on Zig-Zag scan

5. Huffman coding

6. The array of compressed blocks that constitute the image is stored in a

reduced amount of space

7. When desired, the image is reconstructed through decompression; this process

uses IDCT (inverse DCT).

In this project, a set of 1024x1024 grayscale images is taken and read in Matlab.

One of these taken pictures is the following:

6

When read in Matlab, the image forms a 1024x1024 matrix. This matrix is

divided into 8x8 sub-blocks.

IV. D is c r ete Co si n e T r a n s f o r m a t i on ( DC T):

A Discrete Cosine Transform (DCT) expresses a sequence of finitely many data

points in terms of a sum of cosine functions oscillating at different frequencies. The

JPEG process is a widely used form of lossy image compression that centers on the

Discrete Cosine Transform. DCT and Fourier transforms convert images from time-

domain to frequency-domain to decorrelate pixels. The DCT transformation is reversible.

The DCT works by separating images into parts of differing frequencies. During a

step called quantization, where part of compression actually occurs, the less important

frequencies are discarded, hence the use of the term “lossy”. Then, only the most

important frequencies that remain are used retrieve the image in the decompression

process. As a result, reconstructed images contain some distortion; but as we shall soon

see, these levels of distortion can be adjusted during the compression stage. The JPEG

method is used for both color and black-and-white images.

The Discrete Cosine Transform (DCT) is one of many transforms that takes its

input and transforms it into a linear combination of weighted basis functions. These basis

functions are commonly the frequency. The 2-D Discrete Cosine Transform is just a one

dimensional DCT applied twice, once in the x direction, and again in the y direction. One

can imagine the computational complexity of doing so for a large image. Thus, many

algorithms, such as the Fast Fourier Transform (FFT), have been created to speed the

computation. The two-dimensional DCT is useful in processing two-dimensional signals

such as images.

The DCT is used in JPEG image compression, MJPEG, MPEG, DV, and Theora

video compression. There, the two-dimensional DCT of NxN blocks are computed and

the results are quantized and entropy coded. In this case, N is typically 8 and the DCT-II

formula is applied to each row and column of the block. The result is an 8 × 8 transform

coefficient array in which the (0,0) element (top-left) is the DC (zero-frequency)

component and entries with increasing vertical and horizontal index values represent

higher vertical and horizontal spatial frequencies.

7

The following equations are the idealized mathematical definitions of the 8x8

FDCT and 8x8 IDCT:

V. P os t - Qu a n t i z a t i o n:

Our 8x8 block of DCT coefficients is now ready for compression by quantization.

A remarkable and highly useful feature of the JPEG process is that in this step, varying

levels of image compression and quality are obtainable through selection of specific

quantization matrices. This enables the user to decide on quality levels ranging from 1 to

100, where 1 gives the poorest image quality and highest compression, while 100 gives

the best quality and lowest compression. As a result, the quality/compression ratio can be

tailored to suit different needs.

Subjective experiments involving the human visual system have resulted in the

JPEG standard quantization matrix. With a quality level of 50, this matrix renders both

high compression and excellent decompressed image quality.

8

If, however, another level of quality and compression is desired, scalar multiples

of the JPEG standard quantization matrix may be used. For a quality level greater than 50

(less compression, higher image quality), the standard quantization matrix is multiplied

by (100-quality level)/50.

The quantized DCT coefficients are computed with:

Where G is the unquantized DCT coefficients; Q is the quantization matrix above;

and B is the quantized DCT coefficients.

VI. R LC b as ed on Z ig - Z ag s c a n:

After quantization and before storage, all coefficients of quantized matrix are

converted by an encoder to a stream of binary data (01101011...). After quantization, it is

quite common for most of the coefficients to equal zero. JPEG takes advantage of this by

encoding quantized coefficients in the zigzag sequence shown in Figure as under. The

advantage lies in the consolidation of relatively large runs of zeros, which compress very

well. The sequence continues for the entire 8x8 block.

9

VII. Huff m an c o d i ng ( e n tr o p y c o d i n g ):

This is the last component in the compression model. Till now, we have devised

models for an alternate representation of the image, in which its interpixel redundancies

were reduced. This last model, which is a lossless technique, then aims at eliminating the

coding redundancies, whose notion will be clear by considering an example. Suppose, we

have a domain in an image, where pixel values are uniform or the variation in them is

uniform. Now one requires 8 bpp (bits per pixel) for representing each pixel since the

values range from 0 to 255. Thus representing each pixel with the same (or constant

difference) value will introduce coding redundancy. This can be eliminated, if we

transform the real values into some symbolic form, usually a binary system, where each

symbol corresponds to a particular value.

In information theory, Huffman coding is an entropy encoding algorithm used for

lossless data compression. The term refers to the use of a variable length code table for

encoding a source symbol (such as a character in a file) where the variable-length code

table has been derived in a particular way based on the estimated probability of

occurrence for each possible value of the source symbol.

10

VIII. R e s u l t s:

In Matlab, to apply the Discrete Cosine Transform, we use the function DCT2.

The numerical result of the first 8x8 matrix after applying the DCT and the post-

quantization is:

-1 0 0 -1 -1 -1 0 0

0 -1 -1 -1 0 -1 -1 -1

-1 -1 -1 0 0 -1 -1 -1

-1 -1 -1 -1 0 0 -1 -1

0 0 0 0 0 0 0 0

0 -1 0 -1 -1 0 0 0

0 0 0 -1 -1 -1 0 0

0 0 0 -1 -1 -1 0 0

1. The Huffman coded bina r y str eam for th e first 8x 8 sub-block is the following:

110000011110000110111000001111000011001100000111100001111110000000101

000101110000000101000000111000001111000011101100000111100010011000001

111000010011100000001010000001110000000101000000010111000000010100010

101110000011110001001100000111100001000001100000111100010000111000011

001100000111100010011110001000111011100100111011110101100110001100111

0

11

2. DC sub-field:

DC Sub-field probabilities:

(Dm)

DC Value Number of occurrences Subfield Probability Overall Probability

-4 0 0.0000 0.0000

-3 0 0.0000 0.0000

-2 10 0.0000 0.0000

-1 0 0.0000 0.0000

1 10 0.0000 0.0000

2 248920 0.4975 0.2488

3 120780 0.2414 0.1207

4 8900 0.0178 0.0089

5 60690 0.1213 0.0607

6 29950 0.0599 0.0299

7 18190 0.0364 0.0182

8 5050 0.0101 0.0050

9 2110 0.0042 0.0021

10 2670 0.0053 0.0027

11 1020 0.0020 0.0010

12 410 0.0008 0.0004

13 650 0.0013 0.0006

14 0 0.0000 0.0000

15 0 0.0000 0.0000

16 300 0.0006 0.0003

17 220 0.0004 0.0002

18 150 0.0003 0.0001

12

19 90 0.0002 0.0001

20 0 0.0000 0.0000

21 0 0.0000 0.0000

22 0 0.0000 0.0000

23 40 0.0001 0.0000

24 30 0.0001 0.0000

25 60 0.0001 0.0001

26 10 0.0000 0.0000

27 0 0.0000 0.0000

28 20 0.0000 0.0000

29 0 0.0000 0.0000

30 10 0.0000 0.0000

31 0 0.0000 0.0000

Histogram:

From the Matlab code the following results are obtained:

• Highest probability = 0.4975

• Dmod = 2

• Dinf = -4 (minimum DC component)

• Dsup = 31 (maximum DC component)

13

From the above plot, one can notice that the experimental probabilities have a

form similar to a Gaussian distribution. The DC component that has the major presence

of the DC sub-field is the 2. Therefore, the highest probability in the graph is the one

of D2 .

14

3. R 0 sub-field:

R0 subfield:

(Rm0)

Zero ValueNumber of

successive bits

Number of

occurrences

Subfield

Probability

Overall

Probability

0 1 248720 0.4971 0.2486

0 2 121150 0.2421 0.1211

0 3 60810 0.1215 0.0608

0 4 29980 0.0599 0.0300

0 5 17900 0.0358 0.0179

0 6 8890 0.0178 0.0089

0 7 4950 0.0099 0.0049

0 8 2860 0.0057 0.0029

0 9 1980 0.0040 0.0020

0 10 1040 0.0021 0.0010

0 11 710 0.0014 0.0007

0 12 390 0.0008 0.0004

0 13 310 0.0006 0.0003

0 14 210 0.0004 0.0002

0 15 130 0.0003 0.0001

0 16 100 0.0002 0.0001

0 17 70 0.0001 0.0001

0 18 30 0.0001 0.0000

0 19 20 0.0000 0.0000

0 20 10 0.0000 0.0000

0 21 10 0.0000 0.0000

0 22 10 0.0000 0.0000

15

0 23 10 0.0000 0.0000

0 24 0 0.0000 0.0000

0 25 0 0.0000 0.0000

0 26 0 0.0000 0.0000

Histogram:

The Markov modulated truncated geometry (MMTG) distribution is expressed as

follows:

16

From the Matlab code, the following results are obtained:

• max0 = 26

• Transitions 00 = 548088

• Transitions 01= 500325

• Transitions 10 = 500300

• Transitions 11 = 548439

• P00 = 0.5228

• P01 = 0.4772

• P10 = 0.4770

• P11 = 0.5230

17

The following histograms show respectively the experimental R0 sub-field, the

theoretical MMTG, and the theoretical and experimental combined. The exponential

form is clearly noticeable.

18

19

4. M ean S q u are err o r:

20

From the Matlab code, the following results are obtained:

• P0 = 0.4999

• P1= 0.5001 , p̂ 0 =0.5000 and p̂D =0.5000

MSE= 1.0459e-008 <<<<<<1 then it follows Markov chain

5. Efficienc y

The entropy’s formula is given by:

The average length formula is given by :

The efficiency is given by:

21

The entropy calculated in Matlab is:

• H= 3.0875 [bits/sec]

The average code length calculated in Matlab is:

• L

= 3.0937 [binits/sec]

The efficiency is

• 99.8024 %

22

IX. Ma tla b Co de:

clear all close all%%%%%%%%%%%%% Quantization matrix %%%%%%%%%%%%% Q =[16 11 10 16 24 40 51 61

12 12 14 19 26 58 60 5514 13 16 24 40 57 69 5614 17 22 29 51 87 80 6218 22 37 56 68 109 103 7724 35 55 64 81 104 113 9249 64 78 87 103 121 120 10172 92 95 98 112 100 103 99];

%%%%%%%%%%%%% images call %%%%%%%%%%%%%disp('enter image number (1 to 4)')number=input('');a=1;for image=1:number if image==1

x = imread('imageA.jpg');elseif image==2

x = imread('imageB.jpg');elseif image==3x = imread('imageC.jpg');elseif image==4x = imread('imageD.jpg');

endx = rgb2gray(x);x= im2double(x);%%%%%%%%%%%%% DCT %%%%%%%%%%%%%X = dct2(x);[row, col] = size(X);%%%%%%%%%%%%% Post Quantization & Zig Zag scan %%%%%%%%%%%%%for u=1:8:row

for v=1:8:rowm8x8= floor(X(u:u+7,v:v+7)./Q); X(u:u+7,v:v+7)=m8x8; y=zeros(64,1);count=1;for s=1:8

if mod(s,2)==0 for m=s:-1:1

y(count)=m8x8(m,s+1-m);count=count+1;

elseend;

for m=1:s y(count)=m8x8(m,s+1-m); count=count+1;

endend

end

flip=1;

23

for s=8+1:2*8-1if mod(flip,2)==0

for m=8:-1:s+1-8 y(count)=m8x8(m,s+1-m); count=count+1;

elseend

for m=8:-1:s+1-8 y(count)=m8x8(s+1-m,m); count=count+1;

end;end;

endflip=flip+1;

sum=1;for loop=2:64

if y(loop)==y(loop-1)sum=sum+1;

end

else

end

RLC(a,1)=y(loop-1); RLC(a,2)=sum;a=a+1;sum=1;

RLC(a,1)=y(loop); RLC(a,2) = sum; a=a+1;if u == 1 && v == 1

RLC8x8 = RLC;

end end

end end

%%%%%%%%%%%%% Run Length Code %%%%%%%%%%%%%B = RLC;row = size(B,1);matrix= [];for n = 1:row i =

1; matrix(n) = 1; D = [];for m = (n+1):row

if B(n,:) == B(m,:)matrix(n) = matrix(n) + 1;D(i) = m;i = i +1;

endend

column = size(D,2); row = row - column; for m = 1:column

B(D(m),:)=[];D = D -1;

endend

24

%%%%%%%%%%%%% occurences %%%%%%%%%%%%%row = size(B,1);B = [ B zeros(row,1)];for n = 1:row

B(n,3) = matrix(n);end%%%%%%%%%%%%% 0 field %%%%%%%%%%%%%row = size(B,1);sumpz = 0;for n = 1:row

if B(n,1) == 0sumpz = sumpz + B(n,3);

endend

%%%%%%%%%%%%% DC field %%%%%%%%%%%%%sumpd = 0;for n=1:row

if B(n,1) ~= 0sumpd = sumpd + B(n,3);

endend

sumtotal = sumpz + sumpd; B = [ [B] zeros(row,3)];

%%%%%%%%%%%%% R0 probability %%%%%%%%%%%%%for n=1:row

if B(n,1) == 0B(n,4) = B(n,3)/sumpz; B(n,5) = sumpz/sumtotal;

endend

%%%%%%%%%%%%% DC probability %%%%%%%%%%%%%for n=1:row

if B(n,1) ~= 0B(n,4) = B(n,3)/sumpd; B(n,5) = sumpd/sumtotal;

endend

%%%%%%%%%%%%% overall probability %%%%%%%%%%%%%for n=1:row

B(n,6) = B(n,4)*B(n,5);end

%%%%%%%%%%%%% Huffman Coding %%%%%%%%%%%%% [dict, avglen] = huffmandict([1:row],B(:,6));%%%%%%%%%%%%% Code for the first 8x8 matrix %%%%%%%%%%%%%bitcode=[];row = size(RLC8x8,1);rows = size(B,1);for n = 1:row

for m = 1:rowsif RLC8x8(n,1:2) == B(m,1:2)

bitcode = [bitcode dict{m,2}];break

endend

end

%%%%%%%%%%%%% R0 & DC subfield histograms %%%%%%%%%%%%%

25

Brecovered = B; A = [];row = size(B,1); RLC0 = [];i = 1;

for n=1:rowif B(n,1) == 0

A(i)= n;i = i +1;

endend

column = size(A,2);i = 1;for n=1:column

RLC0(i,:)=B(A(n),:); i = i+1; B(A(n),:)=[];A = A - 1;

endRLCDC = B;B = Brecovered;

row = size(RLCDC,1); RLCDC=sortrows(RLCDC,1); x = 1:row;figurebar(x,RLCDC(:,4),'g')title('DC sub-field')

Pdmod = max(RLCDC(:,4));for n = 1:row

if Pdmod == RLCDC(n,4)dmod = RLCDC(n,1);break

endend

Dinf = min(B(:,1)); Dsup = max(B(:,1)); syms asum = 0;

for k = Dinf:Dsup if k ~= 0

sum = sum + exp(-((a*k)^2));

endend

f = ((exp(-((a*dmod)^2)))/sum) - Pdmod;sol = solve(f,a);

min0 = min(RLC0(:,2));max0 = max(RLC0(:,2));[row, column] = size(RLC0);m = 1;RLC0 = sortrows(RLC0,2);

26

for n = min0:max0if RLC0(m,2) ~= n

RLC0 = [ RLC0(1:(m-1),:); zeros(1,column); RLC0(m:row,:) ]; RLC0(m,2) = n;RLC0(m,5) = sumpz/sumtotal;row = size(RLC0,1);

end

endm = m+1;

row = size(RLC0,1);x = 1:row;figure bar(x,RLC0(:,4),0.8,'g') title('R0 sub-field (Green)')

% % number of transitions from 0 to 0 & from 0 to 1 row = size(RLC0,1);sum00= 0;sum01= 0;for n = 1:row

sum00 = (RLC0(n,2)-1)*RLC0(n,3) + sum00;sum01 = RLC0(n,3) + sum01;

endsum0trans= sum00+sum01;P00 = sum00/sum0trans; P01 = sum01/sum0trans;

% % number of transitions from 1 to 0 & from 1 to 1 row = size(RLCDC,1);sum11 = 0;sum10 = 0;for n = 1:row

sum11 = (RLCDC(n,2)-1)*RLCDC(n,3) + sum11;sum10 = RLCDC(n,3) + sum10;

endsum1trans= sum11+sum10;P11 = sum11/sum1trans;P10 = sum10/sum1trans;

P0 = P10/(1+P10-P00); P1 = P01/(1+P01-P11);

% % MMTGrow = size(RLC0,1);for m = min0:max0

PR0m(m) = ((P00)^(m-1))*(1-P00)/(1-(P00)^max0);endx = 1:row; figure bar(x,PR0m,0.8,'r')title('theoretical MMTG (Red)')

row = size(RLC0,1);x = 1:row;figure

27

bar(x,RLC0(:,4),0.8,'g')hold onx = 1:row;bar(x,PR0m,0.3,'r')title('R0 sub-field(green),theoretical MMTG (Red)')

%%%%%%%%%%%% Mean Square Error %%%%%%%%%% P0hat = sumpz/sumtotal;P1hat = sumpd/sumtotal;MSE = 0.5*((P0-P0hat)^2 + (P1-P1hat)^2);

%%%%%%%%%%% Efficiency %%%%%%%%%%row = size(B,1); H = 0;for n = 1:row

H = H + (-B(n,6))*log2(B(n,6));endefficiency= (H/avglen)*100;