LEC1: Introduction Digital Communication Systemeee.guc.edu.eg/Courses/Communications/COMM901... · At the end one gets a digital values s[n] ... Digital Communication System ... Error

SOURCE CODING PROF. A.M.ALLAM

9/9/2017 1

LEC1: Introduction

9/9/2017

Digital Communication System

Source of

Information

User of

Information

Source

Encoder

Channel

Encoder

Modulator

Source

Decoder

Channel

Decoder

De-Modulator

Channel

Flash Back on Digital Communication System

Communication systems are designed

to transmit the information generated

by a source to some destination

Analog/Digital

Convert to digital form (ASCII)

It enables the following:

-Amount of information from a

given source

-Minimum storage and bandwidth

needed to transfer data from a

given source

-Limit on the transmission rate of

information for reliable comm.

over a noisy channel

-Data compression

Channels can only transport physical signals, e.g., electrical signals. Therefore, digital signals

must be converted to appropriate formats

The channel introduce errors. Channel coding is used for controlling errors in data

transmission over unreliable (noisy channel) communication channels 1


9/9/2017 2

LEC1: Introduction

9/9/2017


2

Signal

Source Filter Sampler Quantizer

Mapper

Filter: pass the required signal and remove the undesired signal

Sampler: sample and hold; generates discrete signal continuous value; i.e.,

infinite number of levels

Quantizer: generates discrete signal discrete values; i.e., finite (limited) number of levels

(could be uniform or nonuniform) Mapper: sample and hold; generates discrete signal continuous

value; i.e., infinite number of levels

t

x(t) Analogue

t

x(t) Digital; binary sequence

At the end one gets a digital values s[n] which are in practice numbers that

are stored in a computer to be further processed, that is why we do ADC


9/9/2017 3

LEC1: Introduction

9/9/2017


3

Signal


Mapper Source

Encoder

Source Coding and Compression:

-It is created by identifying and using structures that exist in the data, -Is the art or science of representing information in a compact form

Characters in a text file

Numbers that are samples of speech or image waveforms

Sequences of numbers that are generated by other processes

Exs:


9/9/2017 4

LEC1: Introduction

9/9/2017


4

Signal


Mapper Source

Encoder

-Why we are in need to data compression?

These number of bits or bytes required to represent a multimedia data can be huge, Exs:

To digitally represent 1 second of video without compression (using the CCIR 601 format),

we need more than 20 megabytes, or 160 megabits

To represent 2 minutes of uncompressed CD quality music (44.100 samples per second, 16

bits per sample) requires more than 84 million bits. Downloading music from a website at

these rates would take a long time

-All compression methods are computer programs:

Encoder: Mapping of s[n] into a bit stream b

Decoder: Mapping of the bit stream b into the discrete decoded signal s’[n]


9/9/2017

5

LEC1: Introduction

9/9/2017


5

Signal


Mapper Source

Encoder

Channel

Encoder

Parity check (Odd or Even)

-Adding some redundancy in the binary information sequence to overcome the problems of

interference and noise

Tx: 0 → 000 or 1 → 111

Assume Tx: 0 → 000

Rx: 0 0 0 → correct 0

0 0 1 → maybe 0

1 0 0 → maybe 0

1 0 1 → maybe 1

1 1 1 → correct 1

Code Ratio R= 1/3

Channel Coding n > k

k

X

Parity check: (odd or even)

i/p 2bits → o/p 3bits

0 0 → 000

0 1 → 011

1 0 → 101

1 1 → 110

(number of 1’s should be even)

Code Ratio R = 2/3

Code Repetition


9/9/2017 6

LEC1: Introduction

9/9/2017


6

Signal


Mapper Source

Encoder

Channel

Encoder

Digital

Modulator

Digital Modulator:

-It maps the binary information sequences into signal waveforms

0 bit

1 bit

b bits , b=2M

So(t)

S1(t)

Si(t), i=1,2,3,…,M


9/9/2017 7

LEC1: Introduction

9/9/2017


7

The discrete signal is temporally sampled and its amplitude is represented

using k = 3 bits/sample and 8 different levels

temporal signalsone dimensional are typically Speech and audio signals-

spatial temporal signalsthree dimensional are Videos-


9/9/2017 8

LEC1: Introduction

9/9/2017


8

-Pictures are two dimensional spatial signals

-8 x 8, 16 x 16, 32 x 32, and 128 x 128 samples (from left to right)

-Each sample is represented with 8 bits

-Each square represents average of luminance values it covers

Ex: Sampling of picture with different spatial sampling rates

-1, 2, 4, 8 bits/sample

-The spatial sampling rate is fixed 128x128

Ex: Quantization of picture with different bits/sample


9/9/2017

LEC1: Introduction

9/9/2017

Source coding and Data Compression

Source coding or compression is required for efficient transmission or storage, leading to one or

both of the following benefits:

• Transmit more data given throughput (channel capacity or storage space)

• Use less throughput given data

Original

Signal Encoder

Compression

Algorithm

Compression Algorithms

sc Compressed

Signal

Original

Signal Decoder

Reconstruction

Algorithm sc Compressed

Signal Fewer bits

Original data can be recovered from the compressed data exactly or not depends on the compression technique

Lossless Compression Technique Lossy Compression Technique

Involve no loss of information

Applied for cases that cannot tolerate any difference

between the original and reconstructed data

(REVERSABLE CODE) s = s’

Involve some loss of information

Compressed data using lossy techniques generally

cannot be recovered or reconstructed exactly

( NONREVERSABLE CODE) s ≠ s’

Higher compression ratios Lower compression ratios

“Do not send money” not to be “Do now send money” The exact value of each sample of speech is not necessary

Uses redundancy reduction Uses redundancy reduction and irrelevancy reduction

For data compression Lempel-Ziv coding(gzip)

For picture and video signals JPEG-LS is well known

For audio coding MPEG-1 Layer 3 (mp3)

For picture coding JPEG

For video coding H.264/AVC 9


9/9/2017

LEC1: Introduction

9/9/2017


-Mobile voice, audio, and video transmission

-Internet voice, audio, and video transmission

-Digital television

-MP3 and portable video players (iPod, ...)

-Digital Versatile Discs (DVDs) and Blu-Ray Discs

Source coding applications

File compression

(text file, office document, program code, ...)

Ex: 80 Mbyte down to 20 Mbyte (25%)

Audio compression

-Stereo with sampling frequency of 44.1 kHz

-Each sample being represented with 16 bits

-Raw data rate: 44.1x16x2 = 1.41 Mbit/s

Typical data rate after compression: 64 kbit/s

(4.5%)

Image compression

-Original picture size: 3000x2000 samples (6

MegaPixel)

-3 color components (red, green, blue) and 1 byte

(8 bit) per sample

- Raw le size: 3000x2000x3 = 18 Mbyte

Typical compressed file size: 1 Mbyte (5.6%)

Video compression

-Picture size of 1920x1080 pixels and frame rate of 50 Hz

-Each sample being digitized with 8 bit

-3 color components (red, green, blue)

Raw data rate: 1920x1080x8x50x3 = 2.49 Gbit/s

Typical compressed data rate: 12 Mbit/s (0.5%)

Digital images are typically compressed (JPEG)

Compression is often done in camera

Picture found on web sites are compressed digital video

data are typically compressed (MPEG-2, H.264/AVC)

-Output of video cameras, optical discs

-Video streaming (YouTube, Internet TV)

10


9/9/2017

LEC1: Introduction

9/9/2017


High compression

Original

Medium compression

Low compression

11


9/9/2017

LEC1: Introduction

9/9/2017


Compression)10 :1JPEG ( Ex: Compression)50 :1JPEG ( Ex: Compression)50 :1/ HEVC (265H. Ex:

12


9/9/2017 9/9/2017

Geometrical Implementation of Compression

LEC1: Introduction Source coding and Data Compression

13


9/9/2017

LEC1: Introduction

9/9/2017


Source coding (compression algorithm) can be evaluated

(characterized) in a number of different ways

1-The relative complexity of the algorithm

2-The memory required to implement the algorithm

3-How fast the algorithm performs on a given machine 4-The amount of compression

7-Fidelity and quality: when we say that the fidelity or quality of a reconstruction is high, we

mean that the difference between the reconstruction and the original is small

5-Throughput of the channel: -Transmission channel bit rate -Amount of protocol - Error correction coding

6-Distortion of the decoded signal: difference between the original and the reconstruction lossy compression

-Source encoder - Channel errors introduced in path to source decoder

Given a maximum allowed delay and a maximum allowed complexity

Achieve an optimal trade off between bit rate and distortion for the

transmission problem in the targeted applications

Practical source coding design problem is posed as follows:

14


9/9/2017

LEC1: Introduction

9/9/2017


Distortion Measures

Lossy compression requires the ability to measure distortion

Perceptual Models Objective Models

The characteristics of human perception

are complex because it is very difficult

quantity to measure

Measures Mean Square Error; MSE and

Signal to Noise Ratio; SNR

Are heavily used in speech and audio

coding, to guide encoding decisions

Listening tests are used to determine

subjective quality of coding results

Limited used in picture and video

coding, to guide encoding decisions

Viewing tests are used to

determine subjective quality of

coding results

MSE SNR

][]['][ nsnsnu

1

0

2 ][1 N

n

nuN

MSE

N is the number of samples

Speech and audio:

],[],['],[ yxsyxsyxu

1

0

1

0

2 ],[1 X

x

Y

y

yxuXY

MSE

X is the picture height,

Y is the picture width

Picture:

N is the number of pictures

Video:

1

0

1 N

N

nMSEN

MSE

Speech and audio:

Picture:

k is the bits per sample

Video:

15


9/9/2017 9/9/2017

Probability

Deterministic experiment: if it has only one outcome

Random experiment: if it has more than one possible outcomes

Experiment : is any procedure that can be infinitely repeated and has a well defined set of

possible outcomes, known as the sample space sample space

Mathematical description of an experiment consists of three parts:

1) Sample space

2) Set of events

3)Assignment of probabilities to the events i.e., a function P mapping from events to

probabilities

Rolling a die gives all possible outcomes ; Sample space Sn={1,2,3,4,5,6}space

Set of events, is a set containing zero or more outcomes (a subset of the sample space)

events, A={2.4}, B={1,3,5,6} are subsets of the sample space

Mutually excusive events, they are not intersected A∩B=Φ

P(A∩B)=Φ, P(AUB)=P(A)+P(B)

LEC1: Introduction

16

https://en.wikipedia.org/wiki/Sample_space




9/9/2017

LEC1: Introduction

9/9/2017

Probability

Joint Probability:

Consider two separate tosses of a single die A,B or one single toss of two separate dice

A,B , then P(A,B) is the probability of all joint outcomes

Conditional Probability:

Consider two separate tosses of a single die A,B or one single toss of two separate dice

A,B , then P(A,B) is the probability of all joint outcomes

)(

),()/(

BP

BAPBAP OR

)(

),()/(

AP

BAPABP

If the two events A,B are statistically independent , then

P(A∩B)=P(A)P(B) )()(

)()()/( AP

BP

BPAPBAP )(

)(

)()()/( BP

AP

BPAPABP

Marginal probability: is the probabilities for any one of the

variables with no reference to any specific ranges of values

for the other variables

Is the probabilities for any subset of the variables conditional on particular values of the

remaining variables

17


9/9/2017

LEC1: Introduction

9/9/2017

Random variable: is a variable whose its value is subject to variations due to chance or

randomness, hence, can take a set of possible different values (similarly to other

mathematical variables), each with an associated probability ( in contrast to other

mathematical variables)

Discrete random variables, taking any of a specified finite or countable list of values,

endowed with a probability mass function, i.e., characteristic of a probability distribution

A sample space is a collection of all possible outcomes of a random experiment

A random variable is a function defined on a sample space

Random process: is a collection of random variables, representing the evolution of some

system of random values over time

Rolling a die gives a sample space S={1,2,3,4,5,6}space

A random variable X=S2ace

1 1

2 4

3 9

4 16

5 25

6 36

Continuous random variables, taking any numerical value in an interval or collection of intervals,

via a probability density function, i.e., characteristic of a probability distribution

18

Probability




9/9/2017 9/9/2017

LEC1: Introduction

Probability Distribution Function / Cumulative/ Math:

1 2 3 4 5 6

1/6

6/6

2/6

x

F(x) F(x)=P(X ≤ x)

19

Probability


9/9/2017 9/9/2017

LEC1: Introduction

Probability Density Function :

x

F(x)

F(x)=P(X ≤ x)

1

Cumulative Distribution Function

F(∞)=1 F(-∞)=0

x

F(x)

f(x) =dF(x)/dx

Probability Density Function

)()(

)()(

12

21

2

1

xFxF

dxxfxXxP

x

x

20

Probability

Documents

LEC1: Introduction Digital Communication Systemeee.guc.edu.eg/Courses/Communications/COMM901... · At the end one gets a digital values s[n] ... Digital Communication System ... Error