INTRODUCTION TO INFORMATION THEORY

AND APPLICATIONS

2014

LOVELY PROFESSIONAL UNIVERSITY

SUBMITTED BY SUBMITTED TO NAME :- BIKASH THAPA ROLL NO :- A55 SECTION :- 1407 GROUP :- 1 COURSE :- BSC.IT REG NO :- 11413230

“TRANSFORMING EDUCATION TRASNSFORMING INDIA”

Cap 125 Introduction to information theory and applications

2

Acknowledgement

It was really an awesome experience to be a participant in writing term

paper. I really enjoyed using my own earned skilled and experience in

the class as well as book study to accomplish the term paper topic. As

can as I can do I have done my all the best and I am hoping it will bring

satisfactory result.

In the course of being involved in writing term

paper, I got lots of help and support from my teachers, colleagues and all

my friends. So I can’t remain without thanking them for their seen and

unseen help so I would like to thanks all of them. I also owe to those

authors whose books proved to be a milestone to me to accomplish the

term paper.

At last I like to thanks my parents for their great support to

complete my term paper and let me to study BSc.IT (information

technology) in Lovely Professional University.

Bikash Thapa

Cap 125 Introduction to information theory and applications

3

Table of Contents 1.1 Analog signal ........................................................................................... 5

1.2 Digital signal ............................................................................................ 9

1.3 Difference between digital and analog signal ..................................... 12

Abstract

Cap 125 Introduction to information theory and applications

4

ANALOG AND DIGITAL SIGNAL

Cap 125 Introduction to information theory and applications

5

Signal:

A signal is defined as a function of one or more variables

which conveys information on the nature of a physical phenomenon. The

value of the function can be a real valued scalar quantity, a complex

valued quantity, or perhaps a vector.

System:

A system is defined as an entity that manipulates one or

more signals to accomplish a function, thereby yielding new signals.

Continuous-time signal:

A signal x (t) is said to be a continuous time

signal if it is defined for all time t.

Discrete-time signal:

A discrete time signal x [nT] has values

specified only at discrete points in time.

Signal processing:

A system characterized by the type of

operation that it performs on the signal. For example, if the operation is

linear, the system is called linear. If the operation is non-linear, the

system is said to be non-linear, and so forth. Such operations are usually

referred to as “Signal Processing”

1.1 Analog signal

An analog or analogue signal is any continuous signal for

Cap 125 Introduction to information theory and applications

6

which the time varying feature (variable) of the signal is a representation

of some other time varying quantity, i.e., analog to another time varying

signal. For example, in an analog audio signal, the

instantaneous voltage of the signal varies continuously with

the pressure of the sound waves. It differs from a digital signal, in which

a continuous quantity is represented by a discrete function which can

only take on one of a finite number of values. The term analog signal

usually refers to electrical signals; however, mechanical, hydraulic, and

other systems may also convey analog signals.

An analog signal uses some property of the medium to convey the

signal's information. For example, an aneroid barometer uses rotary

position as the signal to convey pressure information. In an electrical

signal, the voltage, current, or frequency of the signal may be varied to

represent the information.

Any information may be conveyed by an analog signal; often such a

signal is a measured response to changes in physical phenomena, such

as sound, light, temperature, position, or pressure. The physical variable

is converted to an analog signal by a transducer. For example, in sound

recording, fluctuations in air pressure (that is to say, sound) strike the

diaphragm of a microphone which induces corresponding fluctuations in

the current produced by a coil in an electromagnetic microphone, or the

voltage produced by a condenser microphone. The voltage or the current

is said to be an "analog" of the sound.

An analog signal has a theoretically infinite resolution. In practice an

analog signal is subject to electronic noise and distortion introduced

by communication channels and signal processing operations, which can

progressively degrade the signal-to-noise ratio. In contrast, digital

signals have a finite resolution. Converting an analog signal to digital

form introduces a constant low-level noise called quantization noise into

the signal which determines the noise floor, but once in digital form the

signal can in general be processed or transmitted without introducing

additional noise or distortion. Therefore as analog signal processing

systems become more complex, they may ultimately degrade signal

Cap 125 Introduction to information theory and applications

7

resolution to such an extent that their performance is surpassed by digital

systems. This explains the widespread use of digital signals in

preference to analog in modern technology. In analog systems, it is

difficult to detect when such degradation occurs. However, in digital

systems, degradation can not only be detected but can be corrected as

well.

Example of Analog Signals:

An analog signal can be any time-varying signal.

Minimum and maximum values can be either positive or negative.

They can be periodic (repeating) or non-periodic.

Sine waves and square waves are two common analog signals.

Note that this square wave is not a digital signal because its

minimum value is negative.

Video and Audio

Fig: analog signal

Analog transmission:

Cap 125 Introduction to information theory and applications

8

Transmit analog signals without regard to content.

Attenuation limits length of transmission link

Cascaded amplifiers boost signals energy for longer distances but

cause distortion

Analog data can tolerate distortion.

Introduces errors in digital data.

Pros and Cons of Analog Signals:

Advantages:

Major advantages of the analog signal is infinite amount of data.

Density is much higher.

Easy processing.

Disadvantages:

Unwanted noise in recording.

If we transmit data at long distance then unwanted disturbance is

there.

Generation loss is also a big cons of analog signals.

Applications of analog signal are:

Cap 125 Introduction to information theory and applications

9

Thermometer

Photocopiers

Old land-line telephones

Audio tapes

VCRs (same as TV)

1.2 Digital signal

A digital signal is a physical signal that is a representation of a

sequence of discrete values. The term digital signal can refer to either of

the following:

1. any continuous-time waveform signal used in digital

communication, representing a bit stream or other sequence

of discrete values

2. a pulse train signal that switches between a discrete number of

voltage levels or levels of light intensity, also known as a line

coded signal or baseband transmission, for example a signal found

in digital electronics or in serial communications, or a pulse code

modulation (PCM) representation of a digitized analog signal.

A signal that is generated by means of a digital modulation method

(digital pass band transmission), to be transferred between modems, is in

the first case considered as a digital signal, and in the second case as

converted to an analog signal. In computer architecture and

other digital systems, a waveform that switches between two voltage

levels representing the two states of a Boolean value (0 and 1) is referred

to as a digital signal, even though it is an analog voltage waveform,

since it is interpreted in terms of only two levels.

Cap 125 Introduction to information theory and applications

10

Example of Digital signal:

Digital signal are commonly referred to as square waves or clock

signals.

Their minimum value must be 0 volts, and their maximum value

must be 5 volts.

They can be periodic (repeating) or non-periodic.

The time the signal is high (tH) can vary anywhere from 1% of the

period to 99% of the period.

Text and Integers.

Digital Signal Transmission:

Concerned with the content of the signal.

Attenuation endangers integrity of data.

Digital signal

Repeaters achieve greater distance.

Repeaters recover the signal and retransmit.

Pros and Cons of Digital Signal

Cap 125 Introduction to information theory and applications

11

Advantages:

Because of their digital nature they can travel faster in over digital

lines.

Ability to transfer more data as compared to analog.

Disadvantages:

Greater bandwidth is essential.

Systems and processing is more complex.

Applications of Digital Signal are:

PCs, PDAs

Mobile Phones.

Difference in working:

The working of the digital signals are more reliable and accurate

because they use the digital media or the numeric method where as

analog signals are based on the small fluctuations.

Digital signals are used to play that information which is came

from re-sampling, whereas analog are known for playing recorded

data.

Digital signals have priority over analog signals because with the

help of digital signals users have ability to change the visual and

the audio or re-record it but with analog signals you can do such

parameters.

Cap 125 Introduction to information theory and applications

12

1.3 Difference between digital and analog signal

Analog signal

1. It produces a continuous signal.

2. Signal that is continuous in a time

and can assume an infinite number

of values in a given range.

3. It is denoted by sine wave.

4. Analog hardware is not flexible.

5. It Store data in the form of wave

signal.

6. Analog instrument consume more

power.

7. It is of low cost and portable.

For e.g.:- thermometer.

Wave form in analog signal

Digital signal

1. It produce a discrete signal

2. Signal that is continuous in a

time and assumes only a

limited number of values.

3. It is denoted by square wave.

4. Digital hardware is flexible in

implementation

5. It store data in the form of

binary bit.

6. Digital instrument consume

less power.

7. It is expensive and not easily

portable.

For e.g.: computers.

Wave form in digital

signal

Cap 125 Introduction to information theory and applications

13

Reference books 1. Digital Signal Processing By.

Sen M. Kuo & Woon-Seng Gan

2. Digital Signal Processing a Practical Approach. By

Emmanuel C. Ifeachor & Barrie W. Jervis

3. Digital Signal Processing Principles, Algorithms and Applications

By:

John G.Proakis & Dimitris G.Manolakis