Prof. SankarReview of Random Process1 Probability Sample Space (S) –Collection of all possible outcomes of a random experiment Sample Point –Each outcome

Prof. Sankar Review of Random Process 1

Probability• Sample Space (S)

– Collection of all possible outcomes of a random experiment

• Sample Point– Each outcome of the experiment (or)

element in the sample space

• Events are Collection of sample points• Ex: Rolling a die (six sample points), Odd number thrown

in a die (three sample point – a subset), tossing a coin (two sample points: head,tail)


Probability• Null Event (No Sample Point)

• Union (of A and B)– Event which contains all points in A and B

• Intersection (of A and B)– Event that contains points common to A and B

• Law of Large Numbers–

N – number of times the random experiment is repeated

NA- number of times event A occurred

N

N

N

LimAP A

)(AeventofobabilityPr


Probability

• Properties

yprobabilitjoint theis )BA(P)B.and.A(P)AB(P where

)AB(P)B(P)A(P)BA(P

events, exclusivemutually -nonFor

)B.or.A(P)BA(P)B(P)A(P)BA(P

0)AB(PBA ie., events exclusivemutually for ,SB,A)iii

0)(P;1)S(P)ii

1)A(P0)i


Probability

• Conditional Probability– Probability of B conditioned by the fact that A

has occurred

– The two events are statistically independent if

theoremBayesAP

BAPBP

AP

ABPABP

')(

)|()(

)(

)()|(

)()()( BPAPABP


Probability

• Bernoulli’s Trials– Same experiment repeated n times to find the

probability of a particular event occurring exactly k times

)!(!

!

)(

knk

nCk

n

qpknkP

kn

knkn


Random Signals• Associated with certain amount of uncertainty

and unpredictability. Higher the uncertainty about a signal, higher the information content.

– For example, temperature or rainfall in a city– thermal noise

• Information is quantified statistically (in terms of average (mean), variance, etc.)• Generation

– Toss a coin 6 times and count the number of heads – x(n) is the signal whose value is the number of heads

on the nth trial


Random Signals• Mean

• Median: Middle or most central item in an

ordered set of numbers

• Mode = Max{xi}

• Variance

• Standard Deviation measure of spread or deviation from the mean

N

iixN

x1

1

2N

1ii

2x )xx(

1N

1

iancevarx


Random Variables• Probability is a numerical measure of the outcome

of the random experiment• Random variable is a numerical description of the

outcome of a random experiment, i.e., arbitrarily assigned real numbers to events or sample points– Can be discrete or continuous

– For example: head is assigned +1

tail is assigned –1 or 0


Random Variables• Cumulative Distribution Function (CDF)

– Properties:

• Probability Density Function (PDF)

– Properties:

)xX(P)x(FX

dy)y(x

xp)xX(P)x(

xFor

dx

)x(x

dF)x(

xp

1)(F;0)(F;0)x(F XXX

212X1X xxif),x(F)x(F

0)x(p;1dx)x(p xx


Important Distributions

• Binary distribution (Bernoulli distribution)– Random variable has a binary distribution– Partitions the sample space into two distinct

subsets A and B– All elements in A are mapped into one number

say +1 and B to another number say 0.

)(][)(:

)(,)(

1012

xXPxppdf

pqVariancepmMean

pq]P[Xp]P[X

X

xx



• Binomial Distribution– Perform binary experiment n times with

outcome X1,X2,…Xn, if , then X has binomial distribution

i

iXX

npqnpm

)kx(]kX[P)x(p:pdf

qpk

n]kX[PCDF

2xx

n

0k

knk



• Uniform Distribution– Random variable is equally likely– Equally Weighted pdf

12

,2

,0

1)(

22 abab

m

elsewhere

bxaabxp

XX

X

a b

ab 1

)(xpX


Important Distributions• Poisson Distribution

– Random Variable is Poisson distributed with parameter m with

– Approximation to binomial with p << 1,

and k << 1, then

1np

!k

npeqpk

nk

npknk

X

mVariancemMean

kxk

mexppdf

k

mekXPp

x

k

km

X

km

k

20

)(!

)(

!][


Important Distributions• Gaussian Distribution

• Normalized Gaussian pdf - N(0,1)– Zero mean, Unit Variance

dueduupxF

VariancemMeanexp

x mx

x

x

xX

xX

mxx

x

X

x

x

2

2

2

2

2

22

2

1)()(

:,:2

1)(

2

2

2

2

2

1)(

2

1)(

x

X

xz

z

exp

dzex



• Normalized Gaussian pdf

2x

2

2

ex2

1)x(Q

)3x.,ie(xofvaluesearglFor

)functionerrorarycomplement()x(erfc)x(1)x(Q

)functionerror()x(erf)x(

notationseAlternativ

)x(1)x(


Joint and Conditional PDFs

• For two random variables X and Y–

(y)(x)pp(x,y)ptindependen are Yand X If

pdf under the Volume

dxdy(x,y)p)YY,YXXP(X

(x,y)F

yx(x,y)ppdfJoint

y)x,YP(X(x,y)F

Yand X variablesrandom Two

YXYX

X

X

Y

Y

YX2121

YX2

YX

YX

2

1

2

1


Joint and Conditional PDFs• Marginal pdfs

• Conditional pdfs

(x,y)dxp(y)p

(x,y)dyp(x)p

YXY

YXX

(x)p

(x,y)px)|(yp

(y)p

(x,y)py)|(xp

X

YXX|Y

Y

YXY|X


Expectation and Moments

•

Centralized Moment

– Second centralized moment is variance

dx)x(px)x(E:X ofmoment n

)statisticordernd2(dx)x(px)x(E:X ofmoment Second

statistic)order (1st dx)x(px)x(Em:(mean) X ofmoment First

Xnnth

X22

Xx

nX

th

2X

22X

2X

)mX(E: MomentdCentralize nm)X(E)mX(E


Expectations and Moments

• (i,j) joint moment between random variables X and Y

variablesrandom two thebetween iprelationsh of nature theDetermines

nCorrelatioisR)XY(E1)j(imoment joint First

dxdy(x,y)pyx)YX(E

YX

X

X

Y

Y

YXjiji

2

1

2

1


Expectations and Moments• (i,j) joint central moment

y truenecessarilnot converse

ed,uncorrelat be tosaid are Yand X Then

0Y,XCovYEXEYXE

tindependenlly statistica are Y and X If

YXE

YXECY,XCov

CovarianceCYXE

1)j(imoment joint First

dxdy(x,y)fyxYXE

YX

YXYX

YXYX

Y

Y

YXj

Yi

Xj

Yi

X

2

1


Expectations and Moments• Auto-covariance

• Characteristic Function (moment generator)

)cov(

Coeff.n Correlatio

),( 22

arianceNormalized

CP

xEXXCovC

YX

YXYX

XXXX

pair ansformFourier tr)()(

)(2

1)(

)()(

xft

dxetxf

dxxfeeEt

XF

X

jxtXX

Xjxtjxt

X


Random Process

• If a random variable X is a function of another variable, say time t, x(t) is called random process

• Collection of all possible waveforms is called the ensemble

• Individual waveform is called a sample function• Outcome of a random experiment is a sample

function for random process instead of a single value in the case of random variable


Random Process• Random Process X(.,.) is a function of time

variable t and sample point variable s• Each sample point (s) identifies a function of time

X(.,s) referred as “sample function”• Each time point (t) identifies a function of sample

points X(t,.), i.e., a random variable• Random or Stochastic Processes can be

– continuous or discrete time process – continuous or discrete amplitude process


Random Process

• Ensemble statistic : Ensemble average at a particular time

– Temporal average for a sample function

• Random Process Classifications– Stationary Process : Statistical characteristics of the

sample function do not change with time (time-invariant)

dxxfxxEX x)(

2

2

)(1

T

T

dttxTT

LimX


Random Process• Second Order joint pdf

– Autocorrelation is a function of only time difference

• Wide Sense (or Weak) Stationary

– Independent of time up to second order only• Ergodic Process

– Ensemble average = time average

)()(),()]()([ 122121 xxx RttRttRtxtxE

1221 )(),(

constant)]([

ttRttR

txE

xx

x


Random Process

• Mean – Mean of the random process at time t is the mean of the

random variable X(t) • Autocorrelation

• Auto-covariance

)()]([ tmtXE X

),()]()([ 2121 ttRtXtXE XX

)()(),(),(

),()()()()(

212121

212211

tmtmttRttC

ttCtmtXtmtXE

XXXX

XXXX


Random Process• Cross Correlation and covariance

• Power Density Spectrum

][][),(),(

][)(

process, timediscreteFor

)()()()(),(

)()(),(

2121

221121

2121

nnmnXX

nX

YXXY

XY

XXEXXEmnRornnR

XEnm

tmtYtmtXEttC

tYtXEttR

deRRFfS fjXXX

2)()]([)(


Random Process

• Total Average Power

dSordffSR

dttxT

ETLimP

XXX

X

T

T

)()()0(

)(1 2

2

2

Documents

Prof. SankarReview of Random Process1 Probability Sample Space (S) –Collection of all possible outcomes of a random experiment Sample Point –Each outcome