Upload
others
View
28
Download
0
Embed Size (px)
Citation preview
Chapter 2Probability and Stochastic Processes
WITS Lab, NSYSU.2
Table of ContentsProbability
Random Variables, Probability Distributions, and Probability DensitiesFunctions of Random VariablesStatistical Averages of Random VariablesSome Useful Probability DistributionsUpper Bounds on the Tail ProbabilitySums of Random Variables and the Central Limit Theorem
Stochastic ProcessesStatistical AveragesPower Density SpectrumResponse of a Linear Time-Invariant System to a Random Input SignalSampling Theorem for Band-Limited Stochastic ProcessesDiscrete-Time Stochastic Signals and SystemsCyclostationary Processes
WITS Lab, NSYSU.3
Introduction
Probability and stochastic process is important in:The statistical modeling of sources that generate the information;The digitization of the source output;The characterization of the channel through which the digital information is transmitted;The design of the receiver that processes the information-bearing signal from the channel;The evaluation of the performance of the communication system.
WITS Lab, NSYSU.4
2.1 Probability
Sample space or certain event of a die experiment:
The six outcomes are the sample points of the experiment.An event is a subset of S, and may consist of any number of sample points. For example:
The complement of the event A, denoted by , consists of all the sample points in S that are not in A:
6,5,4,3,2,1=S
4,2=AA
6,5,3,1=A
WITS Lab, NSYSU.5
Two events are said to be mutually exclusive if they have no sample points in common – that is, if the occurrence of one event excludes the occurrence of the other. For example:
The union (sum) of two events in an event that consists of all the sample points in the two events. For example:
6,3,1 ;4,2 == BA
SAA
CBDC
=
===
∪
∪ 6,3,2,13,2,1
events. exclusivemutually are and AA
2.1 Probability
WITS Lab, NSYSU.6
The intersection of two events is an event that consists of the points that are common to the two events. For example:
When the events are mutually exclusive, the intersection is the null event, denoted as φ. For example:
3,1== CBE ∩
φ=AA∩
2.1 Probability
WITS Lab, NSYSU.7
Associated with each event A contained in S is its probability P(A).Three postulations:
P(A)≥0.The probability of the sample space is P(S)=1. Suppose that Ai , i =1, 2, …, are a (possibly infinite) number of events in the sample space S such that
Then the probability of the union of these mutually exclusive events satisfies the condition:
,...2,1 ; =≠= jiAA ji φ∩
∑=⎟⎟⎠
⎞⎜⎜⎝
⎛
ii
ii APAP )(∪
2.1 Probability
WITS Lab, NSYSU.8
Joint events and joint probabilities (two experiments)If one experiment has the possible outcomes Ai , i =1,2,…,n, and the second experiment has the possible outcomes Bj , j =1,2,…,m, then the combined experiment has the possible joint outcomes(Ai ,Bj), i =1,2,…,n, j =1,2,…,m.Associated with each joint outcome (Ai ,Bj) is the joint probabilityP (Ai ,Bj) which satisfies the condition:
Assuming that the outcomes Bj , j =1,2,…,m, are mutually exclusive, it follows that:
If all the outcomes of the two experiments are mutually exclusive, then:
1),(0 ≤≤ ji BAP
∑=
=m
jiji APBAP
1)(),(
∑∑= =
=n
i
m
jji BAP
1 11),(
2.1 Probability
WITS Lab, NSYSU.9
Conditional probabilitiesThe conditional probability of the event A given the occurrence of the event B is defined as:
provided P(B)>0.)(
),()|(BP
BAPBAP =
)()|()()|(),( APABPBPBAPBAP ==
If two events and are mutually exclusive, , then ( | ) 0.
A B A BP A B
φ==
∩
.1)|( and have we, ofsubset a is If == BAPBBAAB ∩
. and of occurrence ussimultaneo thedenotes ),(is,That . ofy probabilit theas dinterprete is ),(BABAP
BABAP ∩
2.1 Probability
WITS Lab, NSYSU.10
Bayes’ theorem:
P(Ai) represents their a priori probabilities and P(Ai|B) is the a posteriori probability of Ai conditioned on having observed the received signal B.
i 1
If , 1, 2,..., , are mutually exclusive events such that
and is an arbitrary event with nonzero probability, then( , ) ( | )
( )
i
n
i
ii
A i n
A S
BP A B PP A B
P B
=
=
=
= =
∪
1
( | ) ( )
( | ) ( )
i in
j jj
B A P A
P B A P A=∑
2.1 Probability
WITS Lab, NSYSU.11
Statistical independence
When the events A and B satisfy the relation P(A,B)=P(A)P(B), they are said to be statistically independent.Three statistically independent events A1, A2, and A3 must satisfy the following conditions:
).()|( then , ofoccurrence on the dependnot does of occurrence theIf
APBAPBA=
)()()()|(),( BPAPBPBAPBAP ==
)()()(),,()()(),()()(),()()(),(
321321
3232
3131
2121
APAPAPAAAPAPAPAAPAPAPAAPAPAPAAP
====
2.1 Probability
WITS Lab, NSYSU.12
2.1.1 Random Variables, Probability Distributions, and Probability Densities
The function X(s) is called a random variable.Example 1: If we flip a coin, the possible outcomes are head (H) and tail (T), so S contains two points labeled H and T. Suppose we define a function X(s) such that:
Thus we have mapped the two possible outcomes of the coin-flipping experiment into the two points ( +1,-1) on the real line.Example 2: Tossing a die with possible outcomes S=1,2,3,4,5,6. A random variable defined on this sample space may be X(s)=s, in which case the outcomes of the experiment are mapped into the integers 1,…,6, or, perhaps, X(s)=s2, in which case the possible outcomes are mapped into the integers 1,4,9,16,25,36.
line. real on the numbers ofset a is range whoseand isdomain whose)(funciton a define we, elements
and space sample a having experimentan Given
SsXSs
S∈
⎩⎨⎧
==+
=T)(s 1-H)(s 1
)(sX
WITS Lab, NSYSU.13
The function F(x) is called the probability distribution functionof the random variable X.It is also called the cumulative distribution function (CDF).
∞<<∞≤=
≤∞∞≤
x-xXPxFxF
xXPxxX
X
),()( i.e., ,)(by simply it denote
and )( asevent thisofy porbabilit the writeWe).,(- interval in thenumber realany is where
event heconsider t uslet , variablerandom aGiven
1)(0 ≤≤ xF.1)( and 0)( =∞=−∞ FF
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.14
Examples of the cumulative distribution functions of two discrete random variables.
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.15
An example of the cumulative distribution function of a continuous random variable.
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.16
An example of the cumulative distribution function of a random variable of a mixed type.
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.17
The derivative of the CDF F(x), denoted as p(x), is called the probability density function (PDF) of the random variable X.
When the random variable is discrete or of a mixed type, the PDF contains impulses at the points of discontinuity of F(x):
∫∞−
∞<<∞−=
∞<<∞−=
x
xduupxF
xdx
xdFxp
,)()(
,)()(
∑=
−==n
iii xxxXPxp
1
)()()( δ
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.18
( )
. range in the PDF under the area the
simply is event theofy probabilit The
)(
)()()( )()()(
)()()(
. where, , intervalan in fallsvariablerandomay that probabilitthegDeterminin
21
21
1221
2112
2112
1221
2
1
xXxxXx
dxxp
xFxFxXxPxXxPxFxF
xXxPxXPxXP
xxxxX
x
x
≤<≤<
=
−=≤<⇒≤<+=
≤<+≤=≤
>
∫
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.19
Multiple random variables, joint probability distributions, and joint probability densities: (two random variables)
( )( ) ( ) ( ) .0,,, : thatNote
1,),(
PDFs. called are variables theofoneover gintegratin from obtained )( and )( PDFs The
)(),( )(),(
),(),( :PDFJoint
),(),(),( :CDFJoint
12
2- - 121
21
12212121
2121
2
21
2-
x
- 1212211211 2
=−∞=∞−=−∞∞−
=∞∞=
==
∂∂∂
=
=≤≤=
∫ ∫
∫∫
∫ ∫
∞
∞
∞
∞
∞
∞−
∞
∞−
∞ ∞
xFxFF
Fxddxxxp
xpxp
xpdxxxpxpdxxxp
xxFxx
xxp
udduuupxXxXPxxFx
2.1.1 Random Variables, Probability Distributions, and Probability Densities
marginal
WITS Lab, NSYSU.20
Multiple random variables, joint probability distributions, and joint probability densities: (multidimensional random variables)
( )( ) .0,...,,,,
).,...,,,(,...,,,,
),...,,(),...,,(
),...,,(...
),...,,( PDFJoint
...),...,,(...
),...,,(),...,,( CDFJoint . variablesrandom are 21 that Suppose
41
54141
413221
2121
21
- 2
x
- 121
221121
1 2
=−∞−∞=∞∞
=
∂∂∂∂
=
=
≤≤≤==
∫ ∫
∫ ∫ ∫
∞
∞−
∞
∞−
∞ ∞ ∞−
n
nn
nn
nn
n
n
n
x x
n
nnn
i
xxxFxxxxFxxxF
xxxpdxdxxxxp
xxxFxxx
xxxp
duudduuuup
xXxXxXPxxxF,...,n,,, iX
n
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.21
Conditional probability distribution functions (consider two random variables):
( )
( )( )
( ) ( )
( ) ( )
( ) ( ))()(
,,
,,
)(
,|X
.on dconditione variablerandom that theyprobabilit theis |Xevent theofy Probabilit
222
22121
2222
21212121
22
2121
222211
222211
222211
222
1 1 222
2
22
1 2
22
xxFxFxxxFxxF
duupduup
duduuupduduuup
duup
duduuupxXxxxP
xXxxxXxXxxx
xxx
x x xxx
x
xx
x x
xx
∆−−∆−−
=
⎥⎦⎤
⎢⎣⎡ −
−=
=≤<∆−≤
≤<∆−≤≤<∆−≤
∫∫∫ ∫ ∫∫
∫∫ ∫
∆−
∞−∞−
∞− ∞−
∆−
∞−∞−
∆−
∞− ∆−
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.22
Conditional probability distribution functions (consider two random variables):
( ) ( ) ( )( )
( )1 2
2 2
1 2
1 2 21 1 2 2 1 2
2 2
1 2 1 2
2
Divide both numerator and denominator by and let 0The conditional CDF of given the random variable is given by:
, /X | |
/
,
(
x x
x xX X
F x x xP x X x F x x
F x x
p u u du dux
p
−∞ −∞
∆ ∆ →
∂ ∂≤ = ≡ =
∂ ∂
⎡ ⎤∂ ⎢ ⎥⎣ ⎦∂
=∂
∫ ∫( )( ) ( )
( ) ( )
1
2
1 2 1
22 2
2
2 2
, *
)
Observe that | 0 and | 1.
x
x
p u x du
p xu dux
F x F x
−∞
−∞
=⎡ ⎤⎢ ⎥⎣ ⎦
∂
−∞ = ∞ =
∫∫
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.23
Conditional probability distribution functions (consider two random variables):
( )
( ) ( )( )( )
( ) ( )
1
1 21 2
2
1 2
1 2 2 1
Differentiating Equation * with respect to , we obtain
, |
We may express the joint PDF , in terms of the
conditional PDFs, | or | , as:
x
p x xp x x
p x
p x x
p x x p x x
=
( ) ( )( ) ( )
1 2 1 2 2
2 1 1
, | ( )
|
p x x p x x p x
p x x p x
=
=
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.24
( ) ( ) ( )
( )
1 2 1 2 1 1
1 2 1
Joint PDF of the random variables , 1,2,..., , , ,..., , ,..., | ,..., ,...,
Joint conditional CDF: , ,..., | ,...,
...
i
n k k n k n
k k n
X i np x x x p x x x x x p x x
F x x x x x
p
+ +
+
=
=
=( )
( )( ) ( )( )
1
1 2 1 1 2
1
2 1 2 1
2 1
, ,..., , ,..., ...
,...,
, ,..., , ,..., ,..., , ,...,
, ,..., | ,..., 0
kx x
k k n k
k n
k k n k k n
k k n
u u u x x du du du
p x x
F x x x x F x x x x
F x x x x
+−∞ −∞
+
+ +
+
∞ =
−∞ =
∫ ∫
2.1.1 Random Variables, Probability Distributions, and Probability Densities
Conditional probability distribution functions (consider multidimensional variables):
WITS Lab, NSYSU.25
Statistically independent random variables:If the experiments result in mutually exclusive outcomes, the probability of an outcome is independent of an outcome in any other experiment.The joint probability of the outcomes factors into a product of the probabilities corresponding to each outcome.Consequently, the random variables corresponding to the outcomes in these experiments are independent in the sense that their joint PDF factors into a product of marginal PDFs.The multidimensional random variables are statistically independent if and only if:
( ) ( ) ( ) ( )( ) ( ) ( ) ( )nn
nn
xpxpxpxxxpxFxFxFxxxF
...,...,,...,...,,
2121
2121
==
2.1.1 Random Variables, Probability Distributions, and Probability Densities
WITS Lab, NSYSU.26
Problem: given a random variable X, which is characterized by its PDF p(x), determine the PDF of the random variable Y=g(X), where g(X) is some given function of X.When the mapping g from X to Y is one-to-one, the determination of p(y) is relatively straightforward.However, when the mapping is not one-to-one, as in the case, for example, when Y=X2, we must be very careful in our derivation of p(y).
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.27
2.1.2 Functions of Random Variables
Transformation of a random variable
WITS Lab, NSYSU.28
Example 2.1-1:Y=aX+b; where a and b are constant and assume that a>0.
( )( )
( ) ⎟⎠⎞
⎜⎝⎛ −
=
⎟⎠⎞
⎜⎝⎛ −
==
−≤=≤+=≤=
∫∞
abyp
ayp
abyFdxxp
abyXPybaXPyYPyF
XY
XX
Y
1
)()()()(
ab-y
-
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.29
Example 2.1-1 (cont.):Y=aX+b; where a and b are constant and assume that a>0.
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.30
Example 2.1-2:Y=aX3+b; a>0.The mapping between X and Y is one-to-one.
[ ] ⎥⎥⎦
⎤
⎢⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −
−=
⎥⎥⎦
⎤
⎢⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −
=⎥⎥⎦
⎤
⎢⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −
≤=
≤+=≤=
31
32
31
31
3
)(31)(
PDFs twoebetween th iprelationshdesired theyields respect to ation withDifferenti
)()()(
aby
XY
X
Y
pabya
yp
y
abyF
abyXP
ybaXPyYPyF
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.31
Example 2.1-3:Y=aX2+b; a>0.The mapping between X and Y is not one-to-one.
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.32
Example 2.1-3 (cont.):Y=aX2+b; a>0.
Differentiating with respect to y, we obtain the PDF of Y in terms of the PDF of X in the form:
⎟⎟⎠
⎞⎜⎜⎝
⎛ −−−⎟⎟
⎠
⎞⎜⎜⎝
⎛ −=
⎟⎟⎠
⎞⎜⎜⎝
⎛ −≤=≤+=≤=
abyF
abyFyF
abyXPybaXPyYPyF
XXY
Y
)(
)()()( 2
[ ] [ ][ ( ) ] [ ( ) ]
( )2 ( ) 2 ( )
X XY
p y b a p y b ap y
a y b a a y b a− − −
= +− −
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.33
Example 2.1-3 (cont.):Y=g(X)=aX2+b; a>0 has two solutions:
pY(y) consists of two terms corresponding to these two solutions:
In general, suppose that x1,x2, …, xn are the real roots of g(x)=y. The PDF of the random variable Y=g(X) may be expressed as
abyx
abyx −
−=−
= 21 ,
( ) ( )( )∑
= ′=
n
i i
iXY xg
xpyp1
( ) ( )( )
( )( ) ]/[
]/[]/[]/[
2
2
1
1
abyxgabyxp
abyxgabyxp
yp XXY
−−=′
−−=+
−=′
−==
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.34
Function of one random variable (discrete case):Suppose X is a discrete random variable that can have one of nvalues x1, x2, …, xn.Let g(X) be a scalar-valued function.Y=g(X) is a discrete random variable that can have one of m, m≤n, values y1, y2, …, ym.If g(X) is a one-to-one mapping, then m will be equal to n.If g(X) is many-to-one, then m will be smaller than n.The CDF of Y can be obtained easily from the CDF function of X as:
. tomap that of valuesallover is sum thewhere)()(
ii
ii
yxxXPyYP ∑ ===
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.35
Function of one random variable (continuous case):If X is a continuous random variable, then the pdf of Y=g(X) can be obtained from the pdf of X.Let y be a particular value of Y and let x(1), x(2), …, x(n) be roots of the equation y=g(x). That is y=g(x(1))=g(x(2))= … = y=g(x(n)).
( ) ( )( ) [ ]yyxgyxP
yyyfyyYyP Y
∆+≤<=→∆∆=∆+≤<
: 0 as
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.36
Function of one random variable (continuous case):
2.1.2 Functions of Random Variables
x
=y
WITS Lab, NSYSU.37
Function of one random variable (continuous case):( ) ( ) ( ) ( )( ) ( ) ( ) ( )( )
( ) ( ) ( )( )( )( ) ( ) ( )( ) ( ) ( )( ) ( )
1 1 1 2 2 2
3 3 3
1 1 2 2 3 3
X X X
P y Y y y P x X x x P x x X x
P x X x x
f x x f x x f x x
< < + ∆ = < < + ∆ + −∆ < <
+ < < + ∆
= ∆ + ∆ + ∆
( ) ( )( )
( )( )( )
( )( )( )
( )( )33
22
11
have we,/ is of slope theSince
xgyxxg
yxxgyx
xyxgxg
′∆=∆′
∆=∆′∆=∆
∆∆′
( )( )( )( )( )
( )( )( )( )
( )( )( )( )
( )( )( )( )( )∑
= ′=⇒
∆′
+∆′
+∆′
=∆
k
ii
iX
Y
XXXY
xgxfyf
yxgxfy
xgxfy
xgxfyyf
1
3
3
2
2
1
1
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.38
Function of several random variables (continuous case):Goal: find the joint distribution of n random variables Y1, Y2, …, Yn given the distribution of n related random variables, X1, X2, …, Xn, where
Let’s start with a mapping of two random variables onto two other random variables:
( ) niXXXgY nii ,...,2,1 ,,...,, 21 ==
( ) ( )21222111 , , XXgYXXgY ==( ) ( )( ) ( )( )
( ) ( ) 222122112111
212122
211121
, and ,such that plane
, in theregion thefind toneed We.,y and,y of roots theare ,...,2,1 ,, Suppose
yyxxgyyyxxgy
xxxxgxxgkkixx ii
∆+<<∆+<<
===
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.39
Function of several random variables (continuous case):There are k such regions as shown in the following figure for the case of k=3.
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.40
Function of several random variables (continuous case):
( ) ( )( ) ( )
( )
( )( ) ( )( )
( ) ( )( )∑=
=
∂∂
∂∂
∂∂
∂∂
=
∆∆
k
iii
iiXX
YY
ii
xxJxxf
yyf
YY
xg
xg
xg
xg
xxJ
xxJxxJ
yy
1 21
21,21,
21
2
2
1
2
2
1
1
1
21
2121
21
,,
,
as and of pdfjoint obtain the weregions, all fromon contributi thesummingBy
,
as defined isation transform theofJacobian the
is , where,
toequal is ramparallelog
eachofareatheand ramparallelogaofconsistsregion Each
21
21
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.41
Function of several random variables (continuous case):
( )( ) ( ) ( )( )
( )
( )
1 1 11 1 2 2
1 21
We can generalize this result to the -variate case as
, , ...,, , ...,
where denotes the Jacobian of the transformation defined by the
following determinant when
i i ik X n n
Y n ii
i
n
f x g x g x gf y y y
J
J
− − −
=
= = == ∑
( )
( ) ( ) ( ) ( ) ( )
( )
( ) ( ) ( )1 1 11 21 2
1
1 1 11 2 2 1 2 n
1 1 1
1 2
1 2 at , ,...,
evaluating at the solution
, , ..., of ( ) [ x ,g x , ..., g x ]:
i i in n
i
i in n
ni
n n n
n x g x g x g
i th x
g x g x g y g x g
g g gx x x
Jg g gx x x − − −
− − −
= = =
− =
= = = =
∂ ∂ ∂∂ ∂ ∂
=∂ ∂ ∂∂ ∂ ∂
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.42
Example 2.1-4
Aybxybxybxp
yyypAJ
Ab
,...,n,, iYbXYAX
nnAAXY
niXaY
n
j
n
j
n
jjnjnjjjjX
nY
-ij
n
jjiji
n
jjiji
det1,...,,
),...,,(.det isation transform thisofJacobian The
.matrix inverse theof elements theare
21
matrix. an is e wher
,...,2,1 ,
1 1 12211
21
1
1
1
1
⎟⎟⎠
⎞⎜⎜⎝
⎛⋅==⋅==
=
==⇒=
×=
==
∑ ∑ ∑
∑
∑
= = =
⋅
=
−
=
2.1.2 Functions of Random Variables
WITS Lab, NSYSU.43
The mean or expected value of X, which characterized by its PDF p(x), is defined as:
This is the first moment of random variable X.The n-th moment is defined as:
Define Y=g(X), the expected value of Y is:
∫∞
∞−=≡ dxxxpmXE x )()(
dxxpxXE nn )()( ∫∞
∞−=
[ ] ∫∞
∞−== dxxpxgXgEYE )()()()(
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.44
The n-th central moment of the random variable X is:
When n=2, the central moment is called the varianceof the random variable and denoted as :
In the case of two random variables, X1 and X2, with joint PDF p(x1,x2), we define the joint moment as:
( )[ ] ∫∞
∞−−=−= dxxpmxmXEYE n
xn
x )()()(
[ ] 22222
22
)()()(
)()(
xx
xx
mXEXEXE
dxxpmx
−=−=
−= ∫∞
∞−
σ
σ
2xσ
21212121 ),()( dxdxxxpxxXXE nknk ∫ ∫∞
∞−
∞
∞−=
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.45
The joint central moment is defined as:
If k=n=1, the joint moment and joint central momentare called the correlation and the covariance of the random variables X1 and X2, respectively.The correlation between Xi and Xj is given by the joint moment:
[ ]∫ ∫∞
∞−
∞
∞−−−=
−−
21212211
2211
),()()(
)()(
dxdxxxpmxmx
mXmXEnk
nk
∫ ∫∞
∞−
∞
∞−= jijijiji dxdxxxpxxXXE ),()(
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.46
The covariance between Xi and Xj is given by the joint central moment:
The n×n matrix with elements μij is called the covariance matrix of the random variables, Xi, i=1,2, …, n.
( )( )( )( )( )
jiji
jijijijijiji
jijijiji
jiijijiijji
jiijjii
jjiiij
mmXXE
mmmmmmdxdxxxpxx
mmdxdxxxpxx
dxdxxxpmmmxmxxx
dxdxxxpmxmx
mXmXE
j
j
−=
+−−=
−=
+−−=
−−=
−−≡
∫ ∫∫ ∫∫ ∫∫ ∫
∞
∞−
∞
∞−
∞
∞−
∞
∞−
∞
∞−
∞
∞−
∞
∞−
∞
∞−
)(
),(
),(
),(
),(
][µ
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.47
Two random variables are said to be uncorrelated if E(XiXj)=E(Xi)E(Xj)=mimj.Uncorrelated → Covariance μij = 0.If Xi and Xj are statistically independent, they are uncorrelated.If Xi and Xj are uncorrelated, they are not necessarystatistically independently.Two random variables are said to be orthogonal if E(XiXj)=0.
Two random variables are orthogonal if they are uncorrelated and either one of both of them have zero mean.
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.48
Characteristic functionsThe characteristic function of a random variable X is defined as the statistical average:
Ψ(jv) may be described as the Fourier transform of p(x).The inverse Fourier transform is:
First derivative of the above equation with respect to v:
∫∞
∞−=≡ dxxpejveE jvxjvX )()()( ψ
∫∞
∞−
−= dvejvxp jvx)(21)( ψπ
∫∞
∞−= dxxpxej
dvjvd jvx )()(ψ
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.49
Characteristic functions (cont.)First moment (mean) can be obtained by:
Since the differentiation process can be repeated, n-thmoment can be calculated by:
Suppose the characteristic function can be expanded in a Taylor series about the point v=0:
0
)()()(=
−=v
n
nnn
dvjvdjXE ψ
0
)()(=
−==v
x dvjvdjmXE ψ
∑∑∞
==
∞
=
=⇒⎥⎦
⎤⎢⎣
⎡=
000 !)()()(
!)()(
n
nn
n
vnn
n
njvXEjv
nv
dvjvdjv ψψψ
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.50
Characteristic functions (cont.)Determining the PDF of a sum of statistically independentrandom variables:
( )
[ ]nXY
i
n
iXYnn
nn
n
i
jvxn
i
jvX
n
ii
jvYY
n
ii
jvjv
X
jvjvxpxpxpxxxp
dxdxdxxxxpeeE
XjvEeEjvXY
i
ii
)()(
d)distributey identicall andnt (independe iid are If
)()( )()...()(),...,,(
t,independenlly statistica are variablesrandom theSince
...),...,,(...
exp)()(
12121
212111
11
ψψ
ψψ
ψ
=⇒
=⇒=
⎟⎟⎠
⎞⎜⎜⎝
⎛=⎥
⎦
⎤⎢⎣
⎡=
⎥⎦
⎤⎢⎣
⎡⎟⎠
⎞⎜⎝
⎛==⇒=
∏
∫ ∫ ∏∏
∑∑
=
∞
∞−
∞
∞−==
==
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.51
Characteristic functions (cont.)The PDF of Y is determined from the inverse Fourier transform of ΨY(jv).Since the characteristic function of the sum of n statistically independent random variables is equal to the product of the characteristic functions of the individual random variables, it follows that, in the transform domain, the PDF of Y is the n-fold convolution of the PDFs of the Xi.Usually, the n-fold convolution is more difficult to perform than the characteristic function method in determining the PDF of Y.
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.52
Characteristic functions (for n-dimensional random variables)
If Xi, i=1,2,…,n, are random variables with PDF p(x1,x2,…,xn), the n-dimensional characteristic function is defined as:
For two dimensional characteristic function:
nn
n
iii
n
iiin
dxdxdxxxxpxvj
XvjEjvjvjv
...),...,,(exp...
exp),...,,(
21211
121
∫ ∫ ∑
∑∞
∞−
∞
∞−=
=
⎟⎠
⎞⎜⎝
⎛=
⎥⎦
⎤⎢⎣
⎡⎟⎠
⎞⎜⎝
⎛=ψ
021
2211
21
212
21
2121)(
21
),()(
),(),(
==∂∂
∂−=
= ∫ ∫∞
∞−
∞
∞−
+
vvvv
jvjvXXE
dxdxxxpejvjv xvxvj
ψ
ψ
2.1.3 Statistical Averages of Random Variables
WITS Lab, NSYSU.53
2.1.4 Some Useful Probability Distributions
Binomial distribution:
Let
what is the probability distribution function of Y ?1
where the , 1,2,..., are statistically iid,n
i ii
Y X X i n=
= =∑
( ) ( ) pXPXP ==−== 110
( ) ( ) ( )
( ) ( )
)()1(
)( :is of PDF
!!! 1
0
0
kyppkn
kykYPypY
knkn
kn
ppkn
kYP
knkn
k
n
k
knk
−−⎟⎟⎠
⎞⎜⎜⎝
⎛=
−==
−≡⎟⎟
⎠
⎞⎜⎜⎝
⎛−⎟⎟
⎠
⎞⎜⎜⎝
⎛==
−
=
=
−
∑
∑
δ
δ
WITS Lab, NSYSU.54
Binomial distribution:The CDF of Y is:
where [y] denotes the largest integer m such that m≤y.The first two moments of Y are:
The characteristic function is:
( ) ( )[ ]
knky
kpp
kn
yYPyF −
=
−⎟⎟⎠
⎞⎜⎜⎝
⎛=≤= ∑ )1(
0
( )
)1()1()(
2
222
pnppnpnpYE
npYE
−=
+−=
=
σ
njvpepj )1()( +−=νψ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.55
Uniform DistributionThe first two moments of X are:
The characteristic function is:
( )
22
222
)(121
)(31)(
)(21
ba
abbaYE
baXE
−=
++=
+=
σ( )
22
222
)(121
)(31)(
)(21
ba
abbaXE
baXE
−=
++=
+=
σ
)()(
abjveej
jvajvb
−−
=νψ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.56
Gaussian (Normal) DistributionThe PDF of a Gaussian or normal distributed random variable is:
where mx is the mean and σ2 is the variance of the random variable.The CDF is:
22 2/)(
21)( σ
σπxmxexp −−=
( ) ( )
)2
(211)
2(
21
21
121)(
2/2/ 222
σσ
πσπσσ
xx
mx tx mu
mxerfcmxerf
dteduexF xx
−−=
−+=
== ∫∫−
∞−
−
∞−
−−
2.1.4 Some Useful Probability Distributions
( ) 2 2
xu m dut dtσ σ
−= =
WITS Lab, NSYSU.57
Gaussian (Normal) distribution:
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.58
Gaussian (Normal) Distributionerf( ) and erfc( ) denote the error function and complementary error function, respectively, and are defined as:
erf(-x)=-erf(x), erfc(-x)=2-erfc(x), erf(0)=erfc(∞)=0, and erf(∞)=erfc(0)=1.For x>mx, the complementary error functions is proportional to the area under the tail of the Gaussian PDF.
( ) ( ) )(12 and 2 22
0xerfdtexerfcdtexerf
x
tx t −=== ∫∫∞ −−
ππ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.59
Gaussian (Normal) DistributionThe function that is frequently used for the area under the tailof the Gaussian PDF is denoted by Q(x) and is defined as:
2 / 21 1( ) 022 2
t
x
xQ x e dt erfc xπ
∞ − ⎛ ⎞= = ≥⎜ ⎟⎝ ⎠∫
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.60
Gaussian (Normal) DistributionThe characteristic function of a Gaussian random variable with mean mx and variance σ2 is:
The central moments of a Gaussian random variable are:
The ordinary moments may be expressed in terms of the central moments as:
( ) ( ) ( ) 2222 2/12/
21 σσ
σπψ vjvmmxjvx xx edxeejv −∞
∞−
−− =⎥⎦
⎤⎢⎣
⎡= ∫
[ ]⎩⎨⎧ −⋅⋅⋅⋅
==− ) (odd 0)(even )1(31
)(kkk
mXEk
kk
xσ
µ
[ ] ikix
k
i
k mik
XE −=∑ ⎟⎟
⎠
⎞⎜⎜⎝
⎛= µ
0
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.61
Gaussian (Normal) DistributionThe sum of n statistically independent Gaussian random variables is also a Gaussian random variable.
( ) ( )
. varianceand
mean with ddistribute-Gaussian is Therefore,
and where
2
1
22
1
2/
1
2/
1
1
2222
y
y
n
iiy
n
iiy
vjvmn
i
vjvmn
iXY
n
ii
mY
mm
eejvjv
XY
yyii
i
σ
σσ
ψψ σσ
∑∑
∏∏
∑
==
−
=
−
=
=
==
===
=
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.62
Chi-square distributionIf Y=X2, where X is a Gaussian random variable, Y has a chi-square distribution. Y is a transformation of X.There are two type of chi-square distribution:
Central chi-square distribution: X has zero mean.Non-central chi-square distribution: X has non-zero mean.
Assuming X be Gaussian distributed with zero mean and variance σ2, we can apply (2.1-47) to obtain the PDF of Ywith a=1 and b=0;
( ) ( )( )
( )( ) ]/[
]/[]/[]/[
1
1
1
1
abyxgabyxp
abyxgabyxp
yp XXY
−−=′
−−=+
−=′
−==
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.63
Central chi-square distributionThe PDF of Y is:
The CDF of Y is:
The characteristic function of Y is:
0 ,21)(
22/ ≥= − yey
yp yY
σ
σπ
( ) dueu
duupyF uyy
YY
22/
00
121)( σ
σπ−∫∫ ==
2/12 )21(1)(σ
ψvj
jvY −=
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.64
Chi-square (Gamma) distribution with n degrees of freedom.
The characteristic function is:
The inverse transform of this characteristic function yields the PDF:
.varianceandmean zero with variablesrandomGaussian (iid) ddistributey identicall
andt independenlly statistica are ,,...,2,1 , ,
2
1
2
σ
niXXY i
n
ii ==∑
=
2/2 )21(1)( nY vj
jvσ
ψ−
=
0
212
1)(2212
2/≥
⎟⎠⎞
⎜⎝⎛Γ
= − y,e ynσ
yp σ-y/n/
nnY
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.65
Chi-square (Gamma) distribution with n degrees of freedom (cont.).
When n=2, the distribution yields the exponential distribution.
( )
ππ21
23
21
0 integer an !1-p)(
0 ,)(
:asdefinedfunction,gamma theis )(
0
1
=⎟⎠⎞
⎜⎝⎛Γ=⎟
⎠⎞
⎜⎝⎛Γ
>=Γ
>=Γ
Γ
∫∞ −−
pp
pdtetp
ptp
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.66
Chi-square (Gamma) distribution with n degrees of freedom (cont.).
The PDF of a chi-square distributed random variable for several degrees of freedom.
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.67
Chi-square (Gamma) distribution with n degrees of freedom (cont.).
The first two moments of Y are:
The CDF of Y is:
42
4242
2
22)(
)(
σσ
σσ
σ
nnnYE
nYE
y =
+=
=
( ) ∫ ≥⎟⎠⎞
⎜⎝⎛Γ
= −−y un
nnY ydueu
nyF
0
2/12/
2/0 ,
212
1 2σ
σ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.68
Chi-square (Gamma) distribution with n degrees of freedom (cont.).
The integral in CDF of Y can be easily manipulated into the form of the incomplete gamma function, which is tabulated by Pearson (1965).When n is even, the integral can be expressed in closed form. Let m=n/2, where m is an integer, we can obtain:
0 ,)2
(!
11)( 2
1
0
2/ 2
≥−= ∑−
=
− yyk
eyF km
k
yY σ
σ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.69
Non-central chi-square distributionIf X is Gaussian with mean mx and variance σ2, the random variable Y=X2 has the PDF:
The characteristic function corresponding to this PDF is:
0 ),cosh(21)( 2
2/)( 22
≥= +− ymy
ey
yp xmyY
x
σσπσ
)21/(2/12
22
)21(1)( σ
σψ vjvjm
Yxe
vjjv −
−=
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.70
Non-central chi-square distribution with n degrees of freedom
The characteristic function is:. toequal varianceidentical and ,,...,2,1 ,mean
with variablesrandomGaussian (iid) ddistributey identicall
andt independenlly statistica are ,,...,2,1 , ,
2
1
2
σnim
niXXY
i
i
n
ii
=
==∑=
⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜
⎝
⎛
−−=
∑=
21
2
2/2 21exp
)21(1)(
σσψ
vj
mjv
vjjv
n
ii
nY
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.71
Non-central chi-square distribution with n degrees of freedom
The characteristic function can be inverse Fourier transformed to yield the PDF:
( )
2 2( 2) / 4 ( ) / 2/ 2 12 2 2
2
2 2
1
1( ) ( ) ( ), 02
where, is called the non-centrality parameter:
and is the th-order modified Bessel function
n s yY n
n
ii
y sp y e I y ys
s
s m
I x
σ
α
σ σ
α
− − +−
=
= ≥
=∑
2
0
of thefirst kind, which may be represented by the infinite series:
( / 2) ( ) , 0! ( 1)
k
k
xI x xk k
α
α α
+∞
=
= ≥Γ + +∑
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.72
Non-central chi-square distribution with n degrees of freedom
The CDF is:
The first two moments of a non-central chi-square-distributed random variable are:
( )
∫ −+−
−
⎟⎠⎞
⎜⎝⎛=
y
nus
n
Y dusuIesuyF
0 212/2/)(
4/2
22 )(2
1)(22
σσσ
2242
2222242
22
42)(42)(
)(
snsnsnYE
snYE
y σσσ
σσσ
σ
+=
+++=
+=
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.73
Non-central chi-square distribution with n degrees of freedom
When m=n/2 is an integer, the CDF can be expressed in terms of the generalized Marcum’s Q function:
( )
( ) ( ) ( )
( ) ( ) ( )
2 2
2 2
2 2
1( ) / 2
1
1/ 21
0
/ 21
022 2
22
, ( )
,
where , , 0
By using and let , it is easily shown:
mx a
m mb
kma bk
kk
a bk
k
xQ a b x e I ax dxa
bQ a b e I aba
bQ a b e I ab b aa
u sx a σσ
−∞ − +
−
−− +
=
∞− +
=
⎛ ⎞= ⎜ ⎟⎝ ⎠
⎛ ⎞= + ⎜ ⎟⎝ ⎠⎛ ⎞= > >⎜ ⎟⎝ ⎠
= =
∫
∑
∑
( ) 1 ,Y mysF y Q
σ σ⎛ ⎞
= − ⎜ ⎟⎜ ⎟⎝ ⎠
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.74
Rayleigh distributionRayleigh distribution is frequently used to model the statistics of signals transmitted through radio channels such as cellular radio.Consider a carrier signal s at a frequency ω0 and with an amplitude a:
The received signal sr is the sum of n waves:
)exp( 0tjas ω⋅=
[ ] [ ]
∑
∑
=
=
=
+≡+=
n
iii
n
iiir
jajr
tjrtjas
1
01
0
)exp()exp( where
)(exp)(exp
θθ
θωθω
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.75
Rayleigh distribution
Because (1) n is usually very large, (2) the individual amplitudes ai are random, and (3) the phases θi have a uniform distribution, it can be assumed that (from the central limit theorem) x and y are both Gaussian variables with means equal to zero and variance:
θθ
θθ
θθθ
sin cos :where
sin and cos :have We
sincos)exp( :Define
22211
11
ryrxyxr
ayax
jyxajajr
n
iii
n
iii
n
iii
n
iii
==+=
≡≡
+≡+=
∑∑
∑∑
==
==
222 σσσ ≡= yx
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.76
Rayleigh distributionBecause x and y are independent random variables, the joint distribution p(x,y) is
The distribution p(r,θ) can be written as a function of p(x,y) :
⎟⎟⎠
⎞⎜⎜⎝
⎛ +−== 2
22
2 2exp
21)()(),(
σπσyxypxpyxp
⎟⎟⎠
⎞⎜⎜⎝
⎛−=
=−
=∂∂∂∂∂∂∂∂
≡
=
2
2
2exp
2),(
cossinsincos
////
),(),(
σπσθ
θθθθ
θθ
θ
rrrp
rrr
yryxrx
J
yxpJrp
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.77
Rayleigh distributionThus, the Rayleigh distribution has a PDF given by:
The probability that the envelope of the received signal does not exceed a specified value R is given by the corresponding cumulative distribution function (CDF):
⎪⎩
⎪⎨⎧ ≥==
−
∫otherwise 0
0 ),()(22 2/
22
0
rerdrprp
r
R
σπ
σθθ
0 ,exp1)(2222 2/
0
2/2 ≥−== −−∫ rdueurF r
ru
Rσσ
σ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.78
Rayleigh distributionMean:
Variance:
Median value of r is found by solving:
Monents of R are:
Most likely value:= max pR(r) = σ.
σπσ 2533.12
)(][0
==== ∫∞
drrrpRErmean
22
2
0
2222
4292.02
2
2)(][][
σπσ
πσσ
=⎟⎠⎞
⎜⎝⎛ −=
−=−= ∫∞
drrprREREr
( )∫=medianr
drrp02
1
σ177.1=medianr
( ) ⎟⎠⎞
⎜⎝⎛ +Γ=
212][ 2/2 kRE kk σ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.79
Rayleigh distribution
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.80
Rayleigh distributionProbability That Received Signal Doesn’t Exceed A Certain Level (R)
⎟⎟⎠
⎞⎜⎜⎝
⎛−−=
⎟⎟⎠
⎞⎜⎜⎝
⎛−−=
⎟⎟⎠
⎞⎜⎜⎝
⎛−=
=
∫
∫
2
2
02
2
02
2
2
0
2exp1
2exp
2
exp
)()(
σ
σ
σσ
r
u
duuu
duuprF
r
r
r
R
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.81
Rayleigh distributionMean value:
02 2 2
2 2 20 0
2 2
2 200
2
20
[ ] ( )
exp exp2 2
exp exp2 212 exp
2212 1.25332 2
meanr E R rp r dr
r r rdr rd
r rr dr
r dr
σ σ σ
σ σ
πσσπσ
ππσ σ σ
∞
∞ ∞
∞ ∞
∞
= =
⎛ ⎞ ⎛ ⎞= − = − −⎜ ⎟ ⎜ ⎟
⎝ ⎠ ⎝ ⎠⎛ ⎞ ⎛ ⎞
= − ⋅ − + −⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠
⎛ ⎞= ⋅ −⎜ ⎟
⎝ ⎠
= ⋅ = =
∫
∫ ∫
∫
∫
0-0=0
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.82
Rayleigh distribution:Mean square value:
2
02
22
2
2
002
22
2
2
0
2
02
2
2
3
0
22
22
exp2
2exp2
2exp
2exp
2exp
)(][
σσ
σ
σσ
σσσ
=⎟⎟⎠
⎞⎜⎜⎝
⎛−⋅−=
⎟⎟⎠
⎞⎜⎜⎝
⎛−⋅+⎟⎟
⎠
⎞⎜⎜⎝
⎛−⋅−=
⎟⎟⎠
⎞⎜⎜⎝
⎛−−=⎟⎟
⎠
⎞⎜⎜⎝
⎛−=
=
∞
∞∞
∞∞
∞
∫
∫∫
∫
r
drrrrr
rdrdrrr
drrprRE
0-0=0
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.83
Rayleigh distributionVariance:
22
22
222
4292.02
2
)2
()2(
][][
σπσ
πσσ
σ
=⎟⎠⎞
⎜⎝⎛ −⋅=
⋅−=
−= REREr
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.84
Rayleigh distributionMost likely value
Most Likely Value happens when: dp(r) / dr = 0
σσσσ
σσσσσ
σσ
6065.021exp
2exp)(
02
exp22
2exp1)(
2
2
2
2
2
4
2
2
2
2
=⎟⎠⎞
⎜⎝⎛−
=⎟⎟⎠
⎞⎜⎜⎝
⎛−=⇒
=⇒
=⎟⎟⎠
⎞⎜⎜⎝
⎛−⋅
⋅−⎟⎟⎠
⎞⎜⎜⎝
⎛−=
==
rr
rrrp
r
rrrdr
rdp
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.85
Rayleigh distributionCharacteristic function
( )
( ) ( )
( ) ( ) ( )( ) ( ) ,...2,1,0 ,
!;;
:function etrichyupergeomconfluent theis ;21,1where
21
21;
21,1
sinrcosr
011
11
2/22211
0
2/20
2/2
2/
0 2
22
2222
22
−−≠+ΓΓΓ+Γ
=
⎟⎠⎞
⎜⎝⎛ −
+⎟⎠⎞
⎜⎝⎛ −=
+=
=
∑
∫∫
∫
∞
=
−
∞ −∞ −
−∞
ββα
βαβα
σπσ
σσ
σψ
σ
σσ
σ
k
k
v
rr
jvrrR
kkxkxF
aF
evjvF
drvrejdrvre
dreerjv
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.86
Rayleigh distributionCharacteristic function (cont.)
( )∑∞
=
−
−−=⎟
⎠⎞
⎜⎝⎛ −
⎟⎠⎞
⎜⎝⎛ −
011
11
!12;
21,1
:as expressed
bemay ;21,1 shown that has (1990)Beaulieu
k
ka
kkaeaF
aF
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.87
Generalized Rayleigh distribution
Y=R2 is chi-square distributed with n degrees of freedom.PDF is given by:
. variablesradomGaussianmean zero ddistributey identicall t,independen
llystatistica are ,,...,2,1 , where,Consider 1
2 niXXR i
n
ii == ∑
=
( )( )
0 ,
212
22 2/
2/2
1
≥⎟⎠⎞
⎜⎝⎛Γ
= −
−
−
ren
rrp r
nn
n
Rσ
σ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.88
Generalized Rayleigh distributionWhen n is an even number, i.e., n=2m, the CDF of R can be expressed in the closed form:
For any integer n, the k-th moment of R is:
( ) 0 ,2!
111
02
22/ 22
≥⎟⎟⎠
⎞⎜⎜⎝
⎛−= ∑
−
=
− rrk
erFm
k
kr
R σσ
( ) ( )( )
0 ,
21
21
2 2/2 ≥⎟⎠⎞
⎜⎝⎛Γ
⎥⎦⎤
⎢⎣⎡ +Γ
= kn
knRE kk σ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.89
Rice distributionWhen there is a dominant stationary (non-fading) signal component present, such as a line-of-sight (LOS) propagation path, the small-scale fading envelope distribution is Rice.
θθ
θωωωθω
sincos
)(
)](exp[)exp(])[( )exp()](exp['
22200
esdirect wav
0
wavesscattered
0
ryrAx
yAxr
tjrtjjyAxtjAtjrsr
==+
++=
+≡++≡++=
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.90
Rice distributionBy following similar steps described in Rayleigh distribution, we obtain:
( )2 2
02 2 2
2
0 2 20
exp for A 0,r 0 ( ) 2
0 for 0where
1 cos exp2
is the modified zeroth
r r r
r r
r r A ArIp r
(r )
Ar ArI dπ
σ σ σ
θ θσ π σ
⎧ ⎛ ⎞ ⎛ ⎞+− ≥ ≥⎪ ⎜ ⎟ ⎜ ⎟= ⎨ ⎝ ⎠ ⎝ ⎠
⎪ <⎩
⎛ ⎞ ⎛ ⎞=⎜ ⎟ ⎜ ⎟
⎝ ⎠ ⎝ ⎠∫
00
-order Bessel function.
! 2
i
ii
xI (x)i
∞
=
⎛ ⎞= ⎜ ⎟⋅⎝ ⎠∑
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.91
Rice distributionThe Rice distribution is often described in terms of a parameter K which is defined as the ratio between the deterministic signal power and the variance of the multi-path. It is given by K=A2/(2σ2) or in terms of dB:
The parameter K is known as the Rice factor and completely specifies the Rice distribution.As A 0, K -∞ dB, and as the dominant path decreases in amplitude, the Rice distribution degenerates to a Rayleighdistribution.
[dB] 2
log10)( 2
2
σAdBk ⋅=
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.92
Rice distribution
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.93
Rice distribution
[ ]
[ ][ ] [ ]
∑
∑
∑
=
=
=
=
+≡++≡++=
+≡
+⎥⎦
⎤⎢⎣
⎡=
++=
n
iii
n
iii
n
iiir
jajr
tjrtjjyAxtjAtjr
tjAtjjr
tjAtjja
tjAtjas
1
00
00
00
001
01
0
)exp()exp(' where
)(exp)exp()( )exp()(exp'
)exp()exp()exp('
)exp()()exp(
)exp()(exp
θθ
θωωωθω
ωωθ
ωωθ
ωθω
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.94
Rice distribution
Because (1) n is usually very large, (2) the individual amplitudes ai are random, and (3) the phases θi have a uniform distribution, it can be assumed that (from the central limit theorem) x and y are both Gaussian variables with means equal to zero and variance:
θθ
θθ
θθθ
sin cos )( and
sin and cos :have We
sincos)exp(' :Define
22211
11
ryrAxyAxr
ayax
jyxajajr
n
iii
n
iii
n
iii
n
iii
==+++=
≡≡
+≡+=
∑∑
∑∑
==
==
222 σσσ ≡= yx
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.95
Rice distributionBecause x and y are independent random variables, the joint distribution p(x,y) is
The distribution p(r,θ) can be written as a function of p(x,y) :
⎟⎟⎠
⎞⎜⎜⎝
⎛ +−== 2
22
2 2exp
21)()(),(
σπσyxypxpyxp
rrr
yryxrx
J
yxpJrp
=−
=∂∂∂∂∂∂∂∂
≡
=
θθθθ
θθ
θ
cossinsincos
////
),(),(
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.96
Rice distribution
⎟⎠⎞
⎜⎝⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛ +−=
⎟⎟⎠
⎞⎜⎜⎝
⎛ −+−=
⎟⎟⎠
⎞⎜⎜⎝
⎛ +−−=
⎟⎟⎠
⎞⎜⎜⎝
⎛ +−=
22
22
2
22
2
22
2
22
cosexp21
2exp
2cos2exp
2
2)sin()cos(exp
2
2exp
2),(
σθ
πσσ
σθ
πσ
σθθ
πσ
σπσθ
ArArr
ArArr
rArr
yxrrp
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.97
Rice distribution
⎪⎩
⎪⎨⎧
≥⎟⎠⎞
⎜⎝⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛ +−=
=
∫
∫
otherwise 0
0 cosexp21
2exp
),()(
:bygiven (pdf)function density y probabilitahason distributi RiceThe
2
022
22
2
2
0
rdArArr
drprp
π
π
θσ
θπσσ
θθ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.98
Rice distributionJust as the Rayleigh distribution is related to the central chi-square distribution, the Rice distribution is related to the non-central chi-square distribution.
2 21 2 1 2
2
To illustrate this relation, let , where and are statistically independent Gaussian random variables withmeans , 1, 2, and common variance .i
Y X X X X
m i σ
= +
=
( ) ( )2 2
2 2 21 2
/ 202 2
has a non-central chi-square distribution with non-centrality parameter . The PDF of Y, obtainedfrom Equation 2.1-118 for n 2, is:
1 2 , 02
s yY
Ys m m
p y e I y yσ
σ σ− +
= +=
⎛ ⎞= ≥⎜ ⎟⎝ ⎠
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.99
Rice distribution
This PDF characterizes the statistic of the envelope of a signal corrupted by additive narrowband Gaussian noise.It is also used to model the signal statistics of signals transmitted through some radio channels.The CDF of R is obtained from Equation 2.1-124:
( ) ( ) 0 ,
:is variable,of change simple aby 140-2.1Equation fromobtained , of PDF The . variblerandom new a Define
202/
2
222
≥⎟⎠⎞
⎜⎝⎛=
=
+− rrsIerrp
RyR
srR σσ
σ
( ) 123)-2.1in given is (Q 0 ,,1 11 ≥⎟⎠⎞
⎜⎝⎛−= rrsQrFR σσ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.100
Generalized Rice distribution
. toequal variancesidentical
and ,,...,2,1, means with variablesrandomGaussian
tindependenlly statistica are ,...,2,1, where,
2
1
2
σ
nim
niXXR
i
i
n
ii
=
== ∑=
( ) ( )( ) 0 ,
:118-2.1Equation by given is PDF Its 119.-2.1Equation by given parameter
centrality-non and freedom of degreesn on with distributisquare-chi central-non a has variablerandom The
212/2/
2/22
2/
2
2
222
≥⎟⎠⎞
⎜⎝⎛=
=
−+−
− rrsIesrrp
s
YR
nsr
n
n
R σσσ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.101
Generalized Rice distributionThe CDF is:
In the special case when m=n/2 is an integer, we have:
The k-th moment of R is:
( ) ( ) ( ) ( ) ( )( ) 121.-2.1Equation by given is where
2
22
rF
rFrYPryPrRPrF
Y
YR =≤=≤=≤=
( ) 0 ,,1 ≥⎟⎠⎞
⎜⎝⎛−= rrsQrF mR σσ
( ) ( )( )
0 ;2
;2
,2
21
21
2 2
2
112/2/2 22
≥⎟⎟⎠
⎞⎜⎜⎝
⎛ +
⎟⎠⎞
⎜⎝⎛Γ
⎥⎦⎤
⎢⎣⎡ +Γ
= − ksnknFn
kneRE skk
σσ σ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.102
Nakagami m-distributionFrequently used to characterize the statistics of signals transmitted through multi-path fading channels.PDF is given by Nakagami (1960) as:
( ) ( )( )
( )
figure. fading thecalled moments, of ratio theas defined is mparameter The
21 ,
][
2
22
2
2
/12 2
≥Ω−
Ω=
=Ω
⎟⎠⎞
⎜⎝⎛ΩΓ
= Ω−−
mRE
m
RE
ermm
rp mrmm
R
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.103
Nakagami m-distributionThe n-th moment of R is:
By setting m=1, the PDF reduces to a Rayleigh PDF.
( ) ( )( )
2/2/ nn
mmnmRE ⎟
⎠⎞
⎜⎝⎛ Ω
Γ+Γ
=
PDF for the Nakagami m-distribution.Shown with Ω=1.
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.104
Lognormal distribution:
The PDF of R is given by:
The lognormal distribution is suitable for modeling the effect of shadowing of the signal due to large obstructions, such as tall buildings, in mobile radio communications.
. varianceandmean with ddistributenormally is where,lnLet
2σ
mXRX =
( )( ) ( )
( )
2 2ln / 21 02
0 0
r me rp r r
r
σ
πσ− −⎧ ≥⎪= ⎨
⎪ <⎩
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.105
Multivariate Gaussian distributionAssume that Xi, i=1,2,…,n, are Gaussian random variables with means mi, i=1,2,…,n; variances σi
2, i=1,2,…,n; and covariances μij, i,j=1,2,…,n. The joint PDF of the Gaussian random variables Xi, i=1,2,…,n, is defined as
M denotes the n × n covariance matrix with elements μij;x denotes the n × 1 column vector of the random variables;mx denote the n × 1 column vector of mean values mi, i=1,2,…,n.M-1 denotes the inverse of M.x’ denotes the transpose of x.
( )( ) ( )
( ) ( )⎥⎦⎤
⎢⎣⎡ −′−−= −
xxnnxxxp mxMmxM
1
21exp
det21,...,, 2/12/21 π
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.106
Multivariate Gaussian distribution (cont.)Given v the n-dimensional vector with elements υi, i=1,2,…,n, the characteristic function corresponding to the n-dimentional joint PDF is:
( ) ( ) ⎟⎠⎞
⎜⎝⎛ ′−′== ′ Mvvvmv xv
21exp X
j jeEjψ
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.107
Bi-variate or two-dimensional GaussianThe bivariate Gaussian PDF is given by:
( )
( ) ( )( ) ( )( )( )( )
1 2 21 2
2 22 22 1 1 1 2 1 1 2 2 1 2 2
2 2 21 2
21 1 12
12 1 1 2 222 12 2
21 1 2
21 2 2
1,2 1
2 exp
2 1
, ,
, , 0 1
X
ijij ij
i j
p x x
x m x m x m x m
mE x m x m
m
i j
πσ σ ρ
σ ρσ σ σπσ σ ρ
σ µµ
µ σµ
ρ ρσ σσ ρσ σ
ρσ σ σ
=−
⎡ ⎤− − − − + −⎢ ⎥× −
−⎢ ⎥⎣ ⎦⎡ ⎤⎡ ⎤
= = = − −⎡ ⎤⎢ ⎥⎢ ⎥ ⎣ ⎦⎣ ⎦ ⎣ ⎦
= ≠ ≤ ≤
⎡ ⎤= ⎢⎣ ⎦
m M
M( )
21 2 1 2
22 2 21 2 11 2
1, 1
σ ρσ σρσ σ σσ σ ρ
− ⎡ ⎤−=⎥ ⎢ ⎥−− ⎣ ⎦
M
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.108
Bi-variate or two-dimensional Gaussianρ is a measure of the correlation between X1 and X2.When ρ=0, the joint PDF p(x1,x2) factors into the product p(x1)p(x2), where p(xi), i=1,2, are the marginal PDFs.When the Gaussian random variables X1 and X2 are uncorrelated, they are also statistically independent. This property does not hold in general for other distributions.This property can be extended to n-dimensional Gaussian random variables: if ρij=0 for i≠j, then the random variables Xi, i=1,2,…,n, are uncorrelated and, hence, statistically independent.
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.109
Linear transformation of n Gaussian random variablesLet Y=AX , X=A-1Y, where A is a nonsingular matrix.The Jacobian of this transformation is J=1/det A.The joint PDF of Y is:
The linear transformation of a set of jointly Gaussian random variables results in another set of jointly Gaussian random variables.
( )( ) ( )
( ) ( )
( ) ( )( ) ( )
AAMQAmm
myQmyQ
myAMmyAAM
y
1
1
′==
⎥⎦⎤
⎢⎣⎡ −′−−=
⎥⎦⎤
⎢⎣⎡ −
′−−=
−
−−−
and where21exp
det21
21exp
det det21
y
1/22/
112/12/
x
yyn
xxnp
π
π
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.110
Performing a linear transformation that results in nstatistically independent Gaussian random variables
Gaussian random variables are statistically independent if they are pairwise-uncorrelated, i.e., if the covariance matrix Q is diagonal: Q=AMA’=D, where D is a diagonal matrix.Since M is the covariance matrix, one solution is to select A to be an orthogonal matrix (A’=A-1) consisting of columns that are the eigenvectors of the covariance matrix M. Then, D is a diagonal matrix with diagonal elements equal to the eigenvalues of M.To find the eigenvalues of M, solve the characteristic equation: det(M-λI)=0.
2.1.4 Some Useful Probability Distributions
WITS Lab, NSYSU.111
2.1.5 Upper Bounds on the Tail Probability
Chebyshev inequalitySuppose X is an arbitrary random variable with finite mean mxand finite variance σx
2. For any positive number δ:
Proof:
( ) 2
2
δσδ x
xmXP ≤≥−
2 2 2
| |
2 2
| |
( ) ( ) ( ) ( )
( ) (| | ) x
x
x x xx m
xx m
x m p x dx x m p x dx
p x dx P X m
δ
δ
σ
δ δ δ
∞
−∞ − ≥
− ≥
= − ≥ −
≥ = − ≥
∫ ∫∫
WITS Lab, NSYSU.112
Chebyshev inequalityIt is apparent that the Chebyshev inequality is simply an upper bound on the area under the tails of the PDF p(y), where Y=X-mx, i.e., the area of p(y) in the intervals (-∞,δ) and (δ,∞).Chebyshev inequality may be expressed as:
2
2
2
2
1 [ ( ) ( )]
or, equivalently, as
1 [ ( ) ( )]
xY Y
xX x X x
F F
F m F m
σδ δδ
σδ δδ
− − − ≤
− + − − ≤
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.113
Chebyshev inequalityAnother way to view the Chebyshev bound is working with the zero mean random variable Y=X-mx.Define a function g(Y) as:
Upper-bound g(Y) by the quadratic (Y/δ)2, i.e.
The tail probability
( ) ( )( ) ( )[ ] ( )δ
δδ
≥=⎩⎨⎧
<≥
= YPYgEYg and Y 0Y 1
( )2
⎟⎠⎞
⎜⎝⎛≤δYYg
( )[ ] ( )2
2
2
2
2
2
2
2
δσ
δσ
δδxyYEYEYgE ===⎟⎟
⎠
⎞⎜⎜⎝
⎛≤
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.114
Chebychev inequalityA quadratic upper bound on g(Y) used in obtaining the tail probability (Chebyshev bound)
For many practical applications, the Chebyshev bound is extremely loose.
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.115
Chernoff boundThe Chebyshev bound given above involves the area under the two tails of the PDF. In some applications we are interested only in the area under one tail, either in the interval (δ, ∞) or in the interval (-∞, δ).In such a case, we can obtain an extremely tight upper bound by over-bounding the function g(Y) by an exponential having a parameter that can be optimized to yield as tight an upper bound as possible.Consider the tail probability in the interval (δ, ∞).
( ) ( ) ( ) ( ) ( )( )
1 and is defined as
0 where 0 is the parameter to be optimized.
v Y Y δg Y e g Y g Y
Y δv
δ− ≥⎧⎪≤ = ⎨ <⎪⎩≥
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.116
Chernoff bound
The expected value of g(Y) is
This bound is valid for any υ ≥0.
( )[ ] ( ) ( )( )δδ −≤≥= YveEYPygE
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.117
Chernoff boundThe tightest upper bound is obtained by selecting the value of that minimizes E(eυ(Y-δ)).A necessary condition for a minimum is:
( )( ) 0=−δYveEdvd
( ) ( )
( )
( )
( ) ( )
[( ) ] [ ( ) ( )] 0
v Y v Y
v Y
v v Y vY
d dE e E edv dv
E Y ee E Ye E e
δ δ
δ
δ δ
δ
δ
− −
−
− −
=
= −
= − =
( ) ( ) 0 =−→ vYvY eEYeE δ
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.118
Chernoff bound
An upper bound on the lower tail probability can be obtained in a similar manner, with the result that
( ) ( )Yvv eEeYP
v
ˆˆ :isy probabilit
tailsided-one on the boundupper thesolution, thebe ˆLet
δδ −≤≥
( ) ( ) 0 ˆˆ <≤≤ − δδ δ Yvv eEeYP
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.119
Chernoff boundExample 2.1-6: Consider the (Laplace) PDF p(y)=e-|y|/2.
The true tail probability is:
( ) δ
δδ −∞ − ==≥ ∫ edyeYP y
21
21
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.120
Chernoff boundExample 2.1-6 (cont.)
( )( ) ( )
( ) ( )( )( ) ( )
( ) ( )( ) ⎟
⎠⎞
⎜⎝⎛≤≥>>
++−≤≥⇒
++−=
=−+=−
−+=
−+=
−
+−
bound Chebyshevfor 1 2
:1for
112
positive) bemust ˆ( 11ˆ
02obtain we,0 Since
111
112
2
11
2
2
2
2
22
2
δδδδ
δδδ
δδ
δδδ
δ
δ
eYP
eYP
vv
vveEYeE
vveE
vvvYeE
vYvY
vYvY
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.121
Chernoff bound for random variable with a nonzero mean:
Let Y=X-mx, we have:
For δ<0
( ) ( ) ( ) ( )mxx XPmXPmXPYP δδδδ ≥≡+≥=≥−=≥
( ) ( )( )( ) ( )
( ) ( )ˆ ˆ
1 Define:
0
and upper-bounded as: We can obtain the result:
m
m
m
m
v X
v vXm
Xg X
X
g X eP X e E e
δ
δ
δδ
δ
−
−
≥⎧⎪= ⎨ <⎪⎩≤
≥ ≤
( ) ( ) ( ) ( )( )( ) ( )Xvv
m
Xvmxx
eEeXP
eEXPmXPmXPm
m
ˆˆ δ
δ
δ
δδδ−
−
≤≤⇒
≤≤≡+≤=≤−
2.1.5 Upper Bounds on the Tail Probability
WITS Lab, NSYSU.122
2.1.6 Sums of Random Variables and the Central Limit Theorem
Sum of random variablesSuppose that Xi, i=1,2,…,n, are statistically independent and identically distributed random variables, each having a finite mean mx and a finite variance σx
2. Let Y be defined as the normalized sum, called the sample mean:
The mean of Y is
∑=
=n
iiX
nY
1
1
( ) ( )
x
n
iiy
m
XEn
mYE
=
== ∑=
11
WITS Lab, NSYSU.123
Sum of random variablesThe variance of Y is:
An estimate of a parameter (in this case the mean mx) that satisfies the conditions that its expected value converges to the true value of the parameter and the variance converges to zero as n→∞ is said to be a consistent estimate.
( ) ( ) ( )
( ) ( ) ( )
( )[ ] ( )nσmmnn
nmσn
n
mXEXEn
XEn
mXXEn
mYEmYEσ
xxxxx
x
n
i
n
jij
ji
n
ii
n
i
n
jxjixyy
222
222
2
2
1 12
1
22
1 1
22
22222
111
11
1
=−−++=
−+=
−=−=−=
∑∑∑
∑∑
=≠==
= =
2.1.6 Sums of Random Variables and the Central Limit Theorem
WITS Lab, NSYSU.124
Weak law of large numbersThe tail probability of Y can be upper-bounded.The Chebyshev inequality applied to Y is:
In the limit as n→∞, the above equation becomes:
The probability that the estimate of the mean differs from the true mean mx by more than δ(δ>0) approaches zero as n approaches infinity. The upper bound converges to zero relatively slowly, i.e., inversely with n.
( ) 2
2
12
2 1 δσδ
δσ
δn
mXn
PmYP xx
n
ii
yy ≤⎟⎟
⎠
⎞⎜⎜⎝
⎛≥−⇒≤≥− ∑
=
01lim1
=⎟⎟⎠
⎞⎜⎜⎝
⎛≥−∑
=∞→
δx
n
iin
mXn
P
2.1.6 Sums of Random Variables and the Central Limit Theorem
WITS Lab, NSYSU.125
The Chernoff bound
( )
( ) ( )[ ]( ). theof oneany is :Note
expexp
.0 and where
exp
1
1
11
11
1
i
nvXvn
i
vXvn
n
ii
vnm
n
ii
xm
m
n
iim
n
ii
x
n
iiy
XX
eEeeEe
XvEenXvE
m
nXvEnXP
mXn
PmYP
mim
m
δδ
δδ
δδδ
δδ
δδ
−
=
−
=
−
=
==
=
==
⎥⎦
⎤⎢⎣
⎡⎟⎠
⎞⎜⎝
⎛=
⎪⎭
⎪⎬⎫
⎪⎩
⎪⎨⎧
⎥⎦
⎤⎢⎣
⎡⎟⎠
⎞⎜⎝
⎛−
>+=
⎪⎭
⎪⎬⎫
⎪⎩
⎪⎨⎧
⎥⎦
⎤⎢⎣
⎡⎟⎠
⎞⎜⎝
⎛−≤⎟
⎠
⎞⎜⎝
⎛≥=
⎟⎠
⎞⎜⎝
⎛≥−=≥−
∏
∑∑
∑∑
∑
2.1.6 Sums of Random Variables and the Central Limit Theorem
(*)
WITS Lab, NSYSU.126
The Chernoff bound (cont.)The parameter υ that yields the tightest upper bound is obtained by differentiating Equation (*) and setting the derivative equal to zero. This yields the equation:
The bound on the upper tail probability is
In a similar manner, the lower tail probability is upper-bounded as:
( ) ( ) 0=− vXm
vX eEXeE δ
( )[ ] xmnXvv
m
n
ii meEeX
nP m >≤⎟
⎠
⎞⎜⎝
⎛≥ −
=∑ δδ δ ,1 ˆˆ
1
( ) ( )[ ] xmnXvv
m mδeEeYP m <≤≤ − ,ˆˆδδ
2.1.6 Sums of Random Variables and the Central Limit Theorem
WITS Lab, NSYSU.127
Central limit theoremConsider the case in which the random variables Xi, i=1,2,…,n, being summed are statistically independent and identically distributed, each having a finite mean mx and a finite variance σx
2.Define a normalized random variable Ui with a zero mean and unit variance.
Let
We wish to determine the CDF of Y in the limit as n→∞.
nimXUx
xii ,...,2,1 , =
−=
σ
nce.unit varia andmean zero has that Note .11
YUn
Yn
ii∑
=
=
2.1.6 Sums of Random Variables and the Central Limit Theorem
WITS Lab, NSYSU.128
Central limit theoremThe characteristic function of Y is:
Where U denotes any of the Ui, which are identically distributed.
( ) ( ) 1
1
exp
i
n
ijvY i
Y
n
Ui
n
U
jv Ujv E e E
n
jvψn
jvn
ψ
ψ
=
=
⎡ ⎤⎛ ⎞⎢ ⎥⎜ ⎟⎢ ⎥⎜ ⎟= =⎢ ⎥⎜ ⎟
⎜ ⎟⎢ ⎥⎝ ⎠⎣ ⎦⎛ ⎞= ⎜ ⎟⎝ ⎠
⎡ ⎤⎛ ⎞= ⎢ ⎥⎜ ⎟⎝ ⎠⎣ ⎦
∑
∏
2.1.6 Sums of Random Variables and the Central Limit Theorem
WITS Lab, NSYSU.129
Central limit theoremExpand the characteristic function of U in a Taylor series:
( ) ( ) ( )( )
( )
( ) ( )
( )
( )
322 3
3
2
2
1 ...2! 3!
Since 0,and 1, the above equation simplifies to:
1 1 ,2
where denotes the remainder. We note that
U
U
jvv v vj j E U E U E Unn n n
E U E U
v vj R v nn nn
R v,n /n R v
ψ
ψ
⎛ ⎞ = + − + −⎜ ⎟⎝ ⎠
= =
⎛ ⎞ = − +⎜ ⎟⎝ ⎠
( )
( ) ( ) ( ) ( )2 2
approaches zero as .
, ,1 ln ln 1
2 2
n
Y Y
,nn
R v n R v nv vjv jv nn n n n
ψ ψ
→∞
⎡ ⎤ ⎡ ⎤= − + ⇒ = − +⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
2.1.6 Sums of Random Variables and the Central Limit Theorem
WITS Lab, NSYSU.130
Central limit theorem
The above equation is the characteristic function of a Gaussian random variable with zero mean and unit variance.We can conclude that the sum of statistically independent and identically distributed random variables with finite mean and variance approaches a Gaussian CDF as n→∞.
( )
( ) ( ) ( )
( ) ( ) 2/Yn
2
222
32
2
lim 21lnlim
...,22
1,2
ln
...31
211ln , of valuesmallFor
vYn
Y
ejvvjv
nnvR
nv
nnvR
nvnjv
xxxxx
−
∞→∞→=⇒−=
⎥⎥⎦
⎤
⎢⎢⎣
⎡+⎟⎟
⎠
⎞⎜⎜⎝
⎛+−−+−=
−+−=+
ψψ
ψ
2.1.6 Sums of Random Variables and the Central Limit Theorem
WITS Lab, NSYSU.131
Central limit theoremAlthough we assumed that the random variables in the sum are identically distributed, the assumption can be relaxed provided that additional restrictions are imposed on the properties of the random variables.If the random variables Xi are i.i.d., sum of n = 30 random variables is adequate for most applications.If the functions fi(x) are smooth, values of n as low as 5 can be used.For more information, referred to Cramer (1946) and Papoulis “Probability, Random Variables, and Stochastic Processes”, pp. 214-221, 3rd Edition.
2.1.6 Sums of Random Variables and the Central Limit Theorem
WITS Lab, NSYSU.132
2.2 Stochastic Processes
Many of random phenomena that occur in nature are functions of time.In digital communications, we encounter stochastic processes in:
The characterization and modeling of signals generated by information sources;The characterization of communication channels used to transmit the information;The characterization of noise generated in a receiver;The design of the optimum receiver for processing the received random signal.
WITS Lab, NSYSU.133
2.2 Stochastic Processes
IntroductionAt any given time instant, the value of a stochastic process is a random variable indexed by the parameter t. We denote such a process by X(t).In general, the parameter t is continuous, whereas X may be either continuous or discrete, depending on the characteristics of the source that generates the stochastic process.The noise voltage generated by a single resistor or a single information source represents a single realization of the stochastic process. It is called a sample function.
WITS Lab, NSYSU.134
2.2 Stochastic ProcessesIntroduction (cont.)
The set of all possible sample functions constitutes an ensemble of sample functions or, equivalently, the stochastic process X(t).In general, the number of sample functions in the ensemble is assumed to be extremely large; often it is infinite.Having defined a stochastic process X(t) as an ensemble of sample functions, we may consider the values of the process at any set of time instants t1>t2>t3>…>tn, where n is any positive integer.
( )( ).,...,, PDFjoint by their lly statistica zedcharacteri
are ,,...,2,1, variablesrandom thegeneral,In
21 n
i
ttt
it
xxxp
nitXX =≡
WITS Lab, NSYSU.135
2.2 Stochastic Processes
Stationary stochastic processes
When the joint PDFs are different, the stochastic process is non-stationary.
( )
( )1 2
Consider another set of random variables ,
1, 2,..., , where is an arbitrary time shift. These random
variables are characterized by the joint PDF , ,..., .
i
n
t t i
t t t t t t
n X X t t
i n t
p x x x
+
+ + +
≡ +
=
( ) ( ).sensestrict in the stationary be tosaid isit , all and allfor
,...,,,...,, wheni.e., identical, arey When theidentical. benot may or may
,21 and variablesrandom theof PDFsjont The
2121
nt
xxxpxxxp
,...,n,,iXX
ttttttttt
ttt
nn
ii
+++
+
=
=
WITS Lab, NSYSU.136
2.2.1 Stochastic ProcessesAverages for a stochastic process are called ensemble averages.
( ) ( )∫∞
∞−=
iiii
i
ttnt
nt
t
dxxpxXE
Xn
:as defined is variablerandom theof momentth The
.on depends of PDF theif instant timeon the depend willmomentth theof value thegeneral,In
iti tXtn
i ( ) ( )
time.oft independen is momentth thee,consequenca as and, time,oft independen is PDF theTherefore,
. allfor ,stationary is process When the
n
txpxpii ttt =+
WITS Lab, NSYSU.137
2.2.1 Stochastic ProcessesTwo random variables:
The correlation is measured by the joint moment:
Since this joint moment depends on the time instants t1 and t2, it is denoted by φ(t1 ,t2).φ(t1 ,t2) is called the autocorrelation function of the stochastic process.For a stationary stochastic process, the joint moment is:
Average power in the process X(t): φ(0)=E(Xt2).
( ) , 1, 2.it iX X t i≡ =
1 2 1 2 1 2( ) ( , ) ( ) ( )t tE X X t t t tφ φ φ τ= = − =
( ) ( )1 2 1 2 1 2 1 2,t t t t t t t tE X X x x p x x dx dx
∞ ∞
−∞ −∞= ∫ ∫
' '1 1 1 1 1 1
( ) ( ) ( ) ( ) ( )t t t t t tE X X E X X E X Xτ τ τ
φ τ φ τ+ + −− = = = =
WITS Lab, NSYSU.138
2.2.1 Stochastic Processes
Wide-sense stationary (WSS)A wide-sense stationary process has the property that the mean value of the process is independent of time (a constant) and where the autocorrelation function satisfies the condition that φ(t1,t2)=φ(t1-t2).Wide-sense stationarity is a less stringent condition than strict-sense stationarity.If not otherwise specified, any subsequent discussion in which correlation functions are involved, the less stringent condition (wide-sense stationarity) is implied.
WITS Lab, NSYSU.139
2.2.1 Stochastic Processes
Auto-covariance functionThe auto-covariance function of a stochastic process is defined as:
When the process is stationary, the auto-covariance function simplifies to:
For a Gaussian random process, higher-order moments can be expressed in terms of first and second moments. Consequently, a Gaussian random process is completely characterized by its first two moments.
[ ][ ] )()(),(
)()(),(2121
2121 21
tmtmtttmXtmXEtt tt
−=−−=
φµ
22121 )()()(),( mtttt −==−= τφτµµµ
WITS Lab, NSYSU.140
2.2.1 Stochastic ProcessesAverages for a Gaussian process
Suppose that X(t) is a Gaussian random process. At time instants t=ti, i=1,2,…,, the random variables Xti, i=1,2,…,n, are jointly Gaussian with mean values m(ti), i=1,2,…,n, and auto-covariances:
If we denote the n × n covariance matrix with elements μ(ti,tj) by M and the vector of mean values by mx, the joint PDF of the random variables Xti, i=1,2,…,n, is given by:
If the Gaussian process is wide-sense stationary, it is also strict-sense stationary.
( )( )[ ] njitmXtmXEtt jtitji ii,......,2,1, , )()(),( =−−=µ
( )( ) ( )
( ) ( )⎥⎦⎤
⎢⎣⎡ −′−−= −
xxnnxxxp mxMmxM
1
21exp
det21,...,, 2/12/21 π
WITS Lab, NSYSU.141
2.2.1 Stochastic ProcessesAverages for joint stochastic processes
Let X(t) and Y(t) denote two stochastic processes and let Xti≣X(ti), i=1,2,…,n, Yt’j
≣Y(t’j), j=1,2,…,m, represent the random variables at times t1>t2>t3>…>tn, and t’1>t’2>t’3>…>t’m , respectively. The two processes are characterized statistically by their joint PDF:
The cross-correlation function of X(t) and Y(t), denoted by φxy(t1,t2), is defined as the joint moment:
The cross-covariance is:
( )''2
'1,21
,...,, ,...,,mn tttttt yyyxxxp
∫ ∫∞
∞−
∞
∞−==
21212121),()(),( 21 ttttttttxy dydxyxpyxYXEtt φ
)()(),(),( 212121 tmtmtttt yxxyxy −= φµ
WITS Lab, NSYSU.142
2.2.1 Stochastic ProcessesAverages for joint stochastic processes
When the process are jointly and individually stationary, we have φxy(t1,t2)=φxy(t1-t2), and μxy(t1,t2)= μxy(t1-t2):
The stochastic processes X(t) and Y(t) are said to be statistically independent if and only if :
for all choices of ti and t’i and for all positive integers n and m.The processes are said to be uncorrelated if
' ' ' '1 1 1 1 1 1( ) ( ) ( ) ( ) ( )xy t t yxt t t t
E X Y E X Y E Y Xτ τ τφ τ φ τ+ − −
− = = = =
),...,,(),...,,(),...,,,,...,,( ''2
'121''
2'121 mnmn tttttttttttt yyypxxxpyyyxxxp =
⇒= )()(),(2121 ttxy YEXEttφ 0),( 21 =ttxyµ
WITS Lab, NSYSU.143
2.2.1 Stochastic ProcessesComplex-valued stochastic process
A complex-valued stochastic process Z(t) is defined as:
where X(t) and Y(t) are stochastic processes.The joint PDF of the random variables Zti≣Z(ti), i=1,2,…,n, is given by the joint PDF of the components (Xti, Yti), i=1,2,…,n. Thus, the PDF that characterizes Zti, i=1,2,…,n, is:
The autocorrelation function is defined as:
)()()( tjYtXtZ +=
),...,, , ,...,,(2121 nn tttttt yyyxxxp
( )( )
1 2 1 1 2 21 2
1 2 1 2 1 2 1 2
1 1( , ) ( )2 21 ( , ) ( , ) ( , ) ( , ) 2
zz t t t t t t
xx yy yx xy
t t E Z Z E X jY X jY
t t t t j t t t t
φ
φ φ φ φ
∗ ⎡ ⎤≡ = + −⎣ ⎦
⎡ ⎤= + + −⎣ ⎦
(**)
WITS Lab, NSYSU.144
2.2.1 Stochastic ProcessesAverages for joint stochastic processes:
When the processes X(t) and Y(t) are jointly and individually stationary, the autocorrelation function of Z(t) becomes:
φZZ(τ)= φ*ZZ(-τ) because from (**):
)()(),( 2121 τφφφ zzzzzz tttt =−=
' ' ' '1 1 1 1 1 1
1 1 1( ) ( ) ( ) ( ) ( )2 2 2zz t t zzt t t t
E Z Z E Z Z E Z Zτ τ τφ τ φ τ∗ ∗ ∗ ∗
− + += = = = −
WITS Lab, NSYSU.145
2.2.1 Stochastic ProcessesAverages for joint stochastic processes:
Suppose that Z(t)=X(t)+jY(t) and W(t)=U(t)+jV(t) are two complex-valued stochastic processes. The cross-correlation functions of Z(t) and W(t) is defined as:
When X(t), Y(t),U(t) and V(t) are pairwise-stationary, the cross-correlation function become functions of the time difference.
( )( )
1 2
1 1 2 2
1 2
1 2 1 2 1 2 1 2
1( , ) ( )2
1 21 ( , ) ( , ) ( , ) ( , ) 2
zw t t
t t t t
xu yu yu xu
t t E Z W
E X jY U jV
t t t t j t t t t
φ
φ φ φ φ
∗≡ ⋅
⎡ ⎤= + −⎣ ⎦
⎡ ⎤= + + −⎣ ⎦
' ' ' '1 1 1 1
1 1 1( ) ( ) ( ) ( ) ( )2 2 2i i
zw t t wzt t t tE Z W E Z W E W Zτ τ τ
φ τ φ τ∗ ∗ ∗ ∗− + +
= = = = −
WITS Lab, NSYSU.146
2.2.2 Power Density SpectrumA signal can be classified as having either a finite (nonzero) average power (infinite energy) or finite energy.The frequency content of a finite energy signal is obtained as the Fourier transform of the corresponding time function.If the signal is periodic, its energy is infinite and, consequently, its Fourier transform does not exist. The mechanism for dealing with periodic signals is to represent them in a Fourier series.
WITS Lab, NSYSU.147
2.2.2 Power Density SpectrumA stationary stochastic process is an infinite energy signal, and, hence, its Fourier transform does not exist.The spectral characteristic of a stochastic signal is obtained by computing the Fourier transform of the autocorrelation function.The distribution of power with frequency is given by the function:
The inverse Fourier transform relationship is:
( ) ( )∫∞
∞−
−=Φ ττφ τπ def fj2
( ) ( )∫∞
∞−Φ= dfef fj τπτφ 2
WITS Lab, NSYSU.148
2.2.2 Power Density Spectrum
Since φ(0) represents the average power of the stochastic signal, which is the area under Φ(f), Φ(f) is the distribution of power as a function of frequency.Φ(f) is called the power density spectrum of the stochastic process.If the stochastic process is real, φ(τ) is real and even, and, hence Φ(f) is real and even.If the stochastic process is complex, φ(τ)=φ*(-τ) and Φ(f) is real because:
( ) ( ) ( ) 00 2 ≥=Φ= ∫∞
∞− tXEdffφ
( ) ( ) ( )
( ) ( )
* * 2 * 2 '
2
' '
j f j f
j f
f e d e d
e d f
π τ π τ
π τ
φ τ τ φ τ τ
φ τ τ
∞ ∞ −
−∞ −∞
∞ −
−∞
Φ = = −
= = Φ
∫ ∫∫
(2.2-13)
WITS Lab, NSYSU.149
2.2.2 Power Density SpectrumCross-power density spectrum
For two jointly stationary stochastic processes X(t) and Y(t), which have a cross-correlation function φxy(τ), the Fourier transform is:
Φxy(f) is called the cross-power density spectrum.
If X(t) and Y(t) are real stochastic processes
( ) ( )∫∞
∞−
−=Φ ττφ τπ def fjxyxy
2
( ) ( ) ( )
( ) ( )fde
dedef
yxfj
yx
fjxy
fjxyxy
Φ==
−==Φ
∫∫∫
∞
∞−
−
∞
∞−
−∞
∞−
ττφ
ττφττφ
τπ
τπτπ
2
2*2**
( ) ( ) ( )* 2j fxy xy xyf e d fπ τφ τ τ
∞
−∞Φ = = Φ −∫ ( ) ( )yx xyf fΦ = Φ −→
(2.2-15)
WITS Lab, NSYSU.150
2.2.3 Response of a Linear Time-Invariant System to a Random Input Signal
Consider a linear time-invariant system (filter) that is characterized by its impulse response h(t) or equivalently, by its frequency response H(f), where h(t) and H(f) are a Fourier transform pair. Let x(t) be the input signal to the system and let y(t) denote the output signal.
Suppose that x(t) is a sample function of a stationary stochastic process X(t). Since convolution is a linear operation performed on the input signal x(t), the expected value of the integral is equal to the integral of the expected value.
The mean value of the output process is a constant.
( ) ( ) ( )y t h x t dτ τ τ∞
−∞= −∫
( ) ( ) ( )
( ) ( ) 0
y
x x
m E Y t h E X t d
m h d m H
τ τ τ
τ τ
∞
−∞
∞
−∞
= = −⎡ ⎤ ⎡ ⎤⎣ ⎦ ⎣ ⎦
= =
∫∫
WITS Lab, NSYSU.151
2.2.3 Response of a Linear Time-Invariant System to a Random Input Signal
The autocorrelation function of the output is:
If the input process is stationary, the output is also stationary:
( ) ( )
( ) ( ) ( ) ( )[ ]( ) ( ) ( )∫ ∫
∫ ∫∞
∞−
∞
∞−
∞
∞−
∞
∞−
−+−=
−−=
=
βαβαφαβ
βααβαβ
φ
ddtthh
ddtXtXEhh
YYEtt
xx
ttyy
21*
2*
1*
*21
21
21,
21
( ) ( ) ( ) ( )*yy xxh h d dφ τ α β φ τ α β α β
∞ ∞
−∞ −∞= + −∫ ∫
WITS Lab, NSYSU.152
2.2.3 Response of a Linear Time-Invariant System to a Random Input Signal
The power density spectrum of the output process is:
(by making τ0=τ+α-β)The power density spectrum of the output signal is the product of the power density spectrum of the input multiplied by the magnitude squared of the frequency response of the system.
( ) ( ) 2j fyy yyf e dπ τφ τ τ
∞ −
−∞Φ = ∫
( ) ( ) ( )* 2j fxxh h e d d dπ τα β φ τ α β τ α β
∞ ∞ ∞ −
−∞ −∞ −∞= + −∫ ∫ ∫
( ) ( ) 2xx f H f= Φ
WITS Lab, NSYSU.153
When the autocorrelation function φyy(τ) is desired, it is usually easier to determine the power density spectrum Φ yy(f) and then to compute the inverse transform.
The average power in the output signal is:
Since φyy(0)=E(|Yt|2) , we have:
( ) ( ) 2j fyy yy f e dfπ τφ τ
∞
−∞= Φ∫
( ) ( ) 2 2j fxx f H f e dfπ τ∞
−∞= Φ∫
( ) ( ) ( ) 20yy xx f H f dfφ
∞
−∞= Φ∫
( ) ( )∫∞
∞−≥Φ 02 dffHfxx
2.2.3 Response of a Linear Time-Invariant System to a Random Input Signal
valid for any H(f ).
WITS Lab, NSYSU.154
Suppose we let |H(f )|2=1 for any arbitrarily small interval f1≤ f ≤f2 , and H( f )=0 outside this interval. Then, we have:
This is possible if an only if Φxx(f )≥0 for all f.
Conclusion: Φxx(f )≥0 for all f.
( )∫ ≥Φ2
1
0f
f xx dff
2.2.3 Response of a Linear Time-Invariant System to a Random Input Signal
WITS Lab, NSYSU.155
Example 2.2-1Suppose that the low-pass filter is excited by a stochastic process x(t) having a power density spectrum Φxx(f)=N0/2 for all f. A stochastic process having a flat power density spectrum is called white noise. Let us determine the power density spectrum of the output process.The transfer function of the low-pass filter is:
2.2.3 Response of a Linear Time-Invariant System to a Random Input Signal
( )RfLjfLjR
RfH/21
12 ππ +
=+
=
( )( )
2
2 2
11 2 /
H fL R fπ
=+
⇒
WITS Lab, NSYSU.156
Example 2.2-1 (cont.)The power density spectrum of the output process is:
The inverse Fourier transform yields the autocorrelation function:
( )( )
202 2
12 1 2 /yy
j fN e dfL R f
π τφ τπ
∞
−∞=
+∫
2.2.3 Response of a Linear Time-Invariant System to a Random Input Signal
( )( )
02 2
12 1 2 /yy
NfL R fπ
Φ =+
( )/0
4R LRN e
Lτ−=
WITS Lab, NSYSU.157
Cross-correlation function between y(t) and x(t)
With t1-t2=τ, we have:
In the frequency domain, we have:
If the input process is white noise, the cross correlation of the input with the output of the system yields the impulse response h(t) to within a scale factor.
( ) ( ) ( ) ( ) ( )[ ]( ) ( ) ( )
( ) ( ) .stationaryjointly are and processes stochastic The
21
21,
2121
2*
1*
21 21
tYtX
ttdtth
dtXtXEhXYEtt
yxxx
ttyx
−=−−=
−==
∫
∫∞
∞−
∞
∞−
φααφα
αααφ
2.2.3 Response of a Linear Time-Invariant System to a Random Input Signal
( ) ( ) ( )∫∞
∞−−= αατφατφ dh xxyx
( ) ( ) ( )fHff xxyx Φ=Φ
Function of t1-t2
WITS Lab, NSYSU.158
2.2.4 Sampling Theorem for Band-Limited Stochastic Processes
A deterministic signal s(t) that has a Fourier transform S(f) is called band-limited if S(f)=0 for |f|>W, where W is the highest frequency contained in s(t).A band-limited signal is uniquely represented by samples of s(t) taken at a rate of fs≥2W samples/s.The minimum rate fN=2W samples/s is called the Nyquist rate.Sampling below the Nyquist rate results in frequency aliasing.
WITS Lab, NSYSU.159
2.2.4 Sampling Theorem for Band-Limited Stochastic Processes
The band-limited signal sampled at the Nyquist rate can be reconstructed from its samples by use of the interpolation formula:
where s(n/2W) are the samples of s(t) taken at t=n/2W, n=0,±1, ±2,….Equivalently, s(t) can be reconstructed by passing the sampled signal through an ideal low-pass filter with impulse response h(t)=(sin2πWt)/2πWt.
( )⎟⎠⎞
⎜⎝⎛ −
⎥⎦
⎤⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −
⎟⎠⎞
⎜⎝⎛= ∑
∞
−∞=
WntW
WntW
Wnsts
n
22
22sin
2 π
π
WITS Lab, NSYSU.160
2.2.4 Sampling Theorem for Band-Limited Stochastic Processes
The signal reconstruction process based on ideal interpolation can be illustrated in the following figure:
WITS Lab, NSYSU.161
2.2.4 Sampling Theorem for Band-Limited Stochastic Processes
A stationary stochastic process X(t) is said to be band-limited if its power density spectrum Φ(f)=0 for |f|>W.Since Φ(f) is the Fourier transform of the autocorrelation function φ(τ), it follows that φ(τ) can be represented as:
where φ(n/2W) are the samples of φ(τ) taken at τ=n/2W, n=0,±1, ±2,….
( )⎟⎠⎞
⎜⎝⎛ −
⎥⎦
⎤⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −
⎟⎠⎞
⎜⎝⎛= ∑
∞
−∞=
WnW
WnW
Wn
n
22
22sin
2 τπ
τπφτφ
WITS Lab, NSYSU.162
2.2.4 Sampling Theorem for Band-Limited Stochastic Processes
If X(t) is a band-limited stationary stochastic process, then X(t) can be represented as:
where X(n/2W) are the samples of X(t) taken at t=n/2W, n=0,±1, ±2,…. This is the sampling representation for a stationary stochastic process.
( )⎟⎠⎞
⎜⎝⎛ −
⎥⎦
⎤⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −
⎟⎠⎞
⎜⎝⎛= ∑
∞
−∞=
WntW
WntW
WnXtX
n
22
22sin
2 π
π(A)
WITS Lab, NSYSU.163
2.2.4 Sampling Theorem for Band-Limited Stochastic Processes
The samples are random variables that are described statistically by appropriate joint probability density functions.The signal representation in (A) is easily established by showing that:
Equality between the sampling representation and the stochastic process X(t) holds in the sense that the mean square error is zero.
( ) 0
22
22sin
2
2
=
⎪⎪
⎭
⎪⎪
⎬
⎫
⎪⎪
⎩
⎪⎪
⎨
⎧
⎟⎠⎞
⎜⎝⎛ −
⎥⎦
⎤⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −
⎟⎠⎞
⎜⎝⎛− ∑
∞
−∞=n
WntW
WntW
WnXtXE
π
π
WITS Lab, NSYSU.164
2.2.5 Discrete-Time Stochastic Signals and Systems
Discrete-time stochastic process X(n) consisting of an ensemble of sample sequences x(n) are usually obtained by uniformly sampling a continuous-time stochastic process.The mth moment of X(n) is defined as:
The autocorrelation sequence is:
The auto-covariance sequences is:
[ ] ( )∫∞
∞−= nn
mn
mn dXXpXXE
( ) ( ) ( )∫ ∫∞
∞−
∞
∞−== knknknkn dXdXXXpXXXXEkn ,
21
21, **φ
( ) ( ) ( ) ( )*
21,, kn XEXEknkn −= φµ
WITS Lab, NSYSU.165
2.2.5 Discrete-Time Stochastic Signals and Systems
For a stationary process, we have φ(n,k)≣φ(n-k), μ(n,k)≣μ(n-k), and
where mx=E(Xn) is the mean value.A discrete-time stationary process has infinite energy but a finite average power, which is given as:
The power density spectrum for the discrete-time process is obtained by computing the Fourier transform of φ(n).
( ) ( )02 φ=nXE
( ) ( ) 2
21
xmknkn −−=− φµ
( ) ( )∑∞
−∞=
−=Φn
fnjenf πφ 2
WITS Lab, NSYSU.166
2.2.5 Discrete-Time Stochastic Signals and Systems
The inverse transform relationship is:
The power density spectrum Φ(f) is periodic with a period fp=1. In other words, Φ(f+k)=Φ(f) for k=0,±1,±2,….The periodic property is a characteristic of the Fourier transform of any discrete-time sequence.
( ) ( )∫− Φ=21
21
2 dfefn fnj πφ
WITS Lab, NSYSU.167
2.2.5 Discrete-Time Stochastic Signals and Systems
Response of a discrete-time, linear time-invariant system to a stationary stochastic input signal.
The system is characterized in the time domain by its unit sample response h(n) and in the frequency domain by the frequency response H(f).
The response of the system to the stationary stochastic input signal X(n) is given by the convolution sum:
( ) ( )∑∞
−∞=
−=n
fnjenhfH π2
( ) ( ) ( )∑∞
−∞=
−=k
knxkhny
WITS Lab, NSYSU.168
2.2.5 Discrete-Time Stochastic Signals and Systems
Response of a discrete-time, linear time-invariant system to a stationary stochastic input signal.
The mean value of the output of the system is:
where H(0) is the zero frequency [direct current (DC)] gain of the system.
( ) ( ) ( )
( ) ( ) 0
yk
x xk
m E y n h k E x n k
m h k m H
∞
=−∞
∞
=−∞
= = −⎡ ⎤ ⎡ ⎤⎣ ⎦ ⎣ ⎦
= =
∑
∑
WITS Lab, NSYSU.169
2.2.5 Discrete-Time Stochastic Signals and Systems
The autocorrelation sequence for the output process is:
By taking the Fourier transform of φyy(k) and substituting the relation in Equation 2.2-49, we obtain the corresponding frequency domain relationship:
Φyy(f), Φxx(f), and H(f) are periodic functions of frequency with period fp=1.
( ) ( ) ( )
( ) ( ) ( ) ( )
( ) ( ) ( )
12
12
yy
i j
xxi j
k E y n y n k
h i h j E x n i x n k j
h i h j k j i
φ
φ
∗
∞ ∞∗ ∗
=−∞ =−∞
∞ ∞∗
=−∞ =−∞
⎡ ⎤= +⎣ ⎦
⎡ ⎤= − + −⎣ ⎦
= − +
∑ ∑
∑ ∑
( ) ( ) ( ) 2yy xxf f H fΦ = Φ
WITS Lab, NSYSU.170
Cyclostationary Processes
For signals that carry digital information, we encounter stochastic processes with statistical averages that are periodic.Consider a stochastic process of the form:
where an is a discrete-time sequence of random variables with mean ma=E(an) for all n and autocorrelation sequence φaa(k)=E(a*nan+k)/2.The signal g(t) is deterministic.The sequence an represents the digital information sequence that is transmitted over the communication channel and 1/Trepresents the rate of transmission of the information symbols.
( ) ( )nn
X t a g t nT∞
=−∞
= −∑
WITS Lab, NSYSU.171
Cyclostationary ProcessesThe mean value is:
The mean is time-varying and it is periodic with period T.The autocorrelation function of X(t) is:
( ) ( ) ( )
( )
nn
an
E X t E a g t nT
m g t nT
∞
=−∞
∞
=−∞
= −⎡ ⎤⎣ ⎦
= −
∑
∑
( ) ( ) ( )
( ) ( ) ( )
( ) ( ) ( )
12
12
,
xx
n mn m
aan m
t t E X t X t
E a a g t nT g t mT
m n g t nT g t mT
φ τ τ
τ
φ τ
∗
∞ ∞∗ ∗
=−∞ =−∞
∞ ∞∗
=−∞ =−∞
⎡ ⎤+ = +⎣ ⎦
= − + −
= − − + −
∑ ∑
∑ ∑
WITS Lab, NSYSU.172
Cyclostationary ProcessesWe observe that
for k=±1,±2,…. Hence, the autocorrelation function of X(t) is also periodic with period T.Such a stochastic process is called cyclostationary or periodically stationary.Since the autocorrelation function depends on both the variables tand τ, its frequency domain representation requires the use of a two-dimensional Fourier transform.The time-average autocorrelation function over a single period is defined as:
( ) ( ), ,xx xxt kT t kT t tφ τ φ τ+ + + = +
( ) ( )2
2
1 ,T
xx xxTt t dt
Tφ τ φ τ
−= +∫
WITS Lab, NSYSU.173
Cyclostationary ProcessesThus, we eliminate the tie dependence by dealing with the average autocorrelation function.The Fourier transform of φxx(τ) yields the average power density spectrum of the cyclostationary stochastic process.This approach allows us to simply characterize cyclostationaryprocess in the frequency domain in terms of the power spectrum.The power density spectrum is:
( ) ( ) 2j fxx xxf e dπ τφ τ τ
∞ −
−∞Φ = ∫