Stochastic lecture 5

Copyright © Syed Ali Khayam 2009

EE 801 – Analysis of Stochastic Systems

Multiple Random Variables

Muhammad Usman IlyasSchool of Electrical Engineering & Computer ScienceNational University of Sciences & Technology (NUST)Pakistan


In this lecture, we will cover: Joint Distributions of Multiple Random Variables Functions of Multiple Random Variables Moments of Multiple Random Variables Jointly Normal Random Variables Sums of Random Variables

What will we cover in this lecture?


Vector Random VariablesPairs of Random Variables

3


Definition of a Jointly Distributed Random Variable A vector random variable is a mapping from an outcome s of a

random experiment to a vector

: nXZ S S Ì

domain Range or image of X

4


Definition of a Jointly Distributed Random Variable

Random Experiment SX

X, Y(x, y)

2,: X YZ S S Ì

Image courtesy of www.buzzle.com/

1

2

3

4

5

6

654321

SY

5


Joint Distributions The joint cumulative distribution function (cdf) of a pair of rvs X

and Y is defined as:

Joint distributions are also called compound distributions

{ }{ }

, ( , ) Pr

Pr , , ,

X YF a b X a Y b

X a Y b a b

= £ £

= £ £ -¥ < < ¥ -¥ < < ¥

6


Examples of Joint Distributions: Throwing Two Dice Experiment

x

y

pX,Y(x,y)

(1,1)

(6,6)

7


Examples of Joint Distributions: A Joint Gaussian Distribution

8


Joint Distributions

Y

Xb

X≤b

Y≤d

d

9


Joint Distributions

b

X≤b

Y≤d

d

,Integrating shaded region gives ( , )X YF b d

Y

X

10


Properties of Joint CDFs F1:

F2:

F3: If X and Y are continuous rvs then FX,Y(a,b) is also continuous

( ),0 , 1, ,X YF a b a b£ £ -¥ < < ¥

( ) ( )1 2 1 2 , 1 1 , 2 2 and , ,X Y X Ya a b b F a b F a b£ £ £

11


Properties of Joint CDFs F4: ( )

( )

and

,

or

,

, 1

, 0

a b

X Y

a b

X Y

F a b

F a b

¥

-¥

¾¾¾¾¾

¾¾¾¾¾

a

b

,Integrating shaded region gives ( , )X YF a b

Y

X

12


Properties of Joint CDFs

( )and

, , 1a b

X YF a b¥

¾¾¾¾¾

a→∞

bb→∞

Y

X

13



( )and

, , 1a b

X YF a b¥

¾¾¾¾¾

a→∞

b

b→∞ Y

X

14



( )and

, , 1a b

X YF a b¥

¾¾¾¾¾

a→∞

b→∞Y

X

15



F5: ( ) ( ), , a

X Y YF a b F b¥¾¾¾¾

b

These distributions are called marginal distributions

a→∞

( ) ( ), , bX Y XF a b F a¥¾¾¾

Y

X

16



( ) { }{ }{ } { }

{ } ( )

,lim , Pr

Pr

Pr Pr

Pr

a X Y

Y

F a b X Y b

X Y b

Y b X X

Y b F b

¥ = £ ¥ £

= -¥ £ £ ¥ -¥ £ £

= -¥ £ £ -¥ £ £ ¥ -¥ £ £ ¥

= -¥ £ £ =

17



( ) { } { } ( ),lim , Pr Pra X Y YF a b X Y b Y b F b¥ = £ ¥ £ = £ =

b

a→∞

Y

X

18



( ) { } { } ( ),lim , Pr Pra X Y YF a b X Y b Y b F b¥ = £ ¥ £ = £ =

b

a→∞

Y

X

19



b

a→∞

( ) { } { } ( ),lim , Pr Pra X Y YF a b X Y b Y b F b¥ = £ ¥ £ = £ =Y

X

20



F6:

{ } , , , ,Pr and c ( , ) ( , ) ( , ) ( , )X Y X Y X Y X Ya X b Y d F b d F a d F b c F a c< £ < £ = - - +

21



b

X≤b

Y≤d

d

Y

X

22



b

X≤b

Y≤d

d

,Integrating shaded region gives ( , )X YF b d

Y

X

23



b

X≤b

Y≤d

d

c

a

{ }We want Pr ca X b Y d< £ < £

Y

X

24



b

X≤b

Y≤d

d

c

aIntegrating the entire region gives FX,Y(b,d)

Y

X

25



b

X≤b

Y≤d

d

c

a

Firs

t sub

tract

F(a

,d)

from

FX

,Y (b

,d)

Y

X

26



b

X≤b

Y≤d

d

c

a

Then subtract FX,Y (b,c) from FX,Y (b,d)-FX,Y (a,d)

Y

X

27



b

X≤b

Y≤d

d

c

a

Finally add FX,Y (a,c) to FX,Y (b,d)-FX,Y (a,d)-FX,Y (b,c)

{ } , , , ,Pr and c ( , ) ( , ) ( , ) ( , )X Y X Y X Y X Ya X b Y d F b d F a d F b c F a c< £ < £ = - - +

Y

X

28


Joint Probability Density Function The joint probability density function is defined as:

Alternatively:

2

, ,( , ) ( , )X Y X Yf x y F x yx y¶

=¶ ¶

, ,( , ) ( , )yx

X Y X YF x y f a b dadb-¥ -¥

= ò ò

29



b

X≤b

Y≤d

d

c

a

{ } ,Pr c ( , )b d

X Y

a c

a X b Y d f x y dydx< £ < £ = ò ò

Y

X

30


Joint Probability Density Function The joint probability density function must satisfy the following

property:

Moreover:

, ( , ) 1X Yf x y dxdy¥ ¥

-¥ -¥

=ò ò

,

,

( ) ( , )

( ) ( , )

X X Y

Y X Y

f x f x y dy

f y f x y dx

¥

-¥¥

-¥

=

=

ò

ò

31


Homework Reading Assignment: Examples 4.1 to 4.9 in the book

Homework Problem 4.3 in the textbook Problem 4.9 in the textbook

32


Independent Random Variables Two random variables X and Y are independent if

Or:

, ( , ) ( ) ( )X Y X YF x y F x F y=

,

,

( , ) ( ) ( ) for discrete rvs

( , ) ( ) ( ) for continuous rvs

X Y X Y

X Y X Y

p x y p x p y

f x y f x f y

=

=

33


Conditional Probability for Random Variables Conditional probability is an important tool when analyzing

random variables

In general, we are interested in finding the probability that a random variable Y takes on a certain value (or a range of values) given that another random variable X has already taken some value

Conditional probability can be expressed in terms of the joint and marginal probabilities

34


Conditional Probability for Random Variables

Random Experiment

What is the probability that the sum of the two dice is odd given that X=2?

X

Y

35


Conditional Probability for Random Variables We can extend the definition of conditional probability to

random variables

( | ) Pr{( ) | ( )}

Pr{ }

Pr{ }

YF y x Y y X x

Y y X x

X x

= £ =

£ ==

=

36


Conditional Probability for Random Variables We can extend the definition of conditional probability to

random variables

What’s wrong with this definition?

( | ) Pr{( ) | ( )}

Pr{( ) ( )}

Pr{ }

YF y x Y y X x

Y y X x

X x

= £ =

£ ==

=

37



What’s wrong with this definition?

This definition will only work for discrete rvs because Pr{X=a}=0for continuous rvs

Therefore, we need to generalize this definition to encompass continuous rvs

Pr{( ) ( )}( | )

Pr{ }Y

Y y X xF y x

X x

£ ==

=

38



b

0

Pr{( ) ( )}( | ) lim

Pr{ }Y h

Y y x X x hF y x

x X x h

£ < £ +=

< £ +

a

hY

X

39


Conditional Probability for Random Variables We need to generalize the definition of conditional probability

to encompass both discrete and continuous rvs

( )

( )

( )

( )

( )

( )

0

, ,

,

Pr{( ) ( )}( | ) lim

Pr{ }

, ,

,

Y h

y yx h

X Y X Y

xx h

XX

xy

X Y

X

Y y x X x hF y x

x X x h

f x y dx dy hf x y dy

hf xf x dx

f x y dy

f x

+

-¥ -¥+

-¥

£ < £ +=

< £ +

¢ ¢ ¢ ¢ ¢ ¢

=

¢ ¢

¢ ¢

=

ò ò ò

ò

ò

40



We can differentiate the conditional CDF to obtain the conditional pdf

( )

( )

, ,

( | )

y

X Y

YX

f x y dy

F y xf x

-¥

¢ ¢

=ò

41

( )( )

, ,( | ) ( | ) X Y

Y YX

f x ydf y x F y x

dy f x= =


Conditional Probability for Random Variables In other words, a joint pdf can be written as a product of

conditional and marginal pdfs:

For a discrete rv, the same condition can be stated as:

42

( ) ( ), , ( | )X Y Y Xf x y f y x f x=

( ) { } ( ), , Pr , ( | )X Y Y Xp x y X x Y y p y x p x= = = =


Conditional Expectation Expectation of a conditional joint distribution is defined as

43

{ }( | ) continuous rvs

|

( | ) discrete rvs

Y

Yy

yf y x dy

E Y x

yp y x

¥

-¥

ìïïïïïï= íïïïïïïî

ò

å


Useful Properties of Conditional Expectation

P1:

P2: For a function h(Y):

The k‐th moment of Y is given by

44

{ }{ } { }|E E Y x E Y=

{ } { }{ }( ) ( ) |E h Y E E h Y x=

{ } { }{ }|k kE Y E E Y x=


Homework Reading Assignment

Example 4.15 to 4.26 in the textbook

45


Vector Random VariablesMultiple Random Variables

46


Joint Distribution of Multiple rvs Most of the ideas that we have studied so far can be directly

extended to more than two jointly‐distributed random variables

Joint pmf of n discrete random variables is:

Conditional pmfs are obtained as

47

( ) { }1 2, , , 1 2 1 1 2 2, , , Pr , , ,

nX X X n n np x x x X x X x X x= = = =

( )( )( )

1 2

1 2 1

, , , 1 2 11 2 1

, , , 1 2 1

, , , ,| , , ,

, , ,n

n

n

X X X n nX n n

X X X n

p x x x xp x x x x

p x x x-

--

-

=


Joint Distribution of Multiple rvs Most of the ideas that we have studied so far can be directly

extended to more than two jointly‐distributed random variables

Joint pmf of n discrete random variables is:

Conditional pmfs are obtained as

48

( ) { }1 2, , , 1 2 1 1 2 2, , , Pr , , ,

nX X X n n np x x x X x X x X x= = = =

( )( )( )

1 2

1 2 1

, , , 1 2 11 2 1

, , , 1 2 1

, , , ,| , , ,

, , ,n

n

n

X X X n nX n n

X X X n

p x x x xp x x x x

p x x x-

--

-

=


Conditinal Distribution of Multiple rvs Conditional pdf of a joint pdf is:

Repeatedly applying this expression gives:

49

( )( )( )

1 2

1, 2 1

, , , 1

1 1

, , 1 1

, ,| , ,

, ,n

n

n

X X X n

X n n

X X X n

f x xf x x x

f x x-

-

-

=

( )

( ) ( ) ( ) ( )1 2

1 2 1

, , , 1

1 1 1 1 2 2 1 1

, ,

| , , | , , |

n

n n

X X X n

X n n X n n X X

f x x

f x x x f x x x f x x f x-- - -=


Conditinal Distribution of Multiple rvs Conditional pdf of a joint pdf is:

Repeatedly applying this expression gives:

50

( )( )( )

1 2

1, 2 1

, , , 11 1

, , 1 1

, ,| , ,

, ,n

n

n

X X X nX n n

X X X n

f x xf x x x

f x x-

--

=

( )

( ) ( ) ( ) ( )1 2

1 2 1

, , , 1

1 1 1 1 2 2 1 1

, ,

| , , | , , |

n

n n

X X X n

X n n X n n X X

f x x

f x x x f x x x f x x f x-- - -=


Joint Distribution of Multiple rvs Marginal pmf of one rv is obtained by summing over the images

of all other rvs

51

( ) { }

( )1

1 2

2

1 1 1

, , , 1 2

Pr

, , ,n

n

X

X X X nx x

p x X x

p x x x

= =

=å å


Joint Distribution of Multiple rvs Marginal pmf of one rv is obtained by summing over the images

of all other rvs

52

( ) { }

( )1

1 2

2

1 1 1

, , , 1 2

Pr

, , ,n

n

X

X X X nx x

p x X x

p x x x

= =

=å å


Joint Distribution of Multiple rvs Joint CDF of n continuous random variables is:

Joint pdf is then obtained as

53

( )

( )

1 2

1 2

1 2

, , , 1 2

, , , 1 2 1

, , ,

, , ,

n

n

n

X X X n

x x x

X X X n n

F x x x

f x x x dx dx-¥ -¥ -¥

¢ ¢ ¢ ¢ ¢= ò ò ò

( ) ( )1 2 1 2, , , 1 2 , , , 1 2

1

, , , , , ,n n

n

X X X n X X X nn

f x x x F x x xx x

¶=

¶ ¶




54

( )

( )

1 2

1 2

1 2

, , , 1 2

, , , 1 2 1

, , ,

, , ,

n

n

n

X X X n

x x x

X X X n n

F x x x

f x x x dx dx-¥ -¥ -¥

¢ ¢ ¢ ¢ ¢= ò ò ò

( ) ( )1 2 1 2, , , 1 2 , , , 1 2

1

, , , , , ,n n

n

X X X n X X X nn

f x x x F x x xx x

¶=

¶ ¶




55

( )

( )

1 2

1 2

1 2

, , , 1 2

, , , 1 2 1

, , ,

, , ,

n

n

n

X X X n

x x x

X X X n n

F x x x

f x x x dx dx-¥ -¥ -¥

¢ ¢ ¢ ¢ ¢= ò ò ò

( ) ( )1 2 1 2, , , 1 2 , , , 1 2

1

, , , , , ,n n

n

X X X n X X X nn

f x x x F x x xx x

¶=

¶ ¶




56

( )

( )

1 2

1 2

1 2

, , , 1 2

, , , 1 2 1

, , ,

, , ,

n

n

n

X X X n

x x x

X X X n n

F x x x

f x x x dx dx-¥ -¥ -¥

¢ ¢ ¢ ¢ ¢= ò ò ò

( ) ( )1 2 1 2, , , 1 2 , , , 1 2

1

, , , , , ,n n

n

X X X n X X X nn

f x x x F x x xx x

¶=

¶ ¶


Joint Distribution of Multiple rvs A single marginal pdf can be obtained as:

Also, a marginal pdf for a sub‐vector rv can be obtained as:

57

( ) ( )1 1 21 , , , 1 2 2, , ,

nX X X X n nf x f x x x dx dx¥ ¥

-¥ -¥

¢ ¢ ¢ ¢= ò ò

( ) ( )1 1 1 2, , 1 1 , , , 1 2 1, , , , , ,

n nX X n X X X n n nf x x f x x x x dx-

¥

- -

-¥

¢ ¢= ò




58

( ) ( )1 1 21 , , , 1 2 2, , ,


-¥ -¥

¢ ¢ ¢ ¢= ò ò

( ) ( )1 1 1 2, , 1 1 , , , 1 2 1, , , , , ,


¥

- -

-¥

¢ ¢= ò




59

( ) ( )1 1 21 , , , 1 2 2, , ,


-¥ -¥

¢ ¢ ¢ ¢= ò ò

( ) ( )1 1 1 2, , 1 1 , , , 1 2 1, , , , , ,


¥

- -

-¥

¢ ¢= ò




60

( ) ( )1 1 21 , , , 1 2 2, , ,


-¥ -¥

¢ ¢ ¢ ¢= ò ò

( ) ( )1 1 1 2, , 1 1 , , , 1 2 1, , , , , ,


¥

- -

-¥

¢ ¢= ò


Summary A vector rv is a mapping from outcomes of a random

experiment to a vector

Joint density and distribution functions of a vector rv are:

Marginal densities can be obtained as:

61

: nXZ S S Ì

2

, ,( , ) ( , ),X Y X Yf x y F x yx y¶

=¶ ¶ , ,( , ) ( , )

yx


= ò ò

( ) ( ), , aX Y YF a b F b¥¾¾¾¾

( ) ( ), , bX Y XF a b F a¥¾¾¾






62

: nXZ S S Ì

2

, ,( , ) ( , ),X Y X Yf x y F x yx y¶

=¶ ¶ , ,( , ) ( , )

yx


= ò ò

( ) ( ), , aX Y YF a b F b¥¾¾¾¾

( ) ( ), , bX Y XF a b F a¥¾¾¾






63

: nXZ S S Ì

2

, ,( , ) ( , ),X Y X Yf x y F x yx y¶

=¶ ¶ , ,( , ) ( , )

yx


= ò ò

( ) ( ), , aX Y YF a b F b¥¾¾¾¾

( ) ( ), , bX Y XF a b F a¥¾¾¾


Summary Conditional density is defined as:

For independent rvs, we have:

64

( ) ( ), , ( | )X Y Y Xf x y f y x f x=

, ( , ) ( ) ( )X Y X YF x y F x F y=

,

,



X Y X Y

X Y X Y

p x y p x p y

f x y f x f y

=

=


Summary Conditional density is defined as:

For independent rvs, we have:

65

( ) ( ), , ( | )X Y Y Xf x y f y x f x=

, ( , ) ( ) ( )X Y X YF x y F x F y=

,

,



X Y X Y

X Y X Y

p x y p x p y

f x y f x f y

=

=



Example 4.28, 4.29, 4.30 in the textbook

66


Functions of Multiple Random Variables

67


Functions of Multiple Random Variables If we have a function of two discrete random variables, Z=g(X,

Y), we can write its pmf as:

68

( ) ( ){ } { } { }

( ) ( ) ( )

Pr ( , ) Pr ( , ) | Pr ,

for all

( , ) |

Z

Z Y Xx

p z y z x X x y z x X x X x

x

p z p y z x x p x

= Ç = = = =

= å


Functions of Multiple Random Variables If we have a function of two random variables, Z=g(X, Y), we can

write it in terms of the joint pdf and pmf as:

69

( ) ( ) ( ) ( ) ( )( , ) | ( , ) |Z Y X X Yf z f y z x x f x dx f x z y y f y dy¥ ¥

-¥ -¥

= =ò ò


Example: Z=X+Y Let Z = g(X, Y) = X+Y If Z is discrete, we can find its pmf as:

Or:

70

( ) { } { }

{ } { }

( ) ( )

Pr Pr

Pr | Pr

Z

x

Y Xx

p z Z z X Y z

Y z x X x X x

p z x p x

= = = + =

= = - = =

= -

å

å

( ) ( ) ( )Z X Yy

p z p z y p y= -å


Example: Z=X+Y Let Z = g(X, Y) = X+Y If Z is continuous, we have:

The same pdf can also be written as:

71

( ) ( ) ( )Z Y Xf z f z x f x dx

¥

-¥

= -ò

( ) ( ) ( )Z X Yf z f z y f y dy¥

-¥

= -ò


Example: Z=X+Y Let Z = g(X, Y) = X+Y

PDF of the sum of two continuous random variables is equal to their convolution

72

( ) ( ) ( )Z Y Xf z f z x f x dx

¥

-¥

= -ò

( ) ( ) ( )Z X Yf z f z y f y dy¥

-¥

= -ò

Convolution integral


Example2: Z=X+Y Let Z = g(X, Y) = X+Y Let X and Y be the packet processing time taken at two routers

connected in series

X and Y are identically‐distributed exponential random variables with parameters λ

Find the pdf of Z.

73

Router 1(Time taken: X)

Router 2(Time taken: Y)


Example2: Z=X+Y

Z=X+Y

The pdf of X is:

74

( )0

0 0

x

X

e xf x

x

ll -ìï ³ïï= íï <ïïî




Example2: Z=X+Y

Z=X+Y

The pdf of Y is:

75

( )0

0 0,

y

Y

e y z xf y

y y z x

ll -ìï £ £ -ïï= íï < > -ïïî




Example2: Z=X+Y

Z=X+Y

In other words, the pdf of Y is:

76

( )( ) 0,

0

z x

Y

e z x x zf y

x z

ll - -ìï - > <ïï= íï >ïïî




Example2: Z=X+Y The pdf of Z is:

77

( ) ( ) ( )

( ) 2

0 0

2

Z Y X

z zz x x z x x

z

f z f z x f x dx

e e dx e e e dx

ze

l l l l l

l

l l l

l

¥

-¥

- - - - -

-

= -

= =

=

ò

ò ò

This is a 2-stage Erlang PDF ( )1

( 1)!

r r t

R

t ef t

r

ll - -

=-



Example 4.31‐4.34 in the textbook; special emphasis on Example 4.34

Homework Assignment Problem 4.31 in textbook Problem 4.38 in textbook Problem 4.41 in textbook

78


Moments of Functions of Multiple Random Variables

79


Expected Value of Functions of Multiple RVs The expected value of a function of multiple rvs, g(X, Y), is given

as:

80

{ }( ) ( )

( ) ( )

,

,

, , , jointly continuous

, , , discrete

X Y

i n X Y i ni n

g x y f x y dxdy X YE Z

g x y p x y X Y

¥ ¥

-¥ -¥


ò ò

åå


Joint Moments of Multiple RVs The jk‐th joint moment of two rvs, X and Y, is given as:

By setting j=0, we can obtain moments of Y Similarly, k=0 yields moments of X

The (j=1, k=1) moment, E{XY}, is generally called the correlation of X and Y

If E{XY}=0 => X and Y are orthogonal or uncorrelated

81

{ }( )

( )

,

,

, , jointly continuous

, , discrete

j kX Y

j k

j ki n X Y i n

i n

x y f x y dxdy X YE X Y

x y p x y X Y

¥ ¥

-¥ -¥


ò ò

åå


Joint Central Moments of Multiple RVs The jk‐th central moment of two rvs, X and Y, is:

By setting j=0 and k=2, gives the variance of Y Similarly, setting j=2 and k=0 gives the variance of X

The (j=1, k=1) central moment, E{(X‐E{X})(Y‐E{Y})}, is generally called the covariance of X and Y

A more convenient representation of COV(X,Y) is:

Independent rvs have zero covariance82

{ }( ) { }( ){ }j kE X E X Y E Y- -

{ } { }( ) { }( ){ },COV X Y E X E X Y E Y= - -

{ } { } { } { },COV X Y E XY E X E Y= -


Correlation Coefficient The correlation coefficient of X and Y is

It can be shown that

Thus correlation coefficient is a normalized measure that quantifies the amount of dependence between two random variables

83

{ } { } { } { },

,X Y

X Y X Y

COV X Y E XY E X E Yr

s s s s-

= =

,1 1X Yr- £ £


Correlation Coefficient

Source: Wikipedia – Pearson product‐moment correlation coefficient, http://en.wikipedia.org/wiki/Pearson_product‐moment_correlation_coefficient























Useful Properties of Moments of Multiple RVs P1:

If all X’s are independent:

92

{ } { } { } { }1 2 1 2n nE X X X E X E X E X+ + + = + + +

( ) ( ) ( ){ } ( ){ } ( ){ } ( ){ }1 2 1 2n nE g X g X g X E g X E g X E g X=


Reading Assignment Section 4.7 in the textbook

Examples 4.39‐4.42

93


Jointly Normal (Gaussian) Random Variables

94


Jointly Normal Random Variables The multi‐variate Gaussian pdf has the following form:

Where

95

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m

1

2

N

m

21 12 1

221 2 2

21 2

N

N

N N N



For the Bivariate case:

96

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m

1

2

m

21 12

221 2




97

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m

1

2

m

21 12

221 2




98

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m

1

2

m

21 12

221 2

1 212

1 2

( , )Cov X X

1 2 12 12 1 2( , )Cov X X




99

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m

1

2

m

21 12 1 2

221 2 1 2

1 212

1 2

( , )Cov X X

1 2 12 12 1 2( , )Cov X X 12 21




100

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m

1

2

m

21 1 2

22 1 2

1 212

1 2

( , )Cov X X

1 2 12 12 1 2( , )Cov X X 12 21


Jointly Normal Random Variables Two random variables X and Y are jointly normal if their pdf has

the following form

101

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m

21 12 1 2

221 2 1 2

121 2

2 2 2 2 21 1 221 2 1 22

2 1 2

21 2 1



the following form

102

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m

21 12 1 2

221 2 1 2

121 2

2 2 2 2 21 1 221 2 1 22

2 1 2

21 2 1



the following form

103

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m

21 1 21

2

22 1 2

11

11

21 12 1 2

221 2 1 2

121 2

2 2 2 2 21 1 221 2 1 22

2 1 2

21 2 1



the following form

104

( )( )

2 2

,2,

, 2,

1exp 2

2 1,

2 1

X X Y YX Y

X X Y YX Y

X Y

X Y X Y

x m x m y m y m

f x y

rs s s sr

ps s r

ì üé ùï ïæ ö æ öæ ö æ öï ï- - - -ê ú- ÷ ÷ ÷ ÷ç ç ç çï ï÷ ÷ ÷ ÷ç ç ç ç- +ê úí ý÷ ÷ ÷ ÷ç ç ç ç÷ ÷ ÷ ÷ï ï÷ ÷ ÷ ÷ç ç ç çê ú- è ø è øè ø è øï ïê úï ïë ûî þ=-

11 2 1

2

1 1( , ,..., ) exp22

T

X Nf x x x x m x m


Jointly Normal Random Variables

105Picture from Prof. Hayder Radha’s lecture notes of ECE863 course

( )( )

2 2

,2,

, 2,

1exp 2

2 1,

2 1

X X Y YX Y

X X Y YX Y

X Y

X Y X Y

x m x m y m y m

f x y

rs s s sr

ps s r


X

Y

Bivariate Gaussian PDF

-4 -2 0 2 4-4

-2

0

2

4




( )( )

2 2

,2,

, 2,

1exp 2

2 1,

2 1

X X Y YX Y

X X Y YX Y

X Y

X Y X Y

x m x m y m y m

f x y

rs s s sr

ps s r


X

Y


-4 -2 0 2 4-4

-2

0

2

4




( )( )

2 2

,2,

, 2,

1exp 2

2 1,

2 1

X X Y YX Y

X X Y YX Y

X Y

X Y X Y

x m x m y m y m

f x y

rs s s sr

ps s r


X

Y


-4 -2 0 2 4-4

-2

0

2

4



108

( )( )

2 2

,2,

, 2,

1exp 2

2 1,

2 1

X X Y YX Y

X X Y YX Y

X Y

X Y X Y

x m x m y m y m

f x y

rs s s sr

ps s r


If we set the exponents involving x and y in the above expression to a constant K, we obtain the equation for an ellipse

( )( )2

,

, 2,

1exp

2 1,

2 1

X Y

X Y

X Y X Y

K

f x yr

ps s r

ì üï ïï ï-ï ïí ýï ï-ï ïï ïî þ=-



109

( )( )

2 2

,2,

, 2,

1exp 2

2 1,

2 1

X X Y YX Y

X X Y YX Y

X Y

X Y X Y

x m x m y m y m

f x y

rs s s sr

ps s r


If we set the exponents involving x and y in the above expression to a constant K, we obtain the equation for an ellipse

( )( )2

,

, 2,

1exp

2 1,

2 1

X Y

X Y

X Y X Y

K

f x yr

ps s r


This is the equation for an ellipse


( )( )2

,

, 2,

1exp

2 1,

2 1

X Y

X Y

X Y X Y

K

f x yr

ps s r



110

The orientation of the ellipse depends on the value of correlation ρ


( )( )2

,

, 2,

1exp

2 1,

2 1

X Y

X Y

X Y X Y

K

f x yr

ps s r



111


When ρ≠0, we have:

Picture from Prof. Hayder Radha’s lecture notes of ECE863 course


( )( )2

,

, 2,

1exp

2 1,

2 1

X Y

X Y

X Y X Y

K

f x yr

ps s r



112





( )( )2

,

, 2,

1exp

2 1,

2 1

X Y

X Y

X Y X Y

K

f x yr

ps s r



113





( )( )2

,

, 2,

1exp

2 1,

2 1

X Y

X Y

X Y X Y

K

f x yr

ps s r



114

When ρ=0 (i.e., X & Y are uncorrelated), we have:

The ellipses are parallel to the X and Y axis



( )( )2

,

, 2,

1exp

2 1,

2 1

X Y

X Y

X Y X Y

K

f x yr

ps s r



115





( )( )2

,

, 2,

1exp

2 1,

2 1

X Y

X Y

X Y X Y

K

f x yr

ps s r



116






117

As ρ=0 increases towards 1, X and Y become more and more correlated and the ellipses become narrower


X

Y


-4 -2 0 2 4-4

-2

0

2

4



118

As ρ=0 increases towards 1, X and Y become more and more correlated and the ellipses become narrower


XY


-4 -2 0 2 4-4

-2

0

2

4


n Jointly Normal Random Variables Random variables X1, X2,…Xn are jointly normal if their pdf has

the following form:

where:

119

( ) ( )( ) ( ){ }( )1 2

1

, , , 1 2 1/ 2/2

1exp

2, , ,2n

T

X X X nX n

x m K x mf x f x x x

Kp

--- -

=

{ } { } { }

{ } { } { }

{ } { } { }

1 2

1 2

1 1 2 1

2 1 1 2

1 2

var

var

var

n

n

n

n

n n n

x x x x

m m m m

X COV X X COV X X

COV X X X COV X XK

COV X X COV X X X

é ù= ê úë ûé ù= ê úë ûé ùê úê úê úê ú= ê úê úê úê úê úë û

Covariance matrix


Reading Assignment Section 4.8 in the textbook

Examples 4.45‐4.48

120

Documents

Stochastic lecture 5