Embed Size (px)
Citation preview
Bivariate Gaussian PDF• Joint (bivariate) PDF of two jointly Gaussianrandom variables x and y is
1
,1 1( , ) exp 22 CC
xx y x y
y
xf x y x y y
2
2
Covariance matrix of andDeter
CC
C
x
y
x x xyx y
y xy y
E x Expectation of xE y Expectation of y
x
xE x yy
y
minant of C
01 00 1C
x y
σy = σx Joint PDF 3-D plot
00.25 0
0 1Cx y
σy > σx Joint PDF 3-D plot
01 00 0.25C
x y
σx > σy Joint PDF 3-D plot
Contour• Contour of a 3-D plot is 2-D plot showing relationship between x and y when fx,y(x,y) = constant • Set f (x,y) = constant Gives an equation of • Set fx,y(x,y) = constant Gives an equation of
x and y Plotting this equation (y versus x) gives the so-called contour
• As constant varies, we get different contours • Let’s plot contours of previous figures
Contour
,22
, 2 2
22
( , )1( , ) exp Constant2 2
: Get Contou
12ln
r equation of wit
2
h 0
C
yx
x y
yxx
y
xy
yx
E f x yy T
y
xam
x
l
f
e
x y
px
2 2 12ln
This is an equation f2
o
Ell
ips
yxx yx y
yxT
22 2
2
2
2 2
12 l
centered @ ( , ) = ,If ==> It becomes
This is equation of centered @ ( , ) = , . Its radius dependn 2
s on
e
C
irc
le
x y
x
y
x
x
y
x
x
x y
xy
yT
T
01 00 1C
x y
σy = σx
1
2
3
0.70.80.91
,
Plot of versus when
( , ) 0.3x y
y x
f x y
Contour iscircle
x-axis
y-axis
-3 -2 -1 0 1 2 3-3
-2
-1
0
0.10.20.30.40.50.6
,
Plot of y versus xwhen
( , ) 0.9x yf x y
1
2
3
0.70.80.9
00.25 0
0 1Cx y
σy > σx
Contour isEllipse withmajor axis on
x-axis
y-axis
-3 -2 -1 0 1 2 3-3
-2
-1
0
0.10.20.30.40.50.6major axis ony-axis andminor axis on x-axisbecause
σy > σx
Contour isEllipse withmajor axis on
01 00 0.25C
x y
σx > σy
1
2
3
0.70.80.91
major axis onx-axis andminor axis on y-axisbecause σx > σy
x-axis
y-axis
-3 -2 -1 0 1 2 3-3
-2
-1
0
0.10.20.30.40.50.6
Effect of Correlation• Now, we saw PDF and contour when σxy = 0, i.e., when x and y are uncorrelated• How would contour look like when x and y are correlated, i.e., σxy ≠ 0 ?• Correlation coefficient ρ = σ / (σ σ )• Correlation coefficient ρ = σxy / (σx σy )-1 < ρ < 1• ρ > 0 x and y are positively correlated, i.e.,
as x increases , y increases• ρ < 0 x and y are negatively correlated, i.e.,
as x increases, y decreases
0.50,1 0.5
0.5 1
C
x y
ρ = 0.5 Contour isRotated Ellipsex y x y
123
0.70.80.9
Major axis
x-axis
y-axis
-3 -2 -1 0 1 2 3-3-2-101
0.10.20.30.40.50.60.7Major axishas positiveslop as ρ > 0
0.50,1 0.50.5 1
C
x y
ρ = - 0.5 Contour isRotated Ellipse
Major axis
x y x y
1
2
3
0.70.80.91
Major axishas negativeslop as ρ < 0
x-axis
y-axis
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
0.10.20.30.40.50.60.7
Effect of Correlation• As correlation ρ increases, knowing one variable gives more information about the other• For large ρ Given any value of x, variance of
y decreases [because more information about y decreases [because more information about y is available]
• This means that y will become more consternated around its mean at any given value of x• See next slide for Contour @ ρ = 0.98
0.980,1 0.98
0.98 1
C
x y
ρ = 0.98 Contour isRotated Ellipse
1
2
3
0.5
0.6
0.7
Compare withContour forρ = 0.5 in
x-axis
y-axis
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
0.1
0.2
0.3
0.4
0.5ρ = 0.5 in slide ρ=0.5where y haslarger variancearound its meanfor any given value of x
y has small variancearound its meanAt any given valueof x
Effect of Correlation• For ρ>0, increasing x makes average level of y
(mean of y) increases• For ρ<0, increasing x makes average level of y• For ρ<0, increasing x makes average level of y
(mean of y) decreases• For ρ=0, increasing/decreasing x does not affect
average level of y (mean of y)
Effect of Correlationy-a
xis 0
1
2
0.4
0.5
0.6E(y|x=1) = Mean of y when x = 1
ρ = 0.98
x-axis -3 -2 -1 0 1 2 3-3
-2
-1
0.1
0.2
0.3
E(y|x=0) = Mean of y when x = 0
E(y|x=1) > E(y|x=0) As x increases, E(y|x) increases ρ>0
Effect of Correlationρ = 0y-a
xis
0
1
2
3
0.60.70.80.9E(y|x=0) = Mean of y when x = 0
E(y|x=1) = E(y|x=0) As x changes, E(y|x) doesn’t change ρ=0x-axis
y-axis
-3 -2 -1 0 1 2 3-3
-2
-1
0
0.10.20.30.40.5
E(y|x=1) =Mean of ywhen x = 1
Effect of Correlation• For ρ>0, E(y|x) increases as x increases• For ρ<0, E(y|x) decreases as x increases• For ρ=0, E(y|x) doesn’t change as x changes E(y|x) not function of x
Conditional PDF• So, we have seen that correlation ρdetermines how E(y|x) changes as function of
x See slide_ ρ _0.98 and slide_ ρ_0• We also saw how magnitude of ρ affects variance of y around its mean E(y|x) at any variance of y around its mean E(y|x) at any given x See Slide_var• Let’s develop these relationship analytically and further verifies it through graphs• We will get fy(y|x) and observe its mean E(y|x)and variance var(y|x)
Conditional PDF
, 00
0 , 0
0 Bayes' Rule,
| , x
x x yy
x yy f x
f x f xf y x x f x y
y dy
,
,
0
0
0
0
0
0
,
Just a scalar to make 1is a scaled version of
|| ,
, Cross section , @ o f =x
y
xy
x y
x y y
y
y
f
f y x xf y x
x y x x
x f x yf x d
f x
y
y
Conditional PDF
, 00
0 , 0
is a scaled version of
To plot we just plot
Since | ,
| , ,
y x y
y yx
f x y
f
f y x x
f y x x x y
0
0
, 0
, 0
To plot we just plot
We plot Scaled version ofagainst for different
|
|
, ,
, [ ]
y y
y y
x
x
ff y x x x y
f x yy
f y x x
Conditional PDF ρ = zero
0.140.16
x = y = 1, = 0, x = 0, y = 0
xo = 0xo = 0.5x = 1ρ = 0
, 0
, 0 , 0
0Vertical axis ==> Scaled version o, [ ] ,
f ==> Cross , @ section of =
|x y
x y
x y
yf y x xf
f x yf x y x y x x
-3 -2 -1 0 1 2 30
0.02
0.040.06
0.08
0.10.12
y-axis
f y( y | x=
x o ) s
calar
xo = 1xo = 1.5
ρ = 0
Conditional PDF ρ = zero• From previous plot of Conditional PDF when ρ=0, we observe:A. fy(y|x=xo) is GaussianB. Location of maximum of fy(y|x=xo), i.e., its Mean, i.e. E(y|x=x ), is fixed regardless of xMean, i.e. E(y|x=xo), is fixed regardless of xo Cross section of fx,y(x=xo,y) is centered around same point regardless of position of cross sectionC. Variance of fy(y|x=xo), i.e., var(y|x=xo) does not depend on xo
0.160.180.2
x = 1, y = 1, = 0.5, x = 0, y = 0 xo = 0
xo = 0.5
Conditional PDF ρ = 0.5
, 0
, 0 , 0
0Vertical axis ==> Scaled version o, [ ] ,
f ==> Cross , @ section of =
|x y
x y
x y
yf y x xf
f x yf x y x y x x
-3 -2 -1 0 1 2 300.020.040.060.080.1
0.120.140.16
y-axis
f y( y | x
=x o ) s
calar
xo = 1xo = 1.5
ρ = 0.5y=0= 0 x ρy=0.25=0.5 ρy=0.5=1 x ρy=0.75=1.5 ρ
Conditional PDF ρ = 0.5• From previous plot of Conditional PDF when ρ=0.5, we observe:A. fy(y|x=xo) has a Gaussian shape ==> GaussianB. Location of maximum of fy(y|x=xo), i.e., its Mean, i.e. E(y|x=xo), increases as xo increases Cross section of fx,y(x=xo,y) @x=xo is Cross section of fx,y(x=xo,y) @x=xo is centered at different positions of the cross sectionC. Location of maximum of fy(y|x=xo), i.e., E(y|x=xo) = ρ xoD. Variance of fy(y|x=xo), i.e., var(y|x=xo) does not depend on xo
10
12x = 1, y = 1, = -0.9999, x = 0, y = 0
xo = 0xo = 0.5xo = 1x = 1.5
Conditional PDF ρ ≈ -1
, 0
, 0 , 0
0Vertical axis ==> Scaled version o, [ ] ,
f ==> Cross , @ section of =
|x y
x y
x y
yf y x xf
f x yf x y x y x x
-3 -2 -1 0 1 2 30
2
4
6
8
y
f y( y | x=
x o ) s
calar
xo = 1.5
ρ ≈ -1y=0= 0 x ρy=-0.5=0.5 ρy=-1=1 x ρy=-1.5=1.5 ρ
Conditional PDF ρ ≈ -1• From previous plot of Conditional PDF when ρ ≈ -1, we observe:A. fy(y|x=xo) has a Gaussian shape ==> GaussianB. Location of maximum of fy(y|x=xo), i.e., its Mean, i.e. E(y|x=xo), increases as xo increases Cross section of fx,y(x=xo,y) @x=xo is centered at section of fx,y(x=xo,y) @x=xo is centered at different positions of the cross sectionC. Location of maximum of fy(y|x=xo), i.e., E(y|x=xo) = ρ xoD. Variance of fy(y|x=xo), i.e., var(y|x=xo) does not depend on xoE. var(y|x=xo) is smaller than case of ρ = 0.5
Conclusion on Conditional PDF1) If , are jointly Gaussian 2) with coefficient
( | ) when 0,
( | ) is also( | ) is in
GaussianL
INEAR
al
x y
E y x x
f y xE y x x
( | ) when 0,3)4) var( | )
var( | ) is function of As depends on ,
NOT
x y x yE y x x
y xy x x
var( | )y x
Analytical expression of fy|x(y|x)
2|,
22 ||
| 2
, 1( | ) exp 22|
y xx yx y xy x
xy xy x y x y yx x
yf x yf y x f xxE y x x
22 2 2 2
|
| 2 function of
var 1
|
x x
xyy x y y
x
x
y x y xy
x
xy x
As var | y x
Analytical expression of fy|x(y|x)• We see that analytical expressions are inline with
our graphical observations:– E(y|x) is linear in x– var(y|x) does not depend on x– var(y|x) decreases as |ρ| increases
• If ρ = 0, we have– E(y|x) = μy Not function of x– var(y|x) = var(y)
MATAB Code (1/2)% User inputsmu_x = 0;mu_y = 0;sigma_x = 1;sigma_y = 1;rho = -0.9999;%% f(x,y) computation %% f(x,y) computation C=[sigma_x^2 rho*sigma_x*sigma_y;rho*sigma_x*sigma_y sigma_y^2];x=[-3*max(sigma_x,sigma_y):0.1:3*max(sigma_x,sigma_y)];y=[-3*max(sigma_x,sigma_y):0.1:3*max(sigma_x,sigma_y)];[X,Y]=meshgrid(x,y);xn = (X-mu_x)/sigma_x;yn = (Y-mu_y)/sigma_y;f_xy = exp(-(xn.^2 -2*rho*xn.*yn +yn.^2)/(2-2*rho^2))/(2*pi*sqrt(det(C))); % f(x,y)
MATAB Code (2/2)%% Plot 3-D bivariate (joint) PDF of x,yfigure; surfc(X,Y,f_xy);colormap hsv%% Plot Contour of bivariate (joint) PDF of x,yfigure; contour(X,Y,f_xy); grid on;%% Plot cross-section of f(x,y) at x=xo, i.e., plot f(xo,y) vs yxo = 1.5;xo = 1.5;figure; plot(Y(abs(X-xo)<1e-2), f_xy(abs(X-xo)<1e-2))xlabel('\ity\rm');ylabel(['f_y( \ity | x=x_o\rm ) \times scalar'])title(['\sigma_x = ' num2str(sigma_x), ', \sigma_y = ' num2str(sigma_y), ', \rho = ' num2str(rho) ', \mu_x = ' num2str(mu_x) ', \mu_y = ' num2str(mu_y)])legend(['\itx_o\rm = ' num2str(xo)])grid on%% Plot cross-section of f(x,y) at y=yo, i.e., plot f(x,yo) vs xyo = 3;figure; plot(X(abs(Y-yo)<1e-2),f_xy(abs(Y-yo)<1e-2))
