2015/6/191 Chap 10 More Expectations and Variances Ghahramani 3rd edition

112/04/18 1

Chap 10 More Expectations and Variances

Ghahramani 3rd edition

p2.

Outline

10.1 Expected values of sums of random variables10.2 Covariance10.3 Correlation10.4 Conditioning on random variables10.5 Bivariate normal distribution

p3.

10.1 Expected values of sums of random variables

n21n21

n21

n

1ii

n

1ii

n21

XXX)XXX(

Then space. sample same

on the variablesrandom be X , ,X ,XLet Corollary

)()(

space, sample same on the

defined X , ,X ,X variablesrandomFor 10.1 Theorem

EEEE

XEXE ii

p4.

Expected values of sums of random variables

.5.522

715 Hence

.2

7

6

16

6

15

6

14

6

13

6

12

6

11X

XXX)X(

.XXXX

Then roll.ith theof

outcome thebe Xlet and outcomes theof sume thebe XLet :Sol

outcomes? theof sum the

of valueexpected theis What times.15 rolled is dieA 10.1Ex

i

1521

1521

i

EX

E

EEEE

p5.


piles? 4

allin matches ofnumber total theof valueexpected

theis What j. is cardjth theif pile ain occursmatch

asay that wely,respective 13, and 12, 11, as king and

queen, jack, Counting each. 13 of piles 4 intorandomly

divided is cards 52 ofdeck ordinary shuffled-A well

10.2Ex

p6.


otherwise,0

occurs A if1

definingby Then j. is pile ith thein

card jth theevent that thebe Alet ,X calculate To

.XXXX EXand

matches, ofnumber total theis XXXXX

pile. ith thein matches ofnumber thebe X Let :Sol

ij

iji

4321

4321

i

ijX

E

EEEE

p7.


.41111XXXX EXThus

.1 EXHence

.13/1)A(0)A(1)Now E(

.X have We

4321

13

1i

cijij

13

1i

EEEE

EX

PPX

X

jij

ij

jij

p8.


.EX and

,Xget e w

otherwise,0

success a is ith trial theif1

letting n, , 2, 1,ifor Thus, rails. t

Bernoullit independenn in successes ofnumber theis X :Sol

EX? is What

p).(n, parameters with random binomial a be XLet 10.6Ex

21

21

npEXEXEX

XXX

X

n

n

i

p9.


EX? is What success.rth theuntil

trialofnumber theis X p,y probabilit successeach with

trialsBernoullit independen of sequence ain Then, p).(r,

parameters with random binomial negative a be XLet

10.7Ex

p10.


.EX

Hence1/p. EX

so and pparameter withgeometric is X that Note

.X

Then on. so and success,

second get the to trialsadditional ofnumber thebe X

success,first theuntil trialsofnumber thebe X Let :Sol

21

i

i

21

2

1

p

rEXEXEX

XXX

r

r

p11.


EX? is What D).-N min(D, n e wher

n, ..., 2, 1, 0, for x )()(

functiony probabilit with random trichypergeome a be XLet 10.8Ex

n

N

xn

DN

x

D

xXPxp

p12.


.N

nD EXnce He

.N

D)P(A1)P(X0)P(X01)P(X1 EX

.X Then

otherwise.0

occurs A if1Xlet

and defective, is drawn item ith theevent that the thebe A Let :Sol

iiiii

21

ii

i

nXXX

p13.


.0EX Hence .0EY-EYEX

..., 2, 1,ifor 1)E(Y sinceThen .Y-YX and

otherwise,0

1/iy probabilitwith iYlet ..., 2, 1,iFor

:exampleCounter

y true.necessarilnot is 10.1 Theorem ,nFor

1i ii1ii

ii1ii

i

p14.


1i 1i ii

i1i i

1i 1i ii

1i 1i

1i 1i

.EX)X E(

theni, allfor enonnegativ

is X ifor |X|E if that shown be canIt

.EX)X E(Therefore

.1)E(-Y)XE(

have weand ,-YX However,

p15.


otherwise,0

iN if1X

Let :Proof

i).P(NE(N)

Then ...}. 3, 2, {1, valuespossible

ofset with variablerandom discrete a be NLet 10.2 Thm

i

1i

p16.


0

1i1i 1iii

1i 1i1ii

1ii

1ii

t)dt.P(X E(X)

then variable,random enonnegativ continuous

a is X iffact that theof analog theis 9.2 Thm that Note

.i)P(N)()E(E(N) So

.01 then

XEX

NXXXN N

NN

p17.


.)E(Y)E(XE(XY)

)()()]([

0)()(4)](2[

0)()(2)(

0)2(

.0)-(X ,number real allFor :Proof

.)E(Y)E(XE(XY)

Y, and X variablesrandomFor

)Inequality Schwarz-(Cauchy 10.3 Thm

22

222

222

222

222

2

22

YEXEXYE

XEYEXYE

XEXYEYE

YXYXE

Y

p18.


).E(X[EX] Thus

.)E(X)E(1)E(XE(XY)E(X) then

1,Ylet ,Inequality sSchwarz'-CauchyIn :Proof

).E(X[E(X)]

X, variablesrandom aFor Corollary

22

22

22

p19.

10.2 Covariance

Motivation:

Var(aX+bY)=E[(aX+bY)-E(aX+bY)]2

=E[(aX+bY)-aEX-bEY]2

=E[a[X-EX]+b[Y-EY]]2

=E[a2[X-EX]2+b2[Y-EY]2

+2ab[X-EX][Y-EY]]

p20.

Covariance

• Def Let X and Y be jointly distributed r. v.’s; then the covariance of X and Y is defined by

Cov(X, Y)=E[(X-EX)(Y-EY)]• Note that Cov(X, X)=Var(X), and also by Cauchy-Schwarz inequality

YXYXEYYEEXXE

EYYEXXEYXCov

2222 ][][

)])([(),(

p21.

Covariance

• Thm 10.4

Var(aX+bY) =a2Var(X)+b2Var(Y)+2abCov(X,Y).

• In particular, if a=b=1,

Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y)

p22.

Covariance

).,(

),(

get we, relation thisUse

.)()(

)(

)(

)(

)])([(),(

YXCovac

dcYbaXCov

EXEYXYEXYE

XYE

EXEYXYE

XYXYE

YXEYXCov

YX

YXXYYX

YXYX

YXYX

YX

p23.

Covariance

1. X and Y are positively correlated if Cov(X,Y) > 0.

2. X and Y are negatively correlated if Cov(X,Y) < 0.

3. X and Y are uncorrelated if Cov(X,Y) = 0.

p24.

Covariance

• If X and Y are independent then Cov(X,Y)=EXY-EXEY=0. But the converse is not true

• Ex 10.9 Let X be uniformly distributed over (-1,1) and Y=X2. Then

Cov(X,Y)=E(X3)-EXE(X2)=0. So X and Y are uncorrelated but surely X and Y

are dependent.

p25.

Covariance

• Ex 10.12 Let X be the lifetime of an electronic system and Y be the lifetime of one of its components. Suppose that the electronic system fails if the component does (but not necessarily vice versa). Furthermore, suppose that the joint density function of X and Y (in years) is given by

elsewhere.0

yx0 if49

1),(

7/

yeyxf

p26.

Covariance

(a) Determine the expected value of the remaining lifetime of the component when the system dies.

(b) Find the covariance of X and Y.

Sol:

798

1

)2/(49

1

49

1)()( )(

0

7/2

0

227/

0 0

7/

dyey

dyyye

dxdyexyXYEa

y

y

y y

p27.

Covariance

.049)14(7147),(

1449

1)(

749

1)(

14798

1

)(49

1

49

1)()( )(

0 0

7/

0 0

7/

0

7/3

0 0

7/

0 0

7/

EXEYEXYYXCov

dxdyeyYE

dxdyexXE

dyey

dyxdxye

dxdyexyXYEb

y y

y y

y

yy

y y

p28.

Covariance

n

ii

n

ii

j

n

i

n

j

j

iii

n

ii

XVarXVar

XXCovXVarXVar

11

n21

1 2

1

11

)()(

thened,uncorrelat pairwise generally,

more or,t independen pairwise are X , ,X ,X If

) ,(2)()(

:tionGeneraliza

p29.

Covariance

• Ex 10.13 Let X be the number of 6’s in n rolls of a fair die. Find Var(X).

p30.

Covariance

Sol:

.36

5)( Therefore

.36/536/16/1)(ar hence and

,6/1)6/5(0)6/1(1)(

,6/1)6/5(0)6/1(1)(But

).Var(X )Var(X )Var(X Var(X)

and ,XXXX Then

otherwise.0

6 lands die theroll ith theon if1XLet

222

n21

n21

i

nXVar

XV

XE

XE

i

i

i

p31.

Covariance

• Ex 10.15 X ~ B(n,p). Find Var(X).

Sol:

).1()( so ,)1()(arBut


and ,XXXX Then

otherwise.0

successa is trialith theif1XLet

n21

n21

i

pnpXVarppXV i

p32.

Covariance

• Ex 10.16 X ~ NB(r,p). Find Var(X).

Sol:

./)1()( so ,/)1()(arBut


and ,XXXX

Then on. so and success,

second get the to trialadditional ofnumber thebe

success,first theuntil trialofnumber thebe XLet

22

r21

r21

i

pprXVarppXV i

p33.

10.3 Correlation Motivation:

Suppose X and Y, when measured in centimeters, Cov(X,Y)=0.15. But if we change the measurements to millimeters, the X1=10X and Y1=10Y and

Cov(X1,Y1)=Cov(10X,10Y)=100Cov(X,Y)=15

This shows that Cov(X,Y) is sensitive to the units of measurement.

p34.

Correlation

.),(

),(

Therefore Y).(X,by denoted is and Yand X between

tcoefficien ncorrelatio thecalled is Yedstandardiz theand X

edstandardiz thebetween covariance The .0 and

0 with variablesrandom twobe Yand XLet

(Def)

2X

2X

YXYX

YXCovEYYEXXCov

p35.

Correlation

Y).(X,22)Var(

Y),(X,22)Var(

Y),(X,

tcoefficienn correlatio with Y and X variablesrandomFor

10.2 Lemma

YX

YX

YX

YX

p36.

Correlation

Y).(X,22

Y),Cov(2)Var(

1)Var(

1

),2Cov()Var()Var()Var(

(Proof)

22

YXYX

YXYXYX

XYX

σ

YXY

σ

XYX

p37.

Correlation

0.a b, a, constants some

for baXY iff 1Y)(X, 1,y probabilit With c)(

0.a b, a, constants some

for baXY iff 1Y)(X, 1,y probabilit With b)(

.1Y)(X,1- (a)

:Y)(X,

t coefficienn correlatio with Y and X variablesrandomFor

10.5 Thm

p38.

Correlation

related.linearly not are Y and Xthen

otherwise,0

1y0 1,x0 if),(

functiondensity joint with the

variablesrandom continuous are Y and X if that Show

10.17Ex

yxyxf

p39.

Correlation

.111/1)12/11)(12/11(

144/1Y)Cov(X,Y)(X, Thus

.12/11)12/7(12/5 ,12/5EYEX

-1/144.2)(7/12)(7/1-1/3Y)Cov(X, therefore

1/3, EXY7/12,EY that EXNote

1.Y)(X, that prove tosufficesit 1,y probabilit

with1Y)(X, iff relatedlinearly are Yand X Since

(Sol)

222

YX

YX

p40.

Correlation

Y)?(X, is What .XY and

(0,1), interval thefromnumber random a be XLet

10.18Ex

2

p41.

Correlation

.968.04

15

53

2

32

11/12

Y)(X, Therefore

.12/1)3/1)(2/1(4/1)E(X)E(X)E(XY)Cov(X, and

,45/4)3/1(5/1)]([)E(X][)E(Y

,12/14/13/1][)E(X

,3/1)E(X EY1/2, EXThus

.1

1)E(X

)Sol(

23

2224222

222

2

1

0

n

XEEY

EX

ndxx

Y

X

n

p42.

Skip

10.4 Conditioning on random variables

10.5 Bivariate normal distribution

Documents

2015/6/191 Chap 10 More Expectations and Variances Ghahramani 3rd edition