Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
week 8 1
Joint Distribution of two or More Random Variables
• Sometimes more than one measurement (r.v.) is taken on each member of the sample space. In cases like this there will be a few random variables defined on the same probability space and we would like to explore their joint distribution.
• Joint behavior of 2 random variable (continuous or discrete), X and Ydetermined by their joint cumulative distribution function
• n – dimensional case
( ) ( ).,,, yYxXPyxF YX ≤≤=
( ) ( ).,,,..., 111,...,1 nnnXX xXxXPxxFn
≤≤= K
week 8 2
Discrete case• Suppose X, Y are discrete random variables defined on the same probability
space.
• The joint probability mass function of 2 discrete random variables X and Yis the function pX,Y(x,y) defined for all pairs of real numbers x and y by
• For a joint pmf pX,Y(x,y) we must have: pX,Y(x,y) ≥ 0 for all values of x,y and
( ) ( )yYandxXPyxp YX ===,,
( ) 1,, =∑∑x y
YX yxp
week 8 3
Example for illustration• Toss a coin 3 times. Define,
X: number of heads on 1st toss, Y: total number of heads.
• The sample space is Ω =TTT, TTH, THT, HTT, THH, HTH, HHT, HHH.
• We display the joint distribution of X and Y in the following table
• Can we recover the probability mass function for X and Y from the joint table?
• To find the probability mass function of X we sum the appropriate rows of the table of the joint probability function.
• Similarly, to find the mass function for Y we sum the appropriate columns.
week 8 4
Marginal Probability Function• The marginal probability mass function for X is
• The marginal probability mass function for Y is
• Case of several discrete random variables is analogous.• If X1,…,Xm are discrete random variables on the same sample space with
joint probability function
The marginal probability function for X1 is
• The 2-dimentional marginal probability function for X1 and X2 is
( ) ( )∑=y
YXX yxpxp ,,
( ) ( )∑=x
YXY yxpyp ,,
( ) ( )mmmXX xXxXPxxpn
=== ,...,,... 111,...1
( ) ( )∑=m
nxx
mXXX xxpxp,...,
1,...12
11,...
( ) ( )∑=m
nxx
mXXXX xxxxpxxp,...,
321,...213
121,...,,,,
week 8 5
Example• Roll a die twice. Let X: number of 1’s and Y: total of the 2 die.
There is no available form of the joint mass function for X, Y. We display the joint distribution of X and Y with the following table.
• The marginal probability mass function of X and Y are
• Find P(X ≤ 1 and Y ≤ 4)
week 8 6
Independence of random variables
• Recall the definition of random variable X: mapping from Ω to R such that for .
• By definition of F, this implies that (X > 1.4) is and event and for the discrete case (X = 2) is an event.
• In general is an event for any set A that is formed by taking unions / complements / intersections of intervals from R.
• DefinitionRandom variables X and Y are independent if the events and are independent.
( ) FxX ∈=Ω∈ ωω :
( ) ( ) AXAX ∈=∈ ωω :
Rx∈
( )AX ∈ ( )BY ∈
week 8 7
Theorem• Two discrete random variables X and Y with joint pmf pX,Y(x,y) and
marginal mass function pX(x) and pY(y), are independent if and only if
• Proof:
• Question: Back to the rolling die 2 times example, are X and Yindependent?
( ) ( ) ( )ypxpyxp YXYX =,,
week 8 8
Conditional Probability on a joint discrete distribution
• Given the joint pmf of X and Y, we want to find
and
• Back to the example on slide 5,
• These 11 probabilities give the conditional pmf of Y given X = 1.
( ) ( )( )yYP
yYandxXPyYxXP=
===== |
( ) ( )( )xXP
yYandxXPxXyYP=
===== |
( ) 01|2 === XYP
( )51
3610
362
1|3 ==== XYP
( ) 01|12 === XYPM
week 8 9
Definition• For X, Y discrete random variables with joint pmf pX,Y(x,y) and marginal mass
function pX(x) and pY(y). If x is a number such that pX(x) > 0, then theconditional pmf of Y given X = x is
• Is this a valid pmf?
• Similarly, the conditional pmf of X given Y = y is
• Note, from the above conditional pmf we get
Summing both sides over all possible values of Y we get
This is an extremely useful application of the law of total probability.• Note: If X, Y are independent random variables then PX|Y(x|y) = PX(x).
( ) ( ) ( )( )xp
yxpxXypxyp
X
YXXYXY
,|| ,
|| ===
( ) ( ) ( )( )yp
yxpyYxpyxp
Y
YXYXYX
,|| ,
|| ===
( ) ( ) ( )ypyxpyxp YYXYX |, |, =
( ) ( ) ( ) ( )∑∑ ==y
YYXy
YXX ypyxpyxpxp |, |,
week 8 10
Example• Suppose we roll a fair die; whatever number comes up we toss a coin that
many times. What is the distribution of the number of heads?Let X = number of heads, Y = number on die.We know that
Want to find pX(x).
• The conditional probability function of X given Y = y is given by
for x = 0, 1, …, y.• By the Law of Total Probability we have
Possible values of x: 0,1,2,…,6.
( ) 6,5,4,3,2,1,61
== yypY
( )2
| 21| ⎟⎠⎞
⎜⎝⎛⎟⎟⎠
⎞⎜⎜⎝
⎛=
xy
yxp YX
( ) ( ) ( ) ∑∑=
⋅⎟⎠⎞
⎜⎝⎛⎟⎟⎠
⎞⎜⎜⎝
⎛==
6
| 61
21|
xy
y
yYYXX x
yypyxpxp
week 8 11
Example• Interested in the 1st and 2nd success in repeated Bernoulli trails with probability of
success p on any trail. Let X = number of failures before 1st success andY = number of failures before 2nd success (including those before the 1st success).
• The marginal pmf of X and Y are for x = 0, 1, 2, ….
• andfor y = 0, 1, 2, ….
• The joint mass function, pX,Y(x,y) , where x, y are non-negative integers and x ≤ y is the probability that the 1st success occurred after x failure and the 2nd success after y – x more failures (after 1st success), e.g.
• The joint pmf is then given by
• Find the marginal probability functions of X, Y. Are X, Y independent? • What is the conditional probability function of X given Y = 5
( ) ( ) xX pqxXPxp ===
( ) ( ) ( ) yY qpyyYPyp 21+===
( ) ( ) 52, 5,2 qpFFSFFFSPp YX ==
( )⎩⎨⎧ ≤≤
=otherwise
yxqpyxp
y
YX 00
,2
,
week 8 12
Some Notes on Multiple Integration
• Recall: simple integral cab be regarded as the limit as N ∞ ofthe Riemann sum of the form
where ∆xi is the length of the ith subinterval of some partition of [a, b].
• By analogy, a double integral
where D is a finite region in the xy-plane, can be regarded as the limit as N ∞ of the Riemann sum of the form
where Ai is the area of the ith rectangle in a decomposition of D into rectangles.
( )∫b
adxxf
( )∑=
ΔN
iii xxf
1
( )∫∫D
dAyxf ,
( ) ( )∑∑==
ΔΔ=ΔN
iiiii
N
iiii yxyxfAyxf
11
,,
week 8 13
Evaluation of Double Integrals
• Idea – evaluate as an iterated integral. Restrict D now to a region such that any vertical line cuts its boundary in 2 points. So we can describe D by yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU
• Theorem
Let D be the subset of R2 defined by yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xUIf f(x,y) is continuous on D then
• Comment: When evaluating a double integral, the shape of the region or the form of f (x,y) may dictate that you interchange the role of x and y and integrate first w.r.t x and then y.
( ) ( )∫ ∫∫∫ ⎥⎦⎤
⎢⎣⎡= U
L
U
L
x
x
y
yD
dxdyyxfdAyxf ,,
week 8 14
Examples1) Evaluate where D is the triangle with vertices (0,0), (2,0) and (0,1).
2) Evaluate where D is the region in the 1st quadrate bounded
by y = 0, y = x and x2 + y2 = 1.
3) Integrate over the set of points in the positive quadrate for which y > 2x.
∫∫ ⋅D
ydAx
∫∫ +D
dAyxxy 22
( ) yxxeyxf 2, −−=
week 8 15
The Joint Distribution of two Continuous R.V’s• Definition
Random variables X and Y are (jointly) continuous if there is a non-negative function fX,Y(x,y) such that
for any “reasonable” 2-dimensional set A.
• fX,Y(x,y) is called a joint density function for (X, Y).
• In particular , if A = (X, Y): X ≤ x, Y ≤ x, the joint CDF of X,Y is
• From Fundamental Theorem of Calculus we have
( )( ) ( )∫∫=∈A
YX dxdyyxfAYXP ,, ,
( ) ( )∫ ∫∞− ∞−=
x y
YXYX dvduvufyxF ,,, ,,
( ) ( ) ( )yxFxy
yxFyx
yxf YXYXYX ,,, ,
2
,
2
. ∂∂∂
=∂∂∂
=
week 8 16
Properties of joint density function
• for all
• It’s integral over R2 is
( ) 0,, ≥yxf YX
( )∫ ∫∞
∞−
∞
∞−=1,, dxdyyxf YX
Ryx ∈,
week 8 17
Example• Consider the following bivariate density function
• Check if it’s a valid density function.
• Compute P(X > Y).
( ) ( )⎪⎩
⎪⎨⎧ ≤≤≤≤+
=otherwise
yxxyxyxf YX
0
10,107
12,
2
,
week 8 18
Properties of Joint Distribution Function
For random variables X, Y , FX,Y : R2 [0,1] given by FX,Y (x,y) = P(X ≤ x,Y ≤ x)
• FX,Y (x,y) is non-decreasing in each variable i.e.
if x1 ≤ x2 and y1 ≤ y2 .
• and
( ) 0,lim , =•∞−→∞−→
yxF YXyx
( ) 1,lim , =•∞→∞→
yxF YXyx
( ) ( )22,11, ,, yxFyxF YXYX ≤
( ) ( )yFyxF YYXx=
∞→,lim , ( ) ( )xFyxF XYXy
=∞→
,lim ,
week 8 19
Marginal Densities and Distribution Functions
• The marginal (cumulative) distribution function of X is
• The marginal density of X is then
• Similarly the marginal density of Y is
( ) ( ) ( )∫ ∫∞−∞
∞−=≤=
x
YXX dyduyufxXPxF ,,
( ) ( ) ( )∫∞
∞−== dyyxfxFxf YXXX ,,
'
( ) ( )∫∞
∞−= dxyxfyf YXY ,,
week 8 20
Example• Consider the following bivariate density function
• Check if it is a valid density function.
• Find the joint CDF of (X, Y) and compute P(X ≤ ½ ,Y ≤ ½ ).
• Compute P(X ≤ 2 ,Y ≤ ½ ).
• Find the marginal densities of X and Y.
( )⎩⎨⎧ ≤≤≤≤
=otherwise
yxxyyxf YX 0
10,106,
2
,
week 8 21
Generalization to higher dimensions
Suppose X, Y, Z are jointly continuous random variables with density f(x,y,z), then
• Marginal density of X is given by:
• Marginal density of X, Y is given by :
( ) ( )∫ ∫∞
∞−
∞
∞−= dydzzyxfxf X ,,
( ) ( )∫∞
∞−= dzzyxfyxf YX ,,,,
week 8 22
Example
Given the following joint CDF of X, Y
• Find the joint density of X, Y.
• Find the marginal densities of X and Y and identify them.
( ) 10,10, 2222, ≤≤≤≤−+= yxyxxyyxyxF YX