Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Normal Distribution, Continuous Analogs,Derived Distributions
EECS 126 Spring 2020
February 6, 2020
Agenda
Announcements
Review
Normal (Gaussian) Distribution
Analogs to Discrete Probability / RVs
Derived Distributions
Announcements
I Lab 1 and Lab 0 Self-Grades are due Friday (2/7).
I HW3 will be released today and is due next Wednesday(2/12).
I These slides will be posted later.
Questions
Let R be equal to the distance from the origin of a point randomlysampled on a unit ball. What is the
I CDF of R?
I PDF of R?
I Expectation of R?
Answers
Let R be the distance from the origin of a point randomly sampledon a unit ball. What is the
I CDF of R?FR(r) = P(R ≤ r) = 3
4π ·43πr
3 = r3.
I PDF of R?fR(r) = d
dr r3 = 3r2.
I Expectation of R?∫ 10 r · 3r2 = 3
4 .
Normal Distribution
The Normal distribution is seen abundantly in nature (e.g. examscores). This can be explained by the Central Limit Theorem(CLT), which we will go over later in the course.
The Normal distribution is parameterized by its mean µ andvariance σ2 (denoted N (µ, σ2)). Its PDF and CDF are
fX (x) =1√
2πσ2e−(x−µ)
2/2σ2
FX (x) = Φ(x), (cannot be expressed in elementary functions)
Proof that it Integrates to 1
We will show this when µ = 0 and σ2 = 1. Idea is to show(∫∞−∞ fX (x) dx
)2= 1.
(∫ ∞−∞
fX (x) dx)2
=(∫ ∞−∞
fX (x) dx)(∫ ∞
−∞fX (y) dy
)=
∫ ∞−∞
∫ ∞−∞
fX (x)fX (y) dy dx
=
∫ ∞−∞
∫ ∞−∞
( 1√2π
e−x2
2
)( 1√2π
e−y2
2
)dy dx
=
∫ ∞−∞
∫ ∞−∞
1
2πe−
x2+y2
2 dy dx
Proof that it Integrates to 1 (cont.)
Key is now to use polar integration: x2 + y2 = r2 anddy dx = r dr dθ.∫ ∞
−∞
∫ ∞−∞
1
2πe−
x2+y2
2 dy dx =
∫ 2π
0
∫ ∞0
1
2πe−
r2
2 r dr dθ
=
∫ ∞0
e−r2
2 r dr
Now letting u = r2
2 so that du = r dr∫ ∞0
e−r2
2 r dr =
∫ ∞0
e−u du =[−e−u
]u=∞u=0
=[−e−∞ − (−e−0)
]= 0− (−1) = 1
Properties of the Normal DistributionI The sum of two independent Normals is Normal. If
X ∼ N(µ1, σ21), Y ∼ N(µ2, σ
22), and Z = X + Y , then
Z ∼ N(µ1 + µ2, σ21 + σ22)
I The sum of two dependent Normals isn’t always Normal.Consider the following example.
X = N(0, 1)
Y =
{X w.p. 1
2
−X w.p. 12
They are both Normal but X + Y is not Normal.
I A Normal multiplied by a constant is Normal. IfX ∼ N(µ, σ2) and Y = aX , then
Y ∼ N(a · µ, a2 · σ2)
Scaling to the Standard Normal
I The properties on the previous slide allow us to convert anyNormal into the standard Normal.
I If X ∼ N(µ, σ2), then
Z =X − µσ
is distributed with Z ∼ N(0, 1).
I Intuition: I got SDs on midterm 1.
Question
Suppose male height is distributed as N (70, 5) and female heightis distributed as N (64, 4). What’s the probability that a randomlychosen male is taller than a randomly chosen female? You mayexpress your answer in terms of Φ, the standard normal CDF.
Answer
Express X ∼ N (70, 5) and Y ∼ N (64, 4). We want to findP(X > Y ).
P(X > Y ) = P(X + (−Y ) > 0)
= P(N (70, 5) + (−N (64, 4)) > 0)
= P(N (70, 5) +N (−1 · 64, (−1)2 · 4)) > 0)
= P(N (6, 9) > 0)
= P(N (0, 9) > −6)
= P(N (0, 1) >−6
3)
= P(N (0, 1) < 2)
So the answer is Φ(2) ≈ 0.977.
Joint PDFs
We know that multiple discrete RVs have a joint PMF. Similarly,multiple continuous RVs have a joint PDF.
I Discrete PMF: pX ,Y (x , y)
P((X ,Y ) ∈ A) =∑
(x ,y)∈A
pX ,Y (x , y)
I Continuous PDF: fX ,Y (x , y)
P((X ,Y ) ∈ A) =
∫AfX ,Y (x , y)dxdy
I Still needs to be non-negative.
I Still needs to integrate to 1.
Joint CDFs
I Both discrete and continuous RVs have the same definitionfor CDFs and joint CDFs:
FX (x) = P(X ≤ x)
FX ,Y (x , y) = P(X ≤ x ,Y ≤ y)
I Continuous RVs:
fX (x) =d
dxFX (x)
fX ,Y (x , y) =∂2
∂x∂yFX ,Y (x , y)
Marginal Probability Density
I Discrete
pX (x) =∑y∈Y
pX ,Y (x , y)
I Continuous
fX (x) =
∫ ∞−∞
fX ,Y (x , y) dy
I fX (x) is still a density, not a probability.
Conditional Probability Density
I Discrete
pX |Y (x | y) =pX ,Y (x , y)
pY (y)
I Continuous
fX |Y (x | y) =fX ,Y (x , y)
fY (y)
I By definition, Multiplication Rule still holds.
Independence
Similar to discrete, 3 equivalent definitions.
I For all x and y ,
fX ,Y (x , y) = fX (x)fY (y)
I For all x and y ,
fX |Y (x | y) = fX (x)
I For all x and y ,
fY |X (y | x) = fY (y)
Bayes RuleI Discrete (simple form)
pX |Y (x | y) =pY |X (y | x)pX (x)
pY (y)
I Discrete (extended form)
pX |Y (x | y) =pY |X (y | x)pX (x)∑
x ′∈X pY |X (y | x ′)pX (x ′)
I Continuous (simple form)
fX |Y (x | y) =fY |X (y | x)fX (x)
fY (y)
I Continuous (extended form)
fX |Y (x | y) =fY |X (y | x)fX (x)∫∞
−∞ fY |X (y | t)fX (t) dt
Conditional Expectation
I Discrete
E [Y | X = x ] =∑y∈Y
y · pY |X (y | x)
I Continuous
E [Y | X = x ] =
∫ ∞−∞
y · fY |X (y | x) dy
Combining Discrete and Continuous RVs
I You can also have discrete and continuous RVs defined jointly.
I Ex. let X be the outcome of a dice roll and Y be Exp(X ).
∀x ∈ {1, · · · , 6}, pX (x) =1
6fY |X (y | x) = xe−xy
Change of Variables / Derived Distributions
I Let X be a discrete RV with P(X = 0) = 12 , P(X = 1) = 1
2 ,and Y = 2X . Write down the distribution of Y .
I Change of variables in PMF:
P(Y = y) = P(2X = y) = P(X =y
2)
I In general, if X is a discrete RV, Y = f (X ),
P(Y = y) = P(f (X ) = y) = P(X ∈ f −1(y))
Change of Variables / Derived DistributionsI Let X ∼ U[0, 1], and Y = 2X . Then is it true that
fY (y) = P(Y = y) = P(2X = y) = P(X =y
2) = fX (
y
2)
I No, this won’t integrate to 1.
∀x ∈ [0, 1], fX (x) = 1∫ 2
0fY (y)dy =
∫ 2
0fX (
y
2)dy = 2
I You have to use the CDF.
FY (y) = P(Y ≤ y) = P(2X ≤ y) = P(X ≤ y
2) = FX (
y
2)
I Taking the derivative on both sides,
fY (y) =d
dyFX (
y
2) =
1
2· fX (
y
2).
Change of Variables / Derived Distributions
I Let X ∼ U[0, 1], and Y = 2X . Then is it true that
fY (y) = P(Y = y) = P(2X = y) = P(X =y
2) = fX (
y
2)
I Note: PDF does not satisfy fY (y) = P(Y = y).
fY (y)dy = P(y ≤ Y ≤ y + dy)
= P(y ≤ 2X ≤ y + dy)
= P(y
2≤ X ≤ y
2+
dy
2)
= fX (y
2) · dy
2
This implies fY (y) = 12 · fX ( y2 ).
Change of Variables / Derived Distributions
I In general, if X is continuous RV, Y = g(X ). Then we have
FY (y) = P(Y ≤ y) = P(g(X ) ≤ y).
I When g is monotone increasing,
P(g(X ) ≤ y) = P(X ≤ g−1(y)) = FX (g−1(y)) (1)
I When g is monotone decreasing,
P(g(X ) ≤ y) = P(X ≥ g−1(y)) = 1− FX (g−1(y)) (2)
References
Introduction to probability. DP Bertsekas, JN Tsitsiklis - 2002