Embed Size (px)
Citation preview
Functions of Random Variables
Methods for determining the distribution of
functions of Random Variables
1. Distribution function method
2. Moment generating function method
3. Transformation method
Distribution function method
Let X, Y, Z …. have joint density f(x,y,z, …)
Let W = h( X, Y, Z, …)
First step
Find the distribution function of W
G(w) = P[W ≤ w] = P[h( X, Y, Z, …) ≤ w]
Second step
Find the density function of W
g(w) = G'(w).
Example 1
Let X have a normal distribution with mean 0, and
variance 1. (standard normal distribution)
Let W = X2.
Find the distribution of W.
2
21
2
x
f x e
First step
Find the distribution function of W
G(w) = P[W ≤ w] = P[ X2 ≤ w]
if 0P w X w w
2
21
2
w x
w
e dx
F w F w
where 2
21
2
x
F x f x e
d wd w
F w F wdw dw
Second step
Find the density function of W
g(w) = G'(w).
1 1
2 2 2 21 1 1 1
2 22 2
w w
e w e w
1 1
2 21 1
2 2f w w f w w
1
2 21
if 0.2
w
w e w
Thus if X has a standard Normal distribution then
W = X2
has density
1
2 21
if 0.2
w
g w w e w
This distribution is the Gamma distribution with a = ½
and l = ½.
This distribution is also the c2 distribution with n = 1
degree of freedom.
Example 2
Suppose that X and Y are independent random
variables each having an exponential distribution
with parameter l (mean 1/l)
Let W = X + Y.
Find the distribution of W.
1 for 0xf x e xll
2 for 0yf y e yll
1 2,f x y f x f y
2 for 0, 0x y
e x yl
l
First step
Find the distribution function of W = X + Y
G(w) = P[W ≤ w] = P[ X + Y ≤ w]
1 2
0 0
w w x
P X Y w f x f y dydx
2
0 0
w w xx y
e dydxl
l
1 2
0 0
w w x
P X Y w f x f y dydx
2
0 0
w w xx y
e dydxl
l
2
0 0
w w x
x ye e dy dxl ll
2
0 0
w xw yx e
e dxl
lll
02
0
w w x
x e ee dx
lll
l
P X Y w 0
2
0
w w x
x e ee dx
lll
l
0
w
x we e dxl ll
0
wx
wexe
lll
l
0wwe e
wel
lll l
1 w we wel ll
Second step
Find the density function of W
g(w) = G'(w).
1 w wde we
dw
l ll
ww wdw de
e e wdw dw
ll ll l
2w w we e wel l ll l l
2 for 0wwe wll
Hence if X and Y are independent random variables each having an exponential distribution with parameter l then W has density
2 for 0wg w we wll
This distribution can be recognized to be the Gamma distribution with parameters a = 2 and l.
Example: Student’s t distribution
Let Z and U be two independent random
variables with:
1. Z having a Standard Normal distribution
and
2. U having a c2 distribution with n degrees
of freedom
Find the distribution of Z
tUn
The density of Z is:
2
21
2
z
f z e
The density of U is:
2
12 2
1
2
2
u
h u u e
n
n
n
Therefore the joint density of Z and U is:
The distribution function of T is:
Z t
G t P T t P t P Z UU nn
2
2
12 2
1
2,
22
z u
f z u f z h u u e
n
n
n
Therefore:
t
G t P T t P Z Un
2
2
12 2
0
1
2
22
tu
z u
u e dzdu
n
nn
n
Illustration of limits
U
U
z z
t > 0 t < 0
Now:
2
2
12 2
0
1
2( )
22
tu
z u
G t u e dzdu
n
nn
n
and:
2
2
12 2
0
1
2( )
22
tu
z ud
g t G t u e dz dudt
n
nn
n
Using:
b
a
b
a
dxtxFdt
ddxtxF
dt
d),(),(
Using the fundamental theorem of calculus:
( )
x
a
F x f t dt
then
2
2
12 2
0
1
2
22
tu
z ud
g t u e dz dudt
n
nn
n
If then ( )F x f x
2
2
122 2
0
1
2
22
t uuu
u e e du
n
n
n
n n
Hence
221
1
2 2
0
1
2( )
22
tu
g t u e du
n
nn
n n
Using
1
0
xx e dxa l
a
a
l
1
0
1 xx e dxa
a ll
a
or
Hence 2 1
1 21
2 21
20 2
12
2
1
tu
u e du
t
n
nn
n
n
n
and 1
212
2 2
1 12
2 2( ) 1
22
tg t
nn
nn
n n n
or
1 12 22 2
1
2( ) 1 1
2
t tg t K
n nn
n n nn
1
2
2
K
n
nn
where
Student’s t distribution
12 2
( ) 1t
g t K
n
n
1
2
2
K
n
nn
where
Student – W.W. Gosset
Worked for a distillery
Not allowed to publish
Published under the
pseudonym “Student
t distribution
standard normal distribution
Functions of Random Variables
Methods for determining the distribution of
functions of Random Variables
1. Distribution function method
2. Moment generating function method
3. Transformation method
Distribution function method
Let X, Y, Z …. have joint density f(x,y,z, …)
Let W = h( X, Y, Z, …)
First step
Find the distribution function of W
G(w) = P[W ≤ w] = P[h( X, Y, Z, …) ≤ w]
Second step
Find the density function of W
g(w) = G'(w).
Distribution of the Max and Min
Statistics
Let x1, x2, … , xn denote a sample of size n from
the density f(x).
Let M = max(xi) then determine the distribution
of M.
Repeat this computation for m = min(xi)
Assume that the density is the uniform density
from 0 to q.
Hence
10
( )
elsewhere
xf x
and the distribution function
0 0
( ) 0
1
x
xF x P X x x
x
q
Finding the distribution function of M.
( ) max iG t P M t P x t
1 , , nP x t x t
1 nP x t P x t
0 0
0
1
n
t
tt
t
q
Differentiating we find the density function of M.
1
0
0 otherwise
n
n
ntt
g t G tq
q
0
0.1
0.2
0.3
0.4
0.5
0.6
0 2 4 6 8 10
0
0.02
0.04
0.06
0.08
0.1
0.12
0 2 4 6 8 10
f(x) g(t)
Finding the distribution function of m.
( ) min iG t P m t P x t
11 , , nP x t x t
11 nP x t P x t
0 0
1 1 0
1
n
t
tt
t
q
0
0.1
0.2
0.3
0.4
0.5
0.6
0 2 4 6 8 10
Differentiating we find the density function of m.
1
1 0
0 otherwise
nn t
tg t G t
qq q
0
0.02
0.04
0.06
0.08
0.1
0.12
0 2 4 6 8 10
f(x) g(t)
The probability integral transformation
This transformation allows one to convert
observations that come from a uniform
distribution from 0 to 1 to observations that
come from an arbitrary distribution.
Let U denote an observation having a uniform
distribution from 0 to 1.
1 0 1
( )elsewhere
ug u
Find the distribution of X.
1( )X F ULet
Let f(x) denote an arbitrary density function and
F(x) its corresponding cumulative distribution
function.
1( )G x P X x P F U x
P U F x
F x
Hence.
g x G x F x f x
has density f(x).
1( )X F U
Thus if U has a uniform distribution from 0 to 1.
Then
U
1( )X F U
The Transformation Method
Theorem
Let X denote a random variable with
probability density function f(x) and U = h(X).
Assume that h(x) is either strictly increasing
(or decreasing) then the probability density of
U is:
1
1 ( )( )
dh u dxg u f h u f x
du du
Proof
Use the distribution function method.
Step 1 Find the distribution function, G(u)
Step 2 Differentiate G (u ) to find the
probability density function g(u)
G u P U u P h X u
1
1
( ) strictly increasing
( ) strictly decreasing
P X h u h
P X h u h
1
1
( ) strictly increasing
1 ( ) strictly decreasing
F h u h
F h u h
hence
g u G u
1
1
1
1
strictly increasing
strictly decreasing
dh uF h u h
du
dh uF h u h
du
or
1
1 ( )( )
dh u dxg u f h u f x
du du
Example
Suppose that X has a Normal distribution
with mean m and variance s2.
Find the distribution of U = h(x) = eX.
Solution:
2
221
2
x
f x e
m
s
s
1
1ln 1
ln and dh u d u
h u udu du u
hence
1
1 ( )( )
dh u dxg u f h u f x
du du
2
2
ln
21 1
for 02
u
e uu
m
s
s
This distribution is called the log-normal
distribution
log-normal distribution
0
0.02
0.04
0.06
0.08
0.1
0 10 20 30 40
The Transfomation Method
(many variables) Theorem
Let x1, x2,…, xn denote random variables
with joint probability density function
f(x1, x2,…, xn )
Let u1 = h1(x1, x2,…, xn).
u2 = h2(x1, x2,…, xn).
un = hn(x1, x2,…, xn).
define an invertible transformation from the x’s to the u’s
⁞
Then the joint probability density function of
u1, u2,…, un is given by:
1
1 1
1
, ,, , , ,
, ,
n
n n
n
d x xg u u f x x
d u u
1, , nf x x J
where
1
1
, ,
, ,
n
n
d x xJ
d u u
Jacobian of the transformation
1 1
1
1
det
n
n n
n
dx dx
du du
dx dx
du du
Example Suppose that x1, x2 are independent with density
functions f1 (x1) and f2(x2)
Find the distribution of
u1 = x1+ x2
u2 = x1 - x2
Solving for x1 and x2 we get the inverse transformation
1 21
2
u ux
1 22
2
u ux
1 2
1 2
,
,
d x xJ
d u u
The Jacobian of the transformation
1 1
1 2
2 2
1 2
det
dx dx
du du
dx dx
du du
1 1
1 1 1 1 12 2det
1 1 2 2 2 2 2
2 2
The joint density of x1, x2 is
f(x1, x2) = f1 (x1) f2(x2)
Hence the joint density of u1 and u2 is:
1 2 1 21 2
1
2 2 2
u u u uf f
1 2 1 2, ,g u u f x x J
From
1 2 1 21 2 1 2
1,
2 2 2
u u u ug u u f f
We can determine the distribution of u1= x1 + x2
1 1 1 2 2,g u g u u du
1 2 1 21 2 2
1
2 2 2
u u u uf f du
1 2 1 21
2
1put then ,
2 2 2
u u u u dvv u v
du
Hence
1 2 1 21 1 1 2 2
1
2 2 2
u u u ug u f f du
1 2 1f v f u v dv
This is called the convolution of the two
densities f1 and f2.
Example: The ex-Gaussian distribution
1. X has an exponential distribution with
parameter l.
2. Y has a normal (Gaussian) distribution with
mean m and standard deviation s.
Let X and Y be two independent random
variables such that:
Find the distribution of U = X + Y.
This distribution is used in psychology as a model
for response time to perform a task.
Now 1
0
0 0
xe xf x
x
ll
1 2g u f v f u v dv
The density of U = X + Y is :.
2
222
1
2
x
f y e
m
s
s
2
22
0
1
2
u v
ve e dv
m
l sls
or
2
22
02
u vv
g u e dv
ml
sl
s
2 2
2
2
2
02
u v v
e dv
m s l
sl
s
22 2
2
2 2
2
02
v u v u v
e dv
m m s l
sl
s
2 22
2 2
2
2 2
02
v u vu
e e dv
m s lm
s sl
s
or 2 22
2 2
2
2 2
02
v u vu
e e dv
m s lm
s sl
s
2 22 2 2 2 2
2 2
2
2 2
02
u u v u v u
e e dv
m m s l m s l m s l
s sl
s
2 22 2 2 2 2
2 2
2
2 2
0
1
2
u u v u v u
e e dv
m m s l m s l m s l
s sls
22 2
22 0
u u
e P V
m m s l
sl
Where V has a Normal distribution with mean
22
2
21
u ug u e
s ll m m s l
ls
2
V um m s l
and variance s2.
Hence
Where (z) is the cdf of the standard Normal
distribution
0
0.03
0.06
0.09
0 10 20 30
g(u)
The ex-Gaussian distribution
Functions of Random Variables
Methods for determining the distribution of
functions of Random Variables
1. Distribution function method
2. Moment generating function method
3. Transformation method
Distribution function method
Let X, Y, Z …. have joint density f(x,y,z, …)
Let W = h( X, Y, Z, …)
First step
Find the distribution function of W
G(w) = P[W ≤ w] = P[h( X, Y, Z, …) ≤ w]
Second step
Find the density function of W
g(w) = G'(w).
The Transformation Method
Theorem
Let X denote a random variable with
probability density function f(x) and U = h(X).
Assume that h(x) is either strictly increasing
(or decreasing) then the probability density of
U is:
1
1 ( )( )
dh u dxg u f h u f x
du du
The Transfomation Method
(many variables) Theorem
Let x1, x2,…, xn denote random variables
with joint probability density function
f(x1, x2,…, xn )
Let u1 = h1(x1, x2,…, xn).
u2 = h2(x1, x2,…, xn).
un = hn(x1, x2,…, xn).
define an invertible transformation from the x’s to the u’s
⁞
Then the joint probability density function of
u1, u2,…, un is given by:
1
1 1
1
, ,, , , ,
, ,
n
n n
n
d x xg u u f x x
d u u
1, , nf x x J
where
1
1
, ,
, ,
n
n
d x xJ
d u u
Jacobian of the transformation
1 1
1
1
det
n
n n
n
dx dx
du du
dx dx
du du
Use of moment generating
functions
Definition
Let X denote a random variable with probability
density function f(x) if continuous (probability mass
function p(x) if discrete)
Then
mX(t) = the moment generating function of X
tXE e
if is continuous
if is discrete
tx
tx
x
e f x dx X
e p x X
The distribution of a random variable X is
described by either 1. The density function f(x) if X continuous (probability
mass function p(x) if X discrete), or
2. The cumulative distribution function F(x), or
3. The moment generating function mX(t)
Properties
1. mX(0) = 1
0 derivative of at 0.k th
X Xm k m t t 2.
k
k E Xm
2 33211 .
2! 3! !
kkXm t t t t t
k
m mmm 3.
continuous
discrete
k
k
k k
x f x dx XE X
x p x Xm
4. Let X be a random variable with moment
generating function mX(t). Let Y = bX + a
Then mY(t) = mbX + a(t)
= E(e [bX + a]t) = eatE(e X[ bt ])
= eatmX (bt)
5. Let X and Y be two independent random
variables with moment generating function
mX(t) and mY(t) .
Then mX+Y(t) = E(e [X + Y]t) = E(e Xt e Yt)
= E(e Xt) E(e Yt)
= mX (t) mY (t)
6. Let X and Y be two random variables with
moment generating function mX(t) and mY(t)
and two distribution functions FX(x) and
FY(y) respectively.
Let mX (t) = mY (t) then FX(x) = FY(x).
This ensures that the distribution of a random
variable can be identified by its moment
generating function
M. G. F.’s - Continuous distributions
Name
Moment generating
function MX(t)
Continuous
Uniform
ebt-eat
[b-a]t
Exponential l
l t
for t < l
Gamma l
l t
a
for t < l
c2
nd.f.
1
1-2t
n/2
for t < 1/2
Normal etm+(1/2)t2s2
M. G. F.’s - Discrete distributions
Name
Moment
generating
function MX(t)
Discrete
Uniform
et
N etN-1
et-1
Bernoulli q + pet
Binomial (q + pet)N
Geometric pet
1-qet
Negative
Binomial
pet
1-qet k
Poisson el(et-1)
Moment generating function of the
gamma distribution
tX tx
Xm t E e e f x dx
1 0
0 0
xx e xf x
x
aa ll
a
where
tX tx
Xm t E e e f x dx
1
0
tx xe x e dxa
a ll
a
using
1
0
t xx e dx
alal
a
1
0
1a
a bxbx e dx
a
1
0
a bx
a
ax e dx
b
or
then
1
0
t x
Xm t x e dxa
lal
a
t
a
a
al
a l
tt
al
ll
Moment generating function of the
Standard Normal distribution
tX tx
Xm t E e e f x dx
2
21
2
x
f x e
where
thus
2 2
2 21 1
2 2
x xtx
tx
Xm t e e dx e dx
We will use
2
22
0
11
2
x a
be dxb
2
21
2
xtx
Xm t e dx
2 2
21
2
x tx
e dx
22 2 2 22
2 2 2 21 1
2 2
x tx tx t t t
e e dx e e dx
2
2
t
e
Note:
2
2 32 2
2
22 2
12 2! 3!
t
X
t t
tm t e
2 3 4
12! 3! 4!
x x x xe x
2 4 6 2
2 31
2 2 2! 2 3! 2 !
m
m
t t t t
m
Also
2 33211
2! 3!Xm t t t t
mmm
Note:
2
2 32 2
2
22 2
12 2! 3!
t
X
t t
tm t e
2 3 4
12! 3! 4!
x x x xe x
2 4 6 2
2 31
2 2 2! 2 3! 2 !
m
m
t t t t
m
Also 2 33211
2! 3!Xm t t t t
mmm
momentth k
k k x f x dxm
Equating coefficients of tk, we get
21
for 2 then 2 ! 2 !
m
mk m
m m
m
0 if is odd andk km
1 2 3 4hence 0, 1, 0, 3m m m m
Using of moment generating
functions to find the distribution of
functions of Random Variables
Example
Suppose that X has a normal distribution with
mean m and standard deviation s.
Find the distribution of Y = aX + b
2 2
2
tt
Xm t es
m
Solution:
22
2
atat
bt bt
aX b Xm t e m at e e
sm
2 2 2
2
a ta b t
es
m
= the moment generating function of the normal
distribution with mean am + b and variance a2s2.
Thus Z has a standard normal distribution .
Special Case: the z transformation
1XZ X aX b
m m
s s s
10Z a b
mm m m
s s
2
2 2 2 211Z as s s
s
Thus Y = aX + b has a normal distribution with
mean am + b and variance a2s2.
Example Suppose that X and Y are independent each having a normal distribution with means mX and mY , standard deviations sX and sY
Find the distribution of S = X + Y
2 2
2
XX
tt
Xm t es
m
Solution:
2 2
2
YY
tt
Ym t es
m
2 2 2 2
2 2
X YX Y
t tt t
X Y X Ym t m t m t e es s
m m
Now
or
2 2 2
2
X Y
X Y
tt
X Ym t e
s sm m
= the moment generating function of the
normal distribution with mean mX + mY and
variance
2 2
X Ys s
Thus Y = X + Y has a normal distribution
with mean mX + mY and variance
2 2
X Ys s
Example Suppose that X and Y are independent each having a normal distribution with means mX and mY , standard deviations sX and sY
Find the distribution of L = aX + bY
2 2
2
XX
tt
Xm t es
m
Solution:
2 2
2
YY
tt
Ym t es
m
aX bY aX bY X Ym t m t m t m at m bt
Now
2 22 2
2 2
X YX Y
at btat bt
e e
s sm m
or
2 2 2 2 2
2
X Y
X Y
a b ta b t
aX bYm t e
s sm m
= the moment generating function of the
normal distribution with mean amX + bmY
and variance
2 2 2 2
X Ya bs s
Thus L = aX + bY has a normal
distribution with mean amX + bmY and
variance
2 2 2 2
X Ya bs s
Special Case:
Thus Y = X - Y has a normal distribution
with mean mX - mY and variance
2 22 2 2 21 1
X Y X Ys s s s
a = +1 and b = -1.
Example (Extension to n independent RV’s)
Suppose that X1, X2, …, Xn are independent each having a normal distribution with means mi, standard deviations si
(for i = 1, 2, … , n)
Find the distribution of L = a1X1 + a2X2 + …+ anXn
2 2
2
ii
i
tt
Xm t es
m
Solution:
1 1 1 1n n n na X a X a X a Xm t m t m t Now
22 221 1
1 12 2
n nn n
a ta ta t a t
e e
ssm m
(for i = 1, 2, … , n)
1 1 nX X nm a t m a t
or
2 2 2 2 21 1
1 1
1 1
......
2
n n
n n
n n
a a ta a t
a X a Xm t e
s sm m
= the moment generating function of the
normal distribution with mean
and variance
Thus Y = a1X1 + … + anXn has a normal
distribution with mean a1m1 + …+ anmn
and variance
1 1 ... n na am m 2 2 2 2
1 1 ... n na as s
2 2 2 2
1 1 ... n na as s
1 2
1na a a
n
1 2 nm m m m
2 2 2 2
1 2 ns s s s
In this case X1, X2, …, Xn is a sample from a
normal distribution with mean m, and standard
deviations s,and
1 2
1nL X X X
n
the sample meanX
Special case:
Thus
2 2 2 2 2
1 1 ...x n na as s s
and variance
1 1 ...x n na am m m
has a normal distribution with mean
1 1 ... n nY x a x a x
11 1... nx x
n n
1 1...n nm m m
2 2 2 22 2 21 1 1
... nn n n n
ss s s
If x1, x2, …, xn is a sample from a normal
distribution with mean m, and standard
deviations s,then the sample meanx
Summary
22
xn
ss
and variance
xm m
has a normal distribution with mean
standard deviation xn
ss
0
0.1
0.2
0.3
0.4
20 30 40 50 60
Population
Sampling distribution
of x
Suppose x1, x2, …, xn is a sample (independent
identically distributed – i.i.d.) from a
distribution with mean m,
the sample meanx
The Law of Large Numbers
Then
1 as for all 0P x nm
Let
Proof: Previously we used Tchebychev’s Theorem.
This assumes s(s2) is finite.
We will use the following fact:
Let
m1(t), m2(t), …
denote a sequence of moment generating functions
corresponding to the sequence of distribution
functions:
F1(x) , F2(x), …
Let m(t) be a moment generating function
corresponding to the distribution function F(x) then
if
Proof: (use moment generating functions)
lim for all in an interval about 0.ii
m t m t t
lim for all .ii
F x F x x
then
Let x1, x2, … denote a sequence of independent
random variables coming from a distribution with
moment generating function m(t) and distribution
function F(x).
1 2 1 2
=n n nS x x x x x xm t m t m t m t m t
Let Sn = x1 + x2 + … + xn then
=n
m t
1 2now n nx x x Sx
n n
1or
nn
n
x SS
n
t tm t m t m m
n n
using L’Hopitals rule
now ln ln ln
n
x
t tm t m n m
n n
ln where
t m u tu
u n
0
lnThus lim ln limx
n u
t m um t
u
0
0lim
1 0u
m ut
m u mt t
mm
is the moment generating function of
a random variable that takes on the value m with
probability 1.
and lim for all values of .xn
F x F x x
tm t e m
1
i.e. andx
p xx
m
m
0
and distribution function and1
xF x
x
m
m
Thus lim t
xn
m t m t e m
Now
0
since and1
xF x
x
m
m
P x P xm m m
x xF Fm m
1 if 0F Fm m
as n
Q.E.D.
If x1, x2, …, xn is a sample from a distribution
with mean m, and standard deviations s,then
if n is large the sample meanx
The Central Limit theorem
22
xn
ss
and variance
xm m
has a normal distribution with mean
standard deviation xn
ss
We will use the following fact:
Let
m1(t), m2(t), …
denote a sequence of moment generating functions
corresponding to the sequence of distribution
functions:
F1(x) , F2(x), …
Let m(t) be a moment generating function
corresponding to the distribution function F(x) then
if
Proof: (use moment generating functions)
lim for all in an interval about 0.ii
m t m t t
lim for all .ii
F x F x x
then
Let x1, x2, … denote a sequence of independent
random variables coming from a distribution with
moment generating function m(t) and distribution
function F(x).
1 2 1 2
=n n nS x x x x x xm t m t m t m t m t
Let Sn = x1 + x2 + … + xn then
=n
m t
1 2now n nx x x Sx
n n
1or
nn
n
x SS
n
t tm t m t m m
n n
Let x n n
z x
n
m m
s s s
then
nn n
t t
z x
nt ntm t e m e m
n
m m
s s
s s
and ln lnz
n tm t t n m
n
m
s s
Then ln lnz
n tm t t n m
n
m
s s
2 2
2 2 2ln
t tm u
u u
m
s s
2
2 2Let or and
t t tu n n
u un s ss
2
2 2
ln m u ut
u
m
s
0
Now lim ln lim lnz zn u
m t m t
2
2 20
lnlimu
m u ut
u
m
s
2
2 0lim using L'Hopital's rule
2u
m u
m ut
u
m
s
2
22
2 0lim using L'Hopital's rule again
2u
m u m u m u
m ut
s
2
22
2 0lim using L'Hopital's rule again
2u
m u m u m u
m ut
s
2
2
2
0 0
2
m mt
s
222 2
2 2 2
i iE x E xt t
s
2
2
2thus lim ln and lim2
t
z zn n
tm t m t e
2
2Now t
m t e
Is the moment generating function of the standard
normal distribution
Thus the limiting distribution of z is the standard
normal distribution
2
21
i.e. lim2
x u
zn
F x e du
Q.E.D.
The Central Limit theorem
illustrated
If x1, x2, …, xn is a sample from a distribution
with mean m, and standard deviations s,then
if n is large the sample meanx
The Central Limit theorem
22
xn
ss
and variance
xm m
has a normal distribution with mean
standard deviation xn
ss
If x1, x2 are independent from the uniform
distirbution from 0 to 1. Find the distribution
of: the sample meanx
The Central Limit theorem illustrated
1 21 2 and
2 2
x xSS x x x
let
1 2G s P S s P x x s Now
2
2
0 0
0 12
21 1 2
2
1 1
s
s s
ss
s
0 1
2 1 2
0 otherwise
s s
g s G s s s
Now: 12
2
Sx S aS
The density of is:x
2 2dS
h x g S g xdx
12
12
2 0 2 1 2 0
2 2 1 2 2 2 1 1
0 otherwise 0 otherwise
x x x x
x x x x
n = 1
1 0
1 0
n = 2
n = 3
1 0
Distributions of functions of
Random Variables
Gamma distribution, c2 distribution,
Exponential distribution
Therorem
Let X and Y denote a independent random variables
each having a gamma distribution with parameters
(l,a1) and (l,a2). Then W = X + Y has a gamma
distribution with parameters (l, a1 + a2).
Proof:
1 2
and X Ym t m tt t
a al l
l l
1 2 1 2
t t t
a a a al l l
l l l
Therefore X Y X Ym t m t m t
Recognizing that this is the moment generating
function of the gamma distribution with parameters
(l, a1 + a2) we conclude that W = X + Y has a
gamma distribution with parameters (l, a1 + a2).
Therorem (extension to n RV’s)
Let x1, x2, … , xn denote n independent random
variables each having a gamma distribution with
parameters (l,ai), i = 1, 2, …, n.
Then W = x1 + x2 + … + xn has a gamma distribution
with parameters (l, a1 + a2 +… + an).
Proof:
1,2...,i
ixm t i nt
al
l
1 2 1 2 ...
...n n
t t t t
a a a a a al l l l
l l l l
1 2 1 2... ...
n nx x x x x xm t m t m t m t
Recognizing that this is the moment generating
function of the gamma distribution with parameters
(l, a1 + a2 +…+ an) we conclude that
W = x1 + x2 + … + xn has a gamma distribution with
parameters (l, a1 + a2 +…+ an).
Therefore
Therorem
Suppose that x is a random variable having a
gamma distribution with parameters (l,a).
Then W = ax has a gamma distribution with
parameters (l/a, a).
Proof:
xm tt
al
l
then ax xam t m at
at ta
aa l
l
ll
1. Let X and Y be independent random variables
having an exponential distribution with parameter
l then X + Y has a gamma distribution with a= 2
and l
Special Cases
2. Let x1, x2,…, xn, be independent random variables
having a exponential distribution with parameter l
then S = x1+ x2 +…+ xn has a gamma distribution
with a= n and l3. Let x1, x2,…, xn, be independent random variables
having a exponential distribution with parameter l
then
has a gamma distribution with a= n and nl
1 nx xSx
n n
0
0.1
0.2
0.3
0.4
0.5
0.6
0 5 10 15 20
pop'n
n = 4
n = 10
n = 15
n = 20
Distribution of
population – Exponential distribution
x
Another illustration of the central limit theorem
4. Let X and Y be independent random variables
having a c2 distribution with n1 and n2 degrees of
freedom respectively then X + Y has a c2
distribution with degrees of freedom n1 + n2.
Special Cases -continued
5. Let x1, x2,…, xn, be independent random variables
having a c2 distribution with n1 , n2 ,…, nn degrees
of freedom respectively then x1+ x2 +…+ xn has a
c2 distribution with degrees of freedom n1 +…+ nn.
Both of these properties follow from the fact that a
c2 random variable with n degrees of freedom is a
random variable with l= ½ and a = n/2.
If z has a Standard Normal distribution then z2 has a
c2 distribution with 1 degree of freedom.
Recall
Thus if z1, z2,…, zn are independent random variables
each having Standard Normal distribution then
has a c2 distribution with n degrees of freedom.
2 2 2
1 2 ...U z z zn
Therorem
Suppose that U1 and U2 are independent random
variables and that U = U1 + U2 Suppose that U1
and U have a c2 distribution with degrees of
freedom n1andn respectively. (n1 < n)
Then U2 has a c2 distribution with degrees of
freedom n2 =n -n1
Proof:
12
1
12
12
Now
v
Um tt
212
12
and
v
Um tt
1 2
Also U U Um t m t m t
2
12 2
12
12
1122
11 22
12
v
vv
v
t
t
t
2
1
Hence U
U
U
m tm t
m t
Q.E.D.
Tables for Standard Normal
distribution
