Upload
jasper-hines
View
251
Download
4
Tags:
Embed Size (px)
Citation preview
Random Variables
• Random Variable (RV): A numeric outcome that results from an experiment
• For each element of an experiment’s sample space, the random variable can take on exactly one value
• Discrete Random Variable: An RV that can take on only a finite or countably infinite set of outcomes
• Continuous Random Variable: An RV that can take on any value along a continuum (but may be reported “discretely”
• Random Variables are denoted by upper case letters (Y)• Individual outcomes for RV are denoted by lower case
letters (y)
Probability Distributions
• Probability Distribution: Table, Graph, or Formula that describes values a random variable can take on, and its corresponding probability (discrete RV) or density (continuous RV)
• Discrete Probability Distribution: Assigns probabilities (masses) to the individual outcomes
• Continuous Probability Distribution: Assigns density at individual points, probability of ranges can be obtained by integrating density function
• Discrete Probabilities denoted by: p(y) = P(Y=y)• Continuous Densities denoted by: f(y)• Cumulative Distribution Function: F(y) = P(Y≤y)
Discrete Probability Distributions
yyF
FF
ypbYPbF
yYPyF
yp
yyp
yYPyp
b
y
y
in increasinglly monotonica is )(
1)(0)(
)()()(
)()(
:(CDF)Function on Distributi Cumulative
1)(
0)(
)()(
:Function (Mass)y Probabilit
all
Example – Rolling 2 Dice (Red/Green)
Red\Green 1 2 3 4 5 6
1 2 3 4 5 6 72 3 4 5 6 7 83 4 5 6 7 8 94 5 6 7 8 9 105 6 7 8 9 10 116 7 8 9 10 11 12
Y = Sum of the up faces of the two die. Table gives value of y for all elements in S
Rolling 2 Dice – Probability Mass Function & CDF
y p(y) F(y)
2 1/36 1/36
3 2/36 3/36
4 3/36 6/36
5 4/36 10/36
6 5/36 15/36
7 6/36 21/36
8 5/36 26/36
9 4/36 30/36
10 3/36 33/36
11 2/36 35/36
12 1/36 36/36
y
t
tpyF
yyp
2
)()(
inresult can die 2 waysof #
tosumcan die 2 waysof #)(
Rolling 2 Dice – Probability Mass FunctionDice Rolling Probability Function
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
2 3 4 5 6 7 8 9 10 11 12
y
p(y
)
Rolling 2 Dice – Cumulative Distribution FunctionDice Rolling - CDF
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4 5 6 7 8 9 10 11 12 13
y
F(y
)
Expected Values of Discrete RV’s
• Mean (aka Expected Value) – Long-Run average value an RV (or function of RV) will take on
• Variance – Average squared deviation between a realization of an RV (or function of RV) and its mean
• Standard Deviation – Positive Square Root of Variance (in same units as the data)
• Notation:– Mean: E(Y) = – Variance: V(Y) = 2
– Standard Deviation:
Expected Values of Discrete RV’s
2
2222
all
2
all all
2
all
22
all
2
222
all
all
:Deviation Standard
)1()(2
)()(2)(
)(2)()(
)())(()( :Variance
)()()( :)(function a ofMean
)()( :Mean
YEYE
ypyypypy
ypyyypy
YEYEYEYV
ypygYgEYg
yypYE
yyy
yy
y
y
Expected Values of Linear Functions of Discrete RV’s
a
aypya
ypyaypaay
ypbabaybaYV
baypbyypa
ypbaybaYE
babaYYg
baY
y
yy
y
yy
y
22
all
22
all
22
all
2
all
2
all all
all
)()(
)()(
)()()(][
)()(
)()(][
)constants ,()( :FunctionsLinear
Example – Rolling 2 Dice
y p(y) yp(y) y2p(y)
2 1/36 2/36 4/36
3 2/36 6/36 18/36
4 3/36 12/36 48/36
5 4/36 20/36 100/36
6 5/36 30/36 180/36
7 6/36 42/36 294/36
8 5/36 40/36 320/36
9 4/36 36/36 324/36
10 3/36 30/36 300/36
11 2/36 22/36 242/36
12 1/36 12/36 144/36
Sum 36/36=1.00
252/36=7.00
1974/36=54.833
4152.28333.5
8333.5)0.7(8333.54
)(
0.7)()(
2
212
2
2222
12
2
y
y
ypyYE
yypYE
Tchebysheff’s Theorem/Empirical Rule
• Tchebysheff: Suppose Y is any random variable with mean and standard deviation . Then: P(-k≤ Y ≤ +k) ≥ 1-(1/k2) for k ≥ 1– k=1: P(-1≤ Y ≤ +1) ≥ 1-(1/12) = 0 (trivial result)– k=2: P(-2≤ Y ≤ +2) ≥ 1-(1/22) = ¾– k=3: P(-3≤ Y ≤ +3) ≥ 1-(1/32) = 8/9
• Note that this is a very conservative bound, but that it works for any distribution
• Empirical Rule (Mound Shaped Distributions)– k=1: P(-1≤ Y ≤ +1) 0.68– k=2: P(-2≤ Y ≤ +2) 0.95– k=3: P(-3≤ Y ≤ +3) 1
Proof of Tchebysheff’s Theorem
2222
2
22
22222
22)(
)(
2222
222
222
)(
2)(
)(
2)(
2
22
11)()(1
1
)(1
)()(
)()()()(
)( : )Region In
)( : )Region In
)()()()()()(
)()()(
:Variance of definition theof use Making
),)[())](),[()])(()
:parts 3 into line real Breaking
kkYkPkYkP
kk
kYkPk
kYPkkYPk
kYPkypykYPk
kykyiii
kykyi
ypyypyypy
ypyYV
kμiiikμμ-kiiμ-k,- i
kμ
μ-k
kμ
kμ
μ-k
μ-k
Moment Generating Functions (I)
...62
1
...6
)(
2
)(1
!
)(
:get we, with Replacing Now,
...!2
210
...!2
1...!3
3
!2
210
:get we, respect to with sderivative by taking that Note
...62
1!
: ofexpansion series heConsider t
3322
32
0
2
2
22
32
0
YtYttY
tYtYtY
i
tYe
tYx
x
dx
ed
ex
xxx
dx
de
x
xxx
i
xe
e
i
itY
x
xx
i
ix
x
Moment Generating Functions (II)
K
t
k
tt
y i
i
y
tytY
tY
tt
tY
ttt
tY
YEtMYEtMYEtM
ypi
tyypeeEtM
tMe
YYtYYdt
ed
YYYt
tYYYttY
Ydt
de
tt
0
)(2
00
all 0 all
22
0
32
0
2
2
0
322
0
322
0
)(...,)(''),()('
)(!
)()(
:)( asfunction labelling and , of valueexpected theTaking
...0...0
...00...!2
...!3
3
!2
20
:0at evaluating and respect to with sderivative Taking
M(t) is called the moment-generating function for Y, and cam be used to derive any non-central moments of the random variable (assuming it exists in a neighborhood around t=0).
Also, useful in determining the distributions of functions of rndom variables
Probability Generating Functions
))1()...(1()(
)1()(''
)()('
:)(Let
))1()...(1(
)1(
:sderivative its and function heConsider t
1
)(
1
1
22
2
1
kYYYEtP
YYEtP
YEtP
tEtP
tkYYYdt
td
tYYdt
td
Ytdt
dt
t
t
k
t
t
Y
kYk
Yk
YY
YY
Y
P(t) is the probability generating function for Y
Discrete Uniform Distribution• Suppose Y can take on any integer value between a and b
inclusive, each equally likely (e.g. rolling a dice, where a=1 and b=6). Then Y follows the discrete uniform distribution.
12
)1)(1(
12
)1)(1()(
2
1)(
: and 1 When :Note
))1((2
)1()1(
))1((6
)12)(1()12)(1()()(
))1((6
)12)(1()12)(1(
6
)12()1(
6
)12)(1(
)1(
1
)1(
1
)1(
1
))1((2
)1()1(
2
)1(
2
)1(
)1(
1
)1(
1
)1(
1)(
1
ofportion integer )int()1(
)1()(int0
)(
)1(
1)(
2
22
1
1
2
1
222
1
11
nnnnYV
nYE
nba
ab
aabb
ab
aaabbbYEYEYV
ab
aaabbb
aaabbb
abyy
ababyYE
ab
aabbaabb
abyy
ababyYE
by
xxbyaab
ayay
yF
byaab
yf
b
ay
a
y
b
y
b
ay
a
y
b
y
Bernoulli Distribution• An experiment consists of one trial. It can result in one of
2 outcomes: Success or Failure (or a characteristic being Present or Absent).
• Probability of Success is p (0<p<1)• Y = 1 if Success (Characteristic Present), 0 if not
)1(
)1()()(
1)1(0
1)1(0)()(
01
1)(
222
222
1
0
pp
ppppYEYEYV
pppYE
pppyypYE
yp
ypyp
y
Binomial Experiment
• Experiment consists of a series of n identical trials• Each trial can end in one of 2 outcomes: Success or
Failure• Trials are independent (outcome of one has no
bearing on outcomes of others)• Probability of Success, p, is constant for all trials• Random Variable Y, is the number of Successes in
the n trials is said to follow Binomial Distribution with parameters n and p
• Y can take on the values y=0,1,…,n• Notation: Y~Bin(n,p)
Binomial Distribution
onDistributiy Probabilit "Legitimate" 1)1()1()(
)( :Expansion Binomial
)1,,,BINOMDIST( :functionby obtained is )(
)0,,,BINOMDIST( :functionby obtained is )(
:Functions EXCEL
,...,1,0)1()()()3
)1( ) )( (and oft arrangemeneach ofy Probabilit )2
)!(!
! positions of sequence ain ) )( (and arranging of waysof # 1)
:GeneralIn
)1()0()0()(0
)1(3)1()1()(1,,
)1(3)2()2()(2,,
)3()3()(3
:Trials 3 with experimentan of outcomesConsider
0 0
0
3
2
2
3
nn
y
n
y
yny
n
i
inin
yny
ynyss
ss
ppppy
nyp
bai
nba
pnyyF
pnyyp
nyppy
nypyYP
ppFynSy
yny
n
y
nnFynSy
ppYPFFFPyFFF
pppYPFFSFSFSFFPyFFSFSFSFF
pppYPFSSSFSSSFPyFSSSFSSSF
ppYPSSSPySSS
Binomial Distribution (n=10,p=0.10)
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0 1 2 3 4 5 6 7 8 9 10
y
p(y
)
Binomial Distribution (n=10, p=0.50)
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0 1 2 3 4 5 6 7 8 9 10
y
p(y
)
Binomial Distribution – Expected Value
npnpppnpqpnp
qpyny
nnpqp
yny
nnYE
nynyyyyy
qpyny
nqp
ynyy
ynYE
y
qpyny
nyqp
yny
nyYE
pqnyqpyny
nyf
nn
n
y
ynyn
y
yny
n
y
ynyn
y
yny
n
y
ynyn
y
yny
yny
)1()1()(
!)1(!
)!1(
!)1(!
)!1()(
1,...,0,...,1 :Note 11Let
)!()!1(
!
)!()!1(
!)(
)0 when 0(Summand
)!(!
!
)!(!
!)(
1,...,1,0)!(!
!)(
11
1
0*
*)1(***
1
0*
)1*(1***
***
11
10
Binomial Distribution – Variance and S.D.
)1(
)1()()1()()(
)1(]1)1[()1()()1(
)1()1()1()()1(
!)2(!
)!2()1(
!)2(!
)!2)(1()1(
2,...,0,...,2 :Note 22Let
)!()!2(
!)1(
)1,0 when 0(Summand
)!(!
!)1(
)!(!
!)1()1(
:not is )()1(but get, toe?)(impossibldifficult is :Note
1,...,1,0)!(!
!)(
22222
2222222
22222
2
0**
**)2(*****
22
0**
)2**(2******
******
2
20
22
pnp
pnpnppnppnYEYEYV
pnppnnpnppnpnnpnppnnYEYYEYE
pnnpppnnqppnn
qpyny
npnnqp
yny
nnnYYE
nynyyyyy
qpyny
nYYE
y
qpyny
nyyqp
yny
nyyYYE
YEYEYYEYE
pqnyqpyny
nyf
nn
n
y
ynyn
y
yny
n
y
yny
n
y
ynyn
y
yny
yny
Binomial Distribution – MGF & PGF
nn
y
yny
n
y
ynyyY
nn
n
tntttnt
tnttnt
ntn
y
ynyt
n
y
ynytytY
pptppty
n
ppy
nttEtP
pnp
pnpnppnppnYEYEYV
pnppnnpnppnpnnp
pppppnnpMYE
npppnpMYE
eppeepeppennptM
eppenppeppentM
ppeppey
n
ppy
neeEtM
)1()1(
)1()(
)1(
)1()()1()()(
)1(1)1(
]1[)1()1()1()1()1()1()1()0(''
)1()1()1()0(')(
)1()1()1()(''
)1()1()('
)1()1(
)1()(
0
0
22222
22222
122
1
12
11
0
0
Geometric Distribution
• Used to model the number of Bernoulli trials needed until the first Success occurs (P(S)=p)– First Success on Trial 1 S, y = 1 p(1)=p – First Success on Trial 2 FS, y = 2 p(2)=(1-p)p – First Success on Trial k F…FS, y = k p(k)=(1-p)k-1 p
1)1(1
1)1()(
,...1,0,...2,1 that noting and 1 Setting
)1()1()(
,...2,1)1()(
0*
*
1
**
1
1
1
1
1
1
p
p
ppppyp
yyyy
ppppyp
yppyp
y
y
y
y
y
y
y
y
y
Geometric Distribution - Expectations
2
222
2
2
22
2222
2333
22
2
1
12
2
12
2
12
2
1
1
222
1
1
111
1
11212)()(
2)1(212)()1(
22
1
2)1()1(2
)1(
1
1
)1()1(
1
)1(
)1(
)1(
)1()1)(1(
1
)(
p
q
p
q
p
p
p
p
pp
pYEYEYV
p
p
p
pp
pp
qYEYYEYE
p
q
p
pq
q
pqqpq
qdq
dpq
q
q
dq
dpq
qqdq
dpqq
dq
dpq
dq
qdpqpqyyYYE
pp
p
q
qqp
q
qqp
q
q
dq
dp
qqdq
dpq
dq
dp
dq
dqppqyYE
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
Geometric Distribution – MGF & PGF
tp
pt
tq
pttq
q
ptq
tqq
pqt
q
ppqttEtP
ep
pe
qe
peqe
q
pqe
qeq
pqe
q
ppqeeEtM
y
y
y
y
y
yy
y
yyY
t
t
t
t
y
ytt
y
yt
y
yty
y
ytytY
)1(11
)(
)1(11
)(
1
1
111
1
1
1
111
1
Negative Binomial Distribution
• Used to model the number of trials needed until the rth Success (extension of Geometric distribution)
• Based on there being r-1 Successes in first y-1 trials, followed by a Success
5)Chapter in Given (Proof )1(
)(
5)Chapter in Given (Proof )(
,...1,)1(1
1)(
2p
prYV
p
rYE
rryppr
yyp ryr
Poisson Distribution
• Distribution often used to model the number of incidences of some characteristic in time or space:– Arrivals of customers in a queue– Numbers of flaws in a roll of fabric– Number of typos per page of text.
• Distribution obtained as follows:– Break down the “area” into many small “pieces” (n pieces)– Each “piece” can have only 0 or 1 occurrences (p=P(1))– Let =np ≡ Average number of occurrences over “area”– Y ≡ # occurrences in “area” is sum of 0s & 1s over “pieces”– Y ~ Bin(n,p) with p = /n– Take limit of Binomial Distribution as n with p = /n
Poisson Distribution - Derivation
)1,,POISSON( :)(
)0,,POISSON( :)(
:Functions EXCEL
onDistributiy Probabilit "Legitimate" 1!!
)(
! :function lexponentia ofexpansion Series
,...2,1,0!!
)(lim
1lim :get weCalculus, From
1lim!
)(lim
fixed allfor 11
lim...lim :Note
11
...1
lim!
1)(
)1)...(1(lim
!
1)!(
)!)(1)...(1(lim
!1
)!(!
!lim)(lim
: aslimit Taking
1)!(!
!)1(
)!(!
!)(
000
0
yyF
yyp
eey
ey
eyp
i
xe
yy
ee
yyp
en
a
nyyp
yn
yn
n
n
nn
yn
n
n
n
n
ynn
ynnn
y
n
n
nynn
ynynnn
ynnyny
nyp
n
nnyny
npp
yny
nyp
y
y
y
y
y
x
ix
yy
n
an
n
n
n
y
n
nn
n
n
yn
yn
y
yn
yn
yyny
nn
ynyyny
Poisson Distribution - Expectations
2222
22
22
2
22
220
1
1
110
][)()(
)()1(
)!2(
)!2(!)1(
!)1()1(
)!1()!1(!!)(
,...2,1,0!
)(
YEYEYV
YEYYEYE
eey
e
y
e
y
eyy
y
eyyYYE
eey
ey
e
y
ey
y
eyYE
yy
eyf
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
Poisson Distribution – MGF & PGF
)1(
0
00
1
0
00
!
!!)(
!
!!)(
tt
y
y
y
y
y
yyY
ee
y
yt
y
yt
y
ytytY
eeey
te
y
te
y
ettEtP
eeey
ee
y
ee
y
eeeEtM
tt
Hypergeometric Distribution• Finite population generalization of Binomial Distribution• Population:
– N Elements– k Successes (elements with characteristic if interest)
• Sample:– n Elements– Y = # of Successes in sample (y = 0,1,,,,,min(n,k)
5)Chapter in (Proof 1
)(
5)Chapter in (Proof )(
),min(,...,1,0)(
N
nN
N
kN
N
knYV
N
knYE
kny
n
N
yn
kN
y
k
yp