Embed Size (px)
Citation preview
Two Interpretations of Probability
The Frequentist Interpretation: Probability (outcome k) = Relative Frequency of k
i.e. if S = set of all outcomes n = # of trials = # of times k occurs in n trials then
n
nNP k
nk
)(lim
The Probabilistic/Axiomatic Approach:Probabilities are numerical values assigned to outcomes of set S
such that axioms of probability are satisfied. The axioms are a) All probabilities are between 0 and 1.
b) The probabilities sum to 1 over the set S.
c) Probabilities of mutually exclusive events are additive.
Definitions
• Random Experiment: An experiment with random results.
• Outcome: An elementary result from a random experiment.
• Sample Space: The space S of all possible outcomes in a random experiment. sample spaces can be: __ Finite or countable infinite __ Uncountable
• Event: A subset of S. Certain event = S Null Event = Ф e.g. Random Experiment : Toss of a coin twice Outcomes = HH,HT,TH,TT Sample Space = {HH,HT,TH,TT} Event : At least one head = {HH,HT,TH}
The Axioms of Probability
1)
2) P(S) = 1
3) If, for ,
then
For Infinite Sample Spaces (3) must be replaced by:
3a) If such that then
Since Axiom 3 only lets us evaluate which may not be equal to
)(0 AP SA
SB,A BA
)()()( BPAPBAP
SAA ,, 21
jiAA ji
11
)(k
kk
k APAP U
n
kk
nAPLim
1
U
1kkAP
Probabilities of disjoint events add together
Some Elementary Consequences
1)
2)
3)
4)
5)
SA1)( AP
ASAwhere ')(1)( ' APAP
0)( P
)()()()(
)....()1(
..............
)(
)()()(
211
11
BAPBPAPBAP
AAAP
AAAP
AAPAPAP
nn
lkjlkj
n
j kjkjj
n
kk
U
)()( BPAP BAIf
Subtract even-numbered combinations, add odd-numbered ones.
A1 A2
A3
S
Conditional Probability:
= Probability that A occurs given B has occurred.
Independent Events: are independent if,
Which leads to
Note that Independence Mutual exclusion
In fact, if P(A), P(B) > 0, independent A,B are not mutually exclusive.
)()|( APBAP
0)(;)(
)()|(
BP
BP
BAPBAP
SBA ,
)|( BAP
)()|( BPABP
)()()( BPAPBAP
Bayes’ Rule:
If and form a partition of S.
The form
is widely used in estimation.
and are called prior probabilities.
Un
kkBS
1 SA
kB
jiifBB ji
n
kkk
jjjj
BPBAP
BPBAP
AP
BAPABP
1
)()|(
)()|(
)(
)()|(
)(
)()|()|(
AP
BPBAPABP jj
j
)( jBP )(AP
Bernoulli Trial: A single trial of an experiment whose only outcomes are “success” and “failure” or {0,1} etc. If p = P(success) then P(k successes in n independent Bernoulli Trials)
This is called binomial probability law.
312502501250!2!3
!5
501503
5
3
23
5
.).)(.(
).(.
)(P
nk ,...,2,1,0
Example: Coin toss with “success” = heads
Fair coin p = P(heads) = 0.5
P (3 heads in 5 tosses)
knkn )p(p
k
n)k(P
1
Logic of the Binomial Law
Multinomial Probability Law:
If is a partition of S
and rk = Number of times Bk occurs in n trials.
then,
The binomial law is the m=2 case.
knkn )p(p
k
n)k(P
1
mkB 1
.BPP
P....PPr.....rr
nr.....,,r,rP
kk
rm
rr
mm
m
)(where
!!!
!)( 21
2121
21
Prob of k successes
Prob of n-k failures
Number of ways to get k successes
in n attempts
Random Variables
A Random Variable X is a function that assigns a real number, X(), to eachOutcome, , of a random experiment: X : S
Example: Random Experiment = 3 coin tosses X Substitute H = 1 and T = 0 and read as a binary number. So X(HHH) = 7 (111) X(TTT) = 0 (000) X(HTH) = 5 (101) etc
Example: Random Experiment = Examine patient
Outcomes = {Healthy, Sick}
X(Healthy) = 0 X(Sick)=1
A random variable is a deterministic function, not a random one.
Why bother?
Cumulative Distribution Function (CDF)
The CDF describes how probability accumulates over the range of a random variable
Thus
e.g. in the coin toss example:
xxXPxFX )()(
}))(:({)( xXPxFX
coinfairafor50
})011010001000({
})({
)3()3(
.
,,,P
THH,THT,TTH,TTTP
XPFX
Properties of CDF:
1)
2)
3)
4) Nondecreasing in x :
5) Right – continuous :
6)
7) The RHS is obtained from
1)(0 xFX
1)(lim
xFXx
0)(lim
xFXx
)()( bFaFba XX
0
)()(lim)(0
hfor
aFhaFaF XXh
X
ba
aFbFbXaP XX
)()()(
)()()( aFaFaXP XX )()(lim)(0
haFaFaXP XXh
If FX (x) is also left-continuous at a P(X=a) = 0 For continuous FX (x) at a,b
If FX (x) is not continuous at a,b, the equalities do not hold.
)bXa(P
)bXa(P
)bXa(P)bXa(P
Three Types of Random Variables
1) Discrete :
2) Continuous:
3) Mixed:
Probability Mass Function (PMF) : For a discrete random variable X with range Probability Density Function(PDF):
for differentiable .
i.e.
,....},{ 21 xxSX
}:)()({ XkkkX SxxXPxpPMF
xd
xFdxf X
X
)()(
)(xFX
0
)()(
has
xhfhxXxP X
dxxfdxxXxP X )()(
Properties of PDF:
1)
2)
3)
4)
Any function g(x) such that can form a valid pdf
For Discrete Random Variables pdf is defined using the delta function
0)( xf X
dxxfbXaPb
a
X )()(
dssfxFx
XX
)()(
1)(
xf X
xxgandcdxxg
0)()(
)(1
)( xgc
xf X
k
kkXX xxxpxf )()()(
pmf cdf
01
00)()(
x
xdttxu
x
)( kxx
kx
1)( dxxx k
kx
k
kkXX xxuxpxF )()()(
Conditional CDF’s and PDF’s
SAAP
AxXPAxFX
;
)(
)}({)|(
)|()|( AxFdx
dAxf XX
Example 1:
txtF
tFxF
tx
tXxF
elsexXtP
txiftXxXP
tXxXPtXxF
X
XXX
X
)(1
)()(
0
)|(
)(
0),(
)|()|(
0 1 2 3 4 5 6 7
1/6
1
0x
Fx (x)
Fx (x | X even)
3121
61
even) (
})2({
even) (
even) }2({even) (
//
/
XP
XP
XP
XXPX|xFX
Example 2: Rolling a fair 6-sided die
