Upload
muhannadsaleh
View
35
Download
1
Embed Size (px)
Citation preview
Chapter 1. Probability
1. Set definitionsβ’ Element, Subset, Proper subset, Set, Class, Universal set=S, empty=null set=π.β’ Countable, uncountable,, finite, infinite.β’ Disjoint, mutually exclusive.
2. Set operationsβ’ Venn diagram, equality, difference, union=sum, intersection=product,
complement.β’ Algebra of sets (commutative, distributive, associative)β’ De Morganβs lawβ’ Duality Principle
3. Probability introduced through sets and relative frequency4. Joint and conditional probability5. Independent events6. Combined experiments7. Bernoulli trials8. Summary
1Dr. Ali Muqaibel
Dr. Ali Muqaibel EE315
EE315 Dr. Ali Muqaibel
β’ β’
-
"Ii
- β’ _ E
β’
-
- 7
β’
β’ β’ β’ β’ β’
.. - . β’
-. ------- ----_ .. -
\0 - .
- _. ---------------------------- -- -
e.",,-
---- - - -
-- - e ...... f \-~) - ILVΒ·U ' -f ~A __ Ss-el . 7 β’
I J:.~'j fvI.- .. . " - -- -
. IM.c~-ktk \ ) 1Β«)10. 'v.le 1"'- ,. w - β’ - β’ β’
IV\. β’ -~
, t~vvq' .11401<'6
, "tA. V '-- ) , ,
β’
&,~ , . 11..~16 - I '" \ -. β’ -
-- - -~ _. -----,---------
- =-
CIS ,
- --β’
-----"-> , ----
"
I
Dr. Ali Muqaibel EE315
EE315 Dr. Ali Muqaibel
-
.
- -
--- -fj .;... .. ::. ---. ---=~ 3> ,-- - _ c ~_ D, - <,c ,s.._K.5 __ _ β’
-
_ _ (Cr, _ D L F _" _. _ - --- ,/ ----------------------, ,
-- -- '"-' _._-
____ r.x-(
____ \u=~~. J6c:us\j<M . ___ _ _ _____ _
sPJ w/J.~ N e1e~do -ifo3-e..r~
~---- --N -------.-------------
- - -- -
- - , - - - β’ β’ - -t~
N . ~J c:2 .-. ~Li ,v .::-~
7 '.
~----- - ------------------
--------------------- - --
'----------------~----------- - ... --- - ._- -β’
. _---_. _. -- ------------_._.----------------_ .. ----------β’
~. . β’ β’ β’
--------------------- -------- -----,
----_. ---- -------------,-------,--
β’
β’
β’
Dr. Ali Muqaibel EE315
EE315 Dr. Ali Muqaibel
I ,
--~-~'-~-'~. ~. ~~' ~~~~~~' ----------------------------
,
-------- ------------------
:...-------- ~----, -, , ---~--;-----------
_____ a
G .. ---- ,.-.s -01\k :Mk E.,,* b
-- __ U ) ___ _
- ---- -- ------ _._-----
'-r-----
,
--------, ---------- ._----
,
? ,
f\ - Q .,6' <q <....\.c .... Β£L lOO t;' β’ . - - β’β’ β’ ,
~ J b ~ ~.s 0.(' I
< 13 - __ a 'I I ' M - β’β’
~, S'
B - < I 0 , _._-
--, - ~.' - t.' < d ::- _ "'. , ~ 8 IS :::::- .,(,...,..- ,- , - --, - -- - .. -,
--------- , , - '--- - ~~ " . -~-- - -'~- - - -- . - --,
--'------ - ---------------
- w~ _____ .. _ ...
~ ( . .,..-b ..... 6Y\O-j''-'-_ , . - -- ------ --t...QV _, __ _ - ----
A ,LLA ,=,",""LL_-'. :.. .:
β’ β’
n
'" ---- -
1'I :: I
1'--= ,
_ ..
---- --
β’
3 - ---------_.
Dr. Ali Muqaibel EE315
EE315 Dr. Ali Muqaibel
I 2 : ! Β£ i β’ 2 2 zi 2
- ' . -'
, ,
- s~_" 'or) -...!.SwL...,;-.:....:. '---;. ,;'_ , ---,-A -
β’
,
u u --- -[)g~-.-(3..fJ 8---'------ -----, ---
lJ f) - tLl) ( ~, u.c -
u _ n , U A f} g) (..f , .1\ n c
a !,> t ) C I ~a.I( (I," t~tH __ _ f1 /) --efl UJs) u (
o (1 (\ () =. _ .-8. LA. (I5.LLL..) - A \1 is ~U G_
------------------------, -------------------
----
-- --- ----
----_ . β’ -
β’
\ f\ \) I~ ( ? 0..' ---- - - - - ---------- .. .I-,J
-, .. β’ β’ ,
β’
,
"
---
---- --β’
-- '----- ~.---,-.' .-
/ -_-,---,A,-,,_U~ ___ __ __ ====
,- ,--, -
β’
rr--.J.."'"l ~l ? 3 -
-::-" _________________________________________________________ ~ ___ -=Β·===:'v~~:' _
__ (1 <: β’ _ --=-~--,,-< S
U A . - n 6r) .,. '- --
_~ _ = ~==== :---=_-IfLO . _= __ &J'Lk
, -
β’
------------ -
Dr. Ali Muqaibel EE315
EE315 Dr. Ali Muqaibel
,
β’ -
t -
β’
-
\
I <:;:
7 7 ==
β’
I .'
'1 , β’
,
,
\
β’
β’
-------_. ------------------------------------.- ----------------_._,- ----------
-
,
I e ,--, ._--- - ~----- - -
,-------- '--------, " --
-=9 -C~b'v:\.\A.;).V'd .-----------,---
,
β’
Dr. Ali Muqaibel EE315
EE315 Dr. Ali Muqaibel
We
β’
β’
β’
p,.~b 1------
β’ .,
./
i
β’
β’
, ------------
-- 2 .. , ) ----------------------
"
, β’
Dr. Ali Muqaibel EE315
EE315 Dr. Ali Muqaibel
_ akL c:.LC"_ . '-= L-_
_______ --'s ':: 0 < ;y <.. \ Co β’
β’ β’ , .
- --
~~--------f~-~- -'~ .. ---------------I .N
β’
--"-. - ;;..,.......
-
(3 -β’
β’
- r B)
f
β’ ..A
(>,,!e .. "' . 1 _____ _
β’
" ,. ~-~J __ _
β’ ..... . 7 β’
/
β’
~ I
" (
" '. , β’ --< it 1 ________ --' (
/ -,
,
1-1-/--, ( " t) , 1 ' . ' . - - _ ' _ -~ I '
~~~3~6---------------------------------------~=-~~~~,
. - ,
" U A !!J
~ -, 1,'''',- '[ , β’
1.:- , '1 ' ::: I
β’ -- -- - - ,
I ~
β’ , I β’
Zf f I -
, ,
A 1:)
-
-
β’
, ,
-(
β’ β’ " , ,
I I β’ :?-( - . - β’ -,
= ,a. ;; .. ... _ -- - --/6
~ >:.-' --- -
Dr. Ali Muqaibel EE315
EE315 Dr. Ali Muqaibel
, VI o by;,..,
1 / l ,.
β’
I I 1
1 I
- β’
. __ =-<~ ~ _____ '-,_. __ :L1- 51
--- '-' _1 -1.) - l2 L?J ..... <2 _ ______ ~ /'Z / Xi) ____ .
- :$ 3 B,u-') _ --UI 4-1- Q -= .. ::.... 11: L2..;:,..<..v _
β’
-
5 β’
r)
β’
-
Dr. Ali Muqaibel EE315
EE315 Dr. Ali Muqaibel
β’
,
-
- A \ -\ B - - f fLOJ~~) ___ _
β’
-= - -
_____ 1 III jS -
- --
β’
--
, ,
\:. ' ~ 'r-I{~ -
-.S..\B_ - y> (. s () IS) .. - 1 PC P,
f ( F\ n )5) \"' ( c.. (\ i3) - _. _. - - .- - -:---------β’
- n:(3)
-
fee,) P(~) β’ ,
-----
, ,
~- - - _______ ~. _____________ . ___ !2 ~
'J
"
r. , I β’
'>
'--_ c..::=--__ .>::..,; '-__ :""'=:"';
- 32.
- 2'2> lao
.-
β’
Total Probability
18
1 1
( ) ( )N N
n
n
n
n
A AB BSA A
11
( ) [ ( )] ( )N N
n n
nn
P A P A B P A B
Dr. Ali Muqaibel
Given π mutually
exclusive events π΅π, π =1,2, β¦ . π, that divides the
universe
π΅π β© π΅π = πfor all π β π
π=1
π
π΅π = π
Total Probability (proof)
mutually exclusive
π π΄ =
π=1
π
π π΄ π΅π π π΅π
Bayesβ Theorem
19
1 1
( ) ( )( )
( ) ( ) ( ) ( )
n n
n
N N
P A B P BP B A
P A B P B P A B P B
Dr. Ali Muqaibel
1.4 Joint and Conditional Probability
π π΅π π΄ =π π΅π β© π΄
π π΄
π π΄ π΅π =π π΄ β© π΅ππ π΅π
π π΅π π΄ =π π΄ π΅π π π΅ππ π΄
β’ Given one conditional probability,π π΄ π΅ , we can find the other, π(π΅|π΄)
β’ Another form using the total probability to represent π(π΄)
Example: Total Prob. & Bayesβ Theorem
20
Ex 1.4-2:
Dr. Ali Muqaibel
1.4 Joint and Conditional Probability
Binary Symmetric Channel (BSC)
π΅1: 1 before the channelπ΅2: 0 before the channelπ΄1: 1 After the channelπ΄2: 0 After the channel
A priori probabilities
π π΅1 = 0.6, π π΅2 = 0.4Conditional Probabilities or transition prob.
π π΄1 π΅1 , π π΄1 π΅2 , π π΄2 π΅1 , π π΄2 π΅2
Find π π΅1 π΄1 π ππππππππππ πππππ π΅2 π΄1 , π π΅1 π΄2 , π π΅2 π΄2
Example: Solution
21
1 1( ) ?P B A
1 1 1 1 1 2 2( ) ( ) ( ) ( ) ( ) 0.9 0.6 0.1 0.4 0.58P A P A B P B P A B P B
1 1 11 11 1
1 1
( ) ( )( ) 0.9 0.6 54( ) 0.931
( ) ( ) 0.58 58
P A B P BP B AP B A
P A P A
2 1 1 1( ) ? 1 ( )P B A P B A
1 2( ) ?P B A 2 2( ) ?P B A
Dr. Ali Muqaibel
π π΅1 π΄1 =π π΄1 π΅1 π π΅1π π΄1
Find π π΄2 =?
Look at examples in the text book 1.4.3 & 1.4.4
1.5 Independent Events
22
( ) ( ) ( ) & independent ( ) ( )
( ) ( )
P B A P B P AA B P B A P B
P A P A
Def: Two events & are (statistically) independent ifA B
( ) ( ) ( ), ( ) 0, ( ) 0P A B P A P B P A P B
( ) ( ) ( )S A AB B B B A B A
(( ) ( ))P B P B A P B A
& independent A B ( ) ( ) ( ) ( ) ( )
( )[1 ( )] ( ) ( )
( ) P B P B A P B P B P A
P B P A
P B
P P
A
B A
& independentA B
Dr. Ali Muqaibel
Statistically independent : the probability of one event is not affected by the occurrence of the other event, π π΄ π΅ = π π΄ , and π(π΅|π΄) = π(π΅)
What about the complement?
Comments about independenceβ’ Sufficient Condition for independence (not just
necessary) π π΄ β© π΅ = π π΄ π π΅
β’ Independence simplify many things βUsually assumed based on physicsβ.
β’ If π π΄ > 0 & π π΅ > 0, π‘βπππ π΄ β© π΅ = 0 β ππ’π‘π’ππππ¦ ππ₯πππ’π ππ£π
π π΄ β© π΅ = π π΄ π(π΅) independent
β’ For two events to be independent π΄ β© π΅ β π, because if they are disjoint occurrence of one exclude the other.
β’ Example:β’ π¨ βselect a kingβ
β’ π© βselect a jack or queenβ
β’ πͺ βselect a heartβ
β’ Find π· π¨ ,π· π© , π· πͺ , π· π¨ β© π© ,π· π¨ β© πͺ , π· π© β© πͺ and test dependence.
Dr. Ali Muqaibel 23
π π΄ =4
52
π(π΅) =8
52
π(πΆ) =13
52
π π΄ β© π΅ = 0 β π π΄ π π΅ =32
522
π π΄ β© πΆ =1
52= π π΄ π πΆ
π π΅ β© πΆ =2
52= π π΅ π πΆ
π΄ is independent of πΆ and π΅ is independent of πΆ , but π΄ & π΅ are dependent.
Multiple Events
24
1 2 3Def: 3 events , , & independent A A A
1 2 1 2( ) ( ) ( )P A A P A P A ( ) 0iP A 2 3 2 3( ) ( ) ( )P A A P A P A
1 3 1 3( ) ( ) ( )P A A P A P A 1 2 3 1 2 3( ) ( ) ( ) ( )P A A A P A P A P A
Ex:21 3{1,2}, {2,3}, {1,3}AAA {1,2,3,4}S
( ) ( ) ( ),i j i jP A A P A P A i j
32 31 21( ) ( ) ( ) ( )A AA AP P P PA A 2 31, , & NOT independentAAA
21 3, , & pairwise independentAAA
1 3 312 2Fact: , , & independent & ( ) independentA AA AA A
1 3 312 2Fact: , , & independent & ( ) independentA AA AA A
Dr. Ali Muqaibel
1.5 Independent Events
Independence by pairs is not sufficient all combinations must be satisfied
Comments about independence of multiple events β’ If π events are independent, then any one of them is independent of
any event formed by unions, intersection, and complements of the others.
π΄1 is independent π΄2 => π΄1 is independent π΄2
β’ If independent intersection is changed to productπ π΄1 β© π΄2 β© π΄3 = π π΄1 π π΄2 β© π΄3 = π π΄1 π π΄2 π π΄3
π π΄1 β© π΄2 βͺ π΄3 = π π΄1 π π΄2 βͺ π΄3β’ This is assuming full independence. Pairwise is not sufficient.
Example: Drawing four ace with replacement and without repleacement
With replacement π π΄1 β© π΄2 β© π΄3 β© π΄4 = π(π΄1)π(π΄2) π(π΄3) π(π΄4) =4
52
4β 3.5 10β5
Without replacement π π΄1 β© π΄2 β© π΄3 β© π΄4 = π(π΄1)π(π΄2|π΄1)
π(π΄3|π΄1 β© π΄2) π(π΄4|π΄1 β© π΄2 β© π΄3) =4
52Γ3
51Γ2
50Γ1
49β 3.69(10β6)
Dr. Ali Muqaibel 25
Combined Experiments
β’ Combined experiments are made of sub-experiments.
β’ Examples (Wind speed, pressure)
β’ Repeating an exp. π times: Flipping a coin π times.
β’ For two independent (π1, Ξ©1, π1) with π elements and (π2, Ξ©2, π2) with π elements, we can form a combined experiment (π, Ξ©, π) with ππelements where π = π1 Γ π2 = π 1, π 2 , π 1 β π1 , π 2 β π2
β’ Example 1: π1 = π», π , π2 = {1,2,3,4,5,6}
β’ π = { π, 1 , π, 2 , π, 3 , π, 4 , π, 5 , π, 6 , π», 1 , π», 2 , π», 3 ,π», 4 , π», 5 , π», 6 }
β’ Example 2: π1 = π», π , π2 = π», π
β’ π = π»,π» , π», π , π, π» , π, π
β’ Events on the combined Space, π 1 β π΄, π 2 β π΅, πΆ = π΄ Γ π΅
β’ π΄ Γ π΅ = π΄ Γ π2 β© (π1 Γ π΅)
Dr. Ali Muqaibel 26
Probability of Combined Experiments
27
3 independent experiments ( , , ), 1,2,3i i iS P i
( , , )S PCan define a combined probability space
1 2 3S S S S
1 2 3 1 1 2 2 3 3( ) ( ) ( ) ( ), i iP A A A P A P A P A A
1 2 3
Permutation
!( 1) ( 1)
!
n
r
nP n n n r
n r
Combination
!
! !
n n
r r n r
0
( )n
n r n r
r
nx y x y
r
binomial expansion
Dr. Ali Muqaibel
Permutation: # of possible sequences (order important) (not replaced)Combination: # of possible sequences (order not important)(not replaced)# decreases by ππ
π =π!
0!= π!
Examples: Permutation and Combination
β’ Example: drawing 4 cards from 52 card deck. How many possibilities are there.
β’π452 =
52!
52β4 != 52(51)(50 49 = 6,497,400
β’ Example: A coach has five athletes and he wants to make a team made of 3.How many teams can he make?
β’πΆ35 =
5!
2!3!= 10 . Same as choosing 2 for the spare team.
πΆππ = πΆπβπ
π .
Other notations are also possible.
Dr. Ali Muqaibel 28
1.7 Bernoulli Trials
29
Basic experiment - 2 possible outcomes ( or )A A
Bernoulli Trials - repeat the basic experiment timesN
( ) ( ) 1P A p P A p
(Assume that elementary events are independent for every trial.)
({ occurs exactly times}) (1 )k N kkk
P AN
p p
Ex 1.7-1: ( ) 0.4P A 3N
2 13
(2 hits) 0.4 (1 0.4) 0.2882
P
Dr. Ali Muqaibel
Hit or miss , win or lose, 0 or 1
We are firing a carrier with torpedoes.π βππ‘ = 0.4. It will sunk if two or more hits. We are firing three torpedoes.
Continue Example :Bernoulli Trials
30
Ex 1.7-3: ( ) 0.4P A 120N
3 03
(3 hits) 0.4 (1 0.4) 0.0643
P
({carrier sunk}) (2 hits) (3 hits) 0.352P P P
50 70120
(50 hits) 0.4 (1 0.4) ?50
P
large N De Moivre-Laplace approximation
Poisson approximation
120! ?
Dr. Ali Muqaibel
0 33
(0 hits) 0.4 (1 0.4) 0.2160
P
1 23
(1 hits) 0.4 (1 0.4) 0.4321
P
Given we are firing for 3 seconds. Firing rate
2400 per minutes. Find π{ππ₯πππ‘ππ¦ 50 βππ‘π }
De Moivre-Laplace & Poisson Approximations
β’ Stirlingβs Formula: π! β 2ππ1
2πππβπ, for large π.
β’ Error less than 1% even for π = 10.
β’ Using Stirlingβs formula, De Moivre-Laplace Approximation
π =ππππ 1 β π πβπ β
1
2πππ 1 β πππ₯π β
π β ππ 2
2ππ 1 β π
β’ π, π, πππ π β π,ππ’π π‘ ππ πππππ , π ππ’π π‘ ππ ππππ ππ to assure small numerator.
β’ If π is very large and p is very small De Moivre-Laplace approximation fails, we can use Poisson approximation:
ππππ 1 β π πβπ β
ππ ππβππ
π!β’ π large and π is small.
Dr. Ali Muqaibel 31
Example: De Moivre-Laplace Approximation
β’ Back to the torpedoes example. Given we are firing for 3 seconds. Firing rate 2400 per minutes. Find π{ππ₯πππ‘ππ¦ 50 βππ‘π }
β’ π = 3π ππ Γ2400ππ’ππππ‘π /πππ
60 π ππ/πππ= 120 ππ’ππππ‘π .
β’ π = 50, ππ = 120 0.4 = 48, π 1 β π = 120 0.6 = 72
β’ π, π & π β π = 70 are all large. π = 50 is near ππ = 48
β’ π{50 βππ‘π }=ππππ 1 β π πβπ β
1
2πππ 1βπππ₯π β
πβππ 2
2ππ 1βπ
=1
2π(48) 0.6ππ₯π β
50 β 48 2
2(48) 0.6= 0.0693
Dr. Ali Muqaibel 32