8/21/2019 Basic Probability Review
1/77
Probability Review
Course : AAOC C312
8/21/2019 Basic Probability Review
2/77
Review of Basic Probability (Ch 12)
Laws of probability
Addition Law
Conditional Probability
Random Variables
Probability Distribution
Joint random variable Some Common Probability Distributions
Binomial, Poisson, Exponential, Normal
8/21/2019 Basic Probability Review
3/77
Probability
Probability provides a measure of uncertaintyassociated with the occurrence of events oroutcomes of a random experiment.
Experiments
Sample Space Events 0 P(E) 1 Impossible Event; P() = 0
Certain Event; P(S) = 1 Mutually Exclusive Events Pair wise Mutually Exclusive Events Equally likely Events
8/21/2019 Basic Probability Review
4/77
Some definitions
An experiment
Any process that yields a result or an
observation Outcome
A particular result of an experiment
Sample space The set of all possible outcomes of an
experiment
8/21/2019 Basic Probability Review
5/77
Some definitions
An event
Any subset of the sample space.
If the event is A, then n (A) is the number
of sample points that belong to event A
If the event is getting heads on a series ofcoin flips, then n (heads) is the number of
heads in the sample of flips
8/21/2019 Basic Probability Review
6/77
Mutually Exclusive Event
Events defined in such a way that theoccurrence of one event precludes theoccurrence of any of the other events
If one of them happens, the other cannot
happen
8/21/2019 Basic Probability Review
7/77
ProbabilityProbability provides a measure of uncertainty
associated with the occurrence of events oroutcomes of a random experiment.
Definition:
If in a n-trial experiment an event E occursm times then the probability of occurrence ofevent E is
By definition,
0
8/21/2019 Basic Probability Review
8/77
Example: What is the probability of getting
even nos. in a rolling a die.
Example: What is the probability of gettingtotal of 7 on two dice?
8/21/2019 Basic Probability Review
9/77
Addition law of Probability
otherwiseEFPFPEP
exclusivemutuallyareFandE
FPEP
FEP
},{}{}{
},{}{
}{
For two events E and F, E + F representsunion, and EF represents intersection.
8/21/2019 Basic Probability Review
10/77
8/21/2019 Basic Probability Review
11/77
Problem: In a certain college, 25 percent of thestudent failed mathematics, 15 percent failed
chemistry, and 10 percent failed bothmathematics and chemistry. A student isselected at random.
(a) if the student failed chemistry, what is the
probability that he failed mathematics? (b) if the student failed mathematics, what is theprobability that he failed chemistry?
(c) what is the probability that he failed chemistry
or mathematics? (d) what is the probability that he failed neither
chemistry nor mathematics?
8/21/2019 Basic Probability Review
12/77
Problem: Two men A and B fire at a target.Suppose P(A) = 1/3 and P(B) = 1/5 denote
their probabilities of hitting the target. ( weassume that the events A and B areindependent). Find the probability that
(a) A does not hit the target (b) Both hit the target
(c) One of them hits the target
(d) Neither hits the target.
8/21/2019 Basic Probability Review
13/77
Bayes Theorem
,}{
}{}|{}|{
BP
APABPBAP
The two events A and B with P[B] > 0, then
Let E be an event in a sample space S, and let A1,A2,.Anbe mutually disjoint event whose union isS. then
,}{
}{}|{}|{
}{}|{...}{}|{}{}|{}{ 2211
EP
APAEPEAP
APAEPAPAEPAPAEPEP
kkk
nn
8/21/2019 Basic Probability Review
14/77
Problem: Three machines A, B and C produce,respectively, 40%, 10% and 50% of the items in
a factory. The percentage of defective itemsproduced by the machines is respectively, 2%,3% and 4%. An item from the factory is selectedat random.
(a) Find the probability that the item is defective
(b) If the item is defective, find the probability thatthe item was produced by (i) machine A, (ii)
machine B, (iii) machine C.
8/21/2019 Basic Probability Review
15/77
15
Random Variables
Definition:A random variableXon a sample space Sis
a rule that assigns a numerical value to each outcome
of Sor in other words a function from Sinto the setR
of real numbers.X : S R
x : value of random variableX
RX : The set of numbers assigned by random variableX,i.e. range space.
8/21/2019 Basic Probability Review
16/77
16
Random Variables (contd)
Classifications of Random VariablesAccording to thenumber of values which they can assume, i.e. numberof elements inRx.
Discrete Random Variables:Random variables whichcan take on only a finite number, or a countableinfinity of values, i.e.Rx is finite or countable infinity.
Continuous Random Variables:When the range space
Rx is a continuum of numbers. For example aninterval or the union of the intervals.
8/21/2019 Basic Probability Review
17/77
17
Random Variables (contd)
Example: Consider the experiment consisting of 4tosses of a coin then sample space is
S = {HHHH, HHHT, HHTH, HTHH, THHH, HHTT,
HTTH, TTHH, THTH, HTHT, THHT, TTTH,TTHT, THTT, HTTT, TTTT}
Let X assign to each (sample) point in S the totalnumber of heads that occurs. Then X is a random
variable with range spaceRX= {0, 1, 2, 3, 4}
Since range space is finite, X is a discrete randomvariable
8/21/2019 Basic Probability Review
18/77
18
Random Variables (contd)
Example:A pointPis chosen at random in
a circle C with radius r. Let X be the
distance of the point from the center of thecircle. Then X is a (continuous) random
variables withRX = [0, r]
r
P
X
O
C
8/21/2019 Basic Probability Review
19/77
19
Probability Distributions
If X is discrete random variable, the function given
by
f(x)= P[X = x]
for each x within the range of X is called the
probability mass function (pmf)ofX.
To express the probability mass function, we
give a table that exhibits the correspondencebetween the values of random variable and the
associated probabilities
8/21/2019 Basic Probability Review
20/77
20
Probability Distributions (contd)
Ex: In the experiment consisting of four tosses
of a coin, assume that all 16 outcomes are
equally likely then probability mass
function for the total number of heads is
x 0 1 2 3 4f(x) 1/16 1/4 3/8 1/4 1/16
8/21/2019 Basic Probability Review
21/77
21
Probability Distributions (contd)
A function can serve as the probability mass function
of a discrete random variable X if and only if its
value,f(x), satisfy the conditions
1. f(x) 0 for all value ofx.
2. 1)(all
x
xf
Example:Check whether the following can defineprobability distributions
.5,4,3,2,1,0for15
)()a( xx
xf
8/21/2019 Basic Probability Review
22/77
22
Probability Distributions (contd)
.5,4,3,2,1for25
1)()d(
.6,5,4,3for41)()c(
.3,2,1,0for6
5)()b(
2
xx
xf
xxf
xx
xf
8/21/2019 Basic Probability Review
23/77
23
Distribution Function
IfX is a discrete random variable, the function given
by
xtfxXPxFxt
for)()()(
wheref(t) is the value of the probability mass functionofX att, is called the distribution function or thecumulative distributionfunction (cdf) ofX.
8/21/2019 Basic Probability Review
24/77
24
Example
Cumulative Distribution function of the total
number of heads obtained in four tosses of a
balanced coin
We know that f(0) = 1/16, f(1) = 4/16, f(2) = 6/16,f(3) = 4/16, f(4) = 1/16. It follows that
F(0) = f(0) = 1/16
F(1) = f(0) +f(1) = 5/16
F(2) = f(0) +f(1) +f(2) =11/16
F(3) = f(0) +f(1) +f(2) +f(3) = 15/16
F(4) = f(0) +f(1) +f(2) +f(3) +f(4) = 1
Th di t ib ti f ti i d fi d t l f th
8/21/2019 Basic Probability Review
25/77
25
4for1
43for16
15
32for
16
11
21for16
5
10for16
1
0for0
)(
x
x
x
x
x
x
xF
The distribution function is given by
The distribution function is defined not only for the
values taken on by the given random variable, but
for all real number.
8/21/2019 Basic Probability Review
26/77
26
1 2 3 40
1/16
5/16
11/16
15/16
1
F(x)
x
Graph of the Distribution function
..
.
. .
8/21/2019 Basic Probability Review
27/77
27
The values F(x) of the distribution function of adiscrete random variableX satisfy the conditions1.F(-) = 0 andF() = 1; that is, it ranges from 0 to
1.2.If a
8/21/2019 Basic Probability Review
28/77
28
Similarly, for continuous random variable X, we associate
a probability density function (pdf) f, such that
( ) ( ) 0, .
( ) ( ) ( ) .
( ) ( ) 1.
( ) ( ) ( ) , .
( ) ( )
b
a
x
a f x x
b f a X b f x dx if a b
c f x dx
d x f x dx x
de f x
dx
for all real
is integrable and P
F for each real
F
Parameters of random variables
8/21/2019 Basic Probability Review
29/77
29
Parameters of random variables
(i)Expectationof a random variable X is
If h is a real valued function of X, then
(ii) Variance
(iii) MomentsThe r-th moment about origin is
The r-th moment about mean is
x = E(X) = x p(x)
2 2 2 2Var(X) = E(X - ) = E(X ) - {E(X)}
'
r
r = E(X )
x
xfxall
)(
xall
xfxhXhE )()())((
])[( rr XE
dxxfx )(or
dxxfxh )()(or
[ ]
8/21/2019 Basic Probability Review
30/77
Probability Density Function (pdf)
Characteristics
Random variable X
Discrete Continuous
Applicable range a, a+1, , b a x b
Conditions forpdf
p(x)0, f(x)0,1)(
b
ax
xp 1)( b
axf
8/21/2019 Basic Probability Review
31/77
Cumulative distribution function(CDF)
X
a
X
ax
continuousxdxxfXF
discretexxpXPXxP
,)()(
,)()(}{
8/21/2019 Basic Probability Review
32/77
Problem:The number of units, x, needed for anitem is discrete from 1 to 5. the probability p(x)is directly proportional to the number of units
needed. The constant of proportionality is K.(a) find the pdf of x,
(b) Find the value of the constant k
(c) determine the CDF, and find the probability thatx is even value.
8/21/2019 Basic Probability Review
33/77
Problem: Consider the following function
(a) find the value of the constant k that will makef(x) a pdf
(b) determine the CDF, and find the probability thatx is (i) larger than 12, and (ii) between 13 and 15.
2010,)(2
x
x
kxf
8/21/2019 Basic Probability Review
34/77
Expectation of Random Variable
Given that h(x) is a real function of arandom variable x, we define the expectedvalue of h(x), E{h(x)}, as the weighted
average with respect to the pdf of x.
continuousxxfxh
discretexxpxhxhE
b
a
b
ax
),()(
),()()}({
8/21/2019 Basic Probability Review
35/77
Moments of Random Variable
The mth moment of a random variable x,denoted by E(Xm), also called theexpected value of Xm, is defined
continuousxxfx
discretexxpxXE
b
a
m
i
b
ax
i
m
im
),(
),(}{
8/21/2019 Basic Probability Review
36/77
Mean
continuousxxxf
discretexxxp
xE b
a
b
ax
),(
),(
}{
The mean of x, E{x}, is a numericmeasure of central tendency of randomvariable.
First moment of x.
8/21/2019 Basic Probability Review
37/77
Variance
}var{}{
),(}){(
),(}){(}{
2
2
xxstdDev
continuousxxfxEx
discretexxpxExxVar
b
a
b
ax
The variance var{x}, is a measure ofdispersion of x around the mean
8/21/2019 Basic Probability Review
38/77
Problems
Consider a random variable X that is equal to 1,2 or3. If we know p(1) =1/2 and p(2) = 1/3 then p(3)=?
Find E{x} and Var{x} where x is the outcome whenwe are roll a fair die.
Suppose the r.v. has a following distribution function
What is the probability that X exceeds 1?
0)exp(1
00)( 2 xx
xxF
8/21/2019 Basic Probability Review
39/77
Problems
A construction firm has recently sent inbids for 3 jobs worth (in profit) 10, 20 and40 (thousand) dollars. If its probabilities of
winning the jobs are respectively 0.2, 0.8and 0.3, what is the firms expected totalprofit?
Some Standard Distributions
8/21/2019 Basic Probability Review
40/77
40
Some Standard Distributions
Bernoullis Distribution A r. v. X is said to haveBernoulli distribution if and only if the correspondingprobability mass function is given by
x 1-xp X = x = p (1 - p) , x = 0,1.
tX
Also, E(X) = p, Var(X) = p(1 - p), and M (t) = 1 - p + pe
Binomial Distribution A r v X is said to have
8/21/2019 Basic Probability Review
41/77
41
Binomial Distribution A r. v. X is said to have
Binomial distribution if and only if the corresponding
probability mass function is given by
x n-xn
p(X = x) = p (1 - p) , x = 0,1, ..., nx
t nXE(X) = np, Var(X) = np(1 - p) and M (t) = (1 - p + pe )
Geometric Distribution A r v X is said to have
8/21/2019 Basic Probability Review
42/77
42
Geometric Distribution A r. v. X is said to haveGeometric distribution if and only if the correspondingprobability mass function is given by
P(X = x) = p.qx-1, x = 1, 2, 3, .; q = 1 - p
Memoryless Property
1
2
)1()()(,1
)( ttX
qepetMp
qXV
pXE
P(X > t + h | X > t) = P(X > h), t > 0, h > 0
Poissons Distribution A random variable is
8/21/2019 Basic Probability Review
43/77
43
Poisson s Distribution A random variable issaid to be Poissonsrandom variable with parameter
if X has the mass points 0,1,2, and its
probability mass function is
x
-P(X = x) = e , x = 0,1, 2,...
x
and
+
> 0
t
(e -1)
X
Inthiscase,
E(X) = , Var(X) = and M (t) = e
TheoremIf X and Y are independent Poissons
random variables with parameters
respectively, then X+Y will be a Poissons random
variable with parameter
Theorem Suppose X has binomial distribution with
8/21/2019 Basic Probability Review
44/77
44
Theorem Suppose X has binomial distribution with
parameters n and p. If n is large and p is small so
that , then X will follow Poissons
distribution with parameter .
= np
8/21/2019 Basic Probability Review
45/77
Exponential Distribution
A continuous r.v. whose probability densityfunction is given, for some l >0, by
0,00,)(
xifxifexf
xl
l
Its CDF is
E[X] =1/, V[X] = 1/2,
0,1)( xexF xl
Markov or Memoryless Property of the Exponential
8/21/2019 Basic Probability Review
46/77
46
Markov or Memoryless Property of the Exponential
Distribution
P(X > t + h | X > t) = P(X > h), t > 0, h > 0
8/21/2019 Basic Probability Review
47/77
If the no. of arrivals at a service facilityduring a specified time period followsPoison distribution, then the distribution
of the time interval between successivearrivals must be Exponentialdistribution.
If is the rate at which events occur,then 1/ is the average time intervalbetween successive events.
47
U if if (Fi 1))( bb
8/21/2019 Basic Probability Review
48/77
Uniform: if (Fig. 1),),,( babaUX
otherwise.0,
,,1
)( bxaabxfX
)(xfX
xa b
ab1
Fig. 1
Exponential: if (Fig. 2))( lEX
otherwise.0,
,0,
)(
xe
xf
x
X
ll
)(xfX
x
Fig. 2
8/21/2019 Basic Probability Review
49/77
Mean of Uniform
Distribution
b
a
dxab
xxxf 1
2
ba
2212
1ab
Normal Distribution:
8/21/2019 Basic Probability Review
50/77
Normal Distribution:
Normal (Gaussian):X is said to be normal or Gaussian r.v,
if
This is a bell shaped curve, symmetric around the
parameter and its distribution function is given by
where is often tabulated. Since
depends on two parameters and the notation will be used to represent
.2
1)(22 2/)(
2
xX exf
,,
2
1)(
22 2/)(
2
xy
X
xGdyexF
dyexG yx
2/2
2
1)(
),(
2
NX)(xfX
x
Fig.
)(xfX
,2
8/21/2019 Basic Probability Review
51/77
The Standard Normal
Distribution
To find P(a < x < b),we need tofind the area under the appropriatenormal curve. There are several
such normal curves, but one ofthem is called standard normalcurve.
Th St d d N l
8/21/2019 Basic Probability Review
52/77
The Standard Normal
Distribution
Definition : The normal distributionwith Mean = 0; Standard deviation = 1is called the standard normaldistribution (standard normal variableis denoted by Z).
8/21/2019 Basic Probability Review
53/77
X
Normal
Distribution
Z X
Normal to Standard Normal
DistributionNormal
Standardized
X=Z +
The Normal Approximation
8/21/2019 Basic Probability Review
54/77
The Normal Approximation
to the Binomial
We can calculate binomial probabilitiesusing
The binomial formula The cumulative binomial tables
When n is large, andp is not too close
to zero or one, areas under thenormal curve with mean npandvariance npq can be used to
a roximate binomial robabilities.
A i ti th Bi i l
8/21/2019 Basic Probability Review
55/77
Approximating the Binomial
While approximating a randomvariable with integer values by acontinuous random variable, use
continuity correction. . In this, the integer value x0 of
discrete random variable is replaced
by the interval (x01/2, x0 +1/2) ofthe continuous random variable.
8/21/2019 Basic Probability Review
56/77
Thus if aand bare integers, andX*isa continuous random variableapproximating discrete random
variable X thenP(a X b) = P(a < X* b + )
8/21/2019 Basic Probability Review
57/77
Make sure that np and nqare bothgreater than 15to avoid inaccurateapproximations!
8/21/2019 Basic Probability Review
58/77
Exercise :The probability that an
electronic component will fail in lessthan 1200 hours of continuous use is0.2. Use normal approximation to find
the probability that among 250 suchcomponents, fewer than 50 will fail inless than 1200 hours of continuous
use.
8/21/2019 Basic Probability Review
59/77
X= no. of electronic components among 250
randomly chosen which fail in less than1200 hours.
X has binomial distribution with n=250, p=
0.2.We can approximate X by normal random
variable X* with mean (250)(0.2)=50 and
variance = (50)(0.8)=40.=6.324.
Z=(X*-50)/6.324 has standard normal dist.
8/21/2019 Basic Probability Review
60/77
P(X
8/21/2019 Basic Probability Review
61/77
Central Limit Theorem
Let x1, x2, , and xnbe independent andidentically distributed random variables,each with mean and standard deviation
, and defined sn= x1+x2+.+xn.As n become large (n), the distributionof snbecomes normal with mean nand
variance n2, regardless of the originaldistribution of x1, x2, , and xn.
8/21/2019 Basic Probability Review
62/77
. -
8/21/2019 Basic Probability Review
63/77
0.2266
8/21/2019 Basic Probability Review
64/77
From (1) and (2) we get z = -.37.
8/21/2019 Basic Probability Review
65/77
(b) P(-z < Z < z) = .9298
or F(z)F(-z) = .9298
or F(z)(1 - F(z)) =.9298
or 2F(z) = 1.9298
or F(z) = P(Z z)= .9649
from table, z =1.81.
8/21/2019 Basic Probability Review
66/77
8/21/2019 Basic Probability Review
67/77
8/21/2019 Basic Probability Review
68/77
73.3
J i t d i bl
8/21/2019 Basic Probability Review
69/77
Joint random variableConsider the two continuous r.vsx1, a1x1b1,
and x2, a2x2 b2. Define f(x1, x2) as the jointpdf of x1 and x2 and f1(x1) and f2(x2) as themarginal pdfs of x1and x2respectively. Then
f(x1,x
2) 0, a
1x
1
b
1, a
2 x
2
b
2
tindependenarexandxifxfxfxxf
dxxxfxf
dxxxfxf
dxxxfdx
b
a
b
a
b
a
b
a
21221121
12122
22111
2211
),()(),(
),()(
),()(
1),(
1
1
2
2
2
2
1
1
8/21/2019 Basic Probability Review
70/77
E[c1x1+c2x2]=c1E[x1]+c2E[x2]
Var[c1x1+c2x2]=c12var[x1]+c2
2Var[x2]
+2c1
c2
cov{x1
x2
}
Cov{x1x2}=E[x1x2]E[x1]E[x2]
Example: The joint pdf of x1 and x2, P(x1,x2), is
8/21/2019 Basic Probability Review
71/77
Example: The joint pdf of x1and x2, P(x1,x2), is
2.002.0
02.002.002.0
753
3
21
222
1
1
1
xxx
x
xx
(a)Find the marginal pdfs p1(x1) and p2(x2).
(b)Are x1and x2independent?
(c)Compute E{x1 + x2}
8/21/2019 Basic Probability Review
72/77
8/21/2019 Basic Probability Review
73/77
8/21/2019 Basic Probability Review
74/77
8/21/2019 Basic Probability Review
75/77
8/21/2019 Basic Probability Review
76/77
Example: 12 3-3
8/21/2019 Basic Probability Review
77/77
Example: 12.3 3
A lot includes four defectives (D) items and
six good (G) ones. You select one itemrandomly and test it. Then, withoutreplacement, you test a second item. Letthe r.v.sx1and x2represents the outcomes
for the first and second item, respectively.a) Determine the joint and marginal pdfs of x1
and x2.
b) Suppose that you get $ 5 for each gooditem you select but pay $ 6 if it is defective.Determine the mean of your revenue aftertwo items have been selected