Basic Probability Review

8/21/2019 Basic Probability Review

1/77

Probability Review

Course : AAOC C312


2/77

Review of Basic Probability (Ch 12)

Laws of probability

Addition Law

Conditional Probability

Random Variables

Probability Distribution

Joint random variable Some Common Probability Distributions

Binomial, Poisson, Exponential, Normal


3/77

Probability

Probability provides a measure of uncertaintyassociated with the occurrence of events oroutcomes of a random experiment.

Experiments

Sample Space Events 0 P(E) 1 Impossible Event; P() = 0

Certain Event; P(S) = 1 Mutually Exclusive Events Pair wise Mutually Exclusive Events Equally likely Events


4/77

Some definitions

An experiment

Any process that yields a result or an

observation Outcome

A particular result of an experiment

Sample space The set of all possible outcomes of an

experiment


5/77

Some definitions

An event

Any subset of the sample space.

If the event is A, then n (A) is the number

of sample points that belong to event A

If the event is getting heads on a series ofcoin flips, then n (heads) is the number of

heads in the sample of flips


6/77

Mutually Exclusive Event

Events defined in such a way that theoccurrence of one event precludes theoccurrence of any of the other events

If one of them happens, the other cannot

happen


7/77

ProbabilityProbability provides a measure of uncertainty

associated with the occurrence of events oroutcomes of a random experiment.

Definition:

If in a n-trial experiment an event E occursm times then the probability of occurrence ofevent E is

By definition,

0


8/77

Example: What is the probability of getting

even nos. in a rolling a die.

Example: What is the probability of gettingtotal of 7 on two dice?


9/77

Addition law of Probability

otherwiseEFPFPEP

exclusivemutuallyareFandE

FPEP

FEP

},{}{}{

},{}{

}{

For two events E and F, E + F representsunion, and EF represents intersection.


10/77


11/77

Problem: In a certain college, 25 percent of thestudent failed mathematics, 15 percent failed

chemistry, and 10 percent failed bothmathematics and chemistry. A student isselected at random.

(a) if the student failed chemistry, what is the

probability that he failed mathematics? (b) if the student failed mathematics, what is theprobability that he failed chemistry?

(c) what is the probability that he failed chemistry

or mathematics? (d) what is the probability that he failed neither

chemistry nor mathematics?


12/77

Problem: Two men A and B fire at a target.Suppose P(A) = 1/3 and P(B) = 1/5 denote

their probabilities of hitting the target. ( weassume that the events A and B areindependent). Find the probability that

(a) A does not hit the target (b) Both hit the target

(c) One of them hits the target

(d) Neither hits the target.


13/77

Bayes Theorem

,}{

}{}|{}|{

BP

APABPBAP

The two events A and B with P[B] > 0, then

Let E be an event in a sample space S, and let A1,A2,.Anbe mutually disjoint event whose union isS. then

,}{

}{}|{}|{

}{}|{...}{}|{}{}|{}{ 2211

EP

APAEPEAP

APAEPAPAEPAPAEPEP

kkk

nn


14/77

Problem: Three machines A, B and C produce,respectively, 40%, 10% and 50% of the items in

a factory. The percentage of defective itemsproduced by the machines is respectively, 2%,3% and 4%. An item from the factory is selectedat random.

(a) Find the probability that the item is defective

(b) If the item is defective, find the probability thatthe item was produced by (i) machine A, (ii)

machine B, (iii) machine C.


15/77

15

Random Variables

Definition:A random variableXon a sample space Sis

a rule that assigns a numerical value to each outcome

of Sor in other words a function from Sinto the setR

of real numbers.X : S R

x : value of random variableX

RX : The set of numbers assigned by random variableX,i.e. range space.


16/77

16

Random Variables (contd)

Classifications of Random VariablesAccording to thenumber of values which they can assume, i.e. numberof elements inRx.

Discrete Random Variables:Random variables whichcan take on only a finite number, or a countableinfinity of values, i.e.Rx is finite or countable infinity.

Continuous Random Variables:When the range space

Rx is a continuum of numbers. For example aninterval or the union of the intervals.


17/77

17


Example: Consider the experiment consisting of 4tosses of a coin then sample space is

S = {HHHH, HHHT, HHTH, HTHH, THHH, HHTT,

HTTH, TTHH, THTH, HTHT, THHT, TTTH,TTHT, THTT, HTTT, TTTT}

Let X assign to each (sample) point in S the totalnumber of heads that occurs. Then X is a random

variable with range spaceRX= {0, 1, 2, 3, 4}

Since range space is finite, X is a discrete randomvariable


18/77

18


Example:A pointPis chosen at random in

a circle C with radius r. Let X be the

distance of the point from the center of thecircle. Then X is a (continuous) random

variables withRX = [0, r]

r

P

X

O

C


19/77

19

Probability Distributions

If X is discrete random variable, the function given

by

f(x)= P[X = x]

for each x within the range of X is called the

probability mass function (pmf)ofX.

To express the probability mass function, we

give a table that exhibits the correspondencebetween the values of random variable and the

associated probabilities


20/77

20

Probability Distributions (contd)

Ex: In the experiment consisting of four tosses

of a coin, assume that all 16 outcomes are

equally likely then probability mass

function for the total number of heads is

x 0 1 2 3 4f(x) 1/16 1/4 3/8 1/4 1/16


21/77

21


A function can serve as the probability mass function

of a discrete random variable X if and only if its

value,f(x), satisfy the conditions

1. f(x) 0 for all value ofx.

2. 1)(all

x

xf

Example:Check whether the following can defineprobability distributions

.5,4,3,2,1,0for15

)()a( xx

xf


22/77

22


.5,4,3,2,1for25

1)()d(

.6,5,4,3for41)()c(

.3,2,1,0for6

5)()b(

2

xx

xf

xxf

xx

xf


23/77

23

Distribution Function

IfX is a discrete random variable, the function given

by

xtfxXPxFxt

for)()()(

wheref(t) is the value of the probability mass functionofX att, is called the distribution function or thecumulative distributionfunction (cdf) ofX.


24/77

24

Example

Cumulative Distribution function of the total

number of heads obtained in four tosses of a

balanced coin

We know that f(0) = 1/16, f(1) = 4/16, f(2) = 6/16,f(3) = 4/16, f(4) = 1/16. It follows that

F(0) = f(0) = 1/16

F(1) = f(0) +f(1) = 5/16

F(2) = f(0) +f(1) +f(2) =11/16

F(3) = f(0) +f(1) +f(2) +f(3) = 15/16

F(4) = f(0) +f(1) +f(2) +f(3) +f(4) = 1

Th di t ib ti f ti i d fi d t l f th


25/77

25

4for1

43for16

15

32for

16

11

21for16

5

10for16

1

0for0

)(

x

x

x

x

x

x

xF

The distribution function is given by

The distribution function is defined not only for the

values taken on by the given random variable, but

for all real number.


26/77

26

1 2 3 40

1/16

5/16

11/16

15/16

1

F(x)

x

Graph of the Distribution function

..

.

. .


27/77

27

The values F(x) of the distribution function of adiscrete random variableX satisfy the conditions1.F(-) = 0 andF() = 1; that is, it ranges from 0 to

1.2.If a


28/77

28

Similarly, for continuous random variable X, we associate

a probability density function (pdf) f, such that

( ) ( ) 0, .

( ) ( ) ( ) .

( ) ( ) 1.

( ) ( ) ( ) , .

( ) ( )

b

a

x

a f x x

b f a X b f x dx if a b

c f x dx

d x f x dx x

de f x

dx

for all real

is integrable and P

F for each real

F

Parameters of random variables


29/77

29

Parameters of random variables

(i)Expectationof a random variable X is

If h is a real valued function of X, then

(ii) Variance

(iii) MomentsThe r-th moment about origin is

The r-th moment about mean is

x = E(X) = x p(x)

2 2 2 2Var(X) = E(X - ) = E(X ) - {E(X)}

'

r

r = E(X )

x

xfxall

)(

xall

xfxhXhE )()())((

])[( rr XE

dxxfx )(or

dxxfxh )()(or

[ ]


30/77

Probability Density Function (pdf)

Characteristics

Random variable X

Discrete Continuous

Applicable range a, a+1, , b a x b

Conditions forpdf

p(x)0, f(x)0,1)(

b

ax

xp 1)( b

axf


31/77

Cumulative distribution function(CDF)

X

a

X

ax

continuousxdxxfXF

discretexxpXPXxP

,)()(

,)()(}{


32/77

Problem:The number of units, x, needed for anitem is discrete from 1 to 5. the probability p(x)is directly proportional to the number of units

needed. The constant of proportionality is K.(a) find the pdf of x,

(b) Find the value of the constant k

(c) determine the CDF, and find the probability thatx is even value.


33/77

Problem: Consider the following function

(a) find the value of the constant k that will makef(x) a pdf

(b) determine the CDF, and find the probability thatx is (i) larger than 12, and (ii) between 13 and 15.

2010,)(2

x

x

kxf


34/77

Expectation of Random Variable

Given that h(x) is a real function of arandom variable x, we define the expectedvalue of h(x), E{h(x)}, as the weighted

average with respect to the pdf of x.

continuousxxfxh

discretexxpxhxhE

b

a

b

ax

),()(

),()()}({


35/77

Moments of Random Variable

The mth moment of a random variable x,denoted by E(Xm), also called theexpected value of Xm, is defined

continuousxxfx

discretexxpxXE

b

a

m

i

b

ax

i

m

im

),(

),(}{


36/77

Mean

continuousxxxf

discretexxxp

xE b

a

b

ax

),(

),(

}{

The mean of x, E{x}, is a numericmeasure of central tendency of randomvariable.

First moment of x.


37/77

Variance

}var{}{

),(}){(

),(}){(}{

2

2

xxstdDev

continuousxxfxEx

discretexxpxExxVar

b

a

b

ax

The variance var{x}, is a measure ofdispersion of x around the mean


38/77

Problems

Consider a random variable X that is equal to 1,2 or3. If we know p(1) =1/2 and p(2) = 1/3 then p(3)=?

Find E{x} and Var{x} where x is the outcome whenwe are roll a fair die.

Suppose the r.v. has a following distribution function

What is the probability that X exceeds 1?

0)exp(1

00)( 2 xx

xxF


39/77

Problems

A construction firm has recently sent inbids for 3 jobs worth (in profit) 10, 20 and40 (thousand) dollars. If its probabilities of

winning the jobs are respectively 0.2, 0.8and 0.3, what is the firms expected totalprofit?

Some Standard Distributions


40/77

40

Some Standard Distributions

Bernoullis Distribution A r. v. X is said to haveBernoulli distribution if and only if the correspondingprobability mass function is given by

x 1-xp X = x = p (1 - p) , x = 0,1.

tX

Also, E(X) = p, Var(X) = p(1 - p), and M (t) = 1 - p + pe

Binomial Distribution A r v X is said to have


41/77

41

Binomial Distribution A r. v. X is said to have

Binomial distribution if and only if the corresponding

probability mass function is given by

x n-xn

p(X = x) = p (1 - p) , x = 0,1, ..., nx

t nXE(X) = np, Var(X) = np(1 - p) and M (t) = (1 - p + pe )

Geometric Distribution A r v X is said to have


42/77

42

Geometric Distribution A r. v. X is said to haveGeometric distribution if and only if the correspondingprobability mass function is given by

P(X = x) = p.qx-1, x = 1, 2, 3, .; q = 1 - p

Memoryless Property

1

2

)1()()(,1

)( ttX

qepetMp

qXV

pXE

P(X > t + h | X > t) = P(X > h), t > 0, h > 0

Poissons Distribution A random variable is


43/77

43

Poisson s Distribution A random variable issaid to be Poissonsrandom variable with parameter

if X has the mass points 0,1,2, and its

probability mass function is

x

-P(X = x) = e , x = 0,1, 2,...

x

and

+

> 0

t

(e -1)

X

Inthiscase,

E(X) = , Var(X) = and M (t) = e

TheoremIf X and Y are independent Poissons

random variables with parameters

respectively, then X+Y will be a Poissons random

variable with parameter

Theorem Suppose X has binomial distribution with


44/77

44

Theorem Suppose X has binomial distribution with

parameters n and p. If n is large and p is small so

that , then X will follow Poissons

distribution with parameter .

= np


45/77

Exponential Distribution

A continuous r.v. whose probability densityfunction is given, for some l >0, by

0,00,)(

xifxifexf

xl

l

Its CDF is

E[X] =1/, V[X] = 1/2,

0,1)( xexF xl

Markov or Memoryless Property of the Exponential


46/77

46

Markov or Memoryless Property of the Exponential

Distribution

P(X > t + h | X > t) = P(X > h), t > 0, h > 0


47/77

If the no. of arrivals at a service facilityduring a specified time period followsPoison distribution, then the distribution

of the time interval between successivearrivals must be Exponentialdistribution.

If is the rate at which events occur,then 1/ is the average time intervalbetween successive events.

47

U if if (Fi 1))( bb


48/77

Uniform: if (Fig. 1),),,( babaUX

otherwise.0,

,,1

)( bxaabxfX

)(xfX

xa b

ab1

Fig. 1

Exponential: if (Fig. 2))( lEX

otherwise.0,

,0,

)(

xe

xf

x

X

ll

)(xfX

x

Fig. 2


49/77

Mean of Uniform

Distribution

b

a

dxab

xxxf 1

2

ba

2212

1ab

Normal Distribution:


50/77

Normal Distribution:

Normal (Gaussian):X is said to be normal or Gaussian r.v,

if

This is a bell shaped curve, symmetric around the

parameter and its distribution function is given by

where is often tabulated. Since

depends on two parameters and the notation will be used to represent

.2

1)(22 2/)(

2

xX exf

,,

2

1)(

22 2/)(

2

xy

X

xGdyexF

dyexG yx

2/2

2

1)(

),(

2

NX)(xfX

x

Fig.

)(xfX

,2


51/77

The Standard Normal

Distribution

To find P(a < x < b),we need tofind the area under the appropriatenormal curve. There are several

such normal curves, but one ofthem is called standard normalcurve.

Th St d d N l


52/77

The Standard Normal

Distribution

Definition : The normal distributionwith Mean = 0; Standard deviation = 1is called the standard normaldistribution (standard normal variableis denoted by Z).


53/77

X

Normal

Distribution

Z X

Normal to Standard Normal

DistributionNormal

Standardized

X=Z +

The Normal Approximation


54/77

The Normal Approximation

to the Binomial

We can calculate binomial probabilitiesusing

The binomial formula The cumulative binomial tables

When n is large, andp is not too close

to zero or one, areas under thenormal curve with mean npandvariance npq can be used to

a roximate binomial robabilities.

A i ti th Bi i l


55/77

Approximating the Binomial

While approximating a randomvariable with integer values by acontinuous random variable, use

continuity correction. . In this, the integer value x0 of

discrete random variable is replaced

by the interval (x01/2, x0 +1/2) ofthe continuous random variable.


56/77

Thus if aand bare integers, andX*isa continuous random variableapproximating discrete random

variable X thenP(a X b) = P(a < X* b + )


57/77

Make sure that np and nqare bothgreater than 15to avoid inaccurateapproximations!


58/77

Exercise :The probability that an

electronic component will fail in lessthan 1200 hours of continuous use is0.2. Use normal approximation to find

the probability that among 250 suchcomponents, fewer than 50 will fail inless than 1200 hours of continuous

use.


59/77

X= no. of electronic components among 250

randomly chosen which fail in less than1200 hours.

X has binomial distribution with n=250, p=

0.2.We can approximate X by normal random

variable X* with mean (250)(0.2)=50 and

variance = (50)(0.8)=40.=6.324.

Z=(X*-50)/6.324 has standard normal dist.


60/77

P(X


61/77

Central Limit Theorem

Let x1, x2, , and xnbe independent andidentically distributed random variables,each with mean and standard deviation

, and defined sn= x1+x2+.+xn.As n become large (n), the distributionof snbecomes normal with mean nand

variance n2, regardless of the originaldistribution of x1, x2, , and xn.


62/77

. -


63/77

0.2266


64/77

From (1) and (2) we get z = -.37.


65/77

(b) P(-z < Z < z) = .9298

or F(z)F(-z) = .9298

or F(z)(1 - F(z)) =.9298

or 2F(z) = 1.9298

or F(z) = P(Z z)= .9649

from table, z =1.81.


66/77


67/77


68/77

73.3

J i t d i bl


69/77

Joint random variableConsider the two continuous r.vsx1, a1x1b1,

and x2, a2x2 b2. Define f(x1, x2) as the jointpdf of x1 and x2 and f1(x1) and f2(x2) as themarginal pdfs of x1and x2respectively. Then

f(x1,x

2) 0, a

1x

1

b

1, a

2 x

2

b

2

tindependenarexandxifxfxfxxf

dxxxfxf

dxxxfxf

dxxxfdx

b

a

b

a

b

a

b

a

21221121

12122

22111

2211

),()(),(

),()(

),()(

1),(

1

1

2

2

2

2

1

1


70/77

E[c1x1+c2x2]=c1E[x1]+c2E[x2]

Var[c1x1+c2x2]=c12var[x1]+c2

2Var[x2]

+2c1

c2

cov{x1

x2

}

Cov{x1x2}=E[x1x2]E[x1]E[x2]

Example: The joint pdf of x1 and x2, P(x1,x2), is


71/77

Example: The joint pdf of x1and x2, P(x1,x2), is

2.002.0

02.002.002.0

753

3

21

222

1

1

1

xxx

x

xx

(a)Find the marginal pdfs p1(x1) and p2(x2).

(b)Are x1and x2independent?

(c)Compute E{x1 + x2}


72/77


73/77


74/77


75/77


76/77

Example: 12 3-3


77/77

Example: 12.3 3

A lot includes four defectives (D) items and

six good (G) ones. You select one itemrandomly and test it. Then, withoutreplacement, you test a second item. Letthe r.v.sx1and x2represents the outcomes

for the first and second item, respectively.a) Determine the joint and marginal pdfs of x1

and x2.

b) Suppose that you get $ 5 for each gooditem you select but pay $ 6 if it is defective.Determine the mean of your revenue aftertwo items have been selected

Documents

Basic Probability Review