2[1]. Discrete Random Variables

8/8/2019 2[1]. Discrete Random Variables

1/67

1

2. Discrete Random Variables

Broadband Network Lab, NCTU, Taiwan


2/67

Broadband Network Lab, NCTU, Taiwan 2

Outline

2.1 Basic Concepts

2.2 Probability Mass Functions

2.3 Functions of Random Variables

2.4 Expectation, Mean, and Variance

2.5 Joint PMFs of Multiple Random

Variables

2.6 Conditioning

2.7 Independence


3/67


2.1 Visualization of a RandomVariable


4/67


2.1 Main Concepts Related toRandom Variables

A random variable is a real-valued function of

the outcome of the experiment.

A function of a random variable definesanother random variable.

We can associate with each random variable

certain averages of interest, such as themean and the variance.

A random variable can be conditioned on anevent or on another random variables.

There is a notion ofindependence of a randomvariable from an event or from anotherrandom variable.


5/67


2.1 Concepts Related to DiscreteRandom Variables

A discrete random variable is a real-valued

function of the outcome of the experimentthat can take a finite or countably infinitenumber of values.

A discrete random variable has an associated

probability mass function (PMF), which givesthe probability of each numerical value thatthe random variable can take.

A function of a discrete random variable

defines another discrete random variable,whose PMF can be obtained from the PMF ofthe original random variable.


6/67


2.2 Probability Mass Functions

X: Discrete random variable.

pX: PMF.

x: Any possible value ofX.

pX(x): Probability mass ofx.

( )

Note that :

For any set S of possible values ofX, we

have :


7/67


2.2 Calculation of the PMF of aRandom Variable X

For each possible value x of X:

Collect all the possible outcomes that giverise to the event {x = X}.

Add their probabilities to obtain pX(x).

Example: Let Xbe the number of heads obtained in

two independent tosses of a fair coin.

The probability of at least one head is


8/67


2.2 The Bernoulli Random Variable

Toss a coin: p(head) = p, p(tail) = 1 p.

The Bernoulli random variable X is

Its PMF is


9/67


2.2 The Binomial Random Variable

Toss a coin n times. Tosses are indep..

p(a head) = p, p(a tail) = 1 p.

X: number of heads in the n-toss sequence.

X is a binomial random variable with

parameters n and p.

PMF:

Normalization property:


10/67


2.2 The Binomial Random Variable


11/67


2.2 The Geometric Random Variable

Repeatedly and independently toss acoin with p(a head) = p, 0 < p < 1.

Geometric random variable X: Numberof tosses need for a head to come up

for the first time. PMF:


12/67


2.2 The Geometric Random Variable

It is a legitimate PMF because

1

1 1

0

( ) (1 )

(1 )

1

1 (1 )

1

k

x

k k

k

k

p k p p

p p

pp

= =

=

=

=

=

=


13/67


2.2 The Poisson Random Variable

PMF:

is a positive parameter characterizing

the PMF.


14/67


2.2 The Poisson Random Variable

Binomial RV with small p and large n

Poisson RV with = np.

Justification: See Problem 12.

Example of Poisson RV: Number of typos in a book.

Number of cars involved in accidents in acity on a given day.


15/67


2.3 Functions of Random Variables

IfY = g(X) is a function of a RV X, then

Y is also a RV.

The PMF ofY is


16/67


17/67


2.3 Example 2.1 (2/2)

Possible values ofy = 0, 1, 2, 3, 4.

pY(0) = pX(0) = 1/9.

pY(1) = pX(-1) + pX(1) = 2/9.

Similar for y = 2, 3, 4.

PMF ofY is

PMF ofZ is


18/67


2.4 Expectation

We define the expected value (also

called the expectation or the mean) of arandom variable X, with PMF pX, by

Physical meaning: Center of gravity of

the PMF.


19/67


2.4 Example 2.2

Example 2.2 : Consider two independent coin

tosses, each with a 3/4 probability of a head,and let X be the number of heads obtained.This is a binomial random variable, its PMF is :

So the mean is :

2 4 Moments Va iance and


20/67


2.4 Moments, Variance, andStandard Deviation

nth moment = E[Xn].

The 1st moment is just the mean.

Variance: var(X) = E[(X - E[X])2].

The variance provides a measure of

dispersion ofX around its mean.Another measure of dispersion is the

standard deviation ofX:

X has the same unit as X.


21/67


2.4 Example 2.3

Recall Example 2.1.

;

2 4 Expected Value Rule for


22/67


2.4 Expected Value Rule forFunctions of Random Variables

Let X be a random variable with PMF pX . Then,

the expected value of the random variable g(X)is given by :

Proof:


23/67


2.4 Example 2.3 (continued)

The variance ofX can also be calculated as :

2 4 Mean and Variance of a Linear


24/67


2.4 Mean and Variance of a LinearFunction of a Random Variable

Let X be a random variable and let Y = aX + b,

where a and b are given scalars. Then,

Proof : [ ] ( ) ( )

( ) ( )

[ ]

X

x

X X

x x

E Y ax b p x

a xp x b p x

aE x b

= +

= +

= +

2 4 Variance in Terms of Moments


25/67


2.4 Variance in Terms of MomentsExpression

Proof:


26/67


2.4 Example 2.4 (1/2)

If the weather is good (which happens with

probability 0.6), Alice walks the 2 miles toclass at a speed ofV = 5 miles per hour, andotherwise drives her motorcycle at a speed ofV = 30 miles per hour. What is the mean of

the time T to get to class? Sol:


27/67


2.4 Example 2.4 (2/2)

However, it is wrong to calculate the mean of

the speed V .

To summarize, in this example we have


28/67


2.4 Example 2.5

Mean and Variance of the Bernoulli.

PMF:

2 4 Discrete Uniformly Distributed


29/67


2.4 Discrete Uniformly DistributedRandom Variable.

Discrete Uniformly Distributed Random

Variable. PMF:

2 4 Discrete Uniformly Distributed


30/67


2.4 Discrete Uniformly DistributedRandom Variable.

Consider the case where a = 1 and b = n.

General case :


31/67


2.4 Example 2.7

The Mean of the Poisson.

PMF:

var(X) = . (See Problem 24.)


32/67


2.4 Example 2.8 (1/2)

Consider a quiz game where a person is given

two questions and must decide which questionto answer first. Question 1 will be answeredcorrectly with probability 0.8, and the personwill then receive as prize $100, while question2 will be answered correctly with probability0.5, and the person will then receive as prize

$200. If the first question attempted isanswered incorrectly, the quiz terminates, i.e.,the person is not allowed to attempt thesecond question. If the first question isanswered correctly, the person is allowed to

attempt the second question. Which questionshould be answered first to maximize theexpected value of the total prize moneyreceived?


33/67


2.4 Example 2.8 (2/2)

Sol:

2 5 Joint PMF of Two Random


34/67


2.5 Joint PMF of Two RandomVariables

X, Y: Two discrete random variables.

pX,Y : Joint PMF ofX and Y.

(x, y): A pair of possible values ofX and Y.

pX,Y (x, y) = P(X = x, Y = y).

Marginal PMFs:

Verify:


35/67


2.5 Example 2.9

The marginal PMF ofX or Y at a given value is

obtained by adding the table entries along acorresponding column or row.

2 5 Functions of Multiple Random


36/67


2.5 Functions of Multiple RandomVariables

Z = g(X, Y) defines another random

variables.

PMF:

Expected value:

When g(X, Y) = aX + bY + c,


37/67


2.5 Example 2.9 (continued)

Now, a new random variable Z is defined by:

Please find E[z]=?

Alternatively, we can find E[z] from the PFM ofvariable Z. (see textbook p.95)

2 Z X Y = +

[ ] [ ] 2 [ ]E Z E X E Y = +

3 6 8 3 51[ ] 1 2 3 4

20 20 20 20 20E X = + + + =

3 7 7 3 50[ ] 1 2 3 4

20 20 20 20 20E Y = + + + =

51 50[ ] 2 7.55

20 20E Z = + =

2 5 More than Two Random


38/67


2.5 More than Two RandomVariables

PMF of three random variables:

Marginal PMFs:

Expected value:

g(X1, X2,, Xn) = a1X1 + a2X2 + +anXn

2 l 2 0


39/67


2.5 Example 2.10

Mean of the Binomial : Your probability class

has 300 students and each student hasprobability 1/3 of getting an A, independentlyof any other student. What is the mean ofX,the number of students that get an A?

Sol:

1 2 1[ ] 1 0

3 3 3iE X = + =

2 5 E l 2 11


40/67


2.5 Example 2.11

The Hat Problem : Suppose that n people

throw their hats in a box and then each picksone hat at random. What is the expectedvalue ofX, the number of people that get backtheir own hat?

Sol: We introduce a random variable Xi that takesthe value 1 if the ith person selects his/her ownhat, and takes the value 0 otherwise.

;

2 6 Conditioning a Random Variable


41/67


2.6 Conditioning a Random Variableon an Event

The conditional PMF of a random variable X,

conditioned on a particular event A withP(A) > 0, is defined by

Note that are disjoint for different x,therefore

Combining above, we can see that

so pX|A is a legitimate PMF.

{X = x} A

2 6 Vi li ti f ( )


42/67


2.6 Visualization ofpX|A(x)

2 6 E l 2 12


43/67


2.6 Example 2.12

Roll a Die. X: The roll of a fair six-sided die.

A: Event that the roll is an even number.Find pX|A(x).

Sol:

2 6 E l 2 13


44/67


2.6 Example 2.13

A student will take a certain test repeatedly,

up to a maximum of n times, each time with aprobability p of passing, independently of thenumber of pervious attempts. What is the PMFof the number of attempts, given that the

student passes the test? Sol : Let A be the event that the student

passes the test.1

1

( ) (1 )n

m

m

P A p p

=

= 1

1

|

1

(1 ), 1,..., .

(1 )( )

0, otherwise

k

nm

X Am

p pif k n

p pP k

=

=

=

2.6 Conditioning one Random


45/67


2.6 Conditioning one RandomVariable on Another

The conditional PMF pX|Y ofX given Y is

Normalization property:

The conditional PMF is often convenient for the

calculation of the joint PMF,

2 6 Vi li ti f ( | )


46/67


2.6 Visualization ofpX|Y(x|y)

2 6 E l 2 14 (1/2)


47/67


2.6 Example 2.14 (1/2)

Professor May B. Right often has her facts

wrong, and answers each of her studentsquestions incorrectly with probability 1/4,independently of other questions. In eachlecture May is asked 0, 1, or 2 questions with

equal probability 1/3. Let X and Y be thenumber of questions May is asked and thenumber of questions she answers wrong in agiven lecture, respectively. Please constructthe joint PMF pX,Y (x, y).

2 6 E ample 2 14 (2/2)


48/67


2.6 Example 2.14 (2/2)

Sol:

2 6 Example 2 15 (1/2)


49/67


2.6 Example 2.15 (1/2)

Consider a transmitter that is sending

messages over a computer network. Definethe following two random variables:

X : the travel time of a given message,

Y : the length of the given message.

Assume that the travel time X of the messagedepends on its length Y. The travel time is

104Y secs with probability 1/2, 103Y secswith probability 1/3, and 102Y secs withprobability 1/6. Please find the PMF of X.

2 6 Example 2 15 (2/2)


50/67


2.6 Example 2.15 (2/2)

Sol: We can know:

and

So:

2 6 Conditional Expectations


51/67


2.6 Conditional Expectations

The conditional expectation ofX given

an event A with P(A) > 0, is defined by

For a function g(X), we have

The conditional expectation ofX given avalue y ofY is defined by

2 6 Total Expectation Theorem


52/67


2.6 Total Expectation Theorem

Total expectation theorem:

Let A1, , An form a partition of thesample space and P(Ai) > 0 for all i, then

Let A1, , An form a partition of anevent B and for all i, theniP(A B) > 0

2.6 Verify the Total Expectation


53/67


y pTheorem

Verify

2 6 Example 2 16


54/67


2.6 Example 2.16

Messages Transmission :

P(Boston to New York) = 0.5.P(Boston to Chicago) = 0.3.

P(Boston to San Francisco) = 0.2.

X: transit time of a message (random).

E[X | New York] = 0.05 s.

E[X | Chicago] = 0.1 s.

E[X | San Francisco] = 0.3 s.

Please find E[X]. Sol:

2 6 Example 2 17 (1/2)


55/67


2.6 Example 2.17 (1/2)

Mean and Variance of the Geometric Random

Variable.

Alternative way to fine E[X] and var(X).

Define A1 = {X = 1} = {first try is a success},

A2 = {X > 1} = {first try is a failure}.

(1)

2 6 Example 2 17 (2/2)


56/67


2.6 Example 2.17 (2/2)

So

(2)

2.7 Independence of a Random


57/67


pVariable from an Event

We say the random variable X is

independent of the event A if

From the definition of the conditional

PMF, we have

If P(A) > 0, independence is the sameas the condition


58/67

2.7 Independence of Random


59/67


pVariables

We say that two random variables X

and Y are independent if

or, equivalently,

X and Y are said to be conditionallyindependent, given P(A) > 0, if

Once more, this is equivalent to

2.7 Facts About Independent


60/67


pRandom Variables

IfX and Y are independent random

variables, then E[XY] = E[X] E[Y].

For any functions g and h, g(X) and h(Y)are independent.

E[g(X) h(Y)] = E[g(X)] E[h(Y)].

var(X + Y) = var(X) + var(Y).


61/67

2.7 Independence of Several


62/67


pRandom Variables

Three random variables X, Y, and Z are

said to be independent if

IfX, Y, and Z are independent random

variables, then f(X), g(Y), and h(Z) are also independent.

g(X, Y) and h(Z) are independent.

g(X, Y) and h(Y, Z) are usually not

independent.

2.7 Variance of the Sum of


63/67


Independent Random Variables

IfX1, X2, , Xn are independent random

variables, then

Example 2.20 :

Xi ~ Bernoulli(p).X = X1 + + Xn ~ Binomial(n, p).

{Xi} are independent.

Sol:

2 7 Example 2 21 (1/2)


64/67


2.7 Example 2.21 (1/2)

We wish to estimate the approval rating of a president,to be called C. To this end, we ask n persons drawn atrandom from the voter population, and we let Xi be arandom variable that encodes the response of the ithperson:

We model X1,X2, . . . , Xn as independent Bernoulli

random variables with common mean p and variancep(1 p). Naturally, we view p as the true approvalrating of C. We average the responses and computethe sample mean Sn, defined as

Please find E[Sn] and var[Sn].

2 7 Example 2 21 (2/2)


65/67


2.7 Example 2.21 (2/2)

Sol:


66/67

2 7 Example 2 22 (2/2)


67/67

2.7 Example 2.22 (2/2)

To see how accurate this process is, consider nindependent Bernoulli random variables X1, . . . , Xn,

each with PMF

In a simulation context, Xi corresponds to the ithoutcome, and takes the value 1 if the ith outcomebelongs to the event A. The value of the randomvariable

is the estimate ofP(A) provided by the simulation.According to Example 2.21, X has mean P(A) andvariance P(A)(1P(A))/n, so that for large n, it providesan accurate estimate ofP(A).

Documents

2[1]. Discrete Random Variables