Chapter 2: Random Variable and Probability Distributions [email protected] Yang Zhenlin

Chapter 2: Random Variable and Probability Distributions

[email protected]://www.mysmu.edu/faculty/zlyang/

Yang ZhenlinYang Zhenlin

Chapter 2

STAT151, Term I 2015-16 © Zhenlin Yang, SMU

Chapter Contents

Random Variable (r.v.)

– Discrete r.v.– Continuous r.v.

Distribution of a Random Variable

– Probability mass function (pmf)– Probability density function (pdf)

Expectation of a Random Variable

Variance and Standard Deviation

Moment Generating Function

2

Chapter 2


Random Variable

In real-world problems, we are often faced with one or more quantities that do not have fixed values.

The values of such quantities depend on random actions and they change from one experiment to another,

Number of babies born in a certain hospital

Number of traffic accidents occurred on a road per month

The amount of rainfall in Singapore per year

The starting price of a stock each day, etc.

In probability, quantities introduced in these diverse examples are called Random Variables.

Study the properties of random variables allows us to have a better understanding on the real-world phenomena, and to control or to predict their behavior, … .

3

Chapter 2


Random Variable

Example 2.1 If in the experiment of rolling two fair dice, X is the sum, then X is a variable. Since the value of X changes from one experiment to another, X is also a "random variable". The possible values of X and the corresponding probabilities are summarized as follows:

X 2 3 4 5 6 7 8 9 10 11 12

pi 36

1

36

2

36

3

36

4

36

5

36

6

36

5

36

4

36

3

36

2

36

1

4

where, for example,

p2 = P(X=2) = P{(1,1)} = 1/36,

p3 = P(X=3) = P{(1,2), (2,1)} = 2/36,

p7 = P(X=7) = P{(1,6), (6,1), (2,5), (5,2), (3,4), (4,3)} = 6/36

Chapter 2


Random Variable

In Example 2.1, numerical values of X depend on the outcomes of the experiment, e.g.,

• if the outcome e = (2,3), then X = 2+3 = 5,

• if e = (5,6), then X = 5+6 = 11, etc.

Thus, X is a real-valued function.

On the other hand, if, for example, X = 3, then it must be that e = (1,2) or (2,1). Therefore, the inverse image of the function X is an event!

Definition 2.1. Let S be the sample space of an experiment. A real-valued function X defined on S and taking values in R (the set of real numbers) is called a random variable (r.v.) if for each interval I R, the inverse image {e: X(e) I } is an event.

Definition 2.1. Let S be the sample space of an experiment. A real-valued function X defined on S and taking values in R (the set of real numbers) is called a random variable (r.v.) if for each interval I R, the inverse image {e: X(e) I } is an event.

5

Chapter 2


Random Variable

Example 2.2 Suppose that we toss a coin having a probability p of coming up heads, until the first head appears. Let N denote the number of flips required, then assuming the outcome of successive flips are independent, N is a random variable taking values 1, 2, 3, …, with respective probabilities

P{N =1} = P(H) = p,

P{N =2} = P(TH) = (1p)p,

P{N =3} = P(TTH) = (1p)2p,

ppHTTTPnNP n

n

1

1

)1()(}{

1

1 1)1(1

)1(n

n

p

pppAll probabilities add up to 1:

6

Chapter 2



F(x) “cumulates” all of the probabilities of the values of X up to and including x.

Definition 2.2 (Cumulative Distribution Function) The cumulative distribution function (CDF) of a random variable X is defined by,

F(x) = P{X ≤ x},for any real x.

Definition 2.2 (Cumulative Distribution Function) The cumulative distribution function (CDF) of a random variable X is defined by,

F(x) = P{X ≤ x},for any real x.

A function F(x) is a CDF if and only if it satisfies:

(a) limx F(x) = 0 and limx F(x) = 1

(b) limh0 F(x+h)= F(x), h > 0 (right continuous)

(c) a < b implies F(a) ≤ F(b) (nondecreasing).

A function F(x) is a CDF if and only if it satisfies:

(a) limx F(x) = 0 and limx F(x) = 1

(b) limh0 F(x+h)= F(x), h > 0 (right continuous)

(c) a < b implies F(a) ≤ F(b) (nondecreasing).

7

Chapter 2



8

Solution: Possible values for X are 1, 2, 3, and 4, and the corresponding probabilities are:

P(X = 1) = P{(1,1)} = 1/16 P(X = 2) = P{(1,2), (2,1), (2,2)} = 3/16 P(X = 3) = P{(1,3), (2,3), (3, 3), (3,2), (3,1)} = 5/16 P(X = 4) = P{(1,4), (2,4), (3,4), (4,4), (4,3), (4,2), (4,1)} = 7/16.

Thus, the CDF of X is,

x 1 2 3 4

F(x) 1/16 4/16 9/16 1

1

x 0 1 2 3 4

9/16

1/4

F(x)

1/16

Example 2.3. A four-sided die has a different number, 1, 2, 3, or 4, affixed to each side. On each roll, each of the four numbers is equally likely to occur. A game consists of rolling the die twice. If X represents the maximum of the two rolls, then X is a random variable. Find and sketch the CDF of X

Example 2.3. A four-sided die has a different number, 1, 2, 3, or 4, affixed to each side. On each roll, each of the four numbers is equally likely to occur. A game consists of rolling the die twice. If X represents the maximum of the two rolls, then X is a random variable. Find and sketch the CDF of X

Chapter 2



Example 2.4. A function is specified as follows.

F(x) =

Example 2.4. A function is specified as follows.

F(x) =

.313221122121104

00

xxxxxx

x

9

1

3/4 2/3 1/2

1/4

0 1 2 3 x

F(x) Solution: (a) … ,

(b) : P(X < 2) = F(2–) = 1/2.

P(X = 2) = F(2) – F(2–)

= (2/12+1/2)–1/2 = 1/6.

P(1 ≤ X < 3) = P(X < 3) – P(X < 1)

= F(3–) – F(1–)

= (3/12+1/2) – 1/4 = 1/2.

(a) Verify that it is a CDF.

(b) Plot F(x) and calculate: P(X < 2), P(X = 2), and P(1 ≤ X < 3).

(a) Verify that it is a CDF.

(b) Plot F(x) and calculate: P(X < 2), P(X = 2), and P(1 ≤ X < 3).

Chapter 2


Note. Random variables can be classified as discrete, continuous, or mixed. The function given in Example 2.4 is a mixed CDF which has “jumps” at certain points. We will concentrate on pure discrete and pure continuous types of random variables.


Definition 2.3 (Discrete R.V. and Probability Mass Function) If the set of all possible values of a random variable, X, is a countable set, x1, x2, . . . , xn, or x1, x2, . . ., then X is called a discrete random variable. The function

p(x) = P(X = x), x = x1, x2, . . .

that assigns the probability to each possible value of X will be called probability mass function (pmf).

Definition 2.3 (Discrete R.V. and Probability Mass Function) If the set of all possible values of a random variable, X, is a countable set, x1, x2, . . . , xn, or x1, x2, . . ., then X is called a discrete random variable. The function

p(x) = P(X = x), x = x1, x2, . . .

that assigns the probability to each possible value of X will be called probability mass function (pmf).

10

So, pmf tells how the probabilities are distributed across the values of r.v. X.So, pmf tells how the probabilities are distributed across the values of r.v. X.

Chapter 2



A function p(x) is a pmf if and only if

(i) p(x) ≥ 0 for all x, and (ii)

A function p(x) is a pmf if and only if

(i) p(x) ≥ 0 for all x, and (ii) .1)( xall

xp

11

Solution: (a) p(x) > 0 for all x = 0, 1, 2, . . . ; and

2/1,2))2/1(2/11()( 2

all kkkxp

x

(b) Since p(0) = k/2, and p(2) = –k/4, the two values of the function have opposite signs no matter what value k takes. Therefore, it is impossible for p(x) to be always nonnegative, and hence p(x) cannot be a pmf.

Example 2.5. Is each of the following functions a pmf? If it is, find the constant k.

(a) p(x) = k(1/2)x , x = 0, 1, 2, . . .

(b) p(x) = k [(1/2)x – 1/2], x = –2, –1, 0, 1, 2.

Example 2.5. Is each of the following functions a pmf? If it is, find the constant k.

(a) p(x) = k(1/2)x , x = 0, 1, 2, . . .

(b) p(x) = k [(1/2)x – 1/2], x = –2, –1, 0, 1, 2.

Chapter 2



The r.v's introduced above take countable number of values. However, many social phenomena can only be described by variables with uncountable values, e.g., arrival time of a train, lifetime of a transistor, etc.

Definition 2.4 (Continuous R.V. and Probability Density Function) A r.v. X is said to be a continuous random variable if there exists a nonnegative function f, defined for all real x (– , ), such that for any set B of real numbers

P{X B} = .

The function f is called the probability density function (pdf) of the r.v. X.

Definition 2.4 (Continuous R.V. and Probability Density Function) A r.v. X is said to be a continuous random variable if there exists a nonnegative function f, defined for all real x (– , ), such that for any set B of real numbers

P{X B} = .

The function f is called the probability density function (pdf) of the r.v. X.

f (x)dxB

12

So, pdf f(x) tells how likely values of r.v. X occurs around x.So, pdf f(x) tells how likely values of r.v. X occurs around x.

Chapter 2


Some important properties can be deduced immediately from Definition 2.4:

i) F(t) = f (x)dx

t

(let B = (–, t] );

ii) P(a ≤ X ≤ b) = f (x)dxa

b

( let B = [a, b]);

iii) P(X = a) = f (x)dxa

a

= 0 (let a = b in ii) )

iv) The pdf can be obtained from CDF by differentiation:

f(x) = F'(x) = d

dxF(x), for all x where F'(x) exists.

Note that it is possible that F'(x) does not exist at finite number of points,

where f(x) can be defined as either ‘left’ or ‘right’ derivatives.


13

Chapter 2


Example 2.6 A CDF has the following expression

F(x) =

Sketch the graph of F(x) and find the pdf f(x).

Example 2.6 A CDF has the following expression

F(x) =

Sketch the graph of F(x) and find the pdf f(x).

xxx

xxx

5.1,15.15.0,5.025.0

5.00,0,0


14

1

.5

.5 1 1.5 0 x

F(x)

Solution: The graph of F(x) is shown below.

xx

xx

xf

5.1,05.15.0,5.0

5.00,10,0

)(

Clearly, F(x) is not differentiable at x = 0, 0.5 and 1.5. Take the ‘right’ derivatives, i.e., f(0) = 1, f(0.5) = 0.5, and f(1.5) = 0, we then have

Chapter 2


Example 2.7 A machine produces copper wire, and occasionally there is a flaw at some point along the wire. The length of the wire (in meters) produced between successive flaws is a continuous r.v. X with pdf of the form:

f(x) = c(1+x)3, x > 0,

where c is a constant. Find c, and CDF.

Example 2.7 A machine produces copper wire, and occasionally there is a flaw at some point along the wire. The length of the wire (in meters) produced between successive flaws is a continuous r.v. X with pdf of the form:

f(x) = c(1+x)3, x > 0,

where c is a constant. Find c, and CDF.

A function f(x) is a pdf if and only if it satisfies the properties

(i) f(x) ≥ 0 for all x and (ii)

A function f(x) is a pdf if and only if it satisfies the properties

(i) f(x) ≥ 0 for all x and (ii)


15

Solution: 1 = f (x)dx

= c(1 x) 3

0

dx = c

2

1, c =2.

Now, P(X ≤ x) = 2(1 t) 30

x

dt = 1 – (1+x)–2, hence

0,)1(10,0

)( 2 xxx

xF

1)(

dxxf

Chapter 2


The expected value of a continuous r.v. with pdf f(x) is defined by

= E(X) =

if the integral is absolutely convergent. Otherwise we say that E(X) does not exist.

The expected value of a continuous r.v. with pdf f(x) is defined by

= E(X) =

if the integral is absolutely convergent. Otherwise we say that E(X) does not exist.


Definition 2.5. (Expectation) The expected value of a discrete r.v. X with pmf p(x) is defined by

= E(X) =

i.e., a weighted average of all possible values of X. It is also called mean of the population represented by X.

Definition 2.5. (Expectation) The expected value of a discrete r.v. X with pmf p(x) is defined by

= E(X) =

i.e., a weighted average of all possible values of X. It is also called mean of the population represented by X.

,)(xall

xpx

,)(

dxxfx

16

Expected value of a discrete r.v. is just the weighted average of the possible values of X, weighted by their chances to occur.Expected value of a discrete r.v. is just the weighted average of the possible values of X, weighted by their chances to occur.

Chapter 2


In Example 2.1, X is the sum of the two numbers shown on the two dice. The expected sum is:

E(X) =


736

1)12(

36

3)4(

36

2)3(

36

1)2()(

xall

xpx

In Example 2.7, we have

E(X) = 2x(1 x) 30

dx = –

x

(1 x)20

+ 1

(1 x)2

0

dx = – 1

(1 x) 0

= 1.

17

Expectation is an extremely important concept in summarizing characteristics of distributions. It gives a measure of central tendency. It has following simple but important property:

Expectation is an extremely important concept in summarizing characteristics of distributions. It gives a measure of central tendency. It has following simple but important property:

If a and b are constants, thenE[aX + b] = a E[X] + b

If a and b are constants, thenE[aX + b] = a E[X] + b

Chapter 2


Function of a Random Variable

The following results are important:

If X is a r.v., then a function of it, u(X), is also a r.v. ,

E[u(X) ] = , if X is a discrete r.v. with pmf p(x),

E[u(X) ] = , if X is a continuous r.v. with pdf f(x).

The following results are important:

If X is a r.v., then a function of it, u(X), is also a r.v. ,

E[u(X) ] = , if X is a discrete r.v. with pmf p(x),

E[u(X) ] = , if X is a continuous r.v. with pdf f(x).

xall

xpxu )()(

dxxfxu )()(

18

Example 2.8 The r.v. X has pmf p(x) = 1/3, for x = 1, 0, 1. Find E(X) and E(X2).

Solution: 0)1()0()1()()(E 31

31

31

xall

xxpX

3

2)1()0()1()()(E 3

12312

31222

xall

xpxX

The above result says that the expectation of the r.v., u(X), can simply be found through the distribution of the original r.v. X ! The above result says that the expectation of the r.v., u(X), can simply be found through the distribution of the original r.v. X !

Chapter 2


Another important quantity of interest is the variance of a r.v. X, denoted by Var(X), which is defined by

Var(X) = E[(X –)2],

where = E(X). A related quantity is the standard deviation,

(X) = {Var(X)}1/2.

Variance and standard deviation (sd) measure how much the values of X vary around its mean .

As (X –)2 is a function of X, i.e., u(X) = (X –)2, from the results given in the last slide, we have

Var(X) = , if X is discrete with pmf p(x),

Var(X) = , if X is continuous with pdf f(x).

As (X –)2 is a function of X, i.e., u(X) = (X –)2, from the results given in the last slide, we have

Var(X) = , if X is discrete with pmf p(x),

Var(X) = , if X is continuous with pdf f(x).

Variance of a Random Variable

19

xall

xpx )()( 2

dxxfx )()( 2

Chapter 2



20

Example 2.9. The time elapsed, in minutes, between the placement of an order of pizza and its delivery is a random variable with pdf

f(x) = 1 15 if 25 x 400 otherwise

(a) Determine the mean and standard deviation of the time it takes for the pizza shop to deliver pizza.

(b) Suppose that it takes 12 minutes to bake pizza. Determine the mean and standard deviation of the time it takes for the delivery person to deliver pizza.

Solution: Let X be the time between order and delivery.

(i) E(X) = 40

25)15/1( dxx = 32.5,

2 = Var(X) = 40

25

2 )15/1()5.32( dxx = 18.75, and = 4.33.

(ii) The time it takes for delivery person to deliver pizza is Y = X –12. Therefore,

E(Y) = E(X – 12) = 40

25)15/1()12( dxx = 32.5 – 12 = 20.5,

Var(Y) = E[(X – 12 – 20.5)2] = 40

25

2 )15/1()5.2012( dxx = 18.75.

Chapter 2


To show the second result,

Var(X) = E[(X –)2]

= E(X2 –2X+ 2)

= E(X2) –2 E(X)+ 2

= E(X2) – 2

If X is a r.v. with mean , and a and b are arbitrary constants, then

E(aX + b) = aE(X) + b

Var(X) = E(X2) – 2

Var(aX + b) = a2Var(X)

(aX + b) = |a|(X).

21

To show third result,

Var(aX + b)

= E[(aX + b – E(aX + b))2]

= E[(aX + b – aE(X) – b)2]

= E[(aX –aE(X))2]

= E[a 2(X –E(X))2]

= a 2E[ (X –E(X))2]

= a2Var(X)

To show third result,

Var(aX + b)

= E[(aX + b – E(aX + b))2]

= E[(aX + b – aE(X) – b)2]

= E[(aX –aE(X))2]

= E[a 2(X –E(X))2]

= a 2E[ (X –E(X))2]

= a2Var(X)

We note in Example 2.9 Var(X) and Var(Y) are the same. Why? We note in Example 2.9 Var(X) and Var(Y) are the same. Why?


Chapter 2

STAT151, Term I 2015-16 © Zhenlin Yang, SMU22


Example 2.10. The monthly sales at a computer store have a mean of $25,000 and a standard deviation of $4,000. Profits are 30% of the sales less fixed costs of $6,000. Find the mean and standard deviation of the monthly profit.

Example 2.10. The monthly sales at a computer store have a mean of $25,000 and a standard deviation of $4,000. Profits are 30% of the sales less fixed costs of $6,000. Find the mean and standard deviation of the monthly profit.

Solution: Let X = Sales. E(X) = 25,000, and Var(X) = 4,0002

Let Y = Profit. Then, Y = .30(X) – 6,000.

E(Profit) = E(Y) = .30 E(X) – 6,000 = (.30)(25,000) – 6,000 = 1,500.

Var(Profit) = Var(Y) = Var[(.30)(X) – 6,000] = (.30)2Var(X) = 1,440,000

(Profit) = (Y) = 0.3(X) = 1,200.

Chapter 2


Moment Generating Function*

Definition 2.12 (Moment-Generating Function) Let X be a random variable, discrete or continuous. If there exists a positive number h such that

M(t) = E(etX)

exists and is finite for –h < t < h, then M(t) is called the Moment-Generating Function (MGF) of X.

Definition 2.12 (Moment-Generating Function) Let X be a random variable, discrete or continuous. If there exists a positive number h such that

M(t) = E(etX)

exists and is finite for –h < t < h, then M(t) is called the Moment-Generating Function (MGF) of X.

23

The mean, variance and standard deviation are important characteristics of a distribution. For some distributions, it is rather difficult to compute these quantities directly. A special function defined below can help. More importantly, the uniqueness property of this function often help to find the distribution of some useful functions of r.v.s.

Chapter 2



24

Why is it called the Moment Generating Function? because it generates moments.

What are the moments?

• the term ‘moment’ is from mechanics, representing the product of a distance and its weight.

• So, xi p(xi) is a moment, and xi p(xi) or E(X) is the moment of the ‘system’.

• In statistics, E(X), E(X2), E(X3), …, are the 1st, 2nd , 3rd, …, moments about the origin;

• E(X), E[(X)2], E[(X)3], etc, are the 1st, 2nd , 3rd central moments, or moments about the mean.

• See p69 of text for details.

Chapter 2


Thus, if MX(t) = MY(t), it must be that

p(vi) = q(vi), i = 1, 2, …


25

Property of MGF:

M.G.F., if it exists, completely determines the distribution function. In other words, if two random variables, assuming the same set of vaules, have the same M.G.F., they must have the same distribution function.

E(Xr) = M(r)(0), the rth derivative of M(t) evaluated at 0.

Property of MGF:

M.G.F., if it exists, completely determines the distribution function. In other words, if two random variables, assuming the same set of vaules, have the same M.G.F., they must have the same distribution function.

E(Xr) = M(r)(0), the rth derivative of M(t) evaluated at 0.

We use a discrete r.v.s to demonstrate these properties: if X and Y have possible values {v1, v2, …}, and pmfs p(x) and q(x), then,

.)()()()(

,)()()()(

21

21

21

21

vqevqeeEtM

vpevpeeEtMvtvttY

Y

vtvttXX

Chapter 2


Thus,Thus,


26

To understand the property E(Xr) = M(r)(0), we have

)()()(

)()()()(

332211

321

321

321

xpxexpxexpxe

xpexpexpetMxtxtxt

xtxtxtdtd

)()()(

)()()()(

3232

221

21

321

321

3212

2

xpxexpxexpxe

xpexpexpetM

xtxtxt

xtxtxt

dtd

)(E)()()()0( 23

232

221

21 XxpxxpxxpxM

In general,

,)()(1

)(

i iri

xtr xpxetM i )(E)()0(1

)( r

i iri

r XxpxM

Thus,Thus, )(E)()()()0( 332211 XxpxxpxxpxM

Chapter 2



27

Example 2.11. If X has the MGF,

then, as the coefficients of the e terms are probabilities, the probabilities must be 3/6, 2/6, and 1/6; the values beside t are the values of X, which are 1, 2, and 3. The pmf of X is

6

1

6

2

6

3)( 32 ttt eeetM

x 1 2 3

p(x) 3/6 2/6 1/6

Example 2.12. Suppose the MGF of X is

Find the distribution of X.

.2ln,21

2)(

t

e

etM t

t

Chapter 2



28

Solution: Until we expand M(t), we cannot detect the coefficients of

Recall:

we have,

.ixte

11,11

1 32

zzzzz

3

3

2

2

3

3

2

2

2

1

2

1

2

1

2221

2

21

2)(

ttt

tttt

t

t

eee

eeee

e

etM

That is, P(X = x) = (1/2)x, for positive integer x; the pmf of X is thus,

That is, P(X = x) = (1/2)x, for positive integer x; the pmf of X is thus,

.,3,2,1,2

1)(

xxp

x

Documents

Chapter 2: Random Variable and Probability Distributions [email protected] Yang Zhenlin