Department of Computer Engineering, Faculty of Engineering ...anan/myhomepage/wp-content/uploads/2018/08/... · •Expected Value of a Derived RV •Variance & Standard Deviation

Department of Computer Engineering, Faculty of Engineering,

Kasetsart University, THAILAND

1st Semester 2018

Assoc. Prof. Anan Phonphoem, Ph.D.http://www.cpe.ku.ac.th/~anan

Department of Computer Engineering, Faculty of Engineering,Kasetsart University, THAILAND

• Expected Value of a Derived RV

• Variance & Standard Deviation

• Conditional Probability Mass Function

• Joint PMF

• Marginal PMF

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 3

4

x SX

Theorem:

E[Y] = g[X]PX(x)

• To Find E[Y]

→Find PY(y)

→Find E[Y]

• In case of interesting only E[Y]

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

5

Theorem: For PX(x),

E[aX + b] = aE[X] + b

Note:

• Linear Transformation

• scale change of quantity (change the unit)

Ex. Celsius Fahrenheit

• Adding score to every one

new E[X] = old E[X] + adding value

• Y = X2 E[Y] (E[X])2 E[g(X)] g(E[X])


http://www.mathsisfun.com/temperature-conversion.html

To convert from Celsius to Fahrenheit,

first multiply by 180/100, then add 32


PR(r) =

1/4 r = 0

3/4 r = 2

0 otherwise

Find E[Y] for Y = g(R) = 2R + 4

E[R] = (1/4)(0) + (3/4)(2)

= 3/2

E[Y] = E[g(R)]

= E[2R + 4]

= 2E[R] + 4

= 2(3/2) + 4 = 7

E[Y] = g(R)PR(r)

= g(0)(1/4) + g(2)(3/4)

= (2*0+4)(1/4) +(2*2+4)(3/4)

= 1 + 6 = 7

7

• From class average, am I doing OK?

• Let Y = g(X) = X – E[X]

E[Y] = E[g(X)]

= (x - X) PX(x)xSx

= x PX(x) –xSx

X PX(x)xSx

= X – X PX(x)xSx

= 0


http://www.google.co.th/url?sa=i&rct=j&q=student+cartoon&source=images&cd=&cad=rja&docid=hvc20UH1pn9WrM&tbnid=X4bip31v6nUrOM:&ved=0CAUQjRw&url=http://jaunisse.wordpress.com/2011/04/15/the-beginning/&ei=YpvcUdTAD4-ViQf49YHAAw&bvm=bv.48705608,d.aGc&psig=AFQjCNHZ_maxqY5ZeGheA1fw_WYzmtEfPA&ust=1373498541907495

8

Theorem: For any random variable X

E[X – x] = 0


• We knew average, E[X], why do we need

these Variance & Standard Deviation?

• How far from the average?

• T = X – x

• E[T] = E[X – x ]

= 0


10

• The useful measurement is E[|T|]

• E[T2] = E[(X – x )2] → Variance

Definition:

Var[X] = E [(X – x )2]


11

Definition:

X = Var[X]

Sigma X

• X x

• Ex. X = 15, Score +6 from mean

OK (middle of the class)

• Ex. X = 3,Score +6 from mean

Very Good (in group of top class)



millimeters

The heights (at the shoulders) are: 600mm, 470mm, 170mm, 430mm and 300mm.

Mean = 600+470+170+430+300

5

= 394


Standard Deviation: σ = √21,704

= 147.32

millimeters


millimeters

So, using the Standard Deviation we have a "standard" way of

knowing what is normal, and what is extra large or extra small.

Rottweilers are tall dogs. And Dachshunds are a bit short.

15

= (x - X)2 PX(x)xSx

Var[X]x2

=

= E [(X – x )2]

= x2 PX(x) – 2 x xPX(x) + x2 PX(x)

xSx

xSx

xSx

= E[X2] – 2 x xPX(x) + x2 PX(x)

xSx

xSx

= E[X2] - 2 x 2 + x

2

Var[X] = E[X2] – x2 = E[X2] – (E[X])2


16

Theorem:

• If X always takes the same value,

Var[X] = 0

• If Y = X+b,

Var[Y] = Var[X]

• If Y = aX,

Var[Y] = a2 Var[X]


17

• Bernoulli – p Var[X] = p(1 – p)

• Geometric – p Var[X] = (1 – p)/p2

• Binomial – n,p Var[X] = np(1 – p)

• Pascal – k,p Var[X] = k(1 – p)/p2

• Poisson – Var[X] =

• Discrete U – k,l Var[X] = (l – k) (l – k+2)/12


18

Definition: Given event B, P[B] > 0

PX|B(x) = P[X=x|B]

P[A|B] = P[X = x|B]

i = 1

n

P[A|Bi]P[Bi]P[A] = Theorem:

i = 1

n

PX|Bi (x)P[Bi]PX(x) = Theorem:


19

• Discrete Uniform Random Variable

• Bernoulli Random Variable

• Geometric Random Variable

• Binomial Random Variable

• Pascal Random Variable

• Poisson Random Variable





• Joint PMF


Experiment (Physical Model)

• Compose of procedure & observation

• From observation, we get outcomes

• From all outcomes, we get a (mathematical)

probability model called “Sample space”

• From the model, we get P[A], A S


From a probability model

• Ex.: 2 traffic lights, observe the seq. of light

S = {R1R2,R1G2,G1R2,G1G2}

• If assign a number to each outcome in S, each number that we observe is called “Random Variable”

• Observe the number of red light

SX = {0,1,2}


How about Observe more than one thing

in an experiment ?

• Each observation a Random Variable

• 2 observations 2 Random Variables

• 2 observations Multiple RVs


• For an experiment, Observe one thing

• Model with one Random Variable

• Describe the prob. model by using PMF

• For the same experiment, Observe 2 things

• 2 Random Variables X and Y

• Joint PMF

• PX,Y (x,y)


25

PX(x) = P[X=x]

X(s) = xEvent of all outcomes s in S

PMF of random variable X

X is a dummy variable

PX(a) or PX(z) or PX(☺)

A Random variable X



PX,Y(x,y) = P[X=x, Y=y]

Definition:

SX,Y = {(x,y) | PX,Y(x,y) > 0 }

• Timing coordination of 2 traffic lights

• P[the second light is the same color as the first] = 0.8

• Assume 1st light is equally likely to be green or red


• Find P[The second light is green] ?• Find P[wait for at least one light] ? • Let observe

• number of G and number of G before 1st R

• Find the Joint PMF


0.5

0.5

0.8

0.2

0.2

0.8

G1

R1

G2

G2

R2

R2

G1G2 = 0.4

G1R2 = 0.1

R1G2 = 0.1

R1R2 = 0.4

• Let

• Count number of G random variable X

• Count number of G before 1st R (1st one = G) Y


0.5

0.5

0.8

0.2

0.2

0.8

G1

R1

G2

G2

R2

R2

G1G2 = 0.4

G1R2 = 0.1

R1G2 = 0.1

R1R2 = 0.4

X = 2, Y = 2

X = 1, Y = 1

X = 1, Y = 0

X = 0, Y = 0

• Let g(s) transforms each outcome a pair of

RV (X,Y)

• g(G1G2) = (2,2) g(G1R2) = (1,1)

• g(R1G2) = (1,0) g(R1R2) = (0,0)

• For each pair of x,y

• PX,Y(x,y) = sum of prob. that X = x and Y = y

• PX,Y(1,0) P[R1G2]


0.5

0.5

0.8

0.2

0.2

0.8

G1

R1

G2

G2

R2

R2

G1G2

G1R2

R1G2

R1R2

X = 2, Y = 2

X = 1, Y = 1

X = 1, Y = 0

X = 0, Y = 0

31

• Joint PMF can be written in 3 forms:

0.4 x=2, y=2

0.1 x=1, y=1

0.1 x=1, y=0

0.4 x=0, y=0

0 Otherwise

PX,Y(x,y) =

PX,Y(x,y) y=0 y=1 y=2

x=0 0.4 0 0

x=1 0.1 0.1 0

x=2 0 0 0.4

0 1 2

2

1

0

y

x

0.4 0.1

0.1

0.4

0 1 2

1

2

x

y0.4

0.4

0.1

0.1


32

PX,Y(x,y) = 1xSX

ySY

PX,Y(x,y) 0





• Joint PMF

• Marginal PMF


34

For any subset B of X,Y plane,

the probability of the event {(X,Y) B} is

P[B] = PX,Y(x,y)(x,y)B

B = {X2 + Y2 2}

Subset B of (X,Y) plane

Point (X,Y) SX,Y


35

PX,Y(x,y) y=0 y=1 y=2

x=0 0.4 0 0

x=1 0.1 0.1 0

x=2 0 0 0.40 1 2

2

1

0

y

x

0.4 0.1

0.1

0.4

P[B] = PX,Y(0,0) + PX,Y(1,1) + PX,Y(2,2)

= 0.4 + 0.1 + 0.4 = 0.9

X = Y

C = event that X > Y

P[C] = PX,Y(1,0) = 0.1

B = event that X equals Y


• In an experiment with 2 RVs, X and Y

• Possible to consider only one (X) and ignore Y

• PX(x)


Theorem: For random variables X and Y with

joint PMF PX,Y(x,y):

PY(y) = PX,Y(x,y)x

PX(x) = PX,Y(x,y)y

37

• Find Marginal PMF of X and Y

• SX = 0,1,2 SY = 0,1,2

PX,Y(x,y) y=0 y=1 y=2

x=0 0.4 0 0

x=1 0.1 0.1 0

x=2 0 0 0.4

PX(0) = PX,Y(0,y) = 0.4y=0

2

PX(1) = PX,Y(1,y) = 0.1+0.1y=0

2

PX(2) = PX,Y(2,y) = 0.4y=0

2

PX(x) = 0 for x 0,1,2


0 1 2

1

2

x

y0.4

0.4

0.1

0.1

38

PX,Y(x,y) y=0 y=1 y=2

x=0 0.4 0 0

x=1 0.1 0.1 0

x=2 0 0 0.4

PX(x)

0.4

0.2

0.4

PY(y) 0.5 0.1 0.4 1

Marginal X

Marginal Y


0 1 2

1

2

x

y0.4

0.4

0.1

0.1Y

39

• The number of bytes, N, in a message is

geometric distribution with parameter (1-p)

• A maximum packet size = M bytes

• Let Q = the number of packets• Let R = the number of left over bytes

…

N bytes

M bytes R bytes

Q Packets


http://www.google.co.th/url?sa=i&rct=j&q=computer+cartoon&source=images&cd=&cad=rja&docid=_Mr6udYEPbfswM&tbnid=ImLdup8Vip3rUM:&ved=0CAUQjRw&url=http://buildingyourownpc.com/should-your-next-computer-be-a-diy-project/&ei=Ja_cUca0BdGeiAe6qoHwDg&psig=AFQjCNHm1sXASIgXiCnw7SCuf6ctou6bow&ust=1373503636246430

http://www.google.co.th/url?sa=i&rct=j&q=wireless+signal+cartoon&source=images&cd=&cad=rja&docid=q74N6-hyN2R4QM&tbnid=5i6wEw29daUvJM:&ved=0CAUQjRw&url=http://solutionsourcesprogramming.com/category/announcement/&ei=a6_cUcenLKiOiAer84DQCw&psig=AFQjCNHa8uyNo9UAeqbXWytESgUGzRRmhg&ust=1373503703808460

40

• Find Joint Probability Mass Function PQ,R(q,r)

Solution:

• N, Q, R, and M

• N = ?

N = QM + R

• SN, SQ,SR = ?

SN = {0,1,2,3,…}

SQ = {0,1,2,3,…}

SR = {0,1,2,3,…,M-1}

…

N bytes

M bytes R bytes

Q Packets


41

• PQ,R(q,r) = PN(n)

= P[N = n]

= P[N = qM + r]

• N = Geometric RV with parameter (1-p)


42

• Two versions of Geometric RV

SX = {1,2,3,…}

SY = {0,1,2,…}

• E[X] = E[Y] ???

No

E[X] = 1/p

E[Y] = (1-p)/p


43

Definition: X is a Geometric Random Variable if

the PMF of X, PX(x), has the form:

where p = (0,1)

PX(x) =p(1 – p)x-1 x = 1,2,3,…

0 Otherwise

PY(y) =p(1 – p)y y = 0,1,2,…

0 Otherwise


44

• PQ,R(q,r) = PN(n)

= P[N = n]

= P[N = qM + r]

• PN(n) = (1-p) (1-(1-p))n

= (1-p) (1-(1-p))qM + r

= (1-p)pqM + r

• PQ,R(q,r) = (1-p)pqM + r


45

• Find PQ(q) = ?

r = 0• PQ(q) = (1 – p)pqM + r

M-1

= (1 – p)pqM pr

r = 0

M-1

= (1 – p)pqM 1 – pM

1 – p

= (1 – pM) (pM)q q = 0,1,2,3,…


46

• Find PR(r) = ?

q = 0• PR(r) = (1 – p)pqM + r

∞

= (1 – p)pr pqM

q = 0

∞

= (1 – p)pr 1

1 – pM

= pr r = 0,1,2,…,M-1(1 – p)

(1 – pM)Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty

• From PX,Y(x,y), we can find

• PX(x)

• PY(y)

• From PX(x) or PY(y), can we find PX,Y(x,y)?

• NO


Documents

Department of Computer Engineering, Faculty of Engineering ...anan/myhomepage/wp-content/uploads/2018/08/... · •Expected Value of a Derived RV •Variance & Standard Deviation