Upload
talasi
View
43
Download
0
Tags:
Embed Size (px)
DESCRIPTION
RANDOM VARIABLES, EXPECTATIONS, VARIANCES ETC. - THEORY. Variable. Recall: Variable: A characteristic of population or sample that is of interest for us. Random variable: A function defined on the sample space S that associates a real number with each outcome in S. - PowerPoint PPT Presentation
Citation preview
RANDOM VARIABLES, EXPECTATIONS,
VARIANCES ETC. - THEORY
1
Variable
• Recall: • Variable: A characteristic of population or
sample that is of interest for us.• Random variable: A function defined on the
sample space S that associates a real number with each outcome in S.
2
DISCRETE RANDOM VARIABLES
• If the set of all possible values of a r.v. X is a countable set, then X is called discrete r.v.
• The function f(x)=P(X=x) for x=x1,x2, … that assigns the probability to each value x is called probability density function (p.d.f.) or probability mass function (p.m.f.)
3
Example
• Discrete Uniform distribution:
• Example: throw a fair die. P(X=1)=…=P(X=6)=1/6
4
,...2,1N;N,...,2,1x;N
1)xX(P
CONTINUOUS RANDOM VARIABLES
• When sample space is uncountable (continuous)
• Example: Continuous Uniform(a,b)
5
.bxaab
1)X(f
CUMULATIVE DENSITY FUNCTION (C.D.F.)
• CDF of a r.v. X is defined as F(x)=P(X≤x).• Note that, P(a<X ≤b)=F(b)-F(a).• A function F(x) is a CDF for some r.v. X iff it
satisfies
6
)b(F)a(Fimpliesba
)x(F)hx(Flim
1)x(Flim
0)x(Flim
0h
x
x
F(x) is continuous from right
F(x) is non-decreasing.
Example• Consider tossing three fair coins.• Let X=number of heads observed.• S={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH}• P(X=0)=P(X=3)=1/8; P(X=1)=P(X=2)=3/8
7
x F(x)
(-∞,0) 0
[0,1) 1/8
[1,2) 1/2
[2,3) 7/8
[3, ∞) 1
Example
• Let
8
0xfor)x1(2)x(f 3
0xfor0
0xfor)x1(1dt)t1(2)xX(P)x(F
x
023
035.0)4.0(F)45.0(Fdx)x(f)45.0X4.0(P45.0
4.0
JOINT DISTRIBUTIONS• In many applications there are more than one
random variables of interest, say X1, X2,…,Xk.
JOINT DISCRETE DISTRIBUTIONS• The joint probability mass function (joint pmf)
of the k-dimensional discrete rv X=(X1, X2,…,Xk) is
kk2211k21 xX,...,xX,xXPx,...,x,xf .Xx,...,x,x k21 of
9
JOINT DISCRETE DISTRIBUTIONS
• A function f(x1, x2,…, xk) is the joint pmf for some vector valued rv X=(X1, X2,…,Xk) iff the following properties are satisfied:
f(x1, x2,…, xk) 0 for all (x1, x2,…, xk)
and
10
.1x,...,x,xf...
1x kxk21
Example
• Tossing two fair dice 36 possible sample points
• Let X: sum of the two dice; Y: |difference of the two dice|• For e.g.:
– For (3,3), X=6 and Y=0. – For both (4,1) and (1,4), X=5, Y=3.
11
Example• Joint pmf of (x,y)
12
x
y
2 3 4 5 6 7 8 9 10 11 12
0 1/36 1/36 1/36 1/36 1/36 1/36
1 1/18 1/18 1/18 1/18 1/18
2 1/18 1/18 1/18 1/18
3 1/18 1/18 1/18
4 1/18 1/18
5 1/18
Empty cells are equal to 0.
e.g. P(X=7,Y≤4)=f(7,0)+f(7,1)+f(7,2)+f(7,3)+f(7,4)=0+1/18+0+1/18+0=1/9
MARGINAL DISCRETE DISTRIBUTIONS
• If the pair (X1,X2) of discrete random variables has the joint pmf f(x1,x2), then the marginal pmfs of X1 and X2 are
13
12
21222111xx
xxfxf and xxfxf ,,
Example
• In the previous example,
–
–
14
36/1)5y,2X(P...)0y,2X(P)y,2X(P5
0y
2)P(X
12
2x
18/4)2Y,x(P)2Y(P
JOINT DISCRETE DISTRIBUTIONS• JOINT CDF:
• F(x1,x2) is a cdf iff
15
.xX,...,xXPx,...,x,xF kk11k21
.andx,xFhx,xFx,hxF
,0c,aFd,aFc,bFd,bF)dXc,bXa(P
1,Fx,xFlim
.x,0,xFx,xFlim
.x,0x,Fx,xFlim
212121
21
21
2x1x
11212x
22211x
2 10h0h
x x ,limlim
d.c and ba
JOINT CONTINUOUS DISTRIBUTIONS
• A k-dimensional vector valued rv X=(X1, X2,…,Xk) is said to be continuous if there is a function f(x1, x2,…, xk), called the joint probability density function (joint pdf), of X, such that the joint cdf can be given as
16
1x 2x kx
k21k21k21 dt...dtdtt,...,t,tf...x,...,x,xF
JOINT CONTINUOUS DISTRIBUTIONS
• A function f(x1, x2,…, xk) is the joint pdf for some vector valued rv X=(X1, X2,…,Xk) iff the following properties are satisfied:
f(x1, x2,…, xk) 0 for all (x1, x2,…, xk)
and
17
.1dx...dxdxx,...,x,xf... k21k21
JOINT CONTINUOUS DISTRIBUTIONS
• If the pair (X1,X2) of discrete random variables has the joint pdf f(x1,x2), then the marginal pdfs of X1 and X2 are
18
.,, 1212222111 dxxxfxf and dxxxfxf
JOINT DISTRIBUTIONS
• If X1, X2,…,Xk are independent from each other, then the joint pdf can be given as
And the joint cdf can be written as
19
k21k21 xf...xfxfx,...,x,xf
k21k21 xF...xFxFx,...,x,xF
CONDITIONAL DISTRIBUTIONS
• If X1 and X2 are discrete or continuous random variables with joint pdf f(x1,x2), then the conditional pdf of X2 given X1=x1 is defined by
• For independent rvs,
20
elsewhere. 0 f that such ,0xx,xf
x,xfxxf 11
1
2112
.
.
121
212
xfxxf
xfxxf
ExampleStatistical Analysis of Employment Discrimination Data (Example
from Dudewicz & Mishra, 1988; data from Dawson, Hankey and Myers, 1982)
21
% promoted (number of employees)
Pay grade Affected class others
5 100 (6) 84 (80)
7 88 (8) 87 (195)
9 93 (29) 88 (335)
10 7 (102) 8 (695)
11 7 (15) 11 (185)
12 10 (10) 7 (165)
13 0 (2) 9 (81)
14 0 (1) 7 (41)
Affected class might be a minority group or e.g. women
Example, cont.
• Does this data indicate discrimination against the affected class in promotions in this company?
• Let X=(X1,X2,X3) where X1 is pay grade of an employee; X2 is an indicator of whether the employee is in the affected class or not; X3 is an indicator of whether the employee was promoted or not
• x1={5,7,9,10,11,12,13,14}; x2={0,1}; x3={0,1}
22
Example, cont.
• E.g., in pay grade 10 of this occupation (X1=10) there were 102 members of the affected class and 695 members of the other classes. Seven percent of the affected class in pay grade 10 had been promoted, that is (102)(0.07)=7 individuals out of 102 had been promoted.
• Out of 1950 employees, only 173 are in the affected class; this is not atypical in such studies.
23
Pay grade Affected class others
10 7 (102) 8 (695)
Example, cont.
• E.g. probability of a randomly selected employee being in pay grade 10, being in the affected class, and promoted: P(X1=10,X2=1,X3=1)=7/1950=0.0036 (Probability function of a discrete 3 dimensional r.v.)
• E.g. probability of a randomly selected employee being in pay grade 10 and promoted:
P(X1=10, X3=1)= (7+56)/1950=0.0323 (Note: 8% of 695 -> 56) (marginal probability function of X1 and X3)
24
Pay grade Affected class others
10 7 (102) 8 (695)
Example, cont.
• E.g. probability that an employee is in the other class (X2=0) given that the employee is in pay grade 10 (X1=10) and was promoted (X3=1):
P(X2=0| X1=10, X3=1)= P(X1=10,X2=0,X3=1)/P(X1=10, X3=1)
=(56/1950)/(63/1950)=0.89 (conditional probability)• probability that an employee is in the affected class
(X2=1) given that the employee is in pay grade 10 (X1=10) and was promoted (X3=1):
P(X2=1| X1=10, X3=1)=(7/1950)/(63/1950)=0.11
25
Production problem• Two companies manufacture a certain type of sophisticated
electronic equipment for the government; to avoid the lawsuits lets call them C and company D. In the past, company C has had 5% good output, whereas D had 50% good output (i.e., 95% of C’s output and 50% of D’s output is not of acceptable quality). The government has just ordered 10,100 of these devices from company D and 11,000 from C (maybe political reasons, maybe company D does not have a large enough capacity for more orders). Before the production of these devices start, government scientists develop a new manufacturing method that they believe will almost double the % of good devices received. Companies C and D are given this info, but its use is optional: they must each use this new method for at least 100 of their devices, but its use beyond that point is left to their discretion.
26
Production problem, cont.• When the devices are received and tested, the
following table is observed:
• Officials blame scientists and companies for producing with the lousy new method which is clearly inferior.
• Scientists still claim that the new method has almost doubled the % of good items.
• Which one is right?
Production method
Standard New
Results Bad 5950 9005
Good 5050 (46%) 1095 (11%)
27
Production problem, cont.• Answer: the scientists rule!
• The new method nearly doubled the % of good items for both companies.
• Company D knew their production under standard method is already good, so they used the new item for only minimum allowed.
• This is called Simpson’s paradox. Do not combine the results for 2 companies in such cases.
Company
C D
Standard New Standard New
Results Bad 950 9000 5000 5
Good 50 (5%) 1000 (10%) 5000 (50%) 95 (95%)
28
29
Describing the Population
• We’re interested in describing the population by computing various parameters.
• For instance, we calculate the population mean and population variance.
30
EXPECTED VALUESLet X be a rv with pdf fX(x) and g(X) be a
function of X. Then, the expected value (or the mean or the mathematical expectation) of g(X)
Xx
X
g x f x , if X is discrete
E g Xg x f x dx, if X is continuous
providing the sum or the integral exists, i.e.,<E[g(X)]<.
31
EXPECTED VALUES
• E[g(X)] is finite if E[| g(X) |] is finite.
Xx
X
g x f x < , if X is discrete
E g Xg x f x dx< , if X is continuous
32
Population Mean (Expected Value)
• Given a discrete random variable X with values xi, that occur with probabilities p(xi), the population mean of X is
ixall
ii )x(px)X(E ixall
ii )x(px)X(E
33
– Let X be a discrete random variable with possible values xi that occur with probabilities p(xi), and let E(xi) = The variance of X is defined by
ixall
i2
i22 )x(p)x()X(E)X(V
ixalli
2i
22 )x(p)x()X(E)X(V
Population Variance
2
isdeviationdardtansThe
2
isdeviationdardtansThe
Unit*Unit
Unit
34
EXPECTED VALUE• The expected value or mean value of a
continuous random variable X with pdf f(x) is
( ) ( )all x
E X xf x dx • The variance of a continuous random
variable X with pdf f(x) is2 2 2
all x
2 2 2 2
all x
( ) ( ) ( ) ( )
( ) ( ) ( )
Var X E X x f x dx
E X x f x dx
35
EXAMPLE• The pmf for the number of defective items in
a lot is as follows0.35, 0
0.39, 1
( ) 0.19, 2
0.06, 3
0.01, 4
x
x
p x x
x
x
Find the expected number and the variance of
defective items.
36
EXAMPLE
• Let X be a random variable. Its pdf isf(x)=2(1-x), 0< x < 1
Find E(X) and Var(X).
37
Laws of Expected Value• Let X be a rv and a, b, and c be constants.
Then, for any two functions g1(x) and g2(x) whose expectations exist,
1 2 1 2)a E ag X bg X c aE g X bE g X c
1 10 , 0.b) If g x for all x then E g X
1 2 1 2) .c If g x g x for all x, then E g x E g x
1 1)d If a g x b for all x, then a E g X b
38
Laws of Expected Value E(c) = c E(X + c) = E(X) + c E(cX) = cE(X)
Laws of Variance V(c) = 0 V(X + c) = V(X) V(cX) = c2V(X)
Laws of Expected Value and Variance
Let X be a rv and c be a constant.
EXPECTED VALUE
39
.
k
iii
k
iii XEaXaE
11
If X and Y are independent,
YhEXgEYhXgE
The covariance of X and Y is defined as
)Y(E)X(E)XY(E
YEYXEXEY,XCov
EXPECTED VALUE
40
If X and Y are independent,
0YXCov ,
The reverse is usually not correct! It is only correct under normal distribution.
If (X,Y)~Normal, then X and Y are independent iff
Cov(X,Y)=0
EXPECTED VALUE
41
212121 2 XXCovXVarXVarXXVar ,
If X1 and X2 are independent,
2121 XVarXVarXXVar
CONDITIONAL EXPECTATION AND VARIANCE
42
.continuous are Y and X if , dyxyyf
discrete. are Y and X if , xyyf
xYEy
22 xYExYExYVar
CONDITIONAL EXPECTATION AND VARIANCE
43
YEXYEE
))X|Y(E(Var))X|Y(Var(E)Y(Var XX
(EVVE rule)
Proofs available in Casella & Berger (1990), pgs. 154 & 158
Example - Advanced
• An insect lays a large number of eggs, each surviving with probability p. Consider a large number of mothers. X: number of survivors in a litter; Y: number of eggs laid
• Assume:
• Find: expected number of survivors, i.e. E(X)
44
)(lExponentia~
)(Poisson~|Y
)p,Y(Binomial~Y|X
Example - solution
EX=E(E(X|Y))=E(Yp) =p E(Y)=p E(E(Y|Λ))=p E(Λ)=pβ
45
46
SOME MATHEMATICAL EXPECTATIONS
• Population Mean: = E(X)• Population Variance:
2 22 2 0Var X E X E X
(measure of the deviation from the population mean)
• Population Standard Deviation: 2 0
• Moments:* kk E X the k-th moment
k
k E X the k-th central moment
47
SKEWNESS• Measure of lack of symmetry in the pdf.
3
33 3/2
2
E XSkewness
If the distribution of X is symmetric around its mean ,
3=0 Skewness=0
48
KURTOSIS
• Measure of the peakedness of the pdf. Describes the shape of the distribution.
4
44 2
2
E XKurtosis
Kurtosis=3 NormalKurtosis >3 Leptokurtic (peaked and fat tails)Kurtosis<3 Platykurtic (less peaked and thinner tails)
KURTOSIS
• What is the range of kurtosis?• Claim: Kurtosis ≥ 1. Why?• Proof:
49
11))((
))((
]))(([))(())((
.)(
)()()(
4
21
421
221
21
41
21
22
XVarKurtosis
XVar
XEXVarXE
XYLet
EYYEYVar
Problems
1. True or false: The mean, median and mode of a normal distribution with mean µ and std deviation σ coincide.
50
Problems
2. True or false: In a symmetrical population, mean, median, and mode coincide. (Kendall & Stuart, 1969, p. 85)
51
Problems
3. True or False: “The mean, median and mode occur in the same order (or reverse order) as in the dictionary; and that the median is nearer to the mean than that to the mode, just as the corresponding words are nearer together in the dictionary. “ (Kendall & Stuart, 1969, p. 39)
52
Problems
4. If X, Y, Z and W are random variables, then find (show the derivations):
a) Cov(X+Y,Z+W)b) Cov(X-Y,Z)
53
Problems
5. Calculate a) the skewness for . Comment.b) the kurtosis for the following pdf and
comment:
54
0,)( xexf x
||
2
1)( xexf
Problems
5. c) Consider the discrete random variable X with pdf given below:
i) Is the distribution of X symmetric around mean?ii) Show that the 3rd central moment, and hence
skewness, are 0. What does this imply?
55
x -3 -1 0 2f(x) 1/4 1/4 1/8 16/)236( 16/23
22
Problem
6. Let X1, X2, X3 be three independent r.v.s each with variance . Define new r.v.s W1, W2, W3 by W1=X1; W2=X1+X2; W3=X2+X3.
Find Cor(W1,W2), Cor(W2,W3), Cor(W1,W3)
56
2