Hypothesis Testing Review

8/13/2019 Hypothesis Testing Review

1/5

BST 140.652 Hypothesis Testing Review notes

1 Hypothesis testing for a single mean

1. The null, or status quo, hypothesis is labeledH0, the alternativeHa or H1orH2 ...

2. Atype I error occurs when we falsely reject the null hypothesis. The probability of a

type I error is usually labeled .

3. Atype II erroroccurs when we falsely fail to reject the null hypothesis. A type II erroris usually labeled.

4. APoweris the probability that we correctly reject the null hypothesis, 1 .5. TheZtest forH0: = 0versusH1 : < 0 orH2 : =0 orH3 : > 0 constructs a

test statisticT S=X0S/n

and rejects the null hypothesis when

H1 T S Z1H2|T S| Z1/2H3 T S Z1respectively.

6. The Z test requires the assumptions of the CLT and for n to be large enough for it toapply.

7. Ifn is small, then a Students T test is performed exactly in the same way, with thenormal quantiles replaced by the appropriate Students Tquantiles andn 1df.

8. Tests define confidence intervals by considering the collection of values of0for whichyou fail to reject a two sided test. This yields exactly theT andZconfidence intervals

respectively.

9. Conversely, confidence intervals define tests by the rule where one rejectsH0if0is notinthe confidence interval.

10. AP-value is the probability of getting evidence as extreme or more extreme than weactually got under the null hypothesis. ForH3above, the P-value is calculated asP(ZT Sobs|= 0)whereT Sobs is the observed value of our test statistic. To get the P-valueforH2, calculate a one sided P-value and double it.

11. The P-value is equal to theattained significance level. That is, the smallest value

for which we would have rejected the null hypothesis. Therefore, rejecting the nullhypothesis if a P-value is less than is the same as performing the rejection region test.

1


2/5

12. The power of aZtest forH3is given by the formula (know how this is obtained)

P(TS > Z1|= 1) =P

Z 0 1/

n + Z1

.

Notice that power required a value for 1, the value under the null hypothesis. Corre-spondingly forH1 we have

PZ0 1

/n Z1 .

ForH2, the power is approximately the appropriate one sided power using /2.

13. Some facts about power.

a. Power goes up as goes up.

b. Power of a one sided test is greater than the power of the associated two sided test.

c. Power goes up as1 gets further away from0.

d. Power goes up asn goes up.

14. The prior formula can be used to calculate the sample size. For example, using thepower formula forH1, settingZ1= 01/n Z1 yields

n=(Z1+ Z1)

22

(0 1)2 ,

which gives the sample size to have power = 1 . This formula applies forH3 also.For the two sided test, H2, replace by/2.

15. Determinants of sample size.

a. ngets larger as gets smaller.

b. ngets larger as the power you want gets larger.

c. ngets lager the closer1 is to0.

2 Binomial confidence intervals and tests

1. Binomial distributions are used to model proportions. IfX Binomial(n, p) then p =X/nis a sample proportion.

2. phas the following properties.

a. It is a sample mean of Bernoulli random variables.

b. It has expected valuep.

c. It has variancep(1p)/n. Note that the largest value thatp(1p)can take is1/4atp= 1/2.

2


3/5

d. Z = ppp(1p)/n

follows a standard normal distribution for large n by the CLT. The

convergence to normality is fastest whenp = .5.

3. The Wald test forH0 : p = p0 versus one ofH1 : p < p0, H2 : p = p0, andH3 : p > p0uses the test statistic

T S= p pp(1 p)/n

which is compared to standard normal quantiles.

4. TheWald confidence intervalfor a binomial proportion is

p Z1/2

p(1 p)/n.

The Wald interval is the interval obtained by inverting the Wald test (and vice versa).

5. TheScore testfor a binomial proportion is

T S= p p

p0(1p0)/n

.

The score test has better finite sample performance than the Wald test.

6. TheScore intervalis obtained by inverting the score test (and vice versa)

p

n

n+Z21/2

+ 12

Z2

1/2

n+Z21/2

Z1/2

1n+Z2

1/2

p(1 p)

n

n+Z21/2

+ 1

4

Z2

1/2

n+Z21/2

.

7. An approximate score interval for = .05 can be obtained by taking p = X+2n+4

and

calculating the Wald interval using pinstead ofp.8. An exact binomial test forH3 can be performed by calculating the exact P-value

P(X xobs|p= p0) =n

k=xobs

nk

pk0(1p0)nk.

wherexobsis the observed success count. For H1 the corresponding exact P-value is

P(X xobs|p= p0) =xobs

k=0

nk

pk0(1 p0)nk.

These confidence intervals areexact, which means that the actual type one error rate is

no larger than. (The actual type one error rate is generally smaller than.) Thereforethese tests are conservative. ForH2, calculate the appropriate one sided P-value anddouble it.

3


4/5

9. Occasionally, someone will try to convince you to obtain an exact Type I error rate using

supplemental randomization. Ignore them.

10. Inverting the exact test, choosing those value ofp0for which we fail to reject H0, yieldsan exact confidence interval. This interval has to be calculated numerically. The cover-

age of the exact binomial interval is no lower than 100(1 )%.

3 Group comparisons

1. For group comparisons, make sure to differentiate whether or not the observations are

paired (or matched) versus independent.

2. For paired comparisons for continuous data, one strategy is to calculate thedifferencesand use the methods for testing and performing hypotheses regarding a single mean.

The resulting tests and confidence intervals are called paired Students T tests andintervals respectively.

3. For independent groups of iid variables, sayXiand Yi,with a constant variance 2 across

groups

Z=

XY (x y)

Sp

1nx

+ 1ny

limits to a standard normal random variable as both nx and ny get large. Here

S2p =(nx 1)S2x+ (ny 1)S2y

nx+ ny 2is the pooled estimate of the variance. Obviously, X, Sx, nx are the sample mean,sample standard deviation and sample size for the Xi and Y, Sy and ny are definedanalogously.

4. If theXi andYi happen to be normal, thenZfollows the Students T distribution withnx+ ny 2degrees of freedom.

5. The test statisticT S = XYSpq

1

nx+ 1

ny

can be used to test the hypothesis thatH0 : x = y

versus the alternativesH1 : x < y, H2 : x= y andH3 : x > y. The test statisticshould be compared to StudentsT quantiles withnx+ ny 2df.

6. S2x/

2x

S2y/2y

follows what is called the Fdistribution withnx 1numerator degrees of free-domand ny 1denominator degrees of freedom.

7. To test the hypothesisH0: 2x=

2y versus th hypothesesH1 :

2x <

2y,H2 :

2x=

2y and

H3 : 2x>

2y compare the statisticT S=S

21/S

22 to theFdistribution. We reject H0if:

H1 ifTS < Fnx1,ny1,,

H2 ifTS < Fnx1,ny1,/2 or TS > Fnx1,ny1,1/2,

4


5/5

H3 ifTS > Fnx1,ny1,1.

8. The F distribution satisfies the property thatFnx1,ny1, = Fny1,nx1,1. So that, itturns out, that our results are consistent whether we put S2x on the top or bottom.

9. Using the fact that

1 = P

Fnx1,ny1,/2S2x/

2x

S2y/2y

Fnx1,ny1,1/2

we can calculate a confidence interval for 2

y2x as

Fnx1,ny1, S

2

xS2y , Fnx1,ny1,1/2

S2

xS2y

.Of

course, the confidence interval for 2x2y

is

Fny1,nx1,S2yS2x

, Fny1,nx1,1/2S2yS2x

.

10. F tests are not robust to the normality assumption.

11. The statisticX Y (x y)

S2xnx

+ S2yny

follows a standard normal distribution for large nx andny. It follows an approximateStudents Tdistribution if theXi andYi are normally distributed. The degrees of free-dom are given below.

12. For testingH0 : x = y in the event where there is evidence to suggest that x= y,the test statistic T S =

XYrS2xnx

+S2yny

follows an approximate StudentsT distribution under

the null hypothesis when Xi andYi are normally distributed. The degrees of freedomare approximated with

(S2x/nx+ S2y/ny)

2

(S2x/nx)2/(nx 1) + (S2y/ny)2/(ny 1)

.

13. The power for aZtest ofH0: x = y versusH3 : x> y is given by

P

Z Z1 x y

2xnx

+ 2yny

while forH1: x< y it is

P

Z Z1 x y

2xnx

+ 2yny

.

14. Sample size calculation assumingnx= ny =n

n=(Z1+ Z1)

2(2x+ 2y)

(x y)2 .

5

Documents

Hypothesis Testing Review