1

Psych 5500/6500

Null Hypothesis Testing: General Concepts

and Introduction to

The t Test for a Single Group Mean

Fall, 2008

2

Introduction

Time to move on to the heart of experimental science, formulating hypotheses to test theories, gathering data, and then deciding which hypothesis is supported by the data.The general approach is called 'null hypothesis testing'.

3

t Test for a Single Group Mean

We will introduce null hypothesis testing by seeing how it is carried out in what is called the t test for a single group mean.

This test is rarely used in statistics, but it makes a very nice lead into null hypothesis testing from our earlier work on the sampling distribution of the mean.

As the semester progresses we will return to null hypothesis testing again and again, each time will simply be a variation on the same theme.

4

t Test for a Single Group Mean

The t test for a single group mean involves making hypotheses about the true value of the mean of the population from which we are sampling. Its limitation is that you need to have some theoretical reason for hypothesizing specific values about the population mean...

5

ExampleWe have an IQ test that has a μ=100 for the

general population of the United States. We want to test a theory which predicts that the population of people in the country of Elbonia should have a different μ than that of the U.S.

The theory simply predicts that the mean IQ of Elbonians is different, it doesn’t predict specifically whether it should be greater or less than that of the U.S. (we will cover that type of hypothesis testing later).

6

General Approach

We will draw a sample of Elbonians and find the mean IQ of that sample. From that we will decide whether or not the mean of the population of Elbonians is different than the mean of the population of the U.S. (i.e. whether μElbonians100)

7

The Challenge

The challenge we face is this, let’s say that the mean of the sample of Elbonians is 106, does that prove the Elbonians have higher IQ’s (i.e. that μElbonians μAmericans )? Or, might it simply be that Elbonians are the same as people in the USA (i.e. μElbonians = μAmericans = 100) but that we just happened to get a high sample mean due to chance (i.e. your sample has random bias). This is exactly the sort of thing that null hypothesis testing was set up to handle.

8

Two Hypotheses

The first step in null hypothesis testing is to divide reality into two mutually exclusive and exhaustive hypotheses (i.e. one, and only one, of these will be true):

1. null hypothesis (H0): the hypothesis that you set up in hopes of rejecting. Often, but not always, it is the hypothesis of 'no difference'.

2. alternative hypothesis (HA): the hypothesis you hope to prove. Often, but not always, it is the hypothesis of 'difference'.

9

The exact form of the null and alternative hypotheses depend upon the statistical procedure you are using. In our example (t test for a single group mean):

H0: μElbonia = 100

HA: μElbonia 100

Note: 1) Hypotheses are always about populations,

2) Together H0 and HA cover all possibilities.

10

Two Possible ResultsThe results are always stated in terms of the null

hypothesis (this is ‘null hypothesis testing’ after all). There are two and only two possible results of an experiment (in this class always give one of the following two as the ‘result of the experiment’):

1. ‘Do not reject H0’

2. ‘Reject H0’

Later we will take a look at why it is more accurate to say ‘do not reject H0’ than ‘accept H0’’.

11

Implications Regarding HA

Although our decision is stated in terms of H0, once we decide about H0 that implies something about HA (as the two hypotheses are mutually exclusive and exhaustive)

‘reject H0’ implies ‘accept HA’.

‘do not reject H0’ implies ‘do not accept HA’.

12

Four Possibilities 

  Reality

1) Ho true 2) Ho false

 

Decision

1) Reject H0Type 1 error

p = ‘alpha’

Correct decision

p = ‘power’

2) Do not reject H0

Correct decision

Type 2 error

p = ‘beta’

13

Type 1 Error and alphaType 1 error: rejecting H0 when in reality H0 was

true.

alpha (α): the probability that you will make a type 1 error. This is a conditional probability:

Correct interpretation of alpha:α=p(you will decide to reject H0 | H0 is actually true).

Incorrect interpretations of alpha: 1) p(H0 is true | you decide to reject H0)

2) p(making a type one error | you decide to reject H0)

3) p(making a type one error | you are doing null hypothesis testing)

14

Type 2 Error and beta

Type 2 error: not rejecting H0 when in reality H0 was false.

beta (β): the probability you will make a type 2 error. This is also a conditional probability:

β = p(deciding to not reject H0 | H0 is actually false)

15

powerpower = p(deciding to reject H0 | H0 is actually false)

Power involves making a correct decision, to reject H0 when in reality H0 was false.

We rarely know the actual values of power and beta, but we do know the following:

power + beta = 1.00 (so we know that when power goes up beta goes down and vice versa). We will devote a whole lecture just on power later in the semester.

16

Basic Approach in Null Hypothesis Testing

1. Determine the probability of getting the data you did if H0 were true. In this case, what is the probability that we would have gotten a sample mean (106) so far away from what H0 predicted (100) if H0 were actually true? i.e. p(sample mean of 6 or more away from 100 | H0 is actually true)

2. If that probability is very small, then reject H0.

17

Coin Example

H0: the coin is fair, i.e. p(Heads)=0.5

HA: the coins is biased, i.e. p(Heads)0.5

1st coin flip = Heads, p(one head|H0)=0.5

2nd coin flip = Heads, p(two heads|H0)=.25

3rd coin flip = Heads, p(three heads|H0)=.125

4th coin flip = Heads, p(four heads|H0)=.0625

5th coin flip = Heads, p(five heads|H0)=.03125

6th coin flip = Heads, p(six heads|H0)=.01563

7th coin flip = Heads, p(seven heads|H0)=.0078

18

Significance LevelAt what point do we say that the data are so

unlikely given H0 that we want to reject H0 in favor of HA?

Where we draw the line is called our ‘significance level’, it is how improbable the results have to be to reject H0. In psychology we usually use p=.05, or if we wish to be conservative and not reject H0 without more proof, we use p=.01

Note: H0 could still be true, we can’t rule that out, we can only say the results are improbable if H0 were true, so we will go with HA instead.

19

Significance Level and alpha

Once you set your significance level, you have also set the value of alpha (‘α’).

alpha = your significance level.

Thus the value of alpha--the probability of making a type one error given H0 is true--is under your direct control based upon the significance level you choose (an a priori decision).

20

In Our Example1) Our hypotheses were:

H0: μElbonia = 100

HA: μElbonia 1002) We set our significance level at .05. 3) The mean of our sample of Elbonians was 106.4) If the probability of getting a sample mean at least six away from 100 if

H0 were true is, say, p=.04, then we would decide to ‘reject H0’, as the probability of getting such a sample mean if H0 were true is less than .05 (our significance level). Our sample mean was improbable if H0 were true.

5) If, however, the probability of getting a sample mean at least six away from 100 if H0 were true is, say, p=.17, then we would decide to ‘not reject H0’, as the sample mean is not all that unlikely if H0 were true (i.e. p > significance level of .05).

You can, however, influence the value of beta and power through how you design your experiment...much more on this later.

21

beta and power

Usually you can not directly set the value of beta (‘β’) or power (but at least you do know that beta+power=1.00)