Further Inference in the Multiple Regression Model Prepared by Vera Tabakova, East Carolina University

Chapter 6

Further Inference in the Multiple Regression Model

Prepared by Vera Tabakova, East Carolina University

Chapter 6: Further Inference in the Multiple Regression Model

6.1 The F-Test

6.2 Testing the Significance of the Model

6.3 An Extended Model

6.4 Testing Some Economic Hypotheses

6.5 The Use of Nonsample Information

6.6 Model Specification

6.7 Poor Data, Collinearity and Insignificance

6.8 Prediction Slide 6-2Principles of Econometrics, 3rd Edition

6.1 The F-Test

Slide 6-3Principles of Econometrics, 3rd Edition

(6.1)

(6.2)

1 2 3i i i iS P A e

0 2: 0H

1 2: 0H

1 3i i iS A e

6.1 The F-Test

If the null hypothesis is not true, then the difference between SSER and SSEU

becomes large, implying that the constraints placed on the model by the null

hypothesis have a large effect on the ability of the model to fit the data.


(6.3)

R U

U

SSE SSE JF

SSE N K

6.1 The F-Test

Hypothesis testing steps:

1. Specify the null and alternative hypotheses:

2. Specify the test statistic and its distribution if the null hypothesis is true

3. Set and determine the rejection region

Using α=.05, the critical value from the -distribution is .

Thus, H0 is rejected if .


0 2: 0H 1 2: 0H

(1, 72)

1

75 3R U

U

SSE SSEF F

SSE

(.95,1,72) 3.97cF F

3.97F

6.1 The F-Test

4. Calculate the sample value of the test statistic and, if desired, the p-

value

5. State your conclusion

Since , we reject the null hypothesis and conclude that price does

have a significant effect on sales revenue. Alternatively, we reject H0 because

.


2961.827 1718.943 152.06

1718.943 75 3R U

U

SSE SSE JF

SSE N K

(1,72)[ 52.06] .0000p P F

52.06 cF F

.0000 .05p

6.1.1 The Relationship Between t- and F-Tests

The elements of an F-test :

1. The null hypothesis consists of one or more equality restrictions J. The null

hypothesis may not include any ‘greater than or equal to’ or ‘less than or equal

to’ hypotheses.

2. The alternative hypothesis states that one or more of the equalities in the null

hypothesis is not true. The alternative hypothesis may not include any ‘greater

than’ or ‘less than’ options.

3. The test statistic is the F-statistic


R U

U

SSE SSE JF

SSE N K

6.1.1 The Relationship Between t- and F-Tests

4. If the null hypothesis is true, F has the F-distribution with J numerator

degrees of freedom and N-K denominator degrees of freedom. The null

hypothesis is rejected if .

5. When testing a single equality null hypothesis it is perfectly correct to use

either the t- or F-test procedure; they are equivalent.


(1 , , ), where c c J N KF F F F



(6.4)

(6.5)

(6.6)

1 2 2 3 3i i i iK K iy x x x e

0 2 3

1

: 0, 0, , 0

: of the is nonzero

for 2,3,

K

k

H

H At least one

k K

1i iy e



(6.7)

* 2 21

1 1

( ) ( )N N

R i ii i

SSE y b y y SST

( ) / ( 1)

/ ( )

SST SSE KF

SSE N K


Example: Big Andy’s sales revenue

1.

2. If the null is true

3. H0 is rejected if

4.

5. Since 29.95>3.12 we reject the null and conclude that price or advertising expenditure or both have an influence on sales.


1 2 3i i i iS P A e

0 2 3: 0, 0H

1 2 3: 0 or 0, or both are nonzeroH

(2,72)

( ) / (3 1)

/ (75 3)

SST SSEF F

SSE

3.12F

( ) / ( 1) (3115.485 1718.943) / 229.25

/ ( ) 1718.943 / (75 3)

SST SSE KF

SSE N K


Figure 6.1 A Model Where Sales Exhibits Diminishing Returns to Advertising Expenditure




(6.8)

(6.9)

(6.10)

1 2 3S P A e

21 2 3 4S P A A e

3 4( held constant)

( ) ( )2

P

E S E SA

A A



(6.11)

2ˆ 109.72 7.640 12.151 2.768

(se) (6.80) (1.046) (3.556) (.941)i i i iS P A A

1 2 2 3 3 4 4i i i i iy x x x e

22 3 4, , ,i i i i i i i iy S x P x A x A


6.4.1 The Significance of Advertising


(6.12)

(6.13)

21 2 3 4i i i i iS P A A e

1 2i i iS P e

0 3 4: 0, 0H

1 3 4: 0 or 0H


Since , we reject the null hypothesis and conclude that advertising does have a significant effect upon sales revenue.


(2,71)

2

75 4R U

U

SSE SSEF F

SSE

(.95,2,71) 3.126cF F

(2,71)[ 8.44] .0005P F

8.44 3.126cF F

6.4.2 The Optimal Level of Advertising

Economic theory tells us that we should undertake all those actions

for which the marginal benefit is greater than the marginal cost. This

optimizing principle applies to Big Andy’s Burger Barn as it attempts

to choose the optimal level of advertising expenditure.


3 4( held constant)

( )2

P

E SA

A

3 42 1oA

ˆ12.1512 2 ( 2.76796) 1oA


Big Andy has been spending $1,900 per month on advertising. He

wants to know whether this amount could be optimal.

The null and alternative hypotheses for this test are


0 3 4 1 3 4: 2 1.9 1 : 2 1.9 1H H

0 3 4 1 3 4: 3.8 1 : 3.8 1H H



3 4

3 4

( 3.8 ) 1

se( 3.8 )

b bt

b b

23 4 3 4 3 4

2

var( 3.8 ) var( ) 3.8 var( ) 2 3.8 cov( , )

12.6463 3.8 .884774 2 3.8 3.288746

.427967

b b b b b b


Because , we cannot reject the null

hypothesis that the optimal level of advertising is $1,900 per month.

There is insufficient evidence to suggest Andy should change his

advertising strategy.


1.6330 1 .633.9676

.65419.427967t

1.994 .9676 1.994



21 2 4 4(1 3.8 )i i i i iS P A A e

21 2 4( ) ( 3.8 )i i i i i iS A P A A e

(1552.286 1532.084) /1.9362

1532.084 / 71F

2 23.976 (1.994)c cF t

(1,71)

(71) (71)

-value [ .9362]

[ .9676] [ .9676] .3365

p P F

P t P t

6.4.2a A One-Tailed Test with More than One Parameter

Reject H0 if t ≥ 1.667.

t = .9676

Because .9676 < 1.667, we do not reject H0.

There is not enough evidence in the data to suggest the optimal level

of advertising expenditure is greater than $1900.


0 3 4

1 3 4

: 3.8 1

: 3.8 1

H

H

6.4.2 Using Computer Software


21 2 3 4

21 2 3 4

( )

6 1.9 1.9

80

E S P A A

0 3 4

1 2 3 4

: 3.8 1 and

6 1.9 3.61 80

H

5.74 and the -value = .0049F p



(6.14)

(6.15)

1 2 3 4 5ln( ) ln( ) ln( ) ln( ) ln( )Q PB PL PR I

1 2 3 4 5

1 2 3 4 5

2 3 4 5

ln( ) ln ln ln ln

ln( ) ln( ) ln( ) ln( )

ln( )

Q PB PL PR I

PB PL PR I



(6.16)

(6.17)

2 3 4 5 0

1 2 3 4 5ln( ) ln( ) ln( ) ln( ) ln( )t t t t t tQ PB PL PR I e





(6.18)

4 2 3 5

1 2 3

2 3 5 5

1 2

3 5

1 2 3 5

ln( ) ln( ) ln( )

ln( ) ln( )

ln( ) ln( )

ln( ) ln( ) ln( ) ln( )

ln ln ln

t t t

t t t

t t

t t t t t

t t tt

t t t

Q PB PL

PR I e

PB PR

PL PR I PR e

PB PL Ie

PR PR PR



(6.19)ln( ) 4.798 1.2994ln 0.1868ln 0.9458ln

(se) (0.166) (0.284) (0.427)

t t tt

t t t

PB PL IQ

PR PR PR

* * * *4 2 3 5

( 1.2994) 0.1868 0.9458

0.1668

b b b b

6.6 Model Specification

6.6.1 Omitted Variables


(6.20)

5534 3132 4523

(se) (11230) (803) (1066)

( -value) (.622) (.000) (.000)

i i iFAMINC HEDU WEDU

p





(6.21)

(6.22)

26191 5155

(se) (8541) (658)

( -value) (.002) (.000)

i iFAMINC HEDU

p

1 2 2 3 3i i i iy x x e



(6.23)

(6.24)

* * 2 32 2 2 3

2

cov( , )bias( ) ( )

var( )

x xb E b

x

7755 3211 4777 14311 6

(se) (11163) (797) (1061) (5004)

( -value) (.488) (.000) (.000) (.004)

i i i iFAMINC HEDU WEDU KL

p

6.6.2 Irrelevant Variables


5 67759 3340 5869 14200 6 889 1067

(se) (11195) (1250) (2278) (5044) (2242) (1982)

( -value) (.500) (.008) (.010) (.005) (.692) (.591)

i i i i i iFAMINC HEDU WEDU KL X X

p

6.6.3 Choosing the Model

1. Choose variables and a functional form on the basis of your

theoretical and general understanding of the relationship.

2. If an estimated equation has coefficients with unexpected signs, or

unrealistic magnitudes, they could be caused by a misspecification

such as the omission of an important variable.


* *2 2where and c x x c

6.6.3 Choosing the Model

3. One method for assessing whether a variable or a group of variables

should be included in an equation is to perform significance tests. That

is, t-tests for hypotheses such as

or F-tests for hypotheses such as .

Failure to reject hypotheses such as these can be an indication that the

variable(s) are irrelevant.

4. The adequacy of a model can be tested using a general specification test

known as RESET.


0 3: 0H

0 3 4: 0H

6.6.3a The RESET Test


(6.25)

1 2 2 3 3i i i iy x x e

1 2 2 3 3î i iy b b x b x



(6.26)

(6.27)

21 2 2 3 3 1 î i i i iy x x y e

2 31 2 2 3 3 1 2ˆ î i i i i iy x x y y e



0 1

0 1 2

: 0 5.984 -value .015

: 0 3.123 -value .045

H F p

H F p

6.7 Poor data, Collinearity and Insignificance

6.7.1 The Consequences of Collinearity


(6.28)

1 2 2 3 3i i i iy x x e

2

22 2

23 2 21

var( )(1 ) ( )

N

ii

br x x


The effects of imprecise information:

1. When estimator standard errors are large, it is likely that the usual t-

tests will lead to the conclusion that parameter estimates are not

significantly different from zero. This outcome occurs despite

possibly high or F-values indicating significant explanatory power

of the model as a whole.



2. The estimators may be very sensitive to the addition or deletion of a

few observations, or the deletion of an apparently insignificant

variable.

3. Despite the difficulties in isolating the effects of individual

variables from such a sample, accurate forecasts may still be

possible if the nature of the collinear relationship remains the same

within the new (future) sample observations.


6.7.2 An Example

MPG = miles per gallon

CYL = number of cylinders

ENG = engine displacement in cubic inches

WGT = vehicle weight in pounds


42.9 3.558

(se) (.83) (.146)

( -value) (.000) (.000)

i iMPG CYL

p

6.7.2 An Example


44.4 .268 .0127 .00571

(se) (1.5) (.413) (.0083) (.0071)

( -value) (.000) (.517) (.125) (.000)

i i i iMPG CYL ENG WGT

p

6.7.3 Identifying and Mitigating Collinearity

Identifying Collinearity

1. Examining pairwise correlations.

2. Using auxiliary regression

If the R2 from this artificial model is high, above .80 say, the implication is

that a large portion of the variation in is explained by variation in the

other explanatory variables.


2 1 1 3 3i i i K iKx a x a x a x error

6.7.3 Identifying and Mitigating Collinearity

Mitigating Collinearity

1. Obtain more information and include it in the analysis.

2. Introduce nonsample information in the form of restrictions on the

parameters.


6.8 Prediction


(6.29)1 2 2 3 3i i i iy x x e

0 1 02 2 03 3 0y x x e

0 1 02 2 03 3y b x b x b

6.8 Prediction


1 2 02 3 03 0 1 2 02 3 03

0 1 2 02 3 03

2 20 1 02 2 03 3

02 1 2 03 1 3 02 03 2 3

var( ) var[( ) ( )]

var( )

var( ) var( ) var( ) var( )

2 cov( , ) 2 cov( , ) 2 cov( , )

f x x e b b x b x

e b b x b x

e b x b x b

x b b x b b x x b b

Keywords


a single null hypothesis with more than one parameter

auxiliary regressions collinearity F-test irrelevant variable nonsample information omitted variable omitted variable bias overall significance of a regression

model regression specification error test

(RESET) restricted least squares

restricted sum of squared errors single and joint null hypotheses unrestricted sum of squared errors

Chapter 6 Appendices


Appendix 6A Chi-Square and F-tests: More Details

Appendix 6B Omitted Variable Bias: A Proof



(6A.1)

(6A.3)

(6A.2)

( ) /

/ ( )R U

U

SSE SSE JF

SSE N K

21 ( )2

( )R UJ

SSE SSEV

21 ( )2

( )ˆˆ

R UJ

SSE SSEV

(6A.4)

22

2 ( )2

ˆ( )N K

N KV



(6A.5)

1 11 2

2 2

/( , )

/

V mF F m m

V m

2

2 ,2

2

ˆ ˆ

R U

R UJ N K

SSE SSEJ SSE SSE J

FN K

N K



1VF

J

0 3 4: 0H

21 2 3 4i i i i iS P A A e

2

8.44 -value .0005

16.88 -value .0002

F p

p



0 3 4: 3.8 1H

2

.936 -value .3365

.936 -value .3333

F p

p



1 2 2i i iy x v

1 2 2 3 3i i i iy x x e



(6B.1)* 2 22 22

2 2

( )( )

( )i i

i i

i

x x y yb w v

x x

2 22

2 2

( )

( )i

ii

x xw

x x

*2 2 3 3i i i ib w x w e



*2 2 3 3

2 2 32 3 2

2 2

2 2 3 32 3 2

2 2

2 32 3 2

2

( )

( )

( )

( )( )

( )

cov( , )

var( )

i i

i i

i

i i

i

E b w x

x x x

x x

x x x x

x x

x x

x

Documents

Further Inference in the Multiple Regression Model Prepared by Vera Tabakova, East Carolina University