Further Inference in the Multiple Regression Model Hill et al Chapter 8

Further Inference in the Multiple Regression Model

Hill et al Chapter 8

The F-TestUsed to test hypotheses on one or more parameters

Unrestricted model:

1 2 3t t t ttr p a e

Restricted model

1 3t t ttr a e

0 2: 0H

1 2: 0H

0R USSE SSE

The F-statisticAre the differences in SSE significant?

R U

U

SSE SSE JF

SSE T K

If the null hypothesis is true, then the statistic F has an F-distribution with J numerator degrees of freedom and T-K denominator degrees of freedom.

Example

USSE = 1805.168 RSSE = 1964.758

1964.758 1805.168 1

1805.168 52 3

4.332

R U

U

SSE SSE JF

SSE T K

p = P[F1,49 4.332] = .0427

Fc= 4.038

Reject the null hypothesis

1 2 3t t t ttr p a e 0 2: 0H 1 2: 0H

1 3t t ttr a e

Testing the significance of a model

1 2 2 3 3t t t tK K ty x x x e

0 2 3

1

: 0, 0, , 0

: of the is nonzeroK

k

H

H at least one

Restricted model 1t ty e *1

tyb y

T

* 2 21( ) ( )R t tSSE y b y y SST

( ) /( 1)

/( )

SST SSE KF

SSE T K

Example1 2 3t t t ttr p a e

0 2 3: 0, 0H 1 2 3: 0, or 0, or both are nonzeroH

ANALYSIS OF VARIANCE

SS DF MS

REGRESSION 11776. 2. 5888.1

ERROR 1805.2 49. 36.840

TOTAL 13581. 51. 266.30

( ) /( 1) (13581.35 1805.168) / 2 5888.09159.83

/( ) 1805.168/(52 3) 36.84

SST SSE KF

SSE T K

Fc = 3.187

An extended model2

1 2 3 4t t t t ttr p a a e

3 4

( held constant)

( ) ( )2t t

tt tp

E tr E tra

a a

2ˆ 104.81 6.582 2.948 0.0017

(6.58) (3.459) (0.786) (0.0361) (s.e.)t t t ttr p a a

2ˆ 110.46 10.198 3.361 0.0268

(3.74) (1.582) (0.422) (0.0159) (s.e.)t t t ttr p a a

The significance of advertising

261.41R U

U

SSE SSE JF

SSE T K

21 2 3 4t t t t ttr p a a e

0 3 4: 0, 0H

1 3 4: 0, or 0, or both are nonzeroH

J=2, T=78, K= 4. SSEU = 2592. SSER = 20907.331

Fc=3.120

The optimal level of advertisingMarginal benefit from advertising:

3 4

( held constant)

( )2t

tt p

E tra

a

Marginal benefit equals marginal cost:

3 42 1ta

ˆ3.361 2( .0268) 1ta ˆ 44.0485ta

Is this significantly different from $40000? T-test

0 3 4: 2 (40) 1H

1 3 4: 2 (40) 1H

3 4

3 4

( 80 ) 1

se( 80 )

b bt

b b

2

3 4 3 4 3 4ˆ ˆ ˆ ˆvar( 80 ) var( ) 80 var( ) 2(80)cov( , ) .76366b b b b b b

1.217 1.248

.76366t

tc = 1.993

Is this significantly different from $40000? F-test

Restricted model obtained by writing the equation under the assumption that the null is true:

0 3 4

0 3 4

: 2 (40) 1

: 1 2 (40)

H

H

21 2 4 4(1 80 )t t t t ttr p a a e

21 2 4( ) ( 80 )t t t t t ttr a p a a e

(2594.533 2592.301) /1.0637

2592.302 / 74R U

U

SSE SSE JF

SSE T K

Fc=3.970

Testing two conjectures

• Optimal advertising is $40000• If advertising is $40000 and price is $2, revenue

will be 175000

0 3 4 1 2 3 4: 2 (40) 1, 2 40 1600 175H Two hypotheses to substitute in to get restricted model

22 4135 2 80 1600t t t t t ttr a p a a e

( ) /1.75

/( )R U

U

SSE SSE JF

SSE T K

Fc=3.120

Incorporating non-sample information

Multiplying each price and income in a demand equation by a constant has no effect on demand

1 2 3 4 5ln ln ln ln lnB L Rq p p p m

1 2 3 4 5

1 2 3 4 5 2 3 4 5

ln ln ln ln ln

ln ln ln ln ln

B L R

B L R

q p p p m

p p p m

2 3 4 5 0

A restricted model

1 2 3 4 5ln ln ln ln lnB L Rq p p p m

4 2 3 5

1 2 3 2 3 5 5

1 2 3 5

1 2 3 5

ln ln ln ln ln

ln ln ln ln ln ln

ln ln ln

t Bt Lt Rt t t

Bt Rt Lt Rt t Rt t

Bt Lt tt

Rt Rt Rt

q p p p m e

p p p p m p e

p p me

p p p

ˆln 4.798 1.2994ln 0.1868ln 0.9458ln

(3.714) (0.166) (0.284) (0.427)

Bt Lt tt

Rt Rt Rt

p p mq

p p p

Omitted and irrelevant variables

• An omitted variable which is correlated with other variables in the regression will lead to bias.

• The omission of ‘insignificant’ variables may lead to bias (remember all you have done is failed to reject a null)

• Including irrelevant variables will inflate the variances of the estimated parameters.

The RESET test: principle

• If we omit variables and they are correlated with existing variables, including a function of these variables may allow us to pick up some of the effect of the omitted variables.

• If we can artificially improve the model by including powers of the predictions of the model, then a better functional form may exist.

• Overall: if we can improve a model by including powers of the predictions the model is inadequate.

The RESET test: practice

1 2 2 3 3t t t ty x x e 1 2 2 3 3ˆt t ty b b x b x

21 2 2 3 3 1 ˆt t t t ty x x y e

0 1: 0H 1 1: 0H

2 31 2 2 3 3 1 2ˆ ˆt t t t t ty x x y y e

0 1 2: 0H 1 1 2: 0 or 0H

In both cases the null is of no mis-specification

The RESET test: example

1 2 3 4 5ln( ) ln( ) ln( ) ln( ) ln( )t Bt Lt Rt t tq p p p m e

1 2 3 4 5t Bt Lt Rt t tq p p p m e Ramsey RESET Test: LOGLOG Model

F-statistic (1 term) 0.0075 Probability 0.9319

F-statistic (2 terms) 0.3581 Probability 0.7028

Ramsey RESET Test: LINEAR Model

F-statistic (1 term) 8.8377 Probability 0.0066

F-statistic (2 terms) 4.7618 Probability 0.0186

The linear model is mis-specified.

Collinear Economic Variables

• Explanatory variables move together in systematic ways.

• Attribute the increase in TR that is the consequence of two types of advertising.

• Identify the effects of increasing input quantities when technology is of the fixed proportions type.

The consequences of collinearity

2

2 22 2 23

var( )(1 )t

bx x r

• Exact collinearity renders OLS inoperable.• Near exact leads to increased standard errors.• R2 may be high but individual coefficients are

likely to be insignificant.• Estimates will be sensitive to the addition of a few

observations.• Accurate prediction may still be possible.

Identifying and mitigating collinearity

• Identifying:– Large standard errors with high R2.– Pairwise correlation coefficients in excess of

0.8– Auxiliary regressions.

• Mitigating– Additional data.– Parameter restrictions