Statistical Inference

Dr. TU Thuy AnhFaculty of International

Economics1

KTEE 310 FINANCIAL ECONOMETRICS

STATISTICAL INFERENCEChap 5 & 7 – S & W

Model: Y = 1 + 2X + u

Null hypothesis:

Alternative hypothesis:

TESTING A HYPOTHESIS RELATING TO A REGRESSION COEFFICIENT

0220 : H0221 : H

We will suppose that we have the standard simple regression model and that we wish to test the hypothesis H0 that the slope coefficient is equal to some

value 20. We test it against the alternative hypothesis H1, which is simply

that 2 is not equal to 20

Model: Y = 1 + 2X + u

Null hypothesis:


Example model: p = 1 + 2w + u

Null hypothesis:



As an illustration, we will consider a model relating p, the rate of growth of prices and w, the rate of growth of wages.

We will test the hypothesis that the rate of price inflation is equal to the rate of wage inflation. The null hypothesis is therefore H0: 2 = 1.0.

0220 : H

0.1: 20 H

0.1: 21 H

0221 : H

We will assume that we know the standard deviation and that it is equal to 0.1. This is a very unrealistic assumption. In practice you have to estimate it.


1.0 1.10.90.80.70.6 1.2 1.3 1.4

probability densityfunction of b2

b2

Distribution of b2 under the null hypothesis H0: 2 =1.0 is true (standard deviation equals 0.1 taken as given)


Suppose that we have a sample of data for the price inflation/wage inflation model and the estimate of the slope coefficient, b2, is 0.9. Would this be

evidence against the null hypothesis 2= 1.0?

And what if b2 =1.4?

1.0 1.10.90.80.70.6 1.2 1.3 1.4


b2

Distribution of b2 under the null hypothesis H0: 2 =1.0 is true (standard deviation equals 0.1 taken as given)


The usual procedure for making decisions is to reject the null hypothesis if it implies that the probability of getting such an extreme estimate is less than some (small) probability p.


b2

Distribution of b2 under the null hypothesis H0: 2 =2 is true (standard deviation taken as given)

0

2 2+sd 2+2sd2-sd2-2sd 2+3sd2-3sd2-4sd 2+4sd00000 0 0 0 0


For example, we might choose to reject the null hypothesis if it implies that the probability of getting such an extreme estimate is less than 0.05 (5%).

According to this decision rule, we would reject the null hypothesis if the estimate fell in the upper or lower 2.5% tails.


b2

Distribution of b2 under the null hypothesis H0: 2 =2 is true (standard deviation taken as given)

0

2 2+sd 2+2sd2-sd2-2sd 2+3sd2-3sd2-4sd 2+4sd00000 0 0 0 0

2.5% 2.5%


The 2.5% tails of a normal distribution always begin 1.96 standard deviations from its mean. Thus we would reject H0 if the estimate were 1.96 standard deviations (or more) above or below the hypothetical mean.

Or if the difference, expressed in terms of standard deviations, were more than 1.96 in absolute terms (positive or negative).

2.5%2.5%


b22+1.96sd2-1.96sd 202-sd 2+sd00 00

Decision rule (5% significance level):reject(1) if (2) if

0220 :H

s.d. 96.1022 b s.d. 96.10

22 b

96.1s.d./)( 022 b 96.1s.d./)( 0

22 b


2.5%2.5%

Decision rule (5% significance level):reject(1) if (2) if(1) if z > 1.96 (2) if z < -1.96

s.d.

022

b

z


b22+1.96sd2-1.96sd 202-sd 2+sd00 0

0

s.d. 96.1022 b s.d. 96.10

22 b

0220 :H

The range of values of b2 that do not lead to the rejection of the null hypothesis is known as the acceptance region. Type II error (the probability of accepting the false hypothesis)

The limiting values of z for the acceptance region are 1.96 and -1.96 (for a 5% significance test).

acceptance region for b2:

s.d. 96.1s.d. 96.1 022

02 b

96.196.1 z


2.5%2.5%


b22+1.96sd2-1.96sd 202-sd 2+sd00 0

0

acceptance region for b2

reject 0220 :H reject 0

220 :H

Rejection of the null hypothesis when it is in fact true is described as a Type I error.

With the present test, if the null hypothesis is true, a Type I error will occur 5% of the time because 5% of the time we will get estimates in the upper or lower 2.5% tails. The significance level of a test is defined to be the probability of making a Type I error if the null hypothesis is true.

Type I error: rejection of H0 when it is in fact true.

Probability of Type I error: in this case, 5%Significance level of the test is 5%.


2.5%2.5%


b22+1.96sd2-1.96sd 202-sd 2+sd00 0

0

acceptance region for b2

reject 0220 :H reject 0

220 :H

We could change the decision rule to “reject the null hypothesis if it implies that the probability of getting the sample estimate is less than 0.01 (1%)”.

The rejection region now becomes the upper and lower 0.5% tails

5% level

1% level

t TEST OF A HYPOTHESIS RELATING TO A REGRESSION COEFFICIENT

We replace the standard deviation in its denominator with the standard error, the test statistic has a t distribution instead of a normal distribution.

We look up the critical value of t and if the t statistic is greater than it, positive or negative, we reject the null hypothesis. If it is not, we do not.

s.d. of b2 known

discrepancy between hypothetical value and sample estimate, in terms of s.d.:

s.d.

022

b

z

5% significance test:

reject H0: 2 = 2 if

z > 1.96 or z < –1.96

s.d. of b2 not known

discrepancy between hypothetical value and sample estimate, in terms of s.e.:

s.e.

022

b

t

5% significance test:

reject H0: 2 = 2 if

t > tcrit or t < –tcrit

0 0

A graph of a t distribution with 10 degrees of freedom. When the number of degrees of freedom is large, the t distribution looks very much like a normal distribution

0

0.1

0.2

0.3

0.4

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6

Normal (0,1)

t, 10 d.f.


t distribution has longer tails than the normal distribution, the difference being the greater, the smaller the number of degrees of freedom

This means that the rejection regions have to start more standard deviations away from zero for a t distribution than for a normal distribution.

normal

0

0.1

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6

t, 10 d.f.

t, 5 d.f.


The 2.5% tail of a t distribution with 10 degrees of freedom starts 2.33 standard deviations from its mean.

That for a t distribution with 5 degrees of freedom starts 2.57 standard deviations from its mean.

normal

0

0.1

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6

-2.57

t, 10 d.f.

t, 5 d.f.


-2.33

-1.96


For this reason we need to refer to a table of critical values of t when performing significance tests on the coefficients of a regression equation.

t Distribution: Critical values of t

Degrees of Two-sided test 10% 5% 2% 1% 0.2% 0.1% freedom One-sided test 5% 2.5% 1% 0.5% 0.1% 0.05%

1 6.314 12.706 31.821 63.657 318.31 636.622 2.920 4.303 6.965 9.925 22.327 31.5983 2.353 3.182 4.541 5.841 10.214 12.9244 2.132 2.776 3.747 4.604 7.173 8.6105 2.015 2.571 3.365 4.032 5.893 6.869… … … … … … …… … … … … … …

18 1.734 2.101 2.552 2.878 3.610 3.92219 1.729 2.093 2.539 2.861 3.579 3.88320 1.725 2.086 2.528 2.845 3.552 3.850… … … … … … …… … … … … … …

600 1.647 1.964 2.333 2.584 3.104 3.3071.645 1.960 2.326 2.576 3.090 3.291

Number of degrees of freedom in a regression

= number of observations – number of parameters estimated.

Example:

)10.0()05.0(

82.021.1ˆ wp

.80.110.0

00.182.0)(s.e. 2

022

bb

t

18freedom of degrees;20 n

101.2%5,crit t

1:;1: 2020 HH

uwp 21

The critical value of t with 18 degrees of freedom is 2.101 at the 5% level. The absolute value of the t statistic is less than this, so we do not reject the null hypothesis.


18

Model 3: OLS, using observations 1899-1922 (T = 24)Dependent variable: q

coefficient std. error t-ratio p-value -------------------------------------------------------- const -4,85518 14,5403 -0,3339 0,7418 l 0,916609 0,149560 6,129 4,42e-06 *** k 0,158596 0,0416823 3,805 0,0010 ***

Mean dependent var 165,9167 S.D. dependent var 43,75318Sum squared resid 2534,226 S.E. of regression 10,98533R-squared 0,942443 Adjusted R-squared 0,936961F(2, 21) 171,9278 P-value(F) 9,57e-14Log-likelihood -89,96960 Akaike criterion 185,9392Schwarz criterion 189,4734 Hannan-Quinn 186,8768rho 0,098491 Durbin-Watson 1,535082

EXAMPLE

19


coefficient std. error t-ratio p-value ----------------------------------------------------------- const -10,7774 16,4164 -0,6565 0,5190 l 0,822744 0,190860 4,311 0,0003 *** k 0,312205 0,195927 1,593 0,1267 sq_k -0,000249224 0,000310481 -0,8027 0,4316

Mean dependent var 165,9167 S.D. dependent var 43,75318Sum squared resid 2455,130 S.E. of regression 11,07955R-squared 0,944239 Adjusted R-squared 0,935875F(3, 20) 112,8920 P-value(F) 1,05e-12Log-likelihood -89,58910 Akaike criterion 187,1782Schwarz criterion 191,8904 Hannan-Quinn 188,4284rho -0,083426 Durbin-Watson 1,737618

EXAMPLE

Hypothesis testing using p-value

Step 1: Calculate tob =

Step 2: Calculate p-value = P (|t| > |tob|)

Step 3: Gor a given α:• Two-tail test: p-value < α reject H0

• One-tail test: p-value/2 < : α reject H0

20

)ˆ(

ˆ

i

ii

se

21




EXAMPLE

22


coefficient std. error t-ratio p-value ----------------------------------------------------------- const -10,7774 16,4164 -0,6565 0,5190 l 0,822744 0,190860 4,311 0,0003 *** k 0,312205 0,195927 1,593 0,1267 sq_k -0,000249224 0,000310481 -0,8027 0,4316

Mean dependent var 165,9167 S.D. dependent var 43,75318Sum squared resid 2455,130 S.E. of regression 11,07955R-squared 0,944239 Adjusted R-squared 0,935875F(3, 20) 112,8920 P-value(F) 1,05e-12Log-likelihood -89,58910 Akaike criterion 187,1782Schwarz criterion 191,8904 Hannan-Quinn 188,4284rho -0,083426 Durbin-Watson 1,737618

EXAMPLE

probability density function of b2

(1) conditional on 2 = 2 being true

(2) conditional on 2 = 2 being true

CONFIDENCE INTERVALS

b222 -sd2 -1.96sd 2

+sd

min minminmin

min

max

The diagram shows the limiting values of the hypothetical values of 2, together with their associated probability distributions for b2.

2 2 + 1.96sd2 - sd 2 + sd

max max maxmax

(1)(2)


b222 -sd2 -1.96sd 2

+sd

min minminmin2 2 + 1.96sd2 - sd 2+sdmax max maxmax

(1)(2)

Any hypothesis lying in the interval from 2min to 2

max would be compatible with the sample estimate (not be rejected by it). We call this interval the 95% confidence interval.

reject any 2 > 2 = b2 + 1.96 sd

reject any 2 < 2 = b2 - 1.96 sd

95% confidence interval:

b2 - 1.96 sd < 2 < b2 + 1.96 sd

max

min


Standard deviation known

95% confidence intervalb2 - 1.96 sd < 2 < b2 + 1.96 sd

99% confidence intervalb2 - 2.58 sd < 2 < b2 + 2.58 sd

Standard deviation estimated by standard error

95% confidence intervalb2 - tcrit (5%) se < 2 < b2 + tcrit (5%) se

99% confidence intervalb2 - tcrit (1%) se < 2 < b2 + tcrit (1%) se

26

TESTING FOR A SINGLE RESTRICTION

H0 H1 Reject H0 if

j = * j ≠ * tob|>tn-k; α/2

j ≥ * j< * tob<-tn-k; α

j ≤ * j > * tob>tn-k; α

)(*

obj

j

bse

bt

27

TESTING BETWEEN 2 COEFFCIENTs

H0 H1 Reject H0 if

j = j ≠ tob|>tn-k; α/2

j ≥ j< tob<-tn-k; α

j ≤ j > tob>tn-k; α

ijij bbb

ij

ij

ij bb

bbse

bbt

,ˆ2ˆˆ)( 22ob

28



Covariance matrix of regression coefficients:

const k l 211,422 0,371062 -2,01166 const 0,00173742 -0,00533574 k 0,0223683 l

MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE

29

TESTING FOR MORE THAN ONE RESTRICTIONS - F TEST

If 2 =3=..=K= 0?

if 2 =4=0

meaning: all variables in the model do not affect y

model is insignificant R2 =0

Meaning: X2 and X4 should not be included

30

SIGNIFICANCE OF MODEL – F TESTIf the model is significant?H0: R2 =0; H1: R2>0

If Fob > Fα(k-1,n-k) reject H0

2

2

/ ( 1)

(1 ) / ( )ob

R kF

R n k

31


coefficient std. error t-ratio p-value --------------------------------------------------------- const -38,7267 14,5994 -2,653 0,0145 ** l 1,40367 0,0982155 14,29 1,29e-012 ***


EXAMPLE

32




EXAMPLE

33

F-TESTY = 1 +2X2+..+5X5 + u (1)

if 2 =4=0?H0: 2 =4=0; H1: at least one of them is nonzero

Step1: run unrestricted model (1) => R2(1)Step 2: run: restricted model: Y = 1 +X3+ 5X5 + u

(2) => R2(2)

If Fob > F α(m,n-k)=> reject H0

Example: H0: 2 =0 : Fob

= 14,477 > F 5%(1,21)=4.32 => reject H0

2 2

2

( (1) (2)) /

(1 (1)) / ( )ob

R R mF

R n k

Documents

Statistical Inference