View
9
Download
0
Category
Preview:
Citation preview
Lecture 6: Nonlinear Regression FunctionsIntroduction to Econometrics,Fall 2020
Zhaopeng Qu
Nanjing University
10/29/2020
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 1 / 76
1 Review of previous lecture
2 Nonlinear Regression Functions:
3 Nonlinear in Xs
4 Polynomials in X
5 Logarithms
6 Interactions Between Independent Variables
7 Summary
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 2 / 76
Review of previous lecture
Section 1
Review of previous lecture
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 3 / 76
Review of previous lecture
OLS Regression
OLS is the most basic and important tool in econometricians’ toolbox.
The OLS estimators is unbiased, consistent and normal distributionsunder key assumptions.
Using the hypothesis testing and confidence interval in OLSregression, we could make a more reliable judgment about therelationship between the treatment and the outcomes.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 4 / 76
Nonlinear Regression Functions:
Section 2
Nonlinear Regression Functions:
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 5 / 76
Nonlinear Regression Functions:
Introduction
Everything what we have learned so far is linear in the X’s. But thelinear approximation is not always a good one.
We can extend our model to be nonlinear into two cases1 Nonlinear in Xs
Polynomials,Logarithms and InteractionsThe multiple regression framework can be extended to handleregression functions that are nonlinear in one or more X.the difference from a standarad multiple OLS regression is how toexplain estimating coefficients.
2 Nonlinear in functionDiscrete Dependent Variables or Limited Dependent VariablesLinear function is not a good prediciton function.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 6 / 76
Nonlinear in Xs
Section 3
Nonlinear in Xs
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 7 / 76
Nonlinear in Xs
The TestScore – STR relation looks linear (maybe)
TestScore^ = c(698.9) − c(−2.28)*STR
630
660
690
14 16 18 20 22 24 26str
test
scr
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 8 / 76
Nonlinear in Xs
But the TestScore – Income relation looks nonlinear
TestScore^ = c(625.4) + c(1.88)*Avginc
600
650
700
10 20 30 40 50avginc
test
scr
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 9 / 76
Nonlinear in Xs
Nonlinear Regression Regression FunctionsSo far our regression model is
𝑌𝑖 = 𝛽0 + 𝛽1𝑋1,𝑖 + ... + 𝛽𝑘𝑋𝑘,𝑖 + 𝑢𝑖
The effect of Y on a change in 𝑋𝑗 by 1 (unit) is constant and equals𝛽𝑗:
𝛽𝑗 =𝜕𝑌𝑖
𝜕𝑋𝑗𝑖But if a relation between Y and X is nonlinear:
The effect on Y of a change in X depends on the value of X – that is,the marginal effect of X is not constant.A linear regression is misspecified – the functional form is wrong.The estimator of the effect on Y of X is biased(a special case of OVB).
The solution to this is to estimate a regression function that isnonlinear in X.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 10 / 76
Nonlinear in Xs
OLS Assumptions Still Hold
General formula for a nonlinear population regression model:
𝑌𝑖 = 𝑓(𝑋1,𝑖, 𝑋2,𝑖, ..., 𝑋𝑘,𝑖) + 𝑢𝑖
Assumptions:1 𝐸[𝑢𝑖|𝑋1,𝑖, 𝑋2,𝑖, ..., 𝑋𝑘,𝑖] = 0 implies that the function is the
conditional expectation of Y given the X’s.2 (𝑋1,𝑖, 𝑋2,𝑖, ..., 𝑋𝑘,𝑖) are i.i.d.3 Large outliers are rare.4 No perfect multicollinearity.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 11 / 76
Nonlinear in Xs
Two Cases:
1 The effect of change in 𝑋1 on Y depends on 𝑋1 itself.eg. the effect of a change in class size on test scores is bigger wheninitial class size is small.
2 The effect of change in 𝑋1 on Y depends on another variable, like 𝑋2eg.the effect of class size on test scores depends on the percentage ofdisadvantaged pupils in the class.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 12 / 76
Nonlinear in Xs
Different Slops
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 13 / 76
Nonlinear in Xs
The Effect on Y of a Change in X in NonlinearFunctions
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 14 / 76
Nonlinear in Xs
A General Approach to Modeling Nonlinearities:Multiple OLS Regression
Identify a possible nonlinear relationship.Specify a nonlinear function and estimate its parameters by OLS.Determine whether the nonlinear model improves upon a linear model.Plot the estimated nonlinear regression function.Estimate the effect on Y of a change in X.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 15 / 76
Nonlinear in Xs
Two Complementary Approaches:
1 Polynomials in XThe population regression function is approximated by a quadratic,cubic, or higher-degree polynomial.
2 Logarithmic transformationsY and/or X is transformed by taking its logarithmthis gives a percentages interpretation that makes sense in manyapplications
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 16 / 76
Polynomials in X
Section 4
Polynomials in X
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 17 / 76
Polynomials in X
Polynomials in X
Approximate the population regression function by a polynomial:
𝑌𝑖 = 𝛽0 + 𝛽1𝑋𝑖 + 𝛽2𝑋2... + 𝛽𝑟𝑋𝑟𝑖 + 𝑢𝑖
This is just the linear multiple regression model – except that theregressors are powers of X!
Estimation, hypothesis testing, etc. proceeds as in the multipleregression model using OLS.
Although, the coefficients are difficult to interpret, the regressionfunction itself is interpretable.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 18 / 76
Polynomials in X
Testing the null hypothesis that the populationregression function is linear
𝐻0 ∶ 𝛽2 = 0, 𝛽3 = 0, ..., 𝛽𝑟 = 0 𝑎𝑛𝑑 𝐻1 ∶ 𝑎𝑡 𝑙𝑒𝑎𝑠𝑡 𝑜𝑛𝑒 𝛽𝑗 ≠ 0
it can be tested using the F-statistic
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 19 / 76
Polynomials in X
Which degree polynomial should I use?
How many powers of X should be included in a polynomial regression?
The answer balances a trade-off between flexibility and statisticalprecision.
In many applications involving economic data, the nonlinear functionsare smooth, that is, they do not have sharp jumps, or “spikes.”
If so, then it is appropriate to choose a small maximum degree for thepolynomial, such as 2, 3, or 4.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 20 / 76
Polynomials in X
Example: the TestScore-Income relation
Quadratic specification:
𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒𝑖 = 𝛽0 + 𝛽1𝐼𝑛𝑐𝑜𝑚𝑒𝑖 + 𝛽2(𝐼𝑛𝑐𝑜𝑚𝑒𝑖)2 + 𝑢𝑖Cubic specification:
𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒𝑖 = 𝛽0+𝛽1𝐼𝑛𝑐𝑜𝑚𝑒𝑖+𝛽2(𝐼𝑛𝑐𝑜𝑚𝑒𝑖)2+𝛽3(𝐼𝑛𝑐𝑜𝑚𝑒𝑖)3+𝑢𝑖
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 21 / 76
Polynomials in X
Estimation of the quadratic specification in R#### Call:## felm(formula = testscr ~ avginc + I(avginc^2), data = ca)#### Residuals:## Min 1Q Median 3Q Max## -44.416 -9.048 0.440 8.348 31.639#### Coefficients:## Estimate Robust s.e t value Pr(>|t|)## (Intercept) 607.30174 2.90175 209.288
Polynomials in X
Estimation of the cubic specification in R#### Call:## felm(formula = testscr ~ avginc + I(avginc^2) + I(avginc3), data = ca)#### Residuals:## Min 1Q Median 3Q Max## -44.28 -9.21 0.20 8.32 31.16#### Coefficients:## Estimate Robust s.e t value Pr(>|t|)## (Intercept) 6.001e+02 5.102e+00 117.615 < 2e-16 ***## avginc 5.019e+00 7.074e-01 7.095 5.61e-12 ***## I(avginc^2) -9.581e-02 2.895e-02 -3.309 0.00102 **## I(avginc3) 6.855e-04 3.471e-04 1.975 0.04892 *## ---## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1#### Residual standard error: 12.71 on 416 degrees of freedom## Multiple R-squared(full model): 0.5584 Adjusted R-squared: 0.5552## Multiple R-squared(proj model): 0.5584 Adjusted R-squared: 0.5552## F-statistic(full model, *iid*):175.4 on 3 and 416 DF, p-value: < 2.2e-16## F-statistic(proj model): 270.2 on 3 and 416 DF, p-value: < 2.2e-16
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 23 / 76
Polynomials in X
Interpreting the estimated regression function
Dependent Variable: Test Score(1) (2) (3)
avginc 1.879∗∗∗ 3.851∗∗∗ 5.019∗∗∗(0.113) (0.267) (0.704)
I(avginc 2̂) −0.042∗∗∗ −0.096∗∗∗(0.005) (0.029)
I(avginc3) 0.001∗∗(0.0003)
Constant 625.384∗∗∗ 607.302∗∗∗ 600.079∗∗∗(1.863) (2.891) (5.078)
Observations 420 420 420Adjusted R2 0.506 0.554 0.555Residual Std. Error 13.387 12.724 12.707F Statistic 430.830∗∗∗ 261.278∗∗∗ 175.352∗∗∗
Note: ∗p
Polynomials in X
Figure: Linear and Quadratic Regression
600
650
700
10 20 30 40 50avginc
test
scr
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 25 / 76
Polynomials in X
Quadratic vs LinearQuestion: Is the quadratic model better than the linear model?
We can test the null hypothesis that the regression function is linearagainst the alternative hypothesis that it is quadratic:
𝐻0 ∶ 𝛽2 = 0 𝑎𝑛𝑑 𝐻1 ∶ 𝛽2 ≠ 0
the t-statistic
𝑡 = (̂𝛽2 − 0)
𝑆𝐸( ̂𝛽2)= −0.04230.0048 = −8.81
Since 8.81 > 2.58, we reject the null hypothesis (the linear model) ata 1% significance level.
the F-test also reject𝐹 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐𝑞=2,𝑑=417 = 261.3, 𝑝 − 𝑣𝑎𝑙𝑢𝑒 ≅ 0.00
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 26 / 76
Polynomials in X
Interpreting the estimated regression function
Let us predict the change in TestScore for a change in Income, thuswhat is the marginal effect of X on Y in a polynomial function.
The regression model now is
𝑌𝑖 = 𝛽0 + 𝛽1𝑋𝑖 + 𝛽2𝑋2𝑖 + 𝑢𝑖
The marginal effect of X on Y
𝜕𝑌𝑖𝜕𝑋𝑖
= 𝛽1 + 2𝛽2𝑋𝑖
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 27 / 76
Polynomials in X
Interpreting the estimated quadratic regressionfunction
Suppose we want to measure an 1000 increase on incomefrom $10,000 per capita to $11,000 per capita:
Δ𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒 = 607.3 + 3.85 × 11 − 0.0423 × (11)2− [607.3 + 3.85 × 10 − 0.0423 × (10)2]= 2.96
from $40,000 per capita to $41,000 per capita:
Δ𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒 = 607.3 + 3.85 × 41 − 0.0423 × (41)2− [607.3 + 3.85 × 40 − 0.0423 × (40)2]= 0.42
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 28 / 76
Polynomials in X
Figure: Cubic and Quadratic Regression
630
660
690
10 20 30 40 50avginc
test
scr
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 29 / 76
Polynomials in X
Quadratic vs CubicQuestion: Is the cubic model better than the quadratic model?
We can test the null hypothesis that the regression function is linearagainst the alternative hypothesis that it is cubic:
𝐻0 ∶ 𝛽3 = 0 𝑎𝑛𝑑 𝐻1 ∶ 𝛽3 ≠ 0
the t-statistic
𝑡 = (̂𝛽3 − 0)
𝑆𝐸( ̂𝛽3)= −0.0010.0003 = −3.33
Since 3.33 > 2.58, we reject the null hypothesis (the linear model) ata 1% significance level.
the F-test also reject𝐹 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐𝑞=3,𝑑=416 = 175.35, 𝑝 − 𝑣𝑎𝑙𝑢𝑒 ≅ 0.00
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 30 / 76
Polynomials in X
Interpreting the estimated cubic regression function
The regression model now is
𝑌𝑖 = 𝛽0 + 𝛽1𝑋𝑖 + 𝛽2𝑋2𝑖 + 𝛽3𝑋3𝑖 + 𝑢𝑖The marginal effect of X on Y
𝜕𝑌𝑖𝜕𝑋𝑖
= 𝛽1 + 2𝛽2𝑋𝑖 + 3𝛽3𝑋2𝑖
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 31 / 76
Polynomials in X
Interpreting the estimated regression function
from 10, 000 per capita to 11, 000 per capita:Δ𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒 = 600.079 + 5.019 × 11 − 0.96 × (11)2 + 0.001 × (11)3
− [600.079 + 5.019 × 10 − 0.96 × (10)2 + 0.001 × (10)3]
from $40,000 per capita to $41,000 per capita:
Δ𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒 = 600.079 + 5.019 × 41 − 0.96 × (41)2 + 0.001 × (41)3− [600.079 + 5.019 × 40 − 0.96 × (40)2 + 0.001 × (40)3]
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 32 / 76
Polynomials in X
Wrap Up
The nonlinear functions in Polynomials in Xs are just a special form ofMultiple OLS Regression.
If the true relationship between X and Y is nonlinear in polynomials inXs, then a fully linear regression is misspecified – the functional formis wrong.
The estimator of the effect on Y of X is biased(a special case ofOVB).
Estimation, hypothesis testing, etc. proceeds as in the multipleregression model using OLS, which can also help us to tell the degreesof polynomial functions.
The big difference is how to explained the estimate coefficients andmake the predicted change in Y with a change in Xs.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 33 / 76
Logarithms
Section 5
Logarithms
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 34 / 76
Logarithms
Logarithmic functions of Y and/or X
Another way to specify a nonlinear regression model is to use thenatural logarithm of Y and/or X.𝐿𝑛(𝑥) = the natural logarithm of x is the inverse function of theexponential function𝑒𝑥, here 𝑒 = 2.71828.
𝑥 = 𝑙𝑛(𝑒𝑥)
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 35 / 76
Logarithms
Review of the Basic Logarithmic functions
If 𝑋 and a are variables, then we have
𝑙𝑛(1/𝑥) = −𝑙𝑛(𝑥)𝑙𝑛(𝑎𝑥) = 𝑙𝑛(𝑎) + 𝑙𝑛(𝑥)
𝑙𝑛(𝑥/𝑎) = 𝑙𝑛(𝑥) − 𝑙𝑛(𝑎)𝑙𝑛(𝑥𝑎) = 𝑎𝑙𝑛(𝑥)
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 36 / 76
Logarithms
Logarithms and percentages
Because
𝑙𝑛(𝑥 + Δ𝑥) − 𝑙𝑛(𝑥) = 𝑙𝑛(𝑥 + Δ𝑥𝑥 )
≅ Δ𝑥𝑥 (𝑤ℎ𝑒𝑛Δ𝑥𝑥 𝑖𝑠 𝑣𝑒𝑟𝑦 𝑠𝑚𝑎𝑙𝑙)
For example:
𝑙𝑛(1 + 0.01) = 𝑙𝑛(101) − 𝑙𝑛(100) = 0.00995 ≅ 0.01
Thus,logarithmic transforms permit modeling relations in percentageterms (like elasticities), rather than linearly.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 37 / 76
Logarithms
The three log regression specifications:
Case Population regression functionI.linear-log 𝑌𝑖 = 𝛽0 + 𝛽1𝑙𝑛(𝑋𝑖) + 𝑢𝑖II.log-linear 𝑙𝑛(𝑌𝑖) = 𝛽0 + 𝛽1𝑋𝑖 + 𝑢𝑖III.log-log 𝑙𝑛(𝑌𝑖) = 𝛽0 + 𝛽1𝑙𝑛(𝑋𝑖) + 𝑢𝑖
The interpretation of the slope coefficient differs in each case.The interpretation is found by applying the general “before and after”rule: “figure out the change in Y for a given change in X.”(KeyConcept 8.1 in S.W.pp301)
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 38 / 76
Logarithms
I. Linear-log population regression function
Regression Model:𝑌𝑖 = 𝛽0 + 𝛽1𝑙𝑛(𝑋𝑖) + 𝑢𝑖
Change X:
Δ𝑌 = [𝛽0 + 𝛽1𝑙𝑛(𝑋 + Δ𝑋)] − [𝛽0 + 𝛽1𝑙𝑛(𝑋)]= 𝛽1[𝑙𝑛(𝑋 + Δ𝑋) − 𝑙𝑛(𝑋)]
≅ 𝛽1Δ𝑋𝑋
Now 100 × Δ𝑋𝑋 = 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒 𝑐ℎ𝑎𝑛𝑔𝑒 𝑖𝑛 𝑋, then
𝛽1 ≅Δ𝑌Δ𝑋𝑋
Interpretation of 𝛽1: a 1 percent increase in X (multiplying X by 1.01 or100 × Δ𝑋𝑋 ) is associated with a 0.01𝛽1 or
𝛽1100 change in Y.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 39 / 76
Logarithms
Example:the TestScore – log(Income) relation
The OLS regression of ln(Income) on Testscore yields
̂𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒 =557.8 + 36.42 × 𝑙𝑛(𝐼𝑛𝑐𝑜𝑚𝑒)(3.8) (1.4)
Interpreation of 𝛽1: a 1% increase in Income is associated with anincrease in TestScore of 0.36 points on the test.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 40 / 76
Example:the
Logarithms
Test scores: linear-log functionlinear−log
630
660
690
10 20 30 40 50avginc
test
scr
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 41 / 76
Logarithms
Case II. Log-linear population regression functionthe regression model is
𝑙𝑛(𝑌𝑖) = 𝛽0 + 𝛽1𝑋𝑖 + 𝑢𝑖
Change X:
𝑙𝑛(Δ𝑌 + 𝑌 ) − 𝑙𝑛(𝑌 ) = [𝛽0 + 𝛽1(𝑋 + Δ𝑋)] − [𝛽0 + 𝛽1𝑋]
𝑙𝑛(1 + Δ𝑌𝑌 ) = 𝛽1Δ𝑋
thenΔ𝑌𝑌 ≅ 𝛽1Δ𝑋
Now 100Δ𝑌𝑌 = 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒 𝑐ℎ𝑎𝑛𝑔𝑒 𝑖𝑛 𝑌 ,so a change in X by oneunit is associated with a 𝛽1% = 𝛽1100 change in Y.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 42 / 76
Logarithms
Earning function: log-linear functions
Example: Age(working experience) and Earnings
The OLS regression of age on earnings yields
̂𝑙𝑛(𝐸𝑎𝑟𝑛𝑖𝑛𝑔𝑠) =2.811 + 0.0096𝐴𝑔𝑒(0.018) (0.0004)
According to this regression, when one more year old, earnings arepredicted to increase by 100 × 0.0096 = 0.96%
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 43 / 76
Logarithms
Case III. Log-log population regression functionthe regression model is
𝑙𝑛(𝑌𝑖) = 𝛽0 + 𝛽1𝑙𝑛(𝑋𝑖) + 𝑢𝑖
Change X:
𝑙𝑛(Δ𝑌 + 𝑌 ) − 𝑙𝑛(𝑌 ) = [𝛽0 + 𝛽1𝑙𝑛(𝑋 + Δ𝑋)] − [𝛽0 + 𝛽1𝑙𝑛(𝑋)]
𝑙𝑛(1 + Δ𝑌𝑌 ) = 𝛽1𝑙𝑛(1 +Δ𝑋𝑋 )
⇒ Δ𝑌𝑌 ≅ 𝛽1Δ𝑋𝑋
Now 100Δ𝑌𝑌 = 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒 𝑐ℎ𝑎𝑛𝑔𝑒 𝑖𝑛 𝑌 and100Δ𝑋𝑋 = 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒 𝑐ℎ𝑎𝑛𝑔𝑒 𝑖𝑛 𝑋so a 1% change in X by one unit is associated with a 𝛽1% change inY,thus 𝛽1 has the interpretation of an elasticity.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 44 / 76
Logarithms
Test scores and income: log-log specifications
̂𝑙𝑛(𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒) =6.336 + 0.055 × 𝑙𝑛(𝐼𝑛𝑐𝑜𝑚𝑒)(0.006) (0.002)
An 1% increase in Income is associated with an increase of .0554% inTestScore.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 45 / 76
Logarithms
Test scores: The log-linear and log-log functions
log−log
log−linear
6.40
6.45
6.50
6.55
6.60
10 20 30 40 50avginc
log.
test
scr
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 46 / 76
Logarithms
Test scores: The linear-log and cubic functions
cubic
linear−log
630
660
690
10 20 30 40 50avginc
test
scr
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 47 / 76
Logarithms
Logarithmic and cubic functions
Dependent Variable: Test Scoretestscr log.testscr testscr
(1) (2) (3)loginc 36.420∗∗∗ 0.055∗∗∗
(0.002)avginc 5.019∗∗∗
(0.704)I(avginc 2̂) −0.096∗∗∗
(0.029)I(avginc 3̂) 0.001∗∗
(0.0003)Constant 557.832∗∗∗ 6.336∗∗∗ 600.079∗∗∗
(5.078) (0.006) (5.078)Observations 420 420 420Adjusted R2 0.561 0.557 0.555Residual Std. Error 12.618 0.019 12.707F Statistic 537.444∗∗∗ 527.238∗∗∗ 175.352∗∗∗
Note: ∗p
Logarithms
Choice of specification should be guided
By economic logic or theories(which interpretation makes the mostsense in your application?).Both t-test and F-test are enough, other formal tests seldom are usedin reality.Plotting predicted values and use 𝑅2 or 𝑆𝐸𝑅 to make furtherjudgment.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 49 / 76
Logarithms
Summary
We can add polynomial terms of any significant variables to a modeland to perform a joint test of significance. If the additional quadraticsare significant, they can be added to the model.We can also change the variables values into logarithms to capturethe nonlinear relationships.In reality, it can be difficult to pinpoint the precise reason forfunctional form misspecification.Fortunately, using logarithms of certain variables and addingquadratic or cubic functions are sufficient for detectingmany(almost) important nonlinear relationships in economics.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 50 / 76
Interactions Between Independent Variables
Section 6
Interactions Between Independent Variables
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 51 / 76
Interactions Between Independent Variables
Introduction
The product of two variables is called an interaction term.
Try to answer how the effect on Y of a change in an independentvariable depends on the value of another independent variable.
Consider three cases:1 Interactions between two binary variables.2 Interactions between a binary and a continuous variable.3 Interactions between two continuous variables.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 52 / 76
Interactions Between Independent Variables
Interactions Between Two Binary Variables
Assume we would like to study the earnings of worker in the labormarket
The population linear regression of 𝑌𝑖 is
𝑌𝑖 = 𝛽0 + 𝛽1𝐷1𝑖 + 𝛽2𝐷2𝑖 + 𝑢𝑖
the Dependent Variable: log earnings(𝑌𝑖,where 𝑌𝑖 = 𝑙𝑛(𝐸𝑎𝑟𝑛𝑖𝑛𝑔𝑠))Independent Variables: two binary variables
𝐷1𝑖 = 1 if the person graduate from college𝐷2𝑖 = 1 if the worker’s gender is female
So 𝛽1 is the effect on log earnings of having a college degree, holdinggender constant, and 𝛽2 is the effect of being female, holdingschooling constant.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 53 / 76
Interactions Between Independent Variables
Interactions Between Two Binary VariablesThe effect of having a college degree in this specification, holdingconstant gender, is the same for men and women. No reason thatthis must be so.
the effect on 𝑌𝑖 of 𝐷1𝑖, holding 𝐷2𝑖 constant, could depend on thevalue of 𝐷2𝑖there could be an interaction between having a college degree andgender so that the value in the job market of a degree is different formen and women.
The new regression model of 𝑌𝑖 is
𝑌𝑖 = 𝛽0 + 𝛽1𝐷1𝑖 + 𝛽2𝐷2𝑖 + 𝛽3(𝐷1𝑖 × 𝐷2𝑖) + 𝑢𝑖
The new regressor, the product 𝐷1𝑖 × 𝐷2𝑖, is called an interactionterm or an interacted regressor,
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 54 / 76
Interactions Between Independent Variables
Interactions Between Two Binary Variables:
The regression model of 𝑌𝑖 is
𝑌𝑖 = 𝛽0 + 𝛽1𝐷1𝑖 + 𝛽2𝐷2𝑖 + 𝛽3(𝐷1𝑖 × 𝐷2𝑖) + 𝑢𝑖
Then the conditional expectation of 𝑌 𝑖 for 𝐷1𝑖 = 0, given a value of𝐷2𝑖𝐸(𝑌𝑖|𝐷1𝑖 = 0, 𝐷2𝑖 = 𝑑2) = 𝛽0+𝛽1×0+𝛽2𝑑2+𝛽3(0×𝑑2) = 𝛽0+𝛽2𝑑2
Then the conditional expectation of 𝑌 𝑖 for 𝐷1𝑖 = 1, given a value of𝐷2𝑖𝐸(𝑌𝑖|𝐷1𝑖 = 1, 𝐷2𝑖 = 𝑑2) = 𝛽0+𝛽1×1+𝛽2𝑑2+𝛽3(1×𝑑2) = 𝛽0+𝛽1+𝛽2𝑑2+𝛽3𝑑2
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 55 / 76
Interactions Between Independent Variables
Interactions Between Two Binary Variables:
The effect of this change is the difference of expected values,which is
𝐸(𝑌𝑖|𝐷1𝑖 = 1, 𝐷2𝑖 = 𝑑2) − 𝐸(𝑌𝑖|𝐷1𝑖 = 0, 𝐷2𝑖 = 𝑑2) = 𝛽1 + 𝛽3𝑑2
In the binary variable interaction specification, the effect of acquiringa college degree (a unit change in 𝐷1𝑖) depends on the person’sgender.
If the person is male,thus 𝐷2𝑖 = 𝑑2 = 0,then the effect is 𝛽1If the person is female,thus 𝐷2𝑖 = 𝑑2 = 1,then the effect is 𝛽1 + 𝛽3So the coefficient 𝛽3 is the difference in the effect of acquiring acollege degree for women versus men.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 56 / 76
Interactions Between Independent Variables
Application: the STR and the English learners
Let 𝐻𝑖𝑆𝑇 𝑅𝑖 be a binary variable for STR𝐻𝑖𝑆𝑇 𝑅𝑖 = 1 if the 𝑆𝑇 𝑅 > 20𝐻𝑖𝑆𝑇 𝑅𝑖 = 0 otherwise
Let 𝐻𝑖𝐸𝐿𝑖 be a binary variable for English learner𝐻𝑖𝐸𝐿𝑖 = 1 if the 𝑒𝑙𝑝𝑐𝑡 > 10𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝐻𝑖𝐸𝐿𝑖 = 0 otherwise
the result iŝ𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒 =664.1 − 1.9𝐻𝑖𝑆𝑇 𝑅 − 18.2𝐻𝑖𝐸𝐿 − 3.5(𝐻𝑖𝑆𝑇 𝑅 × 𝐻𝑖𝐸𝐿)
(1.4) (1.9) (2.3) (3.1)The value of 𝛽3 here(3.5) means that performance gap in test scoresbetween large class(𝑆𝑇 𝑅 > 20) and small class(𝑆𝑇 𝑅 ≤ 20) varies betweenthe “higher-share-immigrant” class and the “lower-share immigrants or morenative” class.More precisely,positively related with the “higher-share-immigrant” classthough insignificantly.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 57 / 76
Interactions Between Independent Variables
Interactions: a Continuous and a Binary Variable
One binary variable: whether the worker has a college degree
the individual’s years of work experience (𝑋𝑖),the first population model is
𝑌𝑖 = 𝛽0 + 𝛽1𝑋𝑖 + 𝛽2𝐷𝑖 + 𝑢𝑖
the second population model is
𝑌𝑖 = 𝛽0 + 𝛽1𝑋𝑖 + 𝛽2(𝐷𝑖 × 𝑋𝑖) + 𝑢𝑖
the third population model is
𝑌𝑖 = 𝛽0 + 𝛽1𝑋𝑖 + 𝛽2𝐷𝑖 + 𝛽3(𝐷𝑖 × 𝑋𝑖) + 𝑢𝑖
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 58 / 76
Interactions Between Independent Variables
Interactions: a Continuous and a Binary Variable
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 59 / 76
Interactions Between Independent Variables
Application: the STR and the English learners
The estimated interaction regression
̂𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒 = 682.2 − 0.97𝑆𝑇 𝑅 + 5.6𝐻𝑖𝐸𝐿 − 1.28(𝑆𝑇 𝑅 × 𝐻𝑖𝐸𝐿)(11.9) (0.59) (19.5) (0.97)
𝑅2 = 0.305
The value of 𝛽3 here(-1.28) means that the effect of class size on testscores varies between the “higher-share-immigrant” class and the“lower-share immigrants or more native” class.
More precisely,negatively related with the “higher-share-immigrant”class though insignificantly.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 60 / 76
Interactions Between Independent Variables
Interactions Between Two Continuous Variables
Now suppose that both independent variables (𝑋1𝑖 and 𝑋2𝑖) arecontinuous.
𝑋1𝑖 is his or her years of work experience𝑋2𝑖 is the number of years he or she went to school.there might be an interaction between these two variables so that theeffect on wages of an additional year of experience depends on thenumber of years of education.
the population regression model
𝑌𝑖 = 𝛽0 + 𝛽1𝑋1𝑖 + 𝛽2𝑋2𝑖 + 𝛽3(𝑋1𝑖 × 𝑋2𝑖) + 𝑢𝑖
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 61 / 76
Interactions Between Independent Variables
Interactions Between Two Continuous VariablesThus the effect on Y of a change in 𝑋1, holding 𝑋2 constant, is
Δ𝑌Δ𝑋1
= 𝛽1 + 𝛽3𝑋2
A similar calculation shows that the effect on Y of a change Δ𝑋1 in𝑋2, holding 𝑋1 constant, is
Δ𝑌Δ𝑋2
= 𝛽2 + 𝛽3𝑋1
That is, if 𝑋1 changes by Δ𝑋1 and 𝑋2 changes by Δ𝑋2, then theexpected change in Y
Δ𝑌 = (𝛽1 + 𝛽3𝑋2)Δ𝑋1 + (𝛽2 + 𝛽3𝑋1)Δ𝑋2 + 𝛽3Δ𝑋1Δ𝑋2
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 62 / 76
Interactions Between Independent Variables
Application: the STR and the English learners
The estimated interaction regression
̂𝑙𝑛(𝑇 𝑒𝑠𝑡𝑆𝑐𝑜𝑟𝑒) =686.3 − 1.12𝑆𝑇 𝑅 − 0.67𝑃𝑐𝑡𝐸𝐿 + 0.0012(𝑆𝑇 𝑅 × 𝑃𝑐𝑡𝐸𝐿)(11.8) (0.059) (0.037) (0.019)
The value of 𝛽3 here means how the effect of class size on test scores variesalong with the share of English learners in the class.More precisely, increase 1 unit of the share of English learners make theeffect of class size on test scores increase extra 0.0012 scores.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 63 / 76
Interactions Between Independent Variables
Application: the STR and the English learnerswhen the percentage of English learners is at themedian(𝑃𝑐𝑡𝐸𝐿 = 8.85), the slope of the line relating test scores andthe STR is
Δ𝑌Δ𝑋1
= 𝛽1 + 𝛽3𝑋2 = −1.12 + 0.0012 × 8.85 = −1.11
when the percentage of English learners is at the 75thpercentile(𝑃𝑐𝑡𝐸𝐿 = 23.0), the slope of the line relating test scoresand the STR is
Δ𝑌Δ𝑋1
= 𝛽1 + 𝛽3𝑋2 = −1.12 + 0.0012 × 23.0 = −1.09
The difference between these estimated effects is not statisticallysignificant.Because?
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 64 / 76
Interactions Between Independent Variables
Application: STR and Test Scores in a Summary
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 65 / 76
Interactions Between Independent Variables
Application: Jia and Ku(2019)
Ruixue Jia and Hyejin Ku, “Is China’s Pollution the Culprit for theChoking of South Korea?Evidence from the Asian Dust”,TheEconomic Journal.
Main Question: Whether the air pollution spillover from China toSouth Korea and affect the health of South Koreans?
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 66 / 76
https://academic.oup.com/ej/advance-article-abstract/doi/10.1093/ej/uez021/5505829?redirectedFrom=fulltexthttps://academic.oup.com/ej/advance-article-abstract/doi/10.1093/ej/uez021/5505829?redirectedFrom=fulltext
Interactions Between Independent Variables
Jia and Ku(2019)
A naive strategy:Dependent variable: Deaths in South Korea(respiratory andcardiovascular mortality)
Independt variable: Chinese pollution(Air Quality Index)
Because the observed or measured air quality (i.e., pollutionconcentration) in Seoul or Tokyo increases in periods when China ismore polluted does not mean that the pollution must have originatedfrom China.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 67 / 76
Interactions Between Independent Variables
Jia and Ku(2019): Asian Dust as a carrier ofpollutants
Asian Dust (also yellow dust, yellow sand, yellow wind or China duststorms) is a meteorological phenomenon which affects much ofEast Asia year round but especially during the spring months.
The dust originates in China, the deserts of Mongolia, and Kazakhstanwhere high-speed surface winds and intense dust storms kick up denseclouds of fine, dry soil particles.
These clouds are then carried eastward by prevailing winds and passover China, North and South Korea, and Japan, as well as parts of theRussian Far East.
In recent decades,Asian dust brings with it China’s man-made pollutionas well as its by-products.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 68 / 76
Interactions Between Independent Variables
Jia and Ku(2019): Asian Dust
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 69 / 76
Interactions Between Independent Variables
Jia and Ku(2019): Asian Dust
1 A clear directional aspect in that the wind which transport Chinesepollutants to Korea but not vice versa.
2 Exogenous to South Korea’s local activities. And wind patterns andtopography generate rich spatial and temporal variation in theincidence.
3 The occurrence of Asian dust is monitored and recorded station bystation in South Korea.(because of its visual salience)
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 70 / 76
Interactions Between Independent Variables
Jia and Ku(2019): Econometric Method: OLS withinteraction term
Dependent variable: Deaths in South Korea(respiratory andcardiovascular mortality of South Koreans)
Independent variable: Chinese pollution(Air Quality Index in China)
Interaction Variable: Aisan dust(the number of Asian dust days inSouth Korea)
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 71 / 76
Interactions Between Independent Variables
Jia and Ku(2019):Estimation Strategy
the impact of Chinese pollution on district-level mortality thatoperates via Asian dust
𝑀𝑜𝑟𝑡𝑎𝑙𝑖𝑡𝑦𝑖𝑗𝑘 = 𝛽0 + 𝛽1𝐴𝑠𝑖𝑎𝑛𝐷𝑢𝑠𝑡𝑖𝑗𝑘 + 𝛽2𝐶ℎ𝑖𝑛𝑒𝑠𝑒𝑃𝑜𝑙𝑙𝑢𝑡𝑖𝑜𝑛𝑗𝑘+ 𝛽3𝐴𝑠𝑖𝑎𝑛𝐷𝑢𝑠𝑡𝑖𝑗𝑘 × 𝐶ℎ𝑖𝑛𝑒𝑠𝑒𝑃𝑜𝑙𝑙𝑢𝑡𝑖𝑜𝑛𝑗𝑘+ 𝛿1𝑋𝑖𝑗𝑘 + 𝑢𝑖𝑗𝑘
Main coefficient of interest is 𝛽3, which measures _the effect ofChinese pollution in year 𝑦 and month 𝑚 on mortality in district 𝑖 ofSouth Korea.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 72 / 76
Interactions Between Independent Variables
Jia and Ku(2019): the result of interaction terms
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 73 / 76
Interactions Between Independent Variables
Jia and Ku(2019): the result of interaction terms
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 74 / 76
Summary
Section 7
Summary
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 75 / 76
Summary
Wrap up
We extend our multiple ols model form linear to nonlinear in Xs(theindependent variables)
Polynomials,Logarithms and InteractionsThe multiple regression framework can be extended to handleregression functions that are nonlinear in one or more X.the difference between a standard multiple OLS regression and anonlinear OLS regression model in Xs is how to explain estimatingcoefficients.
All are very useful and common tools with OLS regressions. You hadbetter understand it very clear.
Zhaopeng Qu (Nanjing University) Lecture 6: Nonlinear Regression Functions 10/29/2020 76 / 76
Review of previous lectureNonlinear Regression Functions:Nonlinear in XsPolynomials in XLogarithmsInteractions Between Independent VariablesSummary
Recommended