Extension to Multiple Regression

Simple regression

• With simple regression, we have a single predictor and outcome, and in general things are straightforward though issues may arise regarding outliers and violations of assumptions

• The basic model is

Y = b0 + b1X + e

Multiple Regression

• Adding more predictors sounds simple enough, and, true enough, the basic model doesn’t change much

Y = b0 + b1X + b2X + b3X … + bnX + e

• The key idea here is that we are getting a linear combination of the predictors1 and assessing the correlation of that combination with the outcome– Squaring that correlation provides our model R2

• However there is much more to deal with

Prediction and Explanation• Focus of the analysis can be seen as falling somewhere on two

continuums of prediction and explanation – Low Prediction-----------------------------------High Prediction– Low Explanation-----------------------------------High Explanation

• Example of high prediction/low explanation– Model R2 must be very strong, exclusive focus on raw coefficients, little

concern about variable importance, and actual prediction on a new set of data

• Example of high explanation/low prediction– Model R2 can even be weak but at least statistically significant, focus on

standardized coefficients and variable importance, little if any validation• Most of psych research tends to fall in low prediction, high

explanation• This is not a good thing as it leads to satisfaction with models that

may be marginally useful at best

Macro Analysis

• Model fit– Statistical significance– Amount of variance accounted for in the DV (R2)– Standard error of estimate

• The interpretation is no different with the addition of predictors

• However, do know that as we have noted, there is never a zero correlation between variables unless forced (e.g. via experimental design)

• As such, in practice adding a variable will always increase R2

• A bias-adjusted R2 becomes even more important to report as model complexity increases.

Micro Analysis

• Raw coefficients• Standardized coefficients

– As if we ran the model after standardizing our predictors first

– This puts them and their subsequent coefficients on the same scale for easier interpretation

– However, just because one is larger than another doesn’t mean it is statistically or meaningfully so

• Interval estimates for coefficients• Measures of variable importance

– E.g. semi-partial correlation

Problems

• Issues with outliers, violations of the assumptions, and overfitting remain and the integrity of the model must be thoroughly examined

• Furthermore, one must be on the look out for things like collinearity (high correlations among predictors) and suppression (unusual coefficients due to predictor-DV relationships)

Multiple Approaches

• Sequential (hierarchical) regression

• Exploratory (stepwise) regression

• Tests of interactions (moderation)

• Mediating effects

• Model comparison

Summary• Most of the regressions you see will exhibit these types of problems:

– Focus on explanation at the expense of prediction– Lack of or inadequately tested assumptions– Lack of bias-adjusted model fit indices– Lack of comparison of results to robust regression nor validation of their

own model (much less comparison it to other theoretically motivated possible models)

– Inadequately examine differences among predictors– Poorly perform exploratory approaches when doing them– Are a product of someone not doing more appropriate and complex

analysis (e.g. path analysis)• In short, while commonly used, MR is also usually poorly performed• Try and do better!