Regression Analysis1 2 INTRODUCTION TO EMPIRICAL MODELS LEAST SQUARES ESTIMATION OF THE PARAMETERS PROPERTIES OF THE LEAST SQUARES ESTIMATORS AND ESTIMATION OF s 2 HYPOTHESIS TESTING IN LINEAR REGRESSION CONFIDENCE INTERVALS IN LINEAR REGRESSION PREDICTION OF NEW OBSERVATIONS ASSESSING THE ADEQUACY OF THE REGRESSION MODEL CHAPTER OUTLINE Regression Analysis3 Definitions Regress The act of reasoning backward Regression A functional relationship between two or more correlated variables that is often empirically determined from data and is used esp. to predict values of one variable when given values of the others. Regression Analysis4 Models Abstraction/simplification of the system used as a proxy for the system itself Can try wide-ranging ideas in the model Make your mistakes on the computer where they don t count, rather for real where they do count Issue of model validity Two types of models Physical (iconic) Logical/Mathematical -- quantitative and logical assumptions, approximations Regression Analysis5 What Do You Do with a Logical Model? If model is simple enough, use traditional mathematics (queueing theory, differential equations, linear programming) to get answers Nice in the sense that you get exact answers to the model But might involve many simplifying assumptions to make the model analytically tractable -- validity?? Many complex systems require complex models for validity simulation needed Regression Analysis6 models theoretical (mechanical) model empirical model scatter diagram INTRODUCTION TO EMPIRICAL MODELS Regression Analysis7 8 9 linear model (equation) probabilistic linear model simple linear regression model regression coefficients Regression Analysis10 multiple regression model multiple linear regression model intercept partial regression coefficients contour plot Regression Analysis11 dependent variable or response y may be related to k independent or regressor variables interaction any regression model that is linear in parameters (the bs) is a linear regression model, regardless of the shape of the surface that it generates. Regression Analysis12 Regression Analysis13 Regression Analysis14 LEAST SQUARES ESTIMATION OF THE PARAMETERS Simple Linear Regression Regression Analysis15 method of least squares least squares normal equations fitted or estimated regression line residual Regression Analysis16 Regression Analysis17 Example 10-1, pp. 436 Regression Analysis18 Regression Analysis19 Regression Analysis20 Multiple Linear Regression Regression Analysis21 Regression Analysis22 Regression Analysis23 Regression Analysis24 PROPERTIES OF THE LEAST SQUARES ESTIMATORS AND ESTIMATION OF s 2 unbiased estimators covariance matrix estimated standard error residual mean square (or error mean square) Regression Analysis25 Hypothesis Testing on 0 and 1, pp. 447 Regression Analysis26 HYPOTHESIS TESTING IN LINEAR REGRESSION Regression Analysis27 Regression Analysis28 * k = p - 1 Regression Analysis29 Regression Analysis30 Tests on Individual Regression Coefficients Regression Analysis31 Confidence Intervals on Individual Regression Coefficients Regression Analysis32 Regression Analysis33 Confidence Interval on the Mean Response Regression Analysis34 Regression Analysis35 Regression Analysis36 PREDICTION OF NEW OBSERVATIONS Regression Analysis37 Regression Analysis38 simple linear regression Regression Analysis39 Regression Analysis40 ASSESSING THE ADEQUACY OF THE REGRESSION MODEL normal probability plot of residuals standardize outlier Regression Analysis41 Regression Analysis42 Regression Analysis43 Regression Analysis44 Regression Analysis45 Regression Analysis46 Regression Analysis47 Regression Analysis48 Regression Analysis49 Coefficient of Multiple Determination Regression Analysis50 Regression Analysis51 Influential Observations Regression Analysis52 Regression Analysis53 Regression Analysis54