STAT 444/844 and CM 464/764:Statistical Learning: Function Estimation

Winter 2013

Instructor: Shojaeddin ChenouriOffice: M3 3124Instructors office hours: MW 11:00 am -12:00 pm.Teaching Assistant: Garcia (Jiaxi) Liang. His office hours are on Fridays 3:30-4:30 in M3 3108 space 1. Aweek prior to assignment dues or exam, his office hours are Fridays 3:30 - 5:30.Lectures: MWF 14:30 -15:20 in Room: MC 1056Evaluation: Assignments (40%), Midterm Exam or Group Project (TBD) (15%), Final Exam (45%).Prerequisites: CM 361/STAT 341 or STAT 331 or STAT 361 or STAT 371

TENTATIVE COURSE OUTLINE

1. Basic concepts: A review on multiple linear regression, nonlinear regression, the bias-variance tradeoff,integrated risk, integrated mean square error, kernels, loss functions, confidence sets, curse of dimension-ality.

2. Nonparametric regression: Linear smoothers, local regression, penalized regression, regularization,smoothing splines, wavelets, additive models, local likelihood and exponential families, regression trees,confidence bands.

3. Nonparametric density estimation: histograms, kernel density estimation, local polynomials, wavelets,multivariate density estimation, confidence bands.

4. Nonparametric intensity estimation:

5. Models assessments: cross validation, bandwidth selection, tuning parameters.

6. High dimensional problems: Variable selection in multiple regression, ridge regression, garrotte, lasso,LARS, Dantzig selector, adaptive lasso, elastic net, group lasso etc.

Some References:

1. Berk, R. A. (2008). “Statistical Learning from a Regression Perspective”. Springer, New York

2. Buhlmann, P. and van de Geer (2011). “Statistics for High-Dimensional Data: Methods, Theory andApplications”. Springer, New York.

3. Clarke, B., Fokoue, E. and Zhang, H. H. (2009). “Principles and Theory for Data Mining and MachineLearning, Springer, New York.

4. Efromovich, S. (1999). “Nonparametric Curve Estimation”. Springer, New York.

5. Green, P. J. and Silverman, B. W. (1994). “Nonparametric Regression and Generalized Linear Models: Aroughness penalty approach”. Chapman & Hall.

6. Hastie, T., Tibshirani, R. and Friedman, J. (2009) “The Elements of Statistical Learning”, 2nd edition,Springer, New York.

7. Scott, D. W. (1992). “Multivariate Density Estimation: Theory, Practice and Visualization”. Wiley, NewYork.

8. Silverman, B. W. (1986). “Density Estimation for Statistics and Data Analysis”, Chapman & Hall

9. Wasserman, L. (2006) “All of Nonparametric Statistics”, Springer, New York.

10. Wahba, G. (1990). “Spline Models for Observational Data”, SIAM, Philadelphia.

11. Wang, Y. (2011), “Smoothing Splines: Methods and Applications”. CRC Press.

12. Venables, W. N. and Ripley (2002). “Modern Applied Statistics with S.” 4th edition, New York, Springer.