Upload
osborn-burns
View
230
Download
3
Tags:
Embed Size (px)
Citation preview
Penalized Maximum Likelihood Logistic Regression
Joseph Coveney
Cobridge Co., Ltd.
Topics
• Separation in Logistic Regression
• Approaches to Separation
• Firth’s Bias-reduced GLMs
• firthlogit: syntax and examples
• Caveats and to-do’s
Separation in Logistic Regression
Dataset adapted from D. W. Hosmer and S. Lemeshow, Applied Logistic Regression Second Edition. (New York: John Wiley & Sons, 2000), pp. 138–39.
Complete Separation
Quasi-complete Separation
Dataset adapted from D. W. Hosmer and S. Lemeshow, Applied Logistic Regression Second Edition. (New York: John Wiley & Sons, 2000), pp. 138–39.
Approaches to Separation
• Remove predictors– Pool groups– Remove interaction terms
• Gather more data
• Use alternatives
Exact Logistic Regression
But . . .
Dataset from D. M. Potter. 2005. A permutation test for inference in logistic regression with small- and moderate-sized data sets. Statistics in Medicine 24:693–708.
[19] D. Firth. 1993. Bias reduction in maximum likelihood estimates. Biometrika 80:27–38.
firthlogit
But . . . redux
But . . . redux, continued
0
.2
.4
.6
.8
1A
ctua
l Pro
babi
lity
(LO
WE
SS
Sm
ooth
ed)
0 .2 .4 .6 .8 1Predicted Probability
bandwidth = .8
Profile Likelihood Ratio CIs
Caveats
• Profile Penalized Likelihood CIs
• Small-sample Behavior
G. Heinze and M. Ploner, A SAS macro, S-PLUS library and R package to perform logistic regression without convergence problems. Technical Report 2/2004. Medical University of Vienna. p. 36.
To-do’s
• Profile Penalized Likelihood CIs
• Modify ml d0