A (poor) Gibbs Sampling Approach to Logistic Regression
Kyle BogdanGrant Brown
Data
Simulated based on known values of parameters (one covariate, ‘dose’).
‘rats’ given different dosages of imaginary chemical, 4 dose groups with 25 rats in each group.
Data generated three times under different parameters, three chains used for each data set.
Gibbs Sampling For Logistic Data?
Traditionally, binomial likelihood, prior on logit.
Full Conditionals have no coherent form.
Attractive, however, because it eliminates the need to reject iterations
Algorithm
Groenewald and Mokgatlhe, 2005 Create Uniform Latent Variables
Based on Y[i,j] = 0, 1 Draws from joint posterior of Betas
and U[i,j] pi[i] = p(uniform(01) <= logit-
1(Beta*x[i])) Written in R, refined in Python Very inefficient
Draw new parameter for each Y[i,j] at each iteration
Implementation
• Three datasets• Three chains per set• 1 Million iterations per chain• Last 500k iterations sent to CODA• 9m total iterations, 4.5 m analyzed
Initial Problems
Sampler Output/Diagostics
Sampler Output/Diagnostics
Sampler Output/Diagnostics
Sampler Output/Diagnostics
Sampler Output/Diagnostics
WinBUGS Model
Y[i,j]’s given binomial (instead of Bernoulli) likelihood
Betas regressed on logit of proportion
Locally uniform priors on beta1 and beta2
WinBUGS Model
model{for (i in 1:N){ r[i] ~ dbin(p[i], n[i]); logit(p[i]) <- (beta1 + beta2*(x[i] - mean(x[]))); r.hat[i] <- (p[i] * n[i]); }beta1 ~ dflat();beta2 ~ dflat();beta1nocenter <- beta1 - beta2*mean(x[]);}
WinBUGS Output: Beta0 (1,0)
WinBUGS Output: Beta0 (1,1)
WinBUGS Output: Beta0 (1,-2)
Comparison
WinBUGS Wins
Uses proportions instead of Individual Y[i,j]’s
Convergence is Better WinBUGS appears more precise
(more trials needed) Also, much faster.
Resources
Groenewald, Pieter C.N., and Lucky Mokgatlhe. "Bayesian computation for logistic regression.“ Computational Statistics & Data Analysis 48 (2005): 857-68. Science Direct. Elsevier. Web. <http://www.sciencedirect.com/>.
Professor Cowles