Introdccion a WinBUGS

8/9/2019 Introdccion a WinBUGS

1/82

ntro uct on to

,


2/82

r e s ory

1 9 8 9 : project began with a Unix version called BUGS

,

Initially developed by the MRC B i o s t a t i s t i cs U n i t in

Sc h o o l o f M e d i c in e at St Mary's, London.

Windows Ba esian inference Usin Gibbs Sam lin

Software for the Bayesian analysis of complex statistical

models using Markov chain Monte Carlo (MCMC) methods


3/82

Who?

Imperial College Faculty of

Medicine, London (UK)

Th o m a s A n d r e w

University of Helsinki,

Helsinki (Finland)MRC Biostatistics Unit

Institute of Public Health

Da v i d Sp i e g e l h a l t e r

am r ge

Institute of Public HealthCambridge (UK)

Freely downloadable from: http://www.mrc-bsu.cam.ac.uk/bugs/winbugs/contents.shtml


4/82

Key princip e Y o u speci y t e prior an ui up t e i e i oo

W i n BUGS computes the posterior by running a Gibbssamp ng a gor m, ase on:

π(θ|D) / L(D|θ) π(θ)

W i n BUGS computes some convergence diagnostics thato u have to check


5/82

A biolo ical exam le throu houtWhite stork (Ciconia ciconia) in Alsace 1948-1970

Demographic components(fecundity, breeding success,

survival, etc…)(Temperature, rainfall, etc…)


6/82

WinBUGS & Linear Regression15.1 67

13.3 52

15.3 88

2.55

1.85

2.05

13.3 61

14.6 32

15.6 36

2.88

3.13

2.21

Y Numberof chicks

T Temp. May (°C)

R Rainf. May (mm).

13.1 43

15.0 92

11.7 32

.

2.69

2.55

2.84

per pairs

15.3 86

14.4 28

14.4 57

2.47

2.69

2.52

.

11.7 66

11.9 26

15.9 28

.

2.07

2.35

2.9813.4 96

14.0 48

13.9 90

1.98

2.53

2.21

.

15.1 7813.0 87

.

1.782.30


7/82

WinBUGS & Linear Regression

1. Do temperature and rainfall affect the number of chicks?

2. Re ression model:

Yi = α + βr Ri + βt Ti + εi , i=1,...,23

εi i.i.d. ~ N(0,σ2

)

i . . . ~ μ i,σ , = ,...,μ

i

= + r R i + t Ti

3. Estimation of parameters: α, βr, βt, σ

4. Frequentist inference uses t-tests


8/82

Linear Regression using Frequentist approach

15.1 67

13.3 52

15.3 88

2.55

1.85

2.05 13.3 61

14.6 32

15.6 36

2.88

3.13

2.21

Number

of chicks

.R Rainf. May (mm)

... ...

13.0 87

...

2.30

Y = 2.451 + 0.031 T - 0.007 R

Estimate Std. Error t value Pr(>|t|)

rainfall -0.007316 0.002897 -2.525 0.02011 *


9/82

Linear Regression using Frequentist approach

15.1 67

13.3 52

15.3 88

2.55

1.85

2.05 13.3 61

14.6 32

15.6 36

2.88

3.13

2.21

Number

of chicks

.R Rainf. May (mm)

... ...

13.0 87

...

2.30

Y = 2.451 + 0.031 T - 0.007 R

Estimate Std. Error t value Pr(>|t|)

rainfall -0.007316 0.002897 -2.525 0.02011 *

I n f l u e n c e o f Ra i n f a l l o n l y


10/82


11/82

Running WinBUGS

The model use the WinBUGScommand 'model'

We use noninformativeor vague or flat priors

don't forget to embrace

ere

Define the likelihood... 2

Monitor any other parameteri r i t i ,

Note: σ2 = 1/τyou'd like to... e.g. σ2 = 1/τ


12/82

Running WinBUGS

Data and initial values

Use 'list' structures (R/Splus syntax)......and 'vector' structures (R/Splus syntax)


13/82

Running WinBUGS

Overall

1 - a model giving the

2 - data

3 - initial values


14/82

Running WinBUGS

-

At last!!

2- load data3- compile model

-

5- generate burn-in values

6- parameters to be monitored

7- perform the sampling to generate posteriors

8- check convergence and display results


15/82

Running WinBUGS

1. Check model


16/82

Running WinBUGS

1. Check model: highlight 'model'


17/82

Running WinBUGS

1. Check model: open the Model Specification Tool


18/82

Running WinBUGS

1. Check model: Now click 'check model'


19/82

Running WinBUGS

1. Check model: Watch out for the confirmation at the foot of the screen


20/82

Running WinBUGS

2. Load data: Now highlight the 'list' in the data window


21/82

Running WinBUGS

2. Load data: then click 'load data'


22/82

Running WinBUGS

2. Load data: watch out for the confirmation at the foot of the screen


23/82

Running WinBUGS

3. Compile model: Next, click 'compile'


24/82

Running WinBUGS

3. Compile model: watch out for the confirmation at the foot of the screen


25/82

Running WinBUGS

4. Load initial values: highlight the 'list' in the data window


26/82

Running WinBUGS

4. Load initial values: click 'load inits'


27/82

Running WinBUGS

4. Load initial values: watch out for the confirmation at the foot of the screen


28/82

Running WinBUGS

5. Generate Burn-in values: Open the Model Update Tool


29/82

Running WinBUGS

5. Generate Burn-in values: Give the number of burn-in iterations (1000)


30/82

Running WinBUGS

5. Generate Burn-in values: click 'update' to do the sampling


31/82

Running WinBUGS

6. Monitor parameters: open the Inference Samples ool


32/82

Running WinBUGS

6. Monitor parameters: Enter 'intercept' in the node box and click 'set'


33/82

Running WinBUGS. on tor parameters: nter s ope_temperature n t e no e ox an c c set


34/82

Running WinBUGS. on tor parameters: nter s ope_ra n a n t e no e ox an c c set


35/82

Running WinBUGS. enerate poster or va ues: enter t e num er o samp es you want to ta e


36/82

Running WinBUGS. enerate poster or va ues: c c up ate to o t e samp ng


37/82

Running WinBUGS

8. Summarize posteriors: Enter '*' in the node box and click 'stats'


38/82

Running WinBUGS

8. Summarize posteriors: mean, median and credible intervals


39/82

Running WinBUGS

8. Summarize posteriors: 95% Credible intervals

tell us the same story

Estimate Std. Error t value Pr(>|t|)temperature 0.031069 0.054690 0.568 0.57629rainfall -0.007316 0.002897 -2.525 0.02011 *


40/82

Running WinBUGS

8. Summarize posteriors: 95% Credible intervals

tell us the same story

Estimate Std. Error t value Pr(>|t|)temperature 0.031069 0.054690 0.568 0.57629rainfall -0.007316 0.002897 -2.525 0.02011 *


41/82

Running WinBUGS

8. Summarize posteriors: click 'history'

R i Wi BUGS


42/82

Running WinBUGS

8. Summarize posteriors: click 'auto cor'

C i i h l i


43/82

Coping with autocorrelation

use standardized covariates

C i i h l i


44/82

Coping with autocorrelation

use standardized covariates

R i Wi BUGS


45/82

Re-running WinBUGS

1,2,...7, and 8. Summarize posteriors: click 'auto cor'

.

0.0

0.5 1.0

0 20 40

-1.0 -0.5

slope.rainfall

lag

-0.5

0.0 0.5 1.0

la

0 20 40

-1.0

a u t o c o r r e l a t i o n O K

R i Wi BUGS


46/82

Re-running WinBUGS

1,2,...7, and 8. Summarize posteriors: click 'density'

slope.rainfall sample: 1000

6.0

8.0

0.0 2.0 .

- . - . - .

slope.temperature sample: 1000

4.0

6.0 8.0

-0.4 -0.2 0.0 0.2

0.0 2.0

Re r nning WinBUGS


47/82

Re-running WinBUGS

slo e.rainfall

1,2,...7, and 8. Summarize posteriors: click 'quantiles'

-0.3

-0.2 -0.1

-2.77556E-17

iteration

1041 1250 1500 1750

-0.4

slope.temperature

-0.1 0.0 0.1 0.2 0.3

iteration

1041 1250 1500 1750

- .

Running WinBUGS


48/82

Running WinBUGS

8. Checking for convergence using the Brooks-Gelman-Rubin criterion

• A way to identify non-convergence is to simulate

multiple sequences for over-dispersed starting points

• Intuitively, the behaviour of all of the chains should bebasicall the same.

• In other words, the variance within the chains should.

• In WinBUGS, stipulate the number of chains after

' ' ' ' ,sets of initial values as chains have to be loaded, orgenerated)

Running WinBUGS


49/82

Running WinBUGS

8. Checking for convergence using the Brooks-Gelman-Rubin criterion

slope.temperature chains 1:2

1.5

slope.rainfall chains 1:2

1.0

0.0

0.5

.

0.0

0.5

iteration

1 5000 10000

iteration

The normalized width of the central 80% interval of the pooledruns is green

The normalized average width of the 80% intervals within theindividual runs is blue

Re running WinBUGS


50/82

Re-running WinBUGS

1,2,...7, and 8. Summarize posteriors: others...

• Click 'coda' to produce lists of data suitable forexternal treatment via the Coda R package

• Click 'trace' to produce dynamic history changing

Another example: logistic regression


51/82

Another example: logistic regression

15.1 6713.3 52

15.3 88

151 / 173105 / 164

73 / 103

13.3 61

14.6 32

15.6 36

107 / 113

113 / 122

87 / 112

Y Proportion

of nests with

T Temp. May (°C)R Rainf. May (mm)

.

13.1 43

15.0 92

11.7 32

108 / 121

118 / 132

122 / 136

success(>0 youngs)

15.3 86

14.4 2814.4 57

112 / 133

120 / 137122 / 145

.

11.7 66

11.9 26

15.9 28

69 / 90

71 / 80

53 / 67

13.4 96

14.0 48

13.9 90

41 / 54

53 / 58

31 / 39

.

15.1 78

13.0 87

14 / 23

18 / 23

Performing a logistic regression with WinBUGS


52/82




53/82

g g g w W

m o d e l

# succ. in year ~ B n pi , tota # coup es in year

where pi the probability of success in year i

logit(pi )=α + βr Ri + βt Ti, i=1,...,23



54/82

g g g

noninformative priors



55/82

g g g

data & initial values



56/82

g g g

t e resu ts

lower upper

• influence of rainfall, but not temperature (see c r e d i b l e in t e r v a l s )



57/82

g g g

t e resu ts

• additional parameters as a by-product of the MCMC samples: just addthem in the model as parameters to be monitored

- geometric mean:

g e o m < - p o w ( p r o d ( p [ ] ) ,1 / N )

- odds-ratio:

o d d s .r a i n f a l l < - e x p ( s l o p e .r a i n f a l l )

o d d s . t e m p e r a t u r e < - e x p ( s l o p e . t e m p e r a t u r e )



58/82

g g g

t e resu ts

• additional parameters as a by-product of the MCMC samples

- .

- o d d s - r a t i o : -16% for an increase of rainfall of 1 unit

Running WinBUGS from R: R2WinBUGS package


59/82

e og s c regress on examp e rev s e

• It may be uneasy to read complex sets of data and initial values

• It is also quite boring to specify the parameters to be monitored ineac run

• It might be interesting to save the output and read it into R for

• solution 1: WinBUGS can be used in batch mode using scripts

• solution 2: R2WinBUGS allows WinBUGS to be run from R

⇒ create graphical displays of data and posterior simulations or use



60/82


To call WinBUGS from R:

1. Write a WinBUGS model in a ASCII file.

2. Go into R.

' '.

4. A WinBUGS window will pop up amd R will freeze up. The .

in the Log window within WinBUGS. When WinBugs is done, itswindow will close and R will work again.

5. If an error message appears, re-run with 'debug = TRUE'.



61/82

- r e e n co e n a e

# This covers logistic regression model using two explicative variables.# The White storks in Baden-Wurttemberg (Germany) data set is providedmodel

for( i in 1 : N){

nom a s r u on as a e oo n success ~ n p ,n pa rs

# The probability of success is a function of both rainfall and temperaturelogit(p[i])


62/82

- repare e npu s o e ugs unc on

Nbsuccess nbpairs temperature rainfall151 173 15.1 67105 164 13.3 52

.107 113 13.3 61113 122 14.6 32

.

… … … …53 58 14.0 48 .

35 42 12.9 8614 23 15.1 78

.



63/82


# Load R2WinBUGS packagelibrary(R2WinBUGS)

a a s ormaN = 23data = read.table("datalogistic.dat",header=T)

datax = list("N","nbsuccess","nbpairs","temperature","rainfall")

nb.iterations = 10000nb.burnin = 1000



64/82


# Initial valuesinit1 = list(intercept=-1,slope.temperature=-1,slope.rainfall=-1)init2 = list(intercept=0,slope.temperature=0,slope.rainfall=0)

n = s n ercep = ,s ope. empera ure= ,s ope.ra n a =inits = list(init1,init2,init3)nb.chains = length(inits)

# Parameters to be monitoredparameters


65/82


# Summarize resultsres.sim$summary # use print(res.sim) alternatively

mean s . aIntercept 1.55122555 0.05396574 1.55100 1.6589750 1.0020641slope.temperature 0.03030854 0.06128879 0.03148 0.1510975 0.9997848s ope.ra n a - . . - . - . .

deviance 204.60259481 2.48898337 203.90000 211.2000000 1.0002280

Running WinBUGS from R: R2WinBUGS packagep


66/82


# Numerical summaries for slope.rainfall?quantile(slope.rainfall,c(0.025,0.975))

. .-0.2693375 -0.0243025

a cu a e e o s-ra o

odds.rainfall


67/82


# Graphical summaries for slope.rainfall?plot(density(slope.rainfall),xlab="",ylab="", main="slope.rainfall a posteriori density")

Recent developments in WinBUGSM d l l ti i RJMCMC


68/82

Model selection using RJMCMC

We consider data relating to population of Whitestorks breeding in Baden Württemberg (Germany).

Interest lies in the impact of climate variation(rainfall ) in their wintering area on their populationynam cs a u surv va ra es .

Mark-recapture data from 1956-71 are available.

T e covariates re ate to t e amount o rain abetween June-September each year from 10

. Interest lies in identifying the given rainfall

.


69/82

Bayesian Mo e Se ection Discriminating between different models

can often be of particular interest, sincethey represent competing biologicalhypotheses.

How do we decide which covariates to use?– often there may be a large number ofpossible covariates.


70/82

Examp e (cont) We express the survival rate as a logistic function

of the covariates:

og t = μ x t εt where x t denotes the set of covariate values at

t ,

εt ~ N(0,σ2). ,

rates for the adults?

-,


71/82

Mo e Se ection In the classical framework, likelihood ratio tests

or information criterion (e.g. AIC) are often used.

T ere is a simi ar Bayesian statistic – t e DI .

This is programmed within WinBUGS – howeveru r r r

models (e.g. random effect models). ,

results in even simple problems.

,a more natural way of dealing with the issue ofmodel discrimination.


72/82

Bayesian Approac We treat the model itself as an unknown

parameter to be estimated.

en, app y ng ayes eorem we o a n eposterior distribution over both parameter and

π(θm, m | data) Ç L(data | θm, m) p(θm) p(m). θ

θm .

LikelihoodPrior on parametersin model m Prior on model m


73/82

Posterior Mo e Pro a i ities The Bayesian approach then allows us to

quantitatively discriminate between competing

π(m | data) = s π(θm, m | data) dθm

Ç p(m) s L(data | θm, m) p(θm) dθm

Note that we need to specify priors on both theparameters and now also on the modelsthemselves.

Thus we need to specify a prior probability foreach model to be considered.


74/82

MCMC- ase estimates We have a posterior distribution (over parameter

and model space) defined up to proportionality:

π m, m a a a a m, m p m m p m

If we can sample from this posterior distribution

summary statistics of interest.

be estimated as the proportion of time that thechain is in each model.

So, all we need to do is define how we constructsuch a Markov chain!!


75/82

Reversi e Jump MCMC The reversible jump MCMC algorithm allows us to

construct a Markov chain with stationary

. It is simply an extension of the Metropolis-

different dimensions.

parameters,θm, in model m, may differ between

models.

Note that this algorithm needs only one Markovchain irrespective of the number of models!!


76/82

Mar ov c ain ac era on o e ar ov c a n essen a y

involves two steps:

– . using standard MCMC moves (Gibbs sampler,Metropolis-Hastings)

. – a reversible jump type move.

Then, standard MCMC-type processes apply,such as using an appropriate burn-in, obtainingsummary statistics of interest etc.

,particular example relating to variable selection(e.g. covariate analysis).


77/82

inBUGS General RJ updates cannot currently be

programmed into WinBUGS.

espo e co e nee s o e wr en ns ea .

However, the recent add-on called “jump” allows

WinBUGS: Variable selection

Splines

See http://www.winbugs-development.org.uk/rjmcmc.html

So, in particular, WinBUGS can be used for modelselection in the White storks example.


78/82


79/82

Examp e: W ite Stor s WinBUGS demonstration.

RJMCMC is performed on the beta’s only,


80/82

Resu tsModels with largest posterior support

. .0000000000 0.07629791141 0.59410374110001010000 0.0474087675 0.6415125086

. .0001000001 0.03085379849 0.71050722971001000000 0.02549001607 0.7359972458

. .0001001000 0.02336699564 0.7831099380001000010 0.0229240303 0.8060339683

. .0101000000 0.01809960982 0.84536837270011000000 0.01540739041 0.86077576311000000000 0.01186825798 0.87264402110000010000 0.01103282075 0.8836768419


81/82

Resu ts Additionally the (marginal) posterior probability that each covariate influencethe survival rates:

node mean sd posterior marg probeffect[1] 0.01 0.04 0.06

. . .effect[3] 0.00 0.02 0.03

effect[4] 0.30 0.17 0.83-

effect[6] 0.01 0.05 0.09effect[7] -0.00 0.03 0.04effect 8 0.01 0.06 0.07effect[9] 0.01 0.04 0.05effect[10] -0.01 0.04 0.05p 0.91 0.01sdeps 0.20 0.14


82/82

Resu ts: surviva rates

1 . 0

urv va ra e or w e s or s

0 . 8

**

*

*

*

*

**

- - -

- -

-

--

-

-

-

-

-

-

-

- -

-

-

. 4

0 . 6

u r v i v a

l r a t e

*

* * * * *

* *

-

--

--

--

-

-

- -

- -

0 . 2

0

*

*

*

* * *

*

** *

*

*

* * *

0 . 0

*

1960 1965 1970

Year

Documents

Introdccion a WinBUGS