Bayesian Analysis for Extreme Events

Pao-Shin Chu and Xin ZhaoDepartment of Meteorology

School of Ocean & Earth Science & Technology University of Hawaii-

Manoa

Why Bayesian inference?

• A rigorous way to make probability statements about the parameters of interest.

• An ability to update these statements as new information is received.

• Recognition that parameters are changing over time rather than forever fixed.

• An efficient way to provide a coherent and rational framework for reducing uncertainties by incorporating diverse information sources (e.g., subjective beliefs, historical records, model simulations). An example: annual rates of US hurricanes (Elsner and Bossak, 2002)

• Uncertainty modeling and learning from data (Berliner, 2003)

Some applications of Bayesian analysis for climate research

• Change-point analysis for extreme events (e.g., tropical cyclones, heavy rainfall, summer heat waves) Why change-point analysis?

• Tropical cyclone prediction (Chu and Zhao, 2007, J. Climate; Lu, Chu, and Chen, 2010, Weather & Forecasting, accepted)

• Clustering of typhoon tracks in the WNP (Chu et al., 2010, Regional typhoon activity as revealed by track patterns and climate

change, in Hurricanes and Climate Change, Elsner et al., Eds., Springer, in press)

Other Examples

• Predicting climate

variations (e.g.,

ENSO)

• Quantifying

uncertainties in

projections of future

climate change

θ : parameter Classical statistics: θ a constant Bayesian inference: θ a random quantity, P(θ|y)

Bayes’ theorem

y: data

θ

θθθ|y

θθ|yyθ

dP

PP

)()(

)()()|(

P(y|θ): likelihood function π(θ): prior probability distribution

Change-point analysis for tropical cyclones

• Given the Poisson intensity parameter (i.e., the mean seasonal TC rates), the probability mass function (PMF) of tropical cyclones occurring in T years is

• where and , . The λ is regarded as a random variable, not a constant.

!

)()exp(),|(h

TTThP

h

,...2,1,0h 0 0T

• Gamma density is known as a conjugate prior and posterior for λ. A functional choice for λ is a gamma distribution

• where λ>0, h´ >0, T´>0. h´ and T´ are prior parameters.

' ' 1'( | ', ') exp( ')

( ')

h hTf h T T

h

• The PDF of h tropical cyclones in T years when the Poisson intensity is codified as a gamma density with prior parameters T’ and h’ is a negative binomial distribution (Epstein, 1985)

0

)','|(),|(),','|( dThfThPTThhP

'( ') '

( ') ! ' '

h hh h T T

h h T T T T

,...1,0h 0'h ' 0T 0T

A hierarchical Bayesian tropical cyclone model

ih

i

ni ,...,2,1

'h 'T

adapted from Elsner and Jagger (2004)

Hypothesis model for change-point analysis (Consider 3 hypo.)

H0

H1

H2

(1) Hypothesis 0H : “A no change of the rate” of the typhoon series:

),|(~ 0 ThPoissonh ii , ni ,...,2,1 . )','(~ 000 Thgamma where the prior knowledge of the

parameters '0h and '0T is given. T = 1.

(2) Hypothesis 1H : “A single change of the rate” of the typhoon series:

),|(~ 11 ThPoissonh ii , when 1,...,2,1 i ),|(~ 12 ThPoissonh ii , when ni ,..., n,...,3,2 , and ),(~ '

11'1111 Thgamma

),(~ '12

'1212 Thgamma

where the prior knowledge of the parameters '11h , '

11T , '12h , '

12T is given. There are two epochs in this model and is defined as the first year of the second epoch, or the change-point.

Markov Chain Monte Carlo (MCMC) approach• Standard Monte Carlo methods produce a set of

independent simulated values according to some probability distribution.

• MCMC methods produce chains in which each of the simulated values is mildly dependent on the preceding value. The basic principle is that once this chain has run sufficiently long enough it will find its way to the desired posterior distribution.

• One of the most widely used MCMC algorithms is the Gibbs sampler for producing chain values. The idea is that if it is possible to express each of the coefficients to be estimated as conditioned on all of the others, then by cycling through these conditional statements, we can eventually reach the true joint distribution of interest.

Θ = [θ1,θ2,…θp]

• Gibbs Sampler (We can generate a value from the conditional distribution for one part of the θ given the values of the rest of other parts of θ; it involves successive drawing from conditional posterior densities P(θk |h, θ1,…,θk-1,θk+1,…,θp) for k from 1 to p)

Bayesian inference under each hypothesis

With the prior knowledge, we can apply the Gibbs sampler to draw samples from the posterior distribution of the model parameters under each respective hypothesis.

Hypothesis Analysis

• Under uniform prior assumption for hypothesis space( | ) ( | )P H P Hh h

]',,,,,,,,[ 2123222112110 θ

( | ) ( )( | )

( | ) ( )H

P H P HP H

P H P H

hh

h

( | ) ( | , ) ( | )P H P H P H dθh h θ θ θ

[ ]

1

1( | ) ( | , )

Ni

i

P H P HN

h h θ )|(~ HP θθ

N

• Annual major hurricane count series for the ENP

• P(H2 |h) = 0.784

• P(H1|h) = 0.195

• P(H0|h) = 0.021

• = 1982 and = 1999, 3 epochs 21

Why RJMCMC?

• Because parameter spaces within different hypotheses are typically different from each other, a simulation has to be run independently for each of the candidate hypotheses.

• If the hypotheses have large dimension, the MCMC approach is not efficient.

• Green (1995)

Reversible jump sampling for moving between spaces of differing dimensions

• A trans-dimensional Markov chain simulation in which the dimension of the parameter space can change from one iteration to the next

• Useful for model or hypothesis selection problems

4 different gamma models (4 epochs with 3 change-points)

151 301 401

4 different gamma models (3 changepoints)

Prior specification

1 2

0 1 0 1

With time series =[ , , ... , ]', we run L independent

iterations. Within the -th iteration, 1 L, we randomly

pick two different points from 1 to n, say, and ( < ).

T

h nh h h

j j

k k k k

1

0

0 1

[ ]1 0

hen we calculate the sample mean of this batch of samples

{ , }, obtaining a realization of the Poisson rate

of this iteration, 1/ ( 1) .

In the end, we obtain

i

kjii k

h k i k

k k h

[ ]

2

[ ] 2 [ ] 2

1 1

a set of samples, { , 1 L}.

' / and ' * '

1 1 where and ( ) .

1

j

L Lj j

j j

j

T m s h m T

m s mL L

Extreme rainfall events in Hawaii

Summary• Why Bayesian analysis• Applications for climate research (extreme events

and climate change)• Change-point analysis

Mathematical model of rare event count series

Hypothesis model

Bayesian inference under each hypothesis

Major hurricane series in the eastern North Pacific

• Recent Advance (RJMCMC)