www.***.com

Part2: Chapter 5 Inferring a Binominal Proportion via Exact Mathematical Analysis�

Haru Negami

03/08/2013�

summary�

! binomial proportion " the likelihood function

! the Bernoulli likelihood function " the prior/posterior distribution

! beta distribution

" estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+z) " comparison ---- best model <-p(D|M)

Binomial Proportion�

! dlcW�Gp�W[W ! binomial or dichotomous ! ��

" 50��m 8nKTW.#X&,�6

ahbbC�

p(θ|q) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence

prior probability�

observation�posterior probability�

Likelihood function�

! Example : coin flipping " � y = {0,1} ex) (0:1, 1:2)

" p(y=1|θ)=f(θ)=θA;mθX�8[0,1]S7/n " p(y=0|θ) = 1-θ

" the Bernoulli distribution " p(y|θ) = θy(1-θ)(1-y)

! �MBθX��CyX��SD^C ! *'��SD^C (ΣyAp(y|θ) = 1)�

Likelihood function�

! Bernoulli likelihood function " p(y|θ) = θy(1-θ)(1-y)

" y`��Bθ`��T3^C ! ��9�XθG�O^KTV7/W�`T^C ! gkela��XyKTV:�)U�`�^C

" ��9�X*'��SXUEJTV$�O^C ! p(y|θ) = θy(1-θ)(1-y)(y=i)`θS+�C

ahbbC�

p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence

prior probability�

observation�posterior probability�

done�

belief (to make a model)�

! ��o " p(θ|y) = p(y|θ)p(θ)/Σyp(y|θ) " p(θ)Tp(y|θ)p(θ)G�N��S!I^C

! �N�`(ER50��`�\OJTG�"^C

! ��p " denominator Σyp(y|θ) G4-SH^C

" J_]`%POp(θ)` a conjugate prior for p(y|θ) TEFC

belief (to make a model)�

! p(θ)A= θa(1-θ)b Xgkela��9�WYTQW���*'��9�

>

p(y|θ)×p(θ)A= θy(1-θ)(1-y) × θa(1-θ)b

= θy+a(1-θ)(1-y+b)

! JW�S1L_^*'��9�`beta distributionT�ZC�

belief (to make a model)�

! beta distribution " a, bW2QWfjilcGD^ (a,b > 0) " p(θ|a,b) = beta(θ|a,b) = θ(a-1)(1-θ)(b-1)/B(a,b) ! B(a,b)Xbeta functionT�ZC

" beta distribution W normalizer ! B(a,b) = ∫01dθAθ(a-1)(1-θ)(b-1)

belief (to make a model)�

! Beta Distribution�

b�

a�

ahbbC�

p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence

prior probability�

observation�posterior probability�

done�

belief in detail (prior)�

! beta distribution : beta(θ|a,b)

" two parameters ! mean :

! standard deviation :

belief in detail (prior)�

! beta distribution : beta(θ|a,b) " guess the values of a and b

! from (observed) data " ex) a=b=1, a=b=4, etc…

! m = a/(a+b), n=(a+b)TO^T a = mn, b = (1-m)n

! from mean and standard deviation

<a@1 & b@1 G�EGB a<1 &/or b<1W� [

ahbbC�

p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence

prior probability�

observation�posterior probability�

done� done�

belief in detail(posterior)�! supposition : N flips, z heads ! prior distribution : beta(θ|a,b)

! posterior distribution : beta(θ|z+a, N-z+b)

belief in detail(posterior)�! supposition : N flips, z heads ! prior distribution : beta(θ|a,b)

! posterior distribution : beta(θ|z+a, N-z+b)

one of the beauties of mathematical approach to

Bayesian inference!�

belief in detail (updated parameters)�

! probability distribution : " prior : beta(θ|a,b)

N flips, z heads

" posterior : beta(θ|z+a, N-z+b)

! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)]

= (z+a)/(N+a+b) !!!

belief in detail (updated parameters)�

! probability distribution : " prior : beta(θ|a,b)

N flips, z heads

" posterior : beta(θ|z+a, N-z+b)

! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)]

= (z+a)/(N+a+b) !!!

belief in detail (updated parameters)�

! probability distribution : " prior : beta(θ|a,b)

N flips, z heads

" posterior : beta(θ|z+a, N-z+b)

! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)]

= (z+a)/(N+a+b) !!!

0� z/N� a/(a+b)�

1-α� α�

α = N/(N+a+b)TO^C�

Discussion(?) Three inferential goals�

! from chapter 4 " estimating the binominal proportion " predicting Data " comparing models

estimation�

! uncertainty of the prior distribution " From the posterior distribution

! HDI : the highest density interval (chapter 3)

HDI L : broad R : narrow

prior dist. L : more uncertain�

estimation�

!  reasonable credibility of a value concerned " From the posterior distribution

! ROPE : region of practical equivalence

coin flipping

θ = 0.5 credible?

ROPE = [0.48,0.52] if 95% HDI ∩ ROPEA= = then θ is incredible

estimation�

!  reasonable credibility of a value concerned " From the posterior distribution

! ROPE : region of practical equivalence

coin flipping

θ = 0.5 credible?

ROPE = [0.48,0.52] if 95% HDI ∩ ROPEA= = then θ is incredible

includes many extra assumptions!�

prediction�

! p(y) = ∫dθp(y|θ)p(θ) <-posterior�

prediction�

! p(y) = ∫dθp(y|θ)p(θ) <-posterior�

0� z/N� a/(a+b)�

1-α� α�

α = N/(N+a+b)TO^C�

prediction (example 1)�

! 1st beta(θ|1,1) mean 1/2 (= p(y)) observation : head (N=1,z=1)

! 2nd beta(θ|2,1) mean 2/3 (= p(y)) observation : head (N=1,z=1)

! 3rd beta(θ|3,1) mean 3/4 (= p(y))

prediction (example 2)�

! 1st beta(θ|100,100) : 1/2 (= p(y)) observation : head (N=1,z=1)

observation : head (N=1,z=1)

! 3rd beta(θ|102,100) : 102/202(?50%)

comparison�

to compare the models,

p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence

prior probability�

observation�posterior probability�

comparison�

! Calculation of evidence " p(D|M) = p(z,N)�

comparison�

uniform strongly peaked� uniform strongly peaked�

N = 14, z = 11� N = 14, z = 7�

p(D|M)=0.000183>p(D|M)=6.86×10-5� p(D|M)=1.94×10-5<p(D|M)=5.9×10-5�

comparison�

! both are important " the prior distribution " the likelihood function

" in detail, see chapter 4

The best model (so far) is not a good model.

summary�

! binomial proportion " the likelihood function

! the Bernoulli likelihood function " the prior/posterior distribution

! beta distribution

" estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+b) " comparison ---- best model <-p(D|M)