Two Types of Empirical Likelihood

Zheng, YanDepartment of BiostatisticsUniversity of California, Los Angeles

Introduction

For the statistics of mean, we can define two types of EL for right censored data Expression by CDF

Expression by hazard rate

ii

ii xFxpEL

1)](1[)(

])(exp[)(

ij

ii

i xhxhAL

Introduction

In the context of tests, the two types have corresponding null hypothesis

Where is a constant and pi’s are positive and sum to 1.

Where is a constant.

)()(:0 ii xpxgH

i

iiKMEi xhxSxgH )()(ˆ)(: 10

Differences between two types

The expression for S(x).

The variation in constraints When h(t) is NAE, two types’ constraints are

same.

ij

ji xhxS )(1)(

for continuous distributions )()( ixH

i exS

for discrete distributions

i i

iiiKMEiiKMEi xpxgxSxpxSxg )()()(ˆ/)()(ˆ)( 11

Similarities between two types

Asymptotically, they both converge to a 1-degree-of-freedom chi-square distribution under the null hypothesis.

Point estimation of theta is same when the hazard function is the NAE.

Questions

Which type outperforms in the confidence interval coverage and chisquare approximation?

Which type has a narrower confidence interval?

What’s the impact of continuous and discrete distribution on their performance?

Empirical Likelihood Ratio Theorem 1

For the right censored data with a F, suppose the constraint equation is , where

is the true value. When n goes to infinity,

where the constant

)()( tdFtg

02

)1(*0 ),(log2 krkELR

2

2

2

2

)(

)()(1

)()(

)1(*)ˆ(.

gkdF

kdFtdGtF

sdFsk

dFGkFgdVarAsy

r

tKM

k

Empirical Likelihood Ratio Under the maximization

of is more complicated than straight use of Lagrange multiplier. The application usually doesn’t give the simple solution for pi..

A modified EM/self-consistent algorithm is proposed.

ii

ii xFxpEL

1)](1[)(

)()(:0 ii xpxgH

EM Algorithm in ELR

Theorem 2 The maximization of w.r.t. p i s.t. the

two constraints: and is given by

where satisfies

ii pw log

1 ip ii pxg )(

j

ij

ii

xgw

wp

))((

0))((

))((

i

j

ij

ii

xgw

xgw

EM Algorithm in ELR E-Step: Given F, the weight wj at location tj can

be computed as

where tj is either a jump point for the given distribution F or an uncensored observation.

The wj is zero at other locations.

EF=

i

iijiXFj ZtIEw ],|[ ][

&1

&0

&0

000

)(1

)(

Else

Zt

Zt

Zt

ZF

tF

iji

iji

iji

i

j

EM Algorithm in ELR

M-Step: With the uncensored pseudo observations X=tj and weights wj from E-step, we then find the probability pj by using Theorem 2. Those probabilities give rise to a new distribution F.

A good initial F to start the EM is the NPMLE without the constraint. For right censored data, KME will be the choice.

ELR Computation

Suppose is the NPMLE from EM algorithm under H0 and is the NPMLE without any constraint,

Find the p-value by Chi square distribution. Thus we can test the hypothesis and construct the confidence intervals.

p̂p~

))]ˆ(log())~([log(2log2 pLpLELR

Poisson Extension of the L

AL is a function of hazard function

Linear Constraint

Notice we have used a formula for S(t)=exp(-H(t)) that is only valid for continuous distribution in the case of a discrete distribution. The difference is small for large n.

])(exp[)(

ij

ii

i xhxhAL

i

iiKMEi xhxSxgH )()(ˆ)(: 10

MLE for AL Apply Lagrange multiplier, we get

Where the is the solution to

))(ˆ)(/()(ˆ)( 11 iKMEiiiiKMEi xSxgYDxSxg

)(ˆ)()(ˆ)(

logˆlog11

0

iKMEii

ii

iKMEii

iH

xSxgY

YD

xSxgY

DLA

ii

i DY

DLA logˆlog

)(ˆ)(

)(ˆ)(2

)(ˆ)(log22

1

1

1 iKMEii

iKMEii

iKMEii

i

xSxgY

xSxgD

xSxgY

YLLR

ALR Properties Notice the summation are only over the

uncensored locations. The last jump of a discrete cumulative hazard

function must be one if survival function decreased to zero at last point.

ALR has an asymptotic 1-df Chi-square distribution when the constraint is

We can construct confidence interval for by chi-square distribution of -2LLR

)()( tdHtg

Simulation for Continuous Cases

Suppose our F=1- e-t, G=1- e-0.035t. Xi=Min{Ti,Ci}, di=1 if Xi<=Ci and di=0 else. Si=KME= and S0=1 where

Di=sum(di) at same xi. Suppose g(x)=e-x,

Sample size=50

)1(*1i

ii

Y

DS

5.0)()](1)[(0 dteetdHtFtg tt

Simulation for Continuous Cases QQ-plot for ELR QQ-plot for ALR

0 2 4 6 8 10 12 140

2

4

6

8

10

12

14

16

18

qchisq(1:1000/1001,df=1)

-2LL

R

-2LLR of ALR with sum(g*h(t))=muq-chisquare

0 2 4 6 8 10 12 140

2

4

6

8

10

12

14

qChi-square (1:1000/1001, df=1)

-2LL

R

-2LLR of ELRquantile of chi-square

Simulation for Continuous Cases

QQ-plot for ALR

with constraint of

)(ˆ)(ˆ)( 1 iiKMEi xhxSxg

0 2 4 6 8 10 120

2

4

6

8

10

12

14

chisq(1:1000/1001, df=1)

-2LL

R

-2LLR of ALRChi Square

Simulation for Continuous Cases

Confidence Interval Coverage For 1000 runs, ELR gives 949 confidence

intervals covering 0.5 and ALR gives 1000 confidence intervals covering 0.5.

The above observation indicated that ALR has wider confidence interval than ELR

Simulation for Continuous Cases Plots of -2LLR vs. Mu

0.2 0.4 0.6 0.8 1 1.20

5

10

15

20

25

30

35

true

-2L

LR

-2logALRchisq=3.84-2logELR

Simulation for Continuous Cases

Confidence Intervals from ALR

0 200 400 600 800 10000.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sample Number

l

u

Simulation for Discrete Cases

Suppose F=poisson(5), G=poisson(7) Xi=Min{Ti,Ci}, di=1 if Xi<=Ci and di=0 else.

Si=KME= and S0=1 where Di=sum(di) at same xi.

Suppose g(x)=x,

Sample size=50, 1000 runs

)1(*1i

ii

Y

DS

0067.55!

5 55

0

ei

ei

i

i

Simulation for Discrete Cases QQ-plot for ALR

0 2 4 6 8 10 120

2

4

6

8

10

12

14

16

18

chisq(1:985/986, df=1)

-2L

LR

*8

-2LLR of ALRChi Square

Simulation for Discrete Cases

Confidence Intervals: In 1000 runs, 985

confidence intervals covered Mu0=5.0067.

0 2 4 6 8 100

1

2

3

4

5

6

7

8

9

-2LL

R

-2logALRchisq=3.84

Conclusions

ELR has narrower confidence interval than ALR which indicates that LRT for ELR should be more powerful than LRT for ALR at same alpha level.

On the other hand, the ALR has more accurate coverage on true Mu than ELR.

They had similar performance on approximation to Chi Square distribution when n is large.

Conclusions

Discrete cases have comparable performance on chi-square approximation and confidence interval coverage as continuous cases when sample size is rather large. However, theoretical insight explores that ALR’s perform will be inferior to the ELR in the discrete cases.