Berger the Likelihho Principle 1988 Book

7/27/2019 Berger the Likelihho Principle 1988 Book

1/252

Institute of Mathematical StatisticsLECTURE NOTES-MONOGRAPH SERIES

The Likelihood PrincipleSecond Edition)James O. Bergerurdue niversityRobert L. Wolpertuke niversity


2/252


3/252

T H E L I K E L I H O O D P R I N C I P L E :A R E V I E W , G E N E R A L I Z A T I O N S , A N D S T A T I S T I C A L I M P L I C A T I O N S

S E C O N D E D I T I O N


4/252


5/252

InstituteofMathematical StatisticsLECTURE NOTES-MONOGRAPH SERIESShantiS Gupta, Series EditorVolume6

The Likelihood PrincipleSecond Edition)JamesO BergerPurdue University

RobertL Wolpert uke University

InstituteofMathematical StatisticsHayward,C alifornia


6/252

Institute of Mathem atical S tatisticsLecture Notes-Monograph Series

Series Editor Shanti S. Gupta Purdue University

The production of the IMS Lecture Notes-Monograph Series isman aged by the IM S Business Office: Nicholas P. Jew ell IMSTreasurer and Jose L. Gonzalez IMS Business Manager.

Library of Congress Catalog Card Number: 88-81456International Standard Book Number 0-940600-13-7Copyright 1988 Institute of Mathem atical Statistics

All rights reservedPrinted in the United States of Am erica


7/252

T o m y p a r e n t s , O r v i s a n d T h e l m a

J a m e s B e r g e r

T o m y w i f e , R u t a

R o b e r t W o l p e r t


8/252


9/252

P R E F E

T h i s m o n o g r a p h b e g a n w i t h r e s e a r c h d e s i g n e d t o p r o v i d e a g e n e r a l i -z a t i o n o f t h e L i k e l i h o o d P r i n c i p l e L P ) t o q u i t e a r b i t r a r y s t a t i s t i c a ls i t u a t i o n s . T h e p u r p o s e o f s e e k i n g s u c h a g e n e r a l i z a t i o n w a s t o p a r t i a l l ya n s w e r c e r t a i n c r i t i c i s m s t h a t h a d b e e n l e v i e d a g a i n s t t h e L P , c r i t i c i s m s w h i c hs e e m e d t o p r e v e n t m a n y s t a t i s t i c i a n s f r o m s e r i o u s l y c o n s i d e r i n g t h e L P a n d i t si m p l i c a t i o n s . T h e r e s e a r c h e f f o r t s e e m e d w o r t h w h i l e b e c a u s e o f t h e s i m p l i c i t y ,c e n t r a l i m p o r t a n c e , a n d f a r r e a c h i n g i m p l i c a t i o n s o f t h e L P .

B a c k g r o u n d r e a d i n g f o r t h e r e s e a r c h r e v e a l e d a w i d e r t h a ne x p e c t e d r a n g e o f p u b l i s h e d c r i t i c i s m s o f t h e L P . I n a n a t t e m p t t o b e c o m p l e t ea n d a d d r e s s a l l s u c h c r i t i c i s m s , t h e r e s e a r c h p a p e r e x p a n d e d c o n s i d e r a b l y .E v e n t u a l l y i t s e e m e d s e n s i b l e t o e n l a r g e t h e p a p e r t o a m o n o g r a p h . T h i s a l s oa l l o w e d f o r d i s c u s s i o n o f c o n d i t i o n i n g i d e a s i n g e n e r a l a n d f o r a r e v i e w o f t h ei m p l i c a t i o n s o f t h e L P . I t w a s d e c i d e d , h o w e v e r , t o s t o p s h o r t o f a g e n e r a lr e v i e w o f c o n d i t i o n a l m e t h o s i n s t a t i s t i c s . I n p a r t i c u l a r , t h e m o n o g r a p h d o e sn o t d i s c u s s t h e m a n y l i k e l i h o o d b a s e d s t a t i s t i c a l m e t h o d o l o g i e s t h a t h a v e b e e nd e v e l o p e d , a l t h o u g h r e f e r e n c e s t o t h e s e m e t h o d o l o g i e s w i l l b e g i v e n . T h i sl i m i t a t i o n w a s , i n p a r t , b e c a u s e s u c h a n e n d e a v o r w o u l d b e f a r t o o a m b i t i o u s ,a n d , i n p a r t , b e c a u s e w e f e e l a n d i n d e e d a r g u e i n C h a p t e r 5 ) t h a t B a y e s i a ni m p l e m e n t a t i o n o f t h e L P i s t h e c o r r e c t c o n d i t i o n a l m e t h o d o l o g y .

T h e m a t h e m a t i c a l l e v e l o f t h e m o n o g r a p h i s , f o r t h e m o s t p a r t ,k e p t a t a n o n t e c h n i c a l l e v e l . T h e m a i n e x c e p t i o n i s t h e g e n e r a l i z a t i o n o f t h eL P i n S e c t i o n 3 . 4 , w h i c h i s n e c e s s a r i l y ) p r e s e n t e d a t a m e a s u r e - t h e o r e t i cl e v e l , b u t c a n b e s k i p p e d w i t h n o l o s s i n c o n t i n u i t y . A l s o , t h e m o n o g r a p h

vii


10/252

v i i i P R E F A C E

p r e s u p p o s e s n o f a m i l i a r i t y w i t h c o n d i t i o n i n g c o n c e p t s . I n d e e d C h a p t e r 2p r o v i d e s a n e l e m e n t a r y r e v i e w o f c o n d i t i o n i n g , w i t h m a n y e x a m p l e s .

T h i s s e c o n d e d i t i o n w a s p r o d u c e d u n d e r t h e r a t h e r s e v e r e c o n s t r a i n tt h a t t h e o r i g i n a l m a n u s c r i p t , u s e d f o r p h o t o - o f f s e t p r i n t i n g , w a s i n a d v e r t a n t l yd e s t r o y e d ; o n l y t h e p h o t o s w e r e k e p t . T h u s c h a n g e s c o u l d o n l y b e m a d e b yr e t y p i n g e n t i r e p a g e s o r i n s e r t i n g n e w p a g e s . A l i s t o f c o r r e c t i o n s t h a t w e r et o o m i n o r t o j u s t i f y t h e r e t y p i n g o f a n e n t i r e p a g e i s g i v e n a t t h e e n d o f t h em o n o g r a p h . I n s e r t e d p a g e s r e c e i v e d d e c i m a l p a g e n u m b e r s : e . g . 7 4 . 1 , 7 4 . 2 . Al i s t o f a d d i t i o n a l r e f e r e n c e s w a s a d d e d , a n d n e w d i s c u s s i o n s w e r e k i n d l y c o n t r i -b u t e d b y M . J . B a y a r r i a n d M . H . D e G r o o t , B r u c e H i l l , a n d L u c i e n L e C a m .

S u b s t a n t i a l c h a n g e s o r a d d i t i o n s w e r e m a d e i n S e c t i o n s 3 . 1 , 3 . 5 ,4 . 2 . 1 , 4 . 4 , a n d 4 . 5 . T h e c h a n g e s i n S e c t i o n 4 . 4 c o r r e c t a g l a r i n g o v e r s i g h t i nt h e f i r s t e d i t i o n : t h e f a i l u r e t o e m p h a s i z e t h e m i s l e a d i n g c o n c l u s i o n s t h a tc a n r e s u l t f r o m v i o l a t i o n o f t h e L i k e l i h o o d P r i n c i p l e i n s i g n i f i c a n c e t e s t i n go f a p r e c i s e h y p o t h e s i s . A n o t h e r v e r y w e a k p a r t o f t h e f i r s t e d i t i o n w a sS e c t i o n 3 . 5 , w h i c h d i s c u s s e d p r e d i c t i o n , d e s i g n , a n d n u i s a n c e p a r a m e t e r s . T h en e w m a t e r i a l i n c o r p o r a t e s r e c e n t s u b s t a n t i v e i n s i g h t s f r o m t h e l i t e r a t u r e .

N u m e r o u s o t h e r m i n o r c h a n g e s a n d l i t e r a t u r e u p d a t i n g s w e r e m a d et h r o u g h o u t t h e m o n o g r a p h . W e d i d n o t a t t e m p t c o m p l e t e c o v e r a g e o f r e c e n t l i t -e r a t u r e , h o w e v e r .

W e a r e g r a t e f u l t o a n u m b e r o f p e o p l e f o r v a l u a b l e d i s c u s s i o n s o nt h i s s u b j e c t a n d / o r f o r c o m m e n t s a n d s u g g e s t i o n s o n o r i g i n a l d r a f t s o r t h ef i r s t e d i t i o n o f t h e m o n o g r a p h . I n p a r t i c u l a r , w e w o u l d l i k e t o t h a n k G e o r g eB a r n a r d , M . J . B a y a r r i , M a r k B e r l i n e r , L a w r e n c e B r o w n , G e o r g e C a s e l l a , M o r r i sD e G r o o t , J . L . F o u l l e y , L e o n G l e s e r , P r e m G o e l , C l y d e H a r d i n , B r u c e H i l l ,J i u n n H w a n g , R a j e e v K a r a n d i k a r , L u c i e n L e C a m , K e r - C h a u L i , D e n n i s L i n d l e y ,G e o r g e M c C a b e , G e o r g e s M o n e t t e , J o h n P r a t t , D o n R u b i n , H e r m a n R u b i n , M y r aS a m u e l s , S t e v e S a m u e l s , a n d T o m S e l l k e . W e a r e e s p e c i a l l y g r a t e f u l t o M . J .B a y a r r i a n d M . H . D e G r o o t f o r a n e x c e p t i o n a l l y c o m p l e t e a n d i n s i g h t f u l s e t o fc o r r e c t i o n s a n d c o m m e n t s o n t h e f i r s t e d i t i o n . W e a r e a l s o g r a t e f u l t oS h a n t i G u p t a f o r t h e e n c o u r a g e m e n t t o t u r n t h e m a t e r i a l i n t o a m o n o g r a p h .


11/252


12/252

T A B L E OF C O N T E N T S

CHAPTER 1. INTRODUCTION 1CHAPTER 2. CONDITIONING 5

2.1. Simple Examples 52 2 Relevant Subsets 112.3. Ancillarity 132.4. Conditional Frequentist Procedures 14

2.4.1. Conditional Confidence 142 . 4 . 2 . Estimated Confidence 16

2.5. Criticisms of Partial Conditioning 16CHAPTER 3. THE LIKELIHOOD PRINCIPLE AND GENERALIZATION S . 19

3.1. Introduction 193 2 History of the Likelihood Principle 223.3. Birnbaum s Development the Discrete Case 24

3.3.1. Evidence Conditionally and Sufficiency 243.3.2. Axiomatic Development 26

3.4. Generalizations Beyond the Discrete Case 283.4.1. Difficulties in the Nondiscrete Case 293.4.2. Evidencej Conditionality> and Sufficiency 313.4.3. The Relative Likelihood Principle 32

3.5. Prediction , Design, Nuisance Paramete rs, and the LP 363.5.1. Introduction 363.5.2. Unobserved Variables: Prediction and Design 373.5.3. Nuisance Variables and Parameters 41

x


13/252

TABLE OF CONTENTS xi

3.6. Criticisms of Birnbaum s Axiomatic Development 423.6.1. The Model Assumption 433.6.2. The Evidence Assumption 453.6.3. The Weak Conditionality Principle 453.6.4. The Sufficiency Principle 46

3 7 Violation of the Likelihood Principl e: Inadmissibility andIncoherency 503.7.1. Introduction 503.7.2. Decision Theoretic Evaluation 523.7.3. Betting Evaluation 59

CHAPTER 4. CONSEQUENCE S AND CRITICISMS OF THE LIKELIHOOD PRINCIPLE AND THERELATIVE LIKELIHOOD PRINCIPLE 65

4.1. Incompatibility with Frequentist Concepts 654.1.1. Introduction 654.1.2. Objectivity 674.1.3. Procedures for Nonspecialists 684.1.4. Repeatability 704.1.5. The Confidence Principle 71

4 The Irrelevance of Stopping Rules 744.2.1. Introduction 744.2.2. The Discrete) Stopping Rule Principle 754.3.2. Positive Implications 774.2.4. Criticisms 804.2.5. Stopping Rules and Inadmissibili ty 834.2.6. The General Stopping Rule Principle 864.2.7. Informative Stopping Rules 88

4.3. The Irrelevance of Censoring Mechanisms4.3.1. Introduction 904.3.2. F i x e d e n s o r i n g and E q u i v a l e n t e n s o r i n g M e c h a n i s m s . . . 9 24 3 3 Random Censoring 954.3.4. Informative Censoring 103


14/252

x TABLE OF CONTENTS

4.4. Significance Testing 1044.4.1. onflictwith theLikelihoodPrinciple 1044.4.2. Averaging Over More Extreme Observations 1054.4.3. Testing a Single Null Model 1094.4.4. Conclusions 109.3

4.5. Randomization Analysis 1104.5.1. Introduction 1104.5.2. FinitePopulationSampling 1124.5.3. RandomizationTests 117

CHAPTER 5. IMPLEMENTATION OF THE LIKELIHOOD PRINCIPLE 1215.1. Introduction 1215.2. Non Bayesian Likelihood Methods 1225.3. Arguments for Bayesian Implementation 124

5.3.1. Generalonsiderations 1245.3.2. TheFraser Monette Ng Stone, and Stein Examples. . . 127

5.4. Robust Bayesian Analysis 1365.5. Conclusions 141

R E F E R E N C E S 1 4 3A D D I T IO N A L O R U P D A T E D R E F E R EN C E S 1 60DISCUSSION BY M. J. BAYARRI AND M. H. DEGROOT 160.3DISCUSSION BY BRUCE HILL 161DISCUSSION BY DAVID LANE 175DISCUSSION BY LUCIEN LE CAM 182REPLY TO THE DISCUSSION 186ADDITIONAL REFERENCES IN THE DISCUSSION 198INDEX OF EXAMPLE S 200AUTH OR INDEX 201SUBJECT INDEX 204

ERRATA AND CLARIFICATIONS 207


15/252

C H A P T E R 1 . I N T R O D U C T I O N

A m o n g a l l p r e s c r i p t i o n s f o r s t a t i s t i c a l b e h a v i o r , t h e L i k e l i h o o dP r i n c i p l e ( L P ) s t a n d s o u t a s t h e s i m p l e s t a n d y e t m o s t f a r r e a c h i n g . I t e s s e n t i a l l y s t a t e s t h a t a l l e v i d e n c e , w h i c h i s o b t a i n e d f r o m a n e x p e r i m e n t , a b o u t a nu n k n o w n q u a n t i t y , is c o n t a i n e d i n t h e l i k e l i h o o d f u n c t i o n o f f o r t h e g i v e nd a t a . T h e i m p l i c a t i o n s o f t h i s a r e p r o f o u n d , s i n c e m o s t n o n B a y e s i a n a p p r o a c h e st o s t a t i s t i c s a n d i n d e e d m o s t s t a n d a r d s t a t i s t i c a l m e a s u r e s o f e v i d e n c e ( s u c ha s c o v e r a g e p r o b a b i l i t y , e r r o r p r o b a b i l i t i e s , s i g n i f i c a n c e l e v e l , f r e q u e n t i s tr i s k , e t c . ) a r e t h e n c o n t r a i n d i c a t e d .

T h e L P w a s a l w a y s i m p l i c i t i n t h e B a y e s i a n a p p r o a c h t o s t a t i s t i c s ,b u t i t s d e v e l o p m e n t a s a s e p a r a t e s t a t i s t i c a l p r i n c i p l e w a s d u e i n l a r g e p a r tt o i d e a s o f R . A . F i s h e r a n d G . B a r n a r d ( s e e S e c t i o n 3 .2 f o r r e f e r e n c e s ) . I tr e c e i v e d m a j o r n o t i c e w h e n B i r n b a u m ( 1 9 6 2 a ) s h o w e d i t t o b e a c o n s e q u e n c e o ft h e m o r e c o m m o n l y t r u s t e d S u f f i c i e n c y P r i n c i p l e ( t h a t a s u f f i c i e n t s t a t i s t i cs u m m a r i z e s t h e e v i d e n c e f r o m a n e x p e r i m e n t ) a n d C o n d i t i o n a l i t y P r i n c i p l e ( t ha te x p e r i m e n t s n o t a c t u a l l y p e r f o r m e d s h o u l d b e i r r e l e v a n t t o c o n c l u s i o n s ) . S i n c et h e n t h e L P h a s b e e n e x t e n s i v e l y d e b a t e d b y s t a t i s t i c i a n s i n t e r e s t e d i n f o u n d a t i o n s , b u t h a s b e e n i g n o r e d b y m o s t s t a t i s t i c i a n s . T h e r e a r e p e r h a p s s e v e r a lr e a s o n s f o r t h i s . F i r s t , t h e c o n s e q u e n c e s o f t h e L P s e e m s o a b s u r d t o m a n yc l a s s i c a l s t a t i s t i c i a n s t h a t t h e y f e el i t a w a s t e o f t i m e t o e v e n s t u d y t h ei s s u e . S e c o n d , a c u r s o r y i n v e s t i g a t i o n o f t h e L P r e v e a l s c e r t a i n o f t s t a t e do b j e c t i o n s , f o r e m o s t o f w h i c h i s t h e a p p a r e n t d e p e n d e n c e o f t h e p r i n c i p l e o na s s u m i n g e x a c t k n o w l e d g e o f t h e ( p a r a m e t r i c ) m o d e l f o r t h e e x p e r i m e n t ( so t h a ta n e x a c t l i k e l i h o o d f u n c t i o n e x i s t s ) . S i n c e t h e m o d e l i s r a r e l y t r u e , ( h a s t y )

1


16/252

2 T H E L I K E L I H O O D P R I N C I P L E

r e j e c t i o n o f t h e L P m a y r e s u l t . T h i r d , t h e L P d o e s n o t s a y h o w o n e i s t o p e r -f o r m a s t a t i s t i c a l a n a l y s i s ; i t m e r e l y g i v e s a p r i n c i p l e t o w h i c h a n y m e t h o d o fa n a l y s i s s h o u l d a d h e r e . I n d e e d B a y e s i a n a n a l y s i s i s o f t e n p r e s e n t e d a s t h e w a yt o i m p l e m e n t t h e L P ( w i t h w h i c h w e e s s e n t i a l l y a g r e e ) , a v e r y u n a p p e a l i n gp r o s p e c t t o m a n y c l a s s i c a l s t a t i s t i c i a n s .

T h e m a j o r p u r p o s e o f t h i s ( m o s t l y r e v i e w ) m o n o g r a p h i s t o a d d r e s st h e s e c o n c e r n s . A s e r i o u s e f f o r t w i l l b e m a d e , t h r o u g h e x a m p l e s a n d a p p e a l s t oc o m m o n s e n s e , t o a r g u e t h a t t h e L P i s i n t u i t i v e l y s e n s i b l e , m o r e s o t h a n t h ec l a s s i c a l m e a s u r e s w h i c h i t i m p u n e s . A l s o , a g e n e r a l i z e d v e r s i o n o f t h e L Pw i l l b e i n t r o d u c e d , a v e r s i o n w h i c h r e m o v e s t h e r e s t r i c t i o n o f a n e x a c t l y k n o w nl i k e l i h o o d f u n c t i o n , a n d y e t h a s e s s e n t i a l l y t h e s a m e i m p l i c a t i o n s . ( O t h e rc r i t i c i s m s o f t h e L P w i l l a l s o b e d i s c u s s e d . ) F i n a l l y , t h e q u e s t i o n o f i m p l e -m e n t a t i o n o f t h e L P w i l l b e c o n s i d e r e d , a n d i t w i l l b e a r g u e d t h a t B a y e s i a na n a l y s i s ( m o r e p r e c i s e l y r o b u s t B a y e s i a n a n a l y s i s ) i s t h e m o s t s e n s i b l e a n dr e a l i s t i c m e t h o d o f i m p l e m e n t a t i o n . A t h o r o u g h d i s c u s s i o n o f t h i s i s s u e i s ,h o w e v e r , o u t s i d e t h e s c o p e o f t h e m o n o g r a p h , s o t h e m a i n t h e s i s w i l l s i m p l y b et h a t t h e L P i s b e l i e v a b l e a n d t h a t b e h a v i o r i n v i o l a t i o n o f i t s h o u l d b ea v o i d e d t o t h e e x t e n t p o s s i b l e .

A c c e p t a n c e o f s u c h a t h e s i s r a d i c a l l y a l t e r s t h e w a y o n e v i e w ss t a t i s t i c s . I n d e e d , t o m a n y B a y e s i a n s , b e l i e f i n t h e L P i s t h e b i g d i f f e r e n c eb e t w e e n B a y e s i a n s a n d f r e q u e n t i s t s , n o t t h e d e s i r e t o i n v o l v e p r i o r i n f o r m a t i o n .T h u s S a v a g e s a i d ( i n t h e D i s c u s s i o n o f B i r n b a u m ( 1 9 6 2 a ) )

I m y s e l f , c a m e t o t a k e . . . B a y e s i a n s t a t i s t i c s . . .s e r i o u s l y o n l y t h r o u g h r e c o g n i t i o n o f t h e l i k e l i -h o o d p r i n c i p l e .

M a n y B a y e s i a n s b e c a m e B a y e s i a n s o n l y b e c a u s e t h e L P l e f t t h e m l i t t l e c h o i c e .S u f f i c i e n t t i m e h a s p a s s e d s i n c e t h e a x i o m a t i c d e v e l o p m e n t o f

B i r n b a u m t o h o p e t h a t a n y v a l i d o b j e c t i o n s t o t h e L P w o u l d b y n o w h a v e b e e nf o u n d . I n d e e d , t h e r e a r e n u m e r o u s a r t i c l e s i n t h e l i t e r a t u r e p r e s e n t i n ge x a m p l e s , c o u n t e r e x a m p l e s , a r g u m e n t s , a n d c o u n t e r a r g u m e n t s f o r t h e L P . W e w i l l


17/252


18/252


b e g i v e n i n s i m p l e a r t i f i c i a l s e t t i n g s , r a t h e r t h a n r e a l i s t i c a l l y c o m p l i c a t e ds t a t i s t i c a l s i t u a t i o n s , a g a i n f o r e a s e o f r e a d i n g a n d b e c a u s e c o m p l i c a t e d s i t

u a t i o n s a r e o f t e n t o o i n v o l v e d t o c l e a r l y r e v e a l k e y i s s u e s . A d v a n c e m e n t o f as u b j e c t u s u a l l y p r o c e e d s by a p p l y i n g t o c o m p l i c a t e d s i t u a t i o n s t r u t h s d i s c o v e r e d i n s i m p l e s e t t i n g s .

T h r o u g h o u t t h e m o n o g r a p h , X wi l l d e n o t e t h e r a n d o m q u a n t i t y t o b eo b s e r v e d , X t h e s a m p l e s p a c e , x ( t h e o b s e r v e d d a t a ) a r e a l i z a t i o n o f X , a n dP () t h e p r o b a b i l i t y d i s t r i b u t i o n o f X o n 2 , w h e r e G i s u n k n o w n . A l t h o u g h w i l l b e c a l l e d t h e p a r a m e t e r a n d t h e p a r a m e t e r s p a c e t h e f a m i l y { P ( )> n e e d n o t b e a t y p i c a l p a r a m e t r i c f a m i l y ; c o u l d j u s t d e n o t e s o m e ( p o s s i b l y n o n p a r a m e t r i c ) i n d e x . A l s o , w i l l b e u n d e r s t o o d t o c o n s i s t o f a l l u n k n o w n f e a t u r e s o f t h e p r o b a b i l i t y d i s t r i b u t i o n . O f t e n , t h e r e f o r e , o n l y p a r t o f w i l l b e o f i n t e r e s t , t h e r e m a i n d e r b e i n g a n u i s a n c e p a r a m e t e r . I n d i s c u s s i n g s e q u e n t i a l a n d p r e d i c t i o n p r o b l e m s i t w i l l s o m e t i m e s b e c o n v e n i e n t t o c o n s i d e r u n o b s e r v e d r a n d o m v a r i a b l e s Z , a s w e l l a s t h e u n k n o w n ; z w i l l t h e n d e n o t e a p o s s i b l e v a l u e o f Z . T o s i m p l i f y t h e e x p o s i t i o n i n t h e m o n o g r a p h , h o w e v e r , w e w i ll u s u a l l y o n l y c o n s i d e r t h e s i m p l e r c a s e i n w h i c h Z i s a b s e n t . N o t et h a t f o r s o m e s t a t i s t i c a l p r o b l e m s i t i s i m p o s s i b l e t o s e p a r a t e Z a n d { P _ ( ) }

S e e S e c t i o n 3 .5 f o r d i s c u s s i o n o f s u c h p r o b l e m s .W h e n n e c e s s a r y , ^ w i l l d e n o t e t h e f i e l d o f m e a s u r a b l e e v e n t s i n X .

I f a d e n s i t y f o r X e x i s t s i t w i l l b e d e n o t e d f Q ( x ) , a n d w e w i l l p r e s u m e t h ee x i s t e n c e o f a s i n g l e d o m i n a t i n g f i n i t e m e a s u r e v ( ) f o r { P ( )> E ) s u c ht h a t P Q ( B ) = J f Q ( x ) v ( d x ) f o r e a c h B e 3 . I n a ll t h e e x a m p l e s v w i l l b e t a k e nt o b e c o u n t i n g m e a s u r e i n t h e d i s c r e t e c a s e a n d L e b e s g u e m e a s u r e i n t h e c o n t i n u o u s c a s e , w h e n X i s a s u b s e t o f E u c l i d e a n s p a c e . U s u a l l y w e w i l l w r i t e t h er e f e r e n c e m e a s u r e s i m p l y a s d x ( i m p l i c i t l y t a k i n g L e b e s g u e m e a s u r e f o r v ) ;t h e f o r m u l a s w i l l r e q u i r e m i n o r c h a n g e s f o r c a s e s ( i n c l u d i n g t h o s e i n v o l v i n gd i s c r e t e d i s t r i b u t i o n s ) i n w h i c h o t h e r r e f e r e n c e m e a s u r e s a r e m o r e c o n v e n i e n t .


19/252

C H A P T E R 2 . C O N D I T I O N I N G

The most commonly used measures o f accura cy o f evidenc e ins t a t i s t i c s a r e p re exp er im en t a l . A p a r t i c u l a r p r oc e dure i s de c ide d upon fo rus e , a nd the a c c ur a c y o f the e v ide nc e f rom a n e x pe r ime nt i s i d e n t i f i e d w i th thelong r un be ha v ior o f the p r oc e d ur e , we re the e x pe r ime nt r e pe a te d l y pe r fo r me d .This long run be hav ior is eva luat ed by averag ing the per formance of the proce dur e ove r the sa mple spac e Z . I n c o nt r a s t , the LP s ta t e s t ha t p o s t exp er im en t a lreasoning should be used, where in only the ac tua l observat ion x (and not theo the r obs e r v a t ions i n % t ha t c ou ld have oc c ure d) i s r e l e v a nt . The re a r e av a r i e t y o f i n t e r m e d i a t e p o s i t io n s w hic h c a l l f o r p a r t i a l c o n d i t io n i n g o n x andp a r t i a l lo ng r un f re q u e nc y i n t e r p r e t a t i o n s . P a r t l y f o r h i s t o r i c a l p u rp o se s ,a nd pa r t l y to i nd i c a te tha t the ca se fo r a t l e a s t some s or t o f c ond i t i on i ng i scom pe l ling, we d iscuss in th is cha pte r var ious co ndi t ion ing v iew points .2 1 SIM PLE EXAMPLES

The fo l lowing s imple examples revea l the necess i ty of a t least somet imest h i n k i n g c o n d i t i o n a l l y , a nd w i l l be im p o r t a n t l a t e r .EXAMPLE 1 . Suppose X, and X 2 are independent and

P ( X = 1 ) = PQ ( X . = + 1 ) = , i = 1 , 2 .H e r e > < < is an u n k n o w n p a r a m e t e r to be e s t i m a t e d f r o m X, and X 2 It ise a s y to see h a t a 7 5 % c o n f i d e n c e set f s m a l l e s t s i z e for is

c x v x 2 =t h e p o i n t y X j + X g ) if X 1f

t h e p o i n t X ^ l if X 1=5


20/252


T h u s , i f r e p e a t e d l y u s e d i n t h i s p r o b l e m , C ( X , , X 2 ) w o u l d c o n t a i n w i t hp r o b a b i l i t y . 7 5 .

N o t i c e , h o w e v e r , t h a t w h e n x , f x i t i s a b s o l u t e l y c e r t a i n t h a t = X n + X p ) , w h i l e w h e n x ^ = x ~ i t i s e q u a l l y u n c e r t a i n w h e t h e r = xj1 o r = x , + l ( a s s u m i n g n o p r i o r k n o w l e d g e a b o u t ) . T h u s , f r o m a p o s t e x p e r i m e n t a lv i e w p o i n t , o n e w o u l d s a y t h a t C ( x 1 9 x 2 ) c o n t a i n s w i t h " c o n f i d e n c e " 1 0 0 % w h e nx , f x 2 , b u t o n l y w i t h " c o n f i d e n c e " 5 0 % w h e n x , = x 2 C o m m o n s e n s e c e r t a i n l ys u p p o r t s t h e p o s t e x p e r i m e n t a l v i e w h e r e . I t i s t e c h n i c a l l y c o r r e c t t o c a l lC U p X o ) a 7 5 % c o n f i d e n c e s e t , b u t , i f a f t e r s e e i n g t h e d a t a w e k n o w w h e t h e r i ti s r e a l l y a 1 0 0 % o r 5 0 % s e t , r e p o r t i n g 7 5 % s e e m s r a t h e r s i l l y .

T h e a b o v e e x a m p l e f o c u s e s t h e i s s u e s o m e w h a t : d o e s i t m a k e s e n s et o r e p o r t a p r e e x p e r i m e n t a l m e a s u r e w h e n i t i s k n o w n t o b e m i s l e a d i n g a f t e rs e e i n g t h e d a t a ? T h e n e x t e x a m p l e a l s o s e e m s i n t u i t i v e l y c l e a r , y e t i s t h e k e yt o a l l t h a t f o l l o w s .E X A M P L E 2 . S u p p o s e a s u b s t a n c e t o b e a n a l y z e d c a n b e s e n t e i t h e r t o al a b o r a t o r y i n N e w Y o r k o r a l a b o r a t o r y i n C a l i f o r n i a . T h e t w o l a b s s e e me q u a l l y g o o d , s o a f a i r c o i n i s f l i p p e d t o c h o o s e b e t w e e n t h e m , w i t h " h e a d s "d e n o t i n g t h a t t h e l a b i n N e w Y o r k w i l l b e c h o s e n . T h e c o i n i s f l i p p e d a n dc o m e s u p t a i l s , s o t h e C a l i f o r n i a l a b i s u s e d . A f t e r a w h i l e , t h e e x p e r i m e n t a lr e s u l t s c o m e b a c k a n d a c o n c l u s i o n m u s t b e r e a c h e d . S h o u l d t h i s c o n c l u s i o nt a k e i n t o a c c o u n t t h e f a c t t h a t t h e c o i n c o u l d h a v e b e e n h e a d s , a n d h e n c e t h a tt h e e x p e r i m e n t i n N e w Y o r k m i g h t h a v e b e e n p e r f o r m e d i n s t e a d ?

T h i s , o f c o u r s e , i s a v a r i a n t o f t h e f a m o u s C o x e x a m p l e ( C o x ( 1 9 5 8 ) s e e a l s o C o r n f i e l d ( 1 9 6 9 ) ) , w h i c h c o n c e r n s b e i n g g i v e n ( a t r a n d o m ) e i t h e r a na c c u r a t e o r a n i n a c c u r a t e m e a s u r i n g i n s t r u m e n t ( a n d k n o w i n g w h i c h w a s g i v e n ) .S h o u l d t h e c o n c l u s i o n r e a c h e d b y e x p e r i m e n t a t i o n d e p e n d o n l y o n t h e i n s t r u m e n ta c t u a l l y u s e d , o r s h o u l d i t t a k e i n t o a c c o u n t t h a t t h e o t h e r i n s t r u m e n t m i g h th a v e b e e n o b t a i n e d ?

I n s y m b o l i c f o r m , w e c a n p h r a s e t h i s e x a m p l e a s a " m i x e d e x p e r i m e n t "


21/252


22/252


N o t e t h a t t h e r e i s n o t h i n g l o g i c a l l y w r o n g w i t h r e p o r t i n g e r r o rp r o b a b i l i t i e s i n E x a m p l e 3 ; i t j u s t s e e m s t o b e a n i n a d e q u a t e r e f l e c t i o n o f t h ee v i d e n c e c o n v e y e d b y t h e d a t a t o r e p o r t = . 0 2 2 8 f o r b o t h x = 0 a n d x = 1 .P r a t t ( 1 9 7 7 ) ( p e r h a p s s o m e w h a t t o n g u e i n c h e e k ) t h u s c o i n s

T H E P R I N C I P L E O F A D E Q U A C Y . A c o n c e p t o f s t a t i s t i c a l e v i d e n c e i s ( v e r y )i n a d e q u a t e i f i t d o e s n o t d i s t i n g u i s h e v i d e n c e o f ( v e r y ) d i f f e r e n t s t r e n g t h s .E X A M P L E 4 a . S u p p o s e X i s 1 , 2 , o r 3 a n d is 1 o r 2 , w i t h P ( x ) g i v e n i n t h ef o l l o w i n g t a b l e :

poP l

1 9

1

X2

1

9 8 9

3 9 9

1

T h e t e s t , w h i c h a c c e p t s P Q w h e n x = 3 a n d a c c e p t s P , o t h e r w i s e , i s a m o s tp o w e r f u l t e s t w i t h b o t h e r r o r p r o b a b i l i t i e s e q u a l t o . 0 1 . H e n c e , i t w o u l d b ev a l i d t o m a k e t h e f r e q u e n t i s t s t a t e m e n t , u p o n o b s e r v i n g x = 1 , " M y t e s t h a sr e j e c t e d P Q a n d t h e e r r o r p r o b a b i l i t y i s . 0 1 . " T h i s s e e m s v er y m i s l e a d i n g ,s i n c e t h e l i k e l i h o o d r a t i o i s a c t u a l l y 9 t o 1 in f a v o r o f P Q , w h i c h i s b e i n gr e j e c t e d .E X A M P L E 4 b . O n e c o u l d o b j e c t i n E x a m p l e 4 a , t h a t t h e . 01 l e v e l t e s t i si n a p p r o p r i a t e , a n d t h a t o n e s h o u l d u s e t h e . 0 01 l e v e l t e s t , w h i c h r e j e c t s o n l yw h e n x = 2 . C o n s i d e r , h o w e v e r , t h e f o l l o w i n g s l i g h t l y c h a n g e d v e r s i o n :

poP l

1 5

5 1

X2

5

. 9 8 4 9

3 9 9 1

A g a i n t h e t e s t w h i c h r e j e c t s P Q w h e n x = 1 o r 2 a n d a c c e p t s o t h e r w i s e h a s e r r o rp r o b a b i l i t i e s e q u a l t o . 0 1 , a n d n o w i t i n d e e d s e e m s s e n s i b l e t o t a k e t h ei n d i c a t e d a c t i o n s ( i f w e s u p p o s e a n a c t i o n m u s t b e t a k e n ) . I t s t i l l s e e m s


23/252

O N I T I O N I N G 9

unreasonable, however, to r e p o r t ane r r o r p r o b a b i li t y of .01 upon re jec t ingPQwhenx = 1, s ince the data provides yery l i t t l e evid en ce in favor of P,.EXAMPLE5. For a dec is ion t he o re t ic example, cons ider the i n t e r e s t i n g S t e inphenomenon, concerned w ith es tim at ion of a p v a r i a t e normal mean (p >_ 3) basedon X 7? ( , I) andunder sum of squares e r r o r lo ss. The usual pre exper imenta lmeasure of the performance of an es t ima to r is the r i s k f u n c t i o n (or expectedloss)

R ) = E0 j . X ) ) 2 .

T h e c l a s s i c a l e s t i m a t o r h e r e is 6 (x) = x, but James and St e in (1960) showedthat

J S x ) = ( 1 _ B z |)

has R( , ) < R( , ) = p f or al l . One can t h us r e p o r t 6 as a lwaysbei n g b e t t e r t h a n f r om a p r e e x p e r i m e n t a l v ie w p o i n t . However, i f p = 3 andx = ( 0 , . 0 1 , . 0 1 ) is o b s e r v e d , t h e n

6JS(x) = (0, 49.99, 49.99),w h i c his ana b s u r d e s t i m a t eof . H e n c e canbet e r r i b l e for c e r t a i nx.O f c o u r s e the p o s i t i v e p a r t v e r s i o nof 6 ,

6J s + ( x , = ( i . f i= | ,+Xi

c o r r ec t s t h i s g l a r i n g p r o b l e m ,but the p o i n t is t h a t a procedure which looksgrea t p re exper imen ta l l y cou ld be t e r r i b l e for p a r t i c u l ar x, and i t may notalways be so obvious when this is the case.

Confidence sets fo r canalso bedeveloped (seeCasel laandHwang (1982)) which have l a r g e r p r o b a b i l i t i e s of coverage than the c l a s s i c a lc o n f i d e n c e e l l i p s o i d s , are never l a r g e r in s i z e , and for smal l | x|c o n s i s t of the s i n g l e p o i n t {0}. Indeed , these sets are of the s imple form


24/252

1 0 THE L I K E L I H O O D P R I N C I P L E J ~ S + ( x ) | 2 p ( l ) } if | x | >

C ( x ) ={ 0 } if | x | < ,

2where x (1 ) i s t he 1 t/z p e r c e n t i l e o f th e c h i s q u a re d i s t r i b u t i o n w i t h pdegrees o f f reedom, and i s s u i t ab l y sm al l . A lt hough t h i s con f idence procedure looks grea t p re exp er im ent a l l y , one wou ld l ook ra the r f o o l i sh to concludewhen p = 3 and x = (0 , .01 , .0 1) t h a t i s t he p o in t {0 } w i th con f idence 95%.

The above examples , though s imple, ind icate most of the in tu i t i vereasons fo r co n d i t io n in g . There are a wide var iet y of oth er such examples.The Un if o rm ( , + ) d i s t r i b u t i o n ( , 3 known) p r ov ides a host o f exam p leswhere co nd i t i ona l reason ing d i f f e r s cons iderab ly f rom pre exper im enta l r easoni n g ( c . f . Welch (1939) and Pr at t (19 61 ) ) . The Ste in 2s tage procedure fo r

2o b t a i n i n ga c o n f i d e n c e i n t e r v a lof f i x e d w i d t h f o r t h e m e a n of a ? ? ( , ) d i s 2t r i b u t i o nis a n o t h e r e x a m p l e . A p r e l i m i n a r y s a m p l e a l l o w s e s t i m a t i o n of ,f r o m w h i c h it is p o s s i b l e to d e t e r m i n e the s a m p l e s i z e n e e d e dfor a s e c o n ds a m p l e in o r d e r to g u a r a n t e e an o v e r a l l p r o b a b i l i t y of c o v e r a g e f o r a f i x e dw i d t h i n t e r v a l . B u t w h a t if t h e s e c o n d s a m p l e i n d i c a t e s t h a t the p r e l i m i n a r y

2e s t i m a t e of was w o e f u l l y l o w ? T h e n o n e w o u l d r e a l l y h a v e m u c h l e s s r e a lc o n f i d e n c ein the p r o p o s e d i n t e r v a l ( c . f . L i n d l e y ( 1 9 5 8 ) a n d S a v a g e e t . l.( 1 9 6 2 ) ) . A n o t h e r e x a m p l e is r e g r e s s i o n on r a n d o m c o v a r i a t e s . It isc o m m o np r a c t i c e to p e r f o r m t h e a n a l y s i s c o n d i t i o n a l l yon t h e o b s e r v e d v a l u e sof thec o v a r i a t e s , r a t h e r t h a n g i v i n g c o n f i d e n c e s t a t e m e n t s , e t c . , v a l i din ana v e r a g e s e n s e o v e r a l l c o v a r i a t e s t h a t c o u l d h a v e b e e n o b s e r v e d . R o b i n s o n( 1 9 7 5 ) a l s o g i v e s e x t r e m e l y c o m p e l l i n g ( t h o u g h a r t i f i c i a l ) e x a m p l e sof then e e d to c o n d i t i o n . P i c c i n a t o ( 1 9 8 1 ) g i v e s s o m e i n t e r e s t i n g d e c i s i o n t h e o r e t i ce x a m p l e s .

A f i n a l i m p o r t a n t e x a m p l e is t h a t of r o b u s t e s t i m a t i o n . A c o n v i n c i n g c a s e can e m a d e t h a t i n f e r e n c e s t a t e m e n t s s h o u l d be m a d e c o n d i t i o n a l l yo n the r e s i d u a l s ; if t h e d a t a l o o k s c o m p l e t e l y l i k e n o r m a l d a t a , use n o r m a lt h e o r y . B a r n a r d ( 1 9 8 1 ) s a y s


25/252

C O N D I T I O N I N G " W e s h o u l d r e c o g n i s e t h a t ' r o b u s t n e s s 1ofi n f e r e n c e is a c o n d i t i o n a l p r o p e r t y s o m ei n f e r e n c e s f r o m s o m e s a m p l e s are r o b u s t .B u t o t h e r i n f e r e n c e s , or the s a m e i n f e r e n c e sf r o m o t h e r s a m p l e s , may d e p e n d s t r o n g l y ond i s t r i b u t i o n a l a s s u m p t i o n s . "

D e m p s t e r ( 1 9 7 5 ) c o n t a i n s w e r y c o n v i n c i n g d i s c u s s i o n and a h o s t of i n t e r e s t i n ge x a m p l e s c o n c e r n i n g t h i s i s s u e . R e l a t e d to c o n d i t i o n a l r o b u s t n e s s is l a r g es a m p l e i n f e r e n c e , w h i c h s h o u l d o f t e nbe d o n e c o n d i t i o n a l l y on s h a p e f e a t u r e so f t h e l i k e l i h o o d f u n c t i o n . T h u s , in u s i n g a s y m p t o t i c n o r m a l t h e o r y for them a x i m u m l i k e l i h o o d e s t i m a t o r , , one h o u l d g e n e r a l l y use I ( ) ~ , the i n v e r s eo f o b s e r v e d F i s h e r i n f o r m a t i o n , as the c o v a r i a n c e m a t r i x , r a t h e r t h a n I ( ) ~ ,t h e i n v e r s e of e x p e c t e d F i s h e r i n f o r m a t i o n . F o r e x t e n s i v e d i s c u s s i o n oft h e s eand r e l a t e d i s s u e s s e e J e f f r e y s ( 1 9 6 1 ) , P r a t t ( 1 9 6 5 ) , A n d e r s e n ( 1 9 7 0 ) ,E f r o nand H i n k l e y ( 1 9 7 8 ) , B a r n d o r f f N i e l s e n ( 1 9 8 0 ) , Cox ( 1 9 8 0 ) , a n d H i n k l e y( 1 9 8 0 a , 1 9 8 2 ) .2 . 2 R E L E V A N T S U B S E T S

Fisher ( c . f . F isher (1956a)) long advocated co nd i t ion ing on what hec a l l e d relevant subsets of (a l so ca l l ed recog n izab le sub sets , r e fe rencese ts , o r co nd i t i ona l exper im enta l frames o f re fe re nc e ) . There i s a cons iderab le l i t e ra tu re on the sub jec t , which tends to be more fo rmal t han thein t u i t i ve t ype o f reason ing presented i n t he examples o f Sec t ion 2 .1 . Thebas ic idea is to f ind subsets of ( o f t e n d e te rm in ed by s t a t i s t i c s ) w h ic h ,when co nd i t ion ed upon, change the pre expe r ime ntal m easure. In Example 1 , f o ri ns t anc e ,

= {x : x^ = x^} U {x : x 1 t x 2 ha n d the c o v e r a g e p r o b a b i l i t i e s of C U ^ ^ ) c o n d i t i o n e d on o b s e r v i n g X in the" r e l e v a n t " s u b s e t s { x : x = x 2 ) or { x : x ]fx 2> are 1 and . 5 , r e s p e c t i v e l y .I n E x a m p l e 2, the two u t c o m e s of t h e c o i n f l i p d e t e r m i n e two r e l e v a n t s u b s e t s .I n E x a m p l e s 3, 4, and 5 it is not c l e a r w h a t s u b s e t s s h o u l d be c o n s i d e r e d


26/252

1 2 THE L I K E L I H O O D P R I N C I P L Er e l e v a n t ,but m a n y r e a s o n a b l e c h o i c e s g i v e c o n d i t i o n a l r e s u l t s q u i t e d i f f e r e n tf r o m thep r e e x p e r i m e n t a l r e s u l t s .

F o r m a l t h e o r i e sofr e l e v a n t s u b s e t s ( c . f . B u e h l e r ( 1 9 5 9 ) ) p r o c e e di naf a s h i o n a n a l o g o u stothe f o l l o w i n g . S u p p o s e C ( x ) is ac o n f i d e n c e p r o c e d u r ew i t h c o n f i d e n c e c o e f f i c i e n t 1 for ll, i.e.,( 2 . 2 . 1 ) P ( C ( X ) c o n t a i n s)= 1 forall.T h e nB isc a l l e dar e l e v a n t s u b s e t of if,for s o m e >0, e i t h e r( 2 . 2 . 2 ) P Q ( C ( X ) c o n t a i n s|X B _ ( 1 ) forallo r( 2 . 2 . 3 ) P ( C ( X ) c o n t a i n s | X B > 1 ) + f o r a l l .When ( 2 . 2 . 2 ) or ( 2 . 2 . 3 ) h o l d s and x G B is o b s e r v e d , i t is q u e s t i o n a b l e w h e t h e r( 2 . 2 . 1 ) s h o u l d be the measure of e v i d e n c e r e p o r t e d . T h i s f o r m e d the b a s i s ofF i sh e r ' s o b j e c t i o n ( F i s h e r ( 1 9 5 6 b ) ) to the A s p in W e l c h ( 1 9 4 9 ) s o l u t i o n to theB e h r e n s F i s h e r p r o b l e m se e a l s o Y a t e s ( 1 9 6 4 ) and C o r n f i e l d ( 1 9 6 9 ) ) . A n o t h e re x a m p l e f o l l o w s . For mo re e x a m p l e s , see C o r n f i e l d ( 1 9 6 9 ) , O lshen ( 1 9 7 7 ) , andF r a s e r ( 1 9 7 7 ) . )EXAMPLE 6 . (B row n ( 1 9 6 7 ) , w i t h e a r l i e r r e l a t e d e x a m p l e s by S t e i n ( 1 9 6 1 ) and

pB u e h l e r and F e d d e r s o n ( 1 9 6 3 ) ) . I f X , , . . . , X n is a sa m p l e f r o m a 7 ?( , )d i s t r i b u t i o n , b o t h and unknown, the u s u a l 1 0 0 ( l ) % c o n f i d e n c e i n t e r v a l f or is

C x.s) = x t / 2 ^ *+ t

/ 2 ^where x and s are the sample mean and standard de viation, re spect ive ly, andt i is the appropriate cr it ic a l value for the t dis tr ibut ion with n1 degreesof freedom. For n = 2 and = . 5 we thus have

P 2 C X,S) contains ) = .5 for all , 2 , 9b u t B r o w n ( 1 9 6 7 ) s h o w e d t h a t

P 2 ( C ( X , S ) c o n t a i n s | X | / S _f for all , 2,


27/252


28/252

1 4 THE L I K E L I H O O D P R I N C I P L Ea n c i l l a r y s t a t i s t i c ( h a v i n ga d i s t r i b u t i o n c l e a r l y i n d e p e n d e n t o f ) . Thec o n d i t i o n a l d i s t r i b u t i o no f T g i v e n S = s is u n i f o r m on the set

X = { ( u , v ) : vu s and 2 < u < + 2 " s } .I n f e r e n c e w i t h re s p e c t t o t h i s c o n d i t io n a l d i s t r i b u t i o n i s s t r a i g h t f o r w a r d .For i ns ta nc e , a 1 0 0 ( l ) % ( c o n d i t i o n a l ) c o nf id e nc e in t e r v a l f o r i s

C(U, V) = 1 (U+ V) l ( l ) O s ) ,one of the solu tion s proposed by Welch ( 1 9 3 9 ) . Th is c ond i t iona l i n t e r v a l i scons idera bly more app ea l ing than var ious op t ima l noncond i t iona l in t e rv a ls ,as d iscussed in Pra t t ( 1 9 6 1 ) .

T he r e a r e a num ber o f d i f f i c u l t i e s i n t h e d e f i n i t i o n and use o fa n c i l l a r y s t a t i s t i c s ( c . f . Basu ( 1 9 6 4 ) and Cox ( 1 9 7 1 ) ) . N e v e r t h e l e s s, c o n d it i on i ng on a nc i l l a r i e s goe s a long way towa rds pr ov id i ng be t t e r c ond i t i ona lpr oc e dure s . A fe w r e fe r e nc e s , fr om whic h o the r s c an be ob t a i ne d , a r e F is he r( 1 9 5 6 a ) , Anderson ( 1 9 7 3 ) , Barnard ( 1 9 7 4 ) , Cox and Hin kle y ( 1 9 7 4 ) , Cox ( 1 9 7 5 ) ,Dawid ( 1 9 7 5 , 1 9 8 1 ) , Efron and Hink ley ( 1 9 7 8 ) , B a r n d o r f f N i e l s e n ( 1 9 7 8 , 1 9 8 0 ) ,Hink ley ( 1 9 7 8 , 1 9 8 0 ) , S e i d e n f e l d ( 1 9 7 9 ) , Grambsch ( 1 9 8 0 ) , Amari ( 1 9 8 2 ) ,B a r n e t t ( 1 9 8 2 ) , and Buehler ( 1 9 8 2 ) .2 . 4 CONDITIONAL FREQUENTIST PROCEDURES

An a mbi t ious a t te m pt to fo r m a l i z e c ond i t i on i ng w i t h i n a f r e qu e nt i s tframework was undertaken by Kiefer ( 1 9 7 7 ) . ( Se e a l so K i e f e r (1 9 7 5 , 1 9 7 6 ) ,Brown ( 1 9 7 7 ) , Brownie and Kie fer ( 1 9 7 7 ) , and Berger ( 1 9 8 4 c , 1 9 8 4 d ). ) Thef o r m a l i z a t i o n was i n tw o d i s t i n c t d i r e c t i o n s , w hic h K i e f e r c a l l e d c o n d i t i o n a lc onf i de nc e a nd e s t i ma te d c onf i de nc e .2 . 4 . 1 Con di t iona l Conf idence

The ba s i c ide a o f c ond i t i ona l c onf ide nc e i s to de f i ne a p a r t i t i o nW $ : s and then associate with each set in the partition th e appropriate conditionalfrequency measure for the procedure considered. In xample 1, the partitionwould be into th e sets X = {x : x^ = x} and X = {x : x-j t x 2 ) I n


29/252

CONDITIONING 15Example 2, the p a rt i t io n would be into the sets where heads and t a i l s areo b s e r v e d , r e s p e c t i v e l y .

When dea l ing wi t h a conf idence procedure {C X )} , the con di t ion alfrequency measure th at would be rep orte d, i f x X were obse rved, is

r ( ) = P_( C( X ) co n t a i n s | X


30/252

1 6 T H E L I K E L I H O O D P R I N C I P L E

2 . 4 . 2 E s t i m a t e d C o n f i d e n c eA n a l t e r n a t i v e a p p r o a c h t o c o n d i t i o n i n g , w h i c h c a n b e j u s t i f i e d

f r o m a f r e q u e n t i s t p e r s p e c t i v e ( c . f . K i e f e r ( 1 9 7 7 ) o r B e r g e r ( 1 9 8 4 c ) ) , i s t op r e s e n t a d a t a d e p e n d e n t c o n f i d e n c e f u n c t i o n . I f a c o n f i d e n c e s e t p r o c e d u r eC ( x ) i s t o b e u s e d , f o r i n s t a n c e , o n e c o u l d r e p o r t l ( x ) a s t h e " c o n f i d e n c e "i n C ( x ) w h e n x i s o b s e r v e d . P r o v i d i n g( 2 . 4 . 2 ) E ( X ) ) < P ( C ( X ) c o n t a i n s ) f o r a ll ,

t h i s " r e p o r t " h a s t h e u s u a l f r e q u e n t i s t v a l i d i t y t h a t , i n r e p e a t e d u s e , C ( X )w i l l c o n t a i n w i t h a t l e a s t t h e a v e r a g e o f t h e r e p o r t e d c o n f i d e n c e s . T h u s ,i n E x a m p l e 2 , o n e c o u l d r e p o r t l ( x ) = 1 o r j a s x , ^ x 2 o r x = x 2 >r e s p e c t i v e l y . E s t i m a t e d c o n f i d e n c e ( o r , m o r e g e n e r a l l y , e s t i m a t e d r i s k ) c a nb e v e r y u s e f u l i n a n u m b e r o f s i t u a t i o n s w h e r e c o n d i t i o n a l c o n f i d e n c e f a i l s( s e e K i e f e r ( 1 9 7 7 ) o r B e r g e r ( 1 9 8 4 c ) ) .2 .5 C R I T I C I S M S O F P A R T I A L C O N D I T I O N I N G

T h e n e e d t o a t l e a s t s o m e t i m e s c o n d i t i o n s e e m s t o b e w e l lr e c o g n i z e d , a s t h e b r i e f r e v i e w i n t h i s c h a p t e r h a s i n d i c a t e d . T h e a p p r o a c h e sd i s c u s s e d i n S e c t i o n s 2 . 2 , 2 . 3 , a n d 2 .4 . 1 c o n s i d e r o n l y p a r t i a l c o n d i t i o n i n g ,h o w e v e r ; o n e s t i l l d o e s a f r e q u e n c y a n a l y s i s , b u t w i t h t h e c o n d i t i o n a l d i s t r i b u t i o n o f X o n a s u b s e t . T h e r e a r e s e v e r a l m a j o r c r i t i c i s m s o f s u c h p a r t i a lc o n d i t i o n i n g . ( T h e e s t i m a t e d c o n f i d e n c e a p p r o a c h i n S e c t i o n 2 . 4 . 2 h a s a q u i t ed i f f e r e n t b a s i s ; c r i t i c i s m o f i t w i l l b e g i v e n a t t h e e n d o f t h i s s e c t i o n . )

F i r s t , t h e c h o i c e o f a r e l e v a n t s u b s e t o r a n a n c i l l a r y s t a t i s t i co r a p a r t i t i o n { z $ : s e S ) c a n b e y e r y u n c e r t a i n . I n d e e d , i t s e e m s f a i r l yc l e a r t h a t i t is h a r d t o a r g u e p h i l o s o p h i c a l l y t h a t o n e s h o u l d c o n d i t i o n o n ac e r t a i n s e t o r p a r t i t i o n , b u t n o t o n a s u b s e t o r s u b p a r t i t i o n . ( A f t e r a l l , i ts e e m s s o m e w h a t s t r a n g e t o o b s e r v e x , n o t e t h a t i t i s i n , s a y , , a n d t h e n f o r g e t a b o u t x a n d p r e t e n d o n l y t h a t i s k n o w n t o h a v e o b t a i n e d . ) R e s e a r c h e r sw o r k i n g w i t h a n c i l l a r i t y a t t e m p t t o d e f i n e " g o o d " a n c i l l a r y s t a t i s t i c s t o c o n d i t i o n u p o n , b u t , a s m e n t i o n e d e a r l i e r , t h e r e a p p e a r t o b e n o c o m p l e t e l ys a t i s f a c t o r y d e f i n i t i o n s . A l s o , a n c i l l a r y s t a t i s t i c s d o n o t e x i s t i n m a n y


31/252

C O N D I T I O N I N G 1 7

s i t u a t i o n s w h e r e i t s e e m s i m p o r t a n t t o c o n d i t i o n , a s t h e f o l l o w i n g s i m p l ee x a m p l e s h o w s .

E X A M P L E 8 . S u p p o s e = [ 0 , ] ) , a n d' w i t h p r o b a b i l i t y 1

X =0 w i t h p r o b a b i l i t y .

(An i n s t r u m e n t m e a s u r e s e x a c t l y , b u t w i l l e r r o n e o u s l y g i v e a z e r o r e a d i n gw i t h p r o b a b i l i t y e q u a l t o . ) C o n s i d e r t h e c o n f i d e n c e p r o c e d u r e C ( x ) = { x }( t h e p o i n t x ) . H e r e P Q ( C ( X ) c o n t a i n s ) = 1 . I t i s c l e a r , h o w e v e r , t h a t o n ew a n t s t o c o n d i t i o n o n { x : x > 0 } , s i n c e C ( x ) = { } f o r s u r e i f x > 0 . B u tt h e r e i s n o a n c i l l a r y s t a t i s t i c w h i c h p r o v i d e s s u c h a c o n d i t i o n i n g .

I n s i t u a t i o n s s u c h a s E x a m p l e s 3 , 4 , 5 , a n d 6 , t h e s e l e c t i o n o f ap a r t i t i o n f o r a c o n d i t i o n a l c o n f i d e n c e a n a l y s i s s e e m s q u i t e a r b i t r a r y . K i e f e r( 1 9 7 7 ) s i m p l y s a y s t h a t t h e c h o i c e o f a p a r t i t i o n m u s t u l t i m a t e l y b e l e f t t ot h e u s e r , a l t h o u g h h e d o e s g i v e c e r t a i n g u i d e l i n e s . T h e d e v e l o p m e n t o fi n t u i t i o n o r t h e o r y f o r t h e c h o i c e o f a p a r t i t i o n s e e m s v e r y h a r d , h o w e v e r( s e e a l s o K i e f e r ( 1 9 7 6 ) , B r o w n ( 1 9 7 7 ) , a n d B e r g e r ( 1 9 8 4 c ) ) .

E v e n m o r e d i s t u r b i n g a r e e x a m p l e s , s u c h a s E x a m p l e 4 ( b ) , w h e r e i ts e e m s i m p o s s i b l e t o p e r f o r m t h e i n d i c a t e d s e n s i b l e t e s t a n d r e p o r t c o n d i t i o n a le r r o r p r o b a b i l i t i e s r e f l e c t i n g t h e t r u e u n c e r t a i n t y w h e n x = 1 i s o b s e r v e d .( A t h r e e p o i n t s c a n n o t b e p a r t i t i o n e d i n t o t w o n o n d e g e n e r a t e s e t s , a n d o n ad e g e n e r a t e s e t t h e c o n d i t i o n a l e r r o r p r o b a b i l i t y m u s t b e z e r o o r o n e . ) A n yt h e o r y w h i c h c a n n o t h a n d l e s u c h a s i m p l e e x a m p l e i s c e r t a i n l y s u s p e c t .

T h e s i t u a t i o n f o r e s t i m a t e d c o n f i d e n c e t h e o r y i s m o r e a m b i g u o u s ,b e c a u s e i t h a s n o t b e e n v e r y e x t e n s i v e l y s t u d i e d . I n p a r t i c u l a r , t h e c h o i c eo f a p a r t i c u l a r e s t i m a t e d c o n f i d e n c e o r r i s k i s v e r y d i f f i c u l t , i n a ll b u t t h es i m p l e s t s i t u a t i o n s . A n d , i n s i t u a t i o n s s u c h a s E x a m p l e s 3 a n d 4 ( b ) ,e s t i m a t e d c o n f i d e n c e f u n c t i o n s w i l l h a v e c e r t a i n u n d e s i r a b l e p r o p e r t i e s . I nE x a m p l e 3 , f o r i n s t a n c e , a n y e s t i m a t e d e r r o r p r o b a b i l i t y , ( x ) , w h i c h i s


32/252


33/252


34/252


E X A M P L E 9. S u p p o s e Y , Y 2 , . . .are . i . d . B e r n o u l l i ( ) r a n d o m v a r i a b l e s . Ine x p e r i m e n t E . . ,af i x e d s a m p l e s i z e of 12o b s e r v a t i o n s is d e c i d e d u p o n , and the

1 2s u f f i c i e n t s t a t i s t i cX.= JY.t u r n s o u t to be x,=9. Ine x p e r i m e n t E ~ , it1 i=l i s d e c i d e dto t a k e o b s e r v a t i o n s u n t i lat o t a l of3z e r o e s h a s b e e n o b s e r v e d ,a t w h i c h p o i n t t h e s u f f i c i e n t s t a t i s t i cX2= Y . t u r n s o u t to 9. T h e d i s t r i b u t i o nof X^in. is b i n o m i a l w i t h d e n s i t y

w h i c h f o r x ,= 9y i e l d s the l i k e l i h o o d f u n c t i o n

* g ( Q ) = ( g 2 ) 9 ( l ) 3 T h e d i s t r i b u t i o nofX2in 2is n e g a t i v e b i n o m i a l w i t h d e n s i t y

f 2 ( 2 )=( X f ) X 2 ( I ) 3 ,which for x2 = 9y i e l d s the l i k e l i h o o d f u n c t i o n

= 191) 9 l )3.

I n t h i s s i t u a t i o n , t h eLP a y s t h a t ( i ) f o r e x p e r i m e n t E.a l o n e ,t h e i n f o r m a t i o n a b o u tis c o n t a i n e d s o l e l y in & Q ( ) ;and( i i ) s i n c e & i ( ) and2& ( ) a r e p r o p o r t i o n a las f u n c t i o n sof, the i n f o r m a t i o n a b o u tin e x p e r i

m e n t sE1andE 2is i d e n t i c a l .T h e s e c o n c l u s i o n s a r e , ofc o u r s e , at o d d s w i t h f r e q u e n t i s t

r e a s o n i n g . T h e b i n o m i a l and e g a t i v e b i n o m i a l d i s t r i b u t i o n s w i l l t e n dtog i v ed i f f e r e n t f r e q u e n t i s t m e a s u r e s . For i n s t a n c e ,ao n e t a i l e d s i g n i f i c a n c e t e s to f H Q : =Yw i l l g i v e s i g n i f i c a n c e l e v e l s of =. 0 7 3 0 a n d =. 0 3 3 8 in the


35/252


36/252


37/252

T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S 2 1 . 2

m o s t f a m i l i a r s t a t i s t i c a l s i t u a t i o n . O f t e n , h o w e v e r , t h e r e a r e u n k n o w n s w h i c ha r e r e l e v a n t t o a s t a t i s t i c a l p r o b l e m b u t w h i c h d o n o t d i r e c t l y a f f e c t t h e d i s t r i b u t i o n o f X . O n e e x a m p l e i s p r e d i c t i o n , i n w h i c h i t i s d e s i r e d t o p r e d i c ta n u n k n o w n r a n d o m v a r i a b l e Z , a f t e r o b s e r v i n g X . O t h e r e x a m p l e s a r i s e i nd e s i g n a n d s e q u e n t i a l a n a l y s i s p r o b l e m s , w h e r e a s y e t u n o b s e r v e d d a t a ca na f f e c t t h e d e c i s i o n t o b e m a d e . E x a m p l e s a r e g i v e n i n S e c t i o n 3 . 5 .

I n g e n e r a l , t h e r e f o r e , t h e L P s h o u l d b e f o r m u l a t e d i n s u c h a w a yt h a t c o n s i s t s o f a l l u n k n o w n v a r i a b l e s a n d p a r a m e t e r s t h a t a r e r e l e v a n t t ot h e s t a t i s t i c a l p r o b l e m . ( A n y a t t e m p t t o p r e c i s e l y d e f i n e " r e l e v a n t t o t h es t a t i s t i c a l p r o b l e m " w o u l d i n v o l v e b o t h d e c i s i o n t h e o r y a n d m o d e l f o r m u l a t i o n ,a n d l e a d u s t o o f a r a s t r a y . ) T h e m a j o r d i f f i c u l t y w i t h w o r k i n g i n s u c h g e n e r a l i t y i s t h a t o f d e f i n i n g w h a t i s t h e n m e a n t b y a l i k e l i h o o d f u n c t i o n f o r ( c f . B a y a r r i , D e G r o o t , a n d K a d a n e ( 1 9 8 7 ) ) . W e h a v e o p t e d f o r d i s c u s s i n g t h i sg e n e r a l s i t u a t i o n o n l y i n S e c t i o n 3 . 5 , t h o u g h w e b e l i e v e t h a t v i r t u a l l y a lli s s u e s r a i s e d f o r t h e s p e c i a l c a s e o f b e i n g t h e m o d e l p a r a m e t e r a l s o a p p l yt o a p p r o p r i a t e f o r m u l a t i o n s o f t h e g e n e r a l s i t u a t i o n . I n a n y c a s e , i t i si m p o r t a n t t o k e e p i n m i n d t h e q u a l i f i c a t i o n t h a t m u s t c o n t a i n a l l u n k n o w n sr e l e v a n t t o t h e p r o b l e m f o r t h e L P t o b e v a l i d i n i ts s i m p l e f o r m .

A s e c o n d q u a l i f i c a t i o n f o r t h e L P i s t h a t i t o n l y a p p l i e s f o r af u l l y s p e c i f i e d m o d e l { f f l} I f t h e r e is u n c e r t a i n t y i n t h e m o d e l , a n d if o n ed e s i r e s t o g a i n i n f o r m a t i o n a b o u t w h i c h m o d e l i s c o r r e c t , t h a t u n c e r t a i n t y m u s tb e i n c o r p o r a t e d i n t o t h e d e f i n i t i o n o f .

A t h i r d q u a l i f i c a t i o n i s t h a t , i n a p p l y i n g t h e L P t o t w o d i f f e r e n te x p e r i m e n t s , i t i s i m p e r a t i v e t h a t b e t h e s a m e u n k n o w n q u a n t i t y i n e a c h .T h u s , i n E x a m p l e 9 , w e a s s u m e d t h a t r e p r e s e n t e d t h e s a m e s u c c e s s p r o b a b i l i t yi n e i t h e r t h e b i n o m i a l o r n e g a t i v e b i n o m i a l e x p e r i m e n t . I n a p p l y i n g t h e L P t ot w o d i f f e r e n t e x p e r i m e n t s , w e a l s o r e q u i r e t h a t t h e c h o i c e o f a n e x p e r i m e n t b en o n i n f o r m a t i v e ( e . g . i m p l e m e n t e d b y a c h a n c e m e c h a n i s m n o t i n v o l v i n g ) ;


38/252


39/252


40/252


41/252

T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S 2 5

e x p e r i m e n t . ( T h e " s t r u c t u r a l t h e o r y " o f F r a s e r a n d t h e " p i v o t a l t h e o r y " o fB a r n a r d d e e m a d d i t i o n a l i n f o r m a t i o n r e l a t i n g X , , a n d t h e r a n d o m n e s s t o b ei m p o r t a n t , h o w e v e r . T h i s i s s u e w i l l b e d i s c u s s e d in S e c t i o n s 3 . 6 . 4 a n d 3 . 7 . )

T h e o u t c o m e o f t h e e x p e r i m e n t i s t h e d a t a X = x , a n d f r o m E a n d xw e a r e t o i n f e r o r c o n c l u d e s o m e t h i n g a b o u t ( o r a b o u t s o m e t h i n g r e l a t e d t o ) . F o l l o w i n g B i r n b a u m ( 1 9 6 2 a ) , w e w i l l c a l l t h i s i n f e r e n c e , c o n c l u s i o n , o rr e p o r t t h e e v i d e n c e a b o u t a r i s i n g f r o m E a n d x , a n d w i l l d e n o t e t h i s b yE v ( E , x ) . W e p r e s u p p o s e n o t h i n g a b o u t w h a t t h i s e v i d e n c e i s ; it c o u l d ( a t t h i ss t a g e ) b e a n y s t a n d a r d m e a s u r e o f e v i d e n c e , o r s o m e t h i n g e n t i r e l y n e w . ( S i n c eE i s a n a r g u m e n t , i t c o u l d c e r t a i n l y b e a f r e q u e n t i s t m e a s u r e . ) A l s o , w e d on o t p r e c l u d e t h e p o s s i b i l i t y t h a t E v ( E , x ) d e p e n d s o n " o t h e r i n f o r m a t i o n , " s u c ha s p r i o r i n f o r m a t i o n a b o u t , o r a l o ss f u n c t i o n i n a d e c i s i o n p r o b l e m . T h ef o c u s w i l l b e o n t h e m a n n e r i n w h i c h t h e " r e p o r t " E v ( E , x ) s h o u l d d e p e n d o n E a n dx . ( D a w i d ( 1 9 7 7 ) p r e f e r s t o t a l k a b o u t m e t h o d s o f i n f e r e n c e b a s e d o n E a n d x ,a n d p r i n c i p l e s w h i c h t h e s e m e t h o d s s h o u l d s a t i s f y . I n a s e n s e , b y l e t t i n gE v ( E , x ) d e n o t e w h a t e v e r c o n c l u s i o n o n e i s g o i n g t o r e p o r t , w e a r e a l s o t a k i n gt h i s v i e w , w h i l e k e e p i n g B i r n b a u m ' s n o t a t i o n . ) A s o n e f i n a l p o i n t , E v ( E , x )c o u l d b e a c o l l e c t i o n o f " e v i d e n c e s " a b o u t , o b v i a t i n g t h e c r i t i c i s m t h a t t h eL P i s b a s e d o n t h e a s s u m p t i o n t h a t a s i n g l e m e a s u r e o f e v i d e n c e e x i s t s .

T h e C o n d i t i o n a l i t y P r i n c i p l e e s s e n t i a l l y s a y s t h a t , i f an e x p e r i m e n t i s s e l e c t e d b y s o m e r a n d o m m e c h a n i s m i n d e p e n d e n t o f , t h e n o n l y t h ee x p e r i m e n t a c t u a l l y p e r f o r m e d is r e l e v a n t . ( T h e s e l e c t i o n m e c h a n i s m i sa n c i l l a r y , s o t h i s i s a v e r s i o n o f c o n d i t i o n i n g o n a n a n c i l l a r y s t a t i s t i c . )T h e g e n e r a l c o n d i t i o n a l i t y p r i n c i p l e i s n o t n e e d e d h e r e . I n d e e d w e n e e d o n l yt h e f o l l o w i n g c o n s i d e r a b l y w e a k e r p r i n c i p l e , n a m e d b y B a s u ( 1 9 7 5 ) .

W E A K C O N D I T I O N A L I T Y P R I N C I P L E ( W C P J . S u p p o s e t h e r e a r e t w o e x p e r i m e n t s1 2

E = ( X , , { f } ) a n d E 9 = ( X 9 , , ( f Q } ) w h e r e o n l y t h e u n k n o w n p a r a m e t e r n e e d b e c o m m o n t o t h e t w o e x p e r i m e n t s . C o n s i d e r t h e m i x e d e x p e r i m e n t E * ,w h e r e b y J = 1 o r 2 i s o b s e r v e d , e a c h h a v i n g p r o b a b i l i t y ^ ( i n d e p e n d e n t o f ,X , , o r X p j j a n d e x p e r i m e n t E , i s t h e n p e r f o r m e d . F o r m a l l y , E * = ( X * , , f } ) >


42/252


w h e r e X * = ( J , X j ) a n d f * ( ( j , X j ) ) = \ f ( j ) hen>E v ( E * , ( j , . ) ) = E v ( E . , . ) ,j J J

i . e . , t h e e v i d e n c e a b o u t f r o m E * i s j u s t t h e e v i d e n c e f r o m t h e e x p e r i m e n ta c t u a l l y p e r f o r m e d .

T h e W C P i s n o t h i n g b u t a f o r m a l i z a t i o n o f E x a m p l e 2 , a n d h e n c e i se s s e n t i a l l y d u e t o C o x ( 1 9 5 8 ) . I t is h a r d to d i s b e l i e v e t h e W C P , y e t , a sm e n t i o n e d a f t e r E x a m p l e 2 , e v e n t h e W C P a l o n e h a s s e r i o u s c o n s e q u e n c e s .

T u r n i n g f i n a l l y t o t h e f a m i l i a r c o n c e p t o f s u f f i c i e n c y , w e s t a t et h e f o l l o w i n g w e a k v e r s i o n ( n a m e d b y D a w i d ( 1 9 7 7 ) ) .W E A K S U F F I C I E N C Y P R I N C I P L E ( W S P ) . C o n s i d e r a n e x p e r i m e n t E = ( X , , { f ) ) , a n ds u p p o s e T ( X ) i s a s u f f i c i e n t s t a t i s t i c f o r . T h e n , i f T ( x ^ ) = T ( x 2 ) ,E v t E ^ ) = E v ( E , x 2 ) .

T h e L P w il l b e s e e n t o f o l l o w d i r e c t l y f r o m t h e W C P a n d W S P . Av a r i e t y o f a l t e r n a t e p r i n c i p l e s a l s o l e a d t o t he L P ( c f . B a s u ( 1 9 7 5 ) , D a w i d( 1 9 7 7 ) , B a r n d o f f N i e l s e n ( 1 9 7 8 ) , B e r g e r ( 1 9 8 4 a ) , B h a v e ( 1 9 8 4 ) , a n d E v a n s ,F r a s e r , a n d M o n e t t e ( 1 9 8 5 c , 1 9 8 6 ) ) . T h e W C P a nd W S P a r e t h e m o s t f a m i l i a r ,h o w e v e r . A n o t h e r p r o m i n e n t p r i n c i p l e is " M a t h e m a t i c a l E q u i v a l e n c e , " g i v e n i nB i r n b a u m ( 1 9 7 2 ) . T h i s p r i n c i p l e i s a w e a k v e r s i o n o f t h e s u f f i c i e n c y p r i n c i p l e ,s t a t i n g t h a t i f , i n a g i v e n e x p e r i m e n t E , f ( x , ) = f ( x 0 ) f o r a l l , t h e nE v ( E , X j ) = E v ( E , x 2 ) . O n e c o u l d b a s e t h e L P o n m a t h e m a t i c a l e q u i v a l e n c e , p l u sa m i n o r g e n e r a l i z a t i o n o f t h e W C P . T h e w e a k e n i n g o f s u f f i c i e n c y i s c a r r i e d tot h e u l t i m a t e i n E v a n s , F r a s e r , a n d M o n e t t e ( 1 9 8 6 ) , w h i c h d e r i v e s t h e L P s o l e l yf r o m a g e n e r a l i z e d v e r s i o n o f t h e c o n d i t i o n a l i t y p r i n c i p l e .3 . 3 .2 A x i o m a t i c D e v e l o p m e n t

T h e f o r m a l s t a t e m e n t o f t h e L P i s a s f o l l o w s .F O R M A L L I K E L I H O O D P R I N C I P L E . C o n s i d e r t w o e x p e r i m e n t s E , = { X , , , { f 1 } ) a n dE = ( X o s { f g } ) > w h e r e i s t h e s a m e q u a n t i t y i n e a c h e x p e r i m e n t . S u p p o s et h a t f o r t h e p a r t i c u l a r r e a l i z a t i o n s x a n d x | f r o m E , a n d E 2 r e s p e c t i v e l y ,


43/252

T H E L I K E L I H O O D P R I N C I P L E AND G E N E R A L I Z A T I O N S 2 7* x * ( ) = * )x 1 x2

cons tan t c ( i . e . , f ( x ) = c f .( x ) / or ) . T/zercE v ( E x * ) = E v ( E2 , x * ) .

LIKELIHOOD PRIN IPLECOROLLARY.If E = (X, , {fQ}) is an experiment, thenEv E,x) should depend on Eand x only through ) .

THEOREM 1 Birnbaum 1962 a) ). The Formal Likelihood Principle follows from theWCPand the SP. The converse is also true.Proof I f E1 and E,, are the two experiments about , consider the mixede x p e r i m e n t E* as defined in the WCP. From the WCP weknow tha t( 3 . 3 . 1 ) Ev E*, j ,x . . ) ) = Ev Ej f Xj ) .

N e x t , t h i n k i n g s o l e l yof i t h r a n d o m o u t c o m e ( J , X j ) , c o n s i d e rt h e s t a t i s t i c

f ( l . x f ) ifJ =2,X2= xT ( J . X j )=

( J , X j ) o t h e r w i s e .( T h u sthe twoo u t c o m e s ( l , x ^ )and( 2 , x ^ ) r e s u l tinthes a m e v a l u eofT.)T isa s u f f i c i e n t s t a t i s t i c for . T h i sisc l e a r , s i n c e

P ( X *=( j , X j ) | T = tf( l , x | ) )=1 if(j.xJ= t

0 o t h e r w i s e ,and

P ( X *=( l , x f ) | T= 1 P ( X *=( 2 , x | ) | T=( l . x f ) )

f ( x ) +\a l lof h i c h arei n d e p e n d e n tof. The WSP t h u s i m p l i e s t h a t( 3 . 3 . 2 ) E v ( E * , ( l , x f ) )=E v ( E * , ( 2 , x * ) ) .


44/252


C o m b i n i n g ( 3 . 3 . 1 ) a n d ( 3 . 3 . 2 ) e s t a b l i s h e s t h e r e s u l t .T o p r o v e t h a t t h e L P i m p l i e s t h e W C P , o b s e r v e t h a t , f o r E * ,

T h i s i s c l e a r l y p r o p o r t i o n a l t o f ? ( x , ) , t h e l i k e l i h o o d f u n c t i o n i n E . w h e n x . j j j

i s o b s e r v e d , s o t h e L P i m p l i e s t h a tE v ( E * , ( j , X j ) ) = E v E j . X j ) .

T o p r o v e t h a t t h e L P i m p l i e s t h e W S P , i t s u f f i c e s t o n o t e t h a t , i fT (x j ) = T ( x 2 ) i n a n e x p e r i m e n t f o r w h i c h T i s s u f f i c i e n t , t h e n x 1 a n d x ^ h a v ep r o p o r t i o n a l l i k e l i h o o d f u n c t i o n s . ||P r o o f o f t h e L P C o r o l l a r y . F o r g i v e n x * X , d e f i n e

1 i f X = x *Y =

0 i f X f x * ,a n d n o t e t h a t Y h a s d i s t r i b u t i o n g i v e n b y( 3 . 3 . 3 ) f j ( l ) = f ( x * ) = l f j ( ) .F o r t h e e x p e r i m e n t E * o f o b s e r v i n g Y , i t f o l l o w s f r o m t h e L P t h a t

E v ( E , x * ) = E v ( E * , l ) .B u t E * , a n d h e n c e E v ( E * , l ) , d e p e n d o n l y o n f . ( x * ) = v * ( ) ( u s i n g ( 3 . 3 . 3 ) ) . ||

T h e a b o v e r e s u l t s a r e w o r t h d w e l l i n g u p o n f o r a m o m e n t . T h e L P ise x t r e m e l y r a d i c a l f r o m t h e v i e w p o i n t o f c l a s s i c a l s t a t i s t i c s , a s w i l l b e s e e ni n C h a p t e r 4 . Y e t t o r e j e c t t h e L P , o n e m u s t l o g i c a l l y r e j e c t e i t h e r t h e W C Po r t h e W S P . B u t t h e W S P i s , i t s e l f , a c o r n e r s t o n e o f c l a s s i c a l s t a t i s t i c s , a n dt h e r e i s n o t h i n g i n s t a t i s t i c s a s " o b v i o u s " as t h e W C P ( o r E x a m p l e 2 ) .

3 .4 G E N E R A L I Z A T I O N S B E Y O N D T H E D I S C R E T E C A S EB a s u ( 1 9 7 5 ) a n d o t h e r s h a v e a r g u e d t h a t t h e s a m p l e s p a c e x i n a n y

p h y s i c a l l y r e a l i z a b l e e x p e r i m e n t m u s t b e f i n i t e , d u e t o o u r i n a b i l i t y t om e a s u r e w i t h i n f i n i t e p r e c i s i o n . T h i s s u g g e s t s t h a t t h e L i k e l i h o o d P r i n c i p l ef o r d i s c r e t e e x p e r i m e n t s ( as i n S e c t i o n 3 . 3 ) is a l l t h a t o n e n e e d s . W e a r e


45/252

THE LIKELIHOOD PRINCIPLE AND GENERALIZATIONS 2 9

p h i l o s o p h i c a l l y i n a g re e m e n t w i t h t h i s .On th e o th e r han d , co n t in u o us and o th e r more g en e r a l p r o b a b i l i ty

d i s t r i b u t i o n s a r e e n or m ou sly u s e f u l in s i m p l i f y i n g s t a t i s t i c a l c o m p ut a t io nsa nd i n p r o v id i n g n u m e r i c a l a p p r o x im a t i o n s w hic h a r e o f t e n q u i t e a c c u r a t e . I ti s p o s s i b le f o r t h e l i k e l i h o o d f u n c t i o n f o r a c o nt in uo u s m ode l t o d i f f e rs t r i k i n g l y f ro m t h a t o f t h e d i s c r e t e m ode l i t i s i n t e n d e d t o a p p r o x im a t e , soit is not obvious t h a t t h e v a l i d i t y o f t h e LP in d i s c r e t e p ro ble m s e x t e nd s t oi t s v a l i d i t y i n t h e a p p r o x im a t in g c o nt in uo us p r o ble m s . I n a ny c a s e , e x t e n s io no f th e LP to more g en e r a l s i tu a t i o n s can o n ly s t r e n g th e n i t s case . Such ane x t e n s io n i s o u r t a s k i n t h e p r e s e n t s e c t i o n .

As i n S e c t io n 3 . 3 , an e x pe r im e n t E = ( X , , { P J ) w i l l be u nd e rs to odto inv olv e t h e o b s e r v a t io n o f t h e random v a r i a b l e X , h a vin g p r o b a b i l i t y d i s t r i b u t i o n P on X9 . ( I t w i l l n o t be n ecessar y to assume th e ex i ste n ce o fa d e n s i t y . ) T he r e i s , u n a vo i d a b ly , m e a s u r e t h e o r e t i c m a t he m a t ic s i n t h i ss e c t i o n , b ut t h e s e c t i o n ca n be s k ip p e d , i f d e s i r e d , w i t h o u t a ny e s s e n t i a ll o s s o f c o n t i n u i t y .

The sample space w i l l b e assumed to b e a l o cal l y co m p a ctH au sd o r f f space whose to p o l o g y ad mi ts a co u n tab l e b ase ( LCCB sp ace , f o r sh o r t ) ,and the P w i l l b e assumed to b e B o r e l m easur es . Of co u r se, X o f te n ar i se s asa n % v a l u e d r a n d om v a r i a b l e o n a p r o b a b i l i t y s p a c e ( , 3 , { } ) e q u i p p e d w i t hua f a m i l yofp r o b a b i l i t y m e a s u r e s i n d e x e dby 6 . S u c h u n d e r l y i n g s t r u c t u r ew i l lnot ber e l e v a n t in oura n a l y s i s , h o w e v e r .3 . 4 . 1 D i f f i c u l t i e s in theN o n d i s c r e t e C a s e

I nane x p e r i m e n t E = (X, ,{ P Q} )for w h i c h t h e r e is an x Xs a t i s f y i n g P {x}) = 0 for e v e r y , it isd i f f i c u l t toa s s i g n any p a r t i c u

l a r m e a n i n g to n E v ( E , x ) " . Fore x a m p l e , B a s u ( 1 9 7 5 ) and J o s h i ( 1 9 7 6 ) h a v eo b s e r v e d t h a t an a i v e a p p l i c a t i o n ofB i r n b a u m ' s ( 1 9 6 2 a ) s u f f i c i e n c y p r i n c i p l ew o u l d s u g g e s t fors u c h an xt h a t E v ( E , x ) =E v ( E , y ) fore v e r yy X> s i n c e them a p T: x- w h i c h t a k e s x o n t o y and l e a v e s all o t h e r p o i n t s ( i n c l u d i n g y )f i x e d is s u f f i c i e n t for . T h i s isp a r t i c u l a r l y d i s t u r b i n g forc o n t i n o u s


46/252

30 THE LIKELIHOOD PRINCIPLE

d i s t r i b u t i o n s , s i n c e t h e n PQ ( { x } ) = 0 f or every x e andeve r y e ; B i r n b a u m ' ss u f f i c i e n c y p r i n c i p l e t h en s ug g e st s t h a t a l l p o s s i b l e o b s e r v a t i o n s l e n dp r e c i s e l y the same evidence and t h e r e f o r e n on e) a b o u t .

T h e u n i q u e s p e c i f i c a t i o n of a l i k e l i h o o d f u n c t i o n c auses s i m i l a rp r o b l e m s . I f t h e r e is no s i n g l e f i n i t e m easure v onz whose n u l l se t sc o i n c i d e w i t h t h o s e B o r e l s e t s N fo r wh i c h P. N) = 0 fo r al l 6 , t h e n nol i k e l i h o o d f u n c t i o n e x i s t s . T h i s is the u s u al s t a t e of a f f a i r s in nonp a r a m e t r i c p ro b le m s ( r e c a l l t h a t c o u l d be an a r b i t r a r y i nd e x set ) and can evena r i s e i n s i m p l e p a r a m e t r i c e x a m p l e s ; f o r e x a m p l e , PQA) =j //\dx+ j U ) > =X = [ 0 , 1 ] , d e s c r i b e s an e x p e r i m e n t in wh i c h X= w i t h p r o b a b i l i t y j andi s o t h e r w i se u n i f o r m l y d i s t r i b u t e d o ve r the u n i t i n t e r v a l ; no f i n i t e m easurev domina tes {P } , and no l i k e l i h o o d f u n c t i o n e x i s t s . ( I n c i d e n t a l l y , t h i s seemst o be a s o u r c e of c o n f u s i o n in c e r t a i n c o u n t e r ex a m p l e s to the LP such as thesecond examp le in S e c t i o n 2.5 of B ir n b au m ( 1 9 6 9 ) . )

Even in p r ob l ems whe r e t he r e is a measure v w i t h the i n d i c a t e dp r o p e r t i e s , the Ra don N ik odym d e r i va t i v es

U ) = f f l x) = P fldx)/v dx)are determined only up to sets of vmeasure zero; these functions of could bespecif ied in an entir ely arbit rary manner for al l x in any set N c z withv N) = 0. One way to salvage a likelihood princ iple in the face of suchambiguity is to specify a part icular version of P dx)/ v dx) for each ; for

example, in case a valmost everywhere) continuous density exists we couldset = {open neighborhoods of x }and put

U ) = in f sup P_ U)/v U))ve U

UcVf o r x in the support of v, i ) = 0 otherwise.

By res tr ic t ing our attent ion to valmost everywhere) continuousdensit ies, continuous suff ic ient s tat ist ics, etc. we could develop versions ofthe conditionality, sufficiency, and likelihood principles very similar tothose in the discrete setting.


47/252


48/252

3 2 THELIKELIHOOD PRINC IPLE

a n o t h e r LCCB s p a c e r . Thes t a t i s t i cT de t e rm ine s a f a m i l y {P } ofB o r e lmeasures on by

P I A = P A ) ) ,a n d h e n c eane x p e r i m e n tE =( T , j, { P } ) U n l e s sTis 11 we e x p e c t (ing e n e r a l ) t h a tE w i l l t e l lusl e s s a b o u tt h a n E,s i n c e d i f f e r e n t o u t c o m e sx w i t h p o s s i b l y d i f f e r e n t e v i d e n t i a l i m p o r t can be m a p p e d o n t o the s a m eT ( x ) e x . The e x c e p t i o n a l c a s e is t h a t in w h i c hTis s u f f i c i e n t .D E F I N I T I O N . For the x p e r i m e n tE s u p p o s e t h e r e e x i s t saf a m i l y { g . :t J }o f B o r e l p r o b a b i l i t y m e a s u r e s on s a t i s f y i n g

P ( A ) = / g t ( A ) p J ( d t ) = / g ( ) ( A ) P ( d x )for all Borel sets A c t ThenTis called sufficient11 (or sometimes11sufficient for ).

N o t e t h a t g. is notp e r m i t t e d to depend upon ;o t h e r w i se g. = PQwould a l wa y s work . Anyo n e t o o n e m e a s u r a b le m a p p in gT is s u f f i c i e n t ; j u s tl e t g tbe a poi nt ma s s at T t )X

The Su f f ic i en cy P r in c i p l e makes prec ise the n o t io n t h a t T(x) in jt e l l s as much about as x in E;

SUFFICIENCY PRINCIPLE (SP). If T: X + J is sufficient, thenE v E , x ) =E v E T , T x ) ) fo r {P } a. e. x X

Again we mayd e l e t e the im po ss ible outcomes whenx is countable toremove the {PQ} - a.e. q u a l i f i c a t i o n andconclude tha t Ev E,x) = E v E , y )w h e n e v e ras u f f i c i e n t s t a t i s t i cTs a t i s f i e s T ( x )=T ( y ) , and sor e c o v e r thed i s c r e t eWSP ofS e c t i o n 3 . 3 . 1 .3 . 4 . 3 . TheR e l a t i v e L i k e l i h o o d P r i n c i p l e

L e t E ]=(X.|, ,{ P ' } ) and Ep=( X 2 , ,{P })be twoe x p e r i m e n t sa n d s u p p o s e ( f o r m o t i v a t i o n a l p u r p o s e s ) t h a t e a c h a d m i t sal i k e l i h o o d f u n c t i o n ,


49/252

THE LIKELIHOOD PRINCIPLE ANDGENERALIZATIONS 33

i . e . a f ini t e m e a s u r e v. on the s a m p l e s p a c e Z and a f a m i l y { f l ) } of1 I i n t e g r a b l e f u n c t i o n s s a t i s f y i n gP Q ( A )=/ f j ( x ) v . ( d x ) , A c z . .

T h e L i k e l i h o o d P r i n c i p l e ( w e r eit toh o l d h e r e ) w o u l d a s s e r t t h a tE v ( E 1 , x 1 )=E v ( E 2 , x 2 )

w h e n e v e r f n ( x )=c f Q ( x o )fora l lands o m e c o n s t a n t c =c ( x , , x 0 ) not1 2d e p e n d i n g on ,i . e . w h e n e v e r t h e r e l a t i v e l i k e l i h o o dc =f Q ( x ) / f Q ( x o ) d o e s I D Cnot dependon . Ourfreedom to s p e c i f y ^ Q X ^ ) a r b i t r a r i l y w he ne ve rv ^ x ^ } ) = 0 makes i t c l e a r t h a t t h i s p r i n c i p l e n e e d s r e f o r m u l a t i o n b e f o r e it iss u i t a b l e for experiments with uncountable sample spaces. (However, at p o in t sx, and x which areatoms of v, and v> r e s p e c t i v e l y , the LP is r e a s o n a b l e ,andca n beshown to fol low from theWCPand SP as in S e c t i o n3 . 3 . )

To develop a s u i t a b l e g e ne r a l p r i n c i p l e , weg e n e r a l iz e theconceptt h a t ther e l a t i v e l ik e l ih o o d of x j and x2 is independent of . B a s i c a l l y , i fa mapping exists between twosubsets of andX for which the RadonNikodymd e r i v a t i v e of the induced measure w ith resp ec t to thee xist in g measure on, say,

Z , ) is independent of , then we cane s t a b l is h an equ ivalen ce of evidencebetween thecorresponding obse rvat ions in thesu b set s . Thereasons forg e n e r a l i z i n g the LP in t h i s d i r e c t i o n are : i ) I t can be s t a t e d in g r e a tg e n e r a l i t y , w i t h o u t r e q u i r i n g m o d e l s or d e n s i t i e s ; i i ) I t w i l l beshown tofol low from theWCPand SP, as did the LP; and i i i ) I t , in t u r n , can beshownt o imply in s u b s ta n t ia l g e n e r a l it y ) theS t o p p in g Ru le P r in cip le and CensoringP r i n c i p l e , b e s i d e s h a v i n g d i r e c t l y i m p o r t a n t i m p l i c a t i o n s of i t s own. Them a j o r l i m i t a t i o n of the RLP(comparedto the LP) is t h a t i t does not provideany such convenient summarization of evidence as the l i k e l i h o o d f u n c t i o n ( w h ic hneednote x i st in theg e n e r a l c a s e ) .RELATIVE LIKELIHOOD PRINCIPLE RLP) . Let : Uj + l be a Borel bimeasurableone toone mapping from U, c %..onto IL


50/252


51/252

T H E L I K E L I H O O D P R I N C I P L E A ND G E N E R A L I Z A T I O N S 35f ( x , ) v a n i s h e s for all, sothe" { P } a . e . " q u a l i f i c a t i o n is

U I u n n e c e s s a r y ) . ||T H E O R E M 3.TheCPan dt h e SP t o g e t h e r i m p l y theRLP.P r o o f . Let E,andEp betwoe x p e r i m e n t s , ab i m e a s u r a b l e m a p p i n g f r o ma B o r e ls e t U^e %1 o n t o U ^ ^ ,and : U^ >( 0,) a m e a s u r a b l e f u n c t i o n s a t i s f y i n g

P < A )= / . ] C V c ( x ) ] p J ( d x ) (A)f o r a l l Bore l A c I L , a l l . Le t E* be t he mi x ture of E, and E2 , andd e f i n e a mapping T: X * X * by

) i f i = 1 a nd x]T ( i , x . ) =

]

e l s e .T h i s d e t e r m i n e sanewe x p e r i m e n t E* T= ( T , X*, { P } ) , w h e r e P I ( A )=

F i r s twe s h o w t h a t T is s u f f i c i e n t . Fore a c h t =(i,x 6X*

d e f i n ea m e a s u r e g.onz * by ( A . ) = e t ( A ) if i = 1 or x.

g t ( A )=( c ( A , ) + ( A 2 ) ) / ( l + c ) if 1 = 2, ,andx., = " ] ( x 2 ) .

H e r e c = c ( x ) ande , , . d e n o t ethe u n i t p o i n t m a s s e s at xj Z- , x 2 Z o >t X*r e s p e c t i v e l y ; A. d e n o t e s { x . X ^ : ( l ^ x ^ ) A } . It is s t r a i g h t f o r w a r dt o v e r i f y t h a t

P * ( A ) = / g t( A ) p J ( d t )f o r e a c h B o r e l A c ^ * , so T is s u f f i c i e n t .

B y the SP we can c o n c l u d e t h a tE v ( E * , ( l , X l ) ) = E v ( E * T , ( 2 , ( . , ) ) ) andE v ( E * , ( 2 , x 2 ) ) = E v ( E *T , ( 2 , x 2 ) )

^ 6 X and { P } a . e . x2 X r In pa rt icu lar , foror - a.e.


52/252

3 6 HE LIKELIHOOD PRINCIPLE

{ P * } a . e . ) x G U and X2 = ^ x ^ we ha ve

E v ( E * , ( l , x 1 ) ) = E v ( E * T , ( 2 , x 2 ) ) = Ev ( E * , ( 2 , x 2 ) ) .

B y t h e WCP we hav e

E v U M l . X j ) ) = EvE^Xj) and E v ( E * , ( 2 , x 2 ) ) = E v ( E 2 > x 2 ) ,

s o we ca n c o n c l u d e t h a t

EvEj.Xj) =Ev E2, f o r { ?h a . e . x , E U | |

T h eRLP w i l l be u s e d inC h a p t e r 4 to e s t a b l i s h g e n e r a l v e r s i o n s ofi m p o r t a n t c o n s e q u e n c e s oft h e L P . T h e o r e m 3 d e m o n s t r a t e s t h a t r e j e c t i o n oft h e s e c o n s e q u e n c e s ( a nd s e v e r a lare u i t e u n p a l a t a b l e f r o m the f r e q u e n t i s tv i e w p o i n t ) i m p l i e s r e j e c t i o nof the W CP r the SP.3 . 5 P R E D I C T I O N , D E S I G N , N U I S A N C E P A R A M E T E R S , A N D T H E LP3 . 5 . 1 I n t r o d u c t i o n

T h e LP ass t a t e d a b o v e has he w e r y i m p o r t a n t q u a l i f i c a t i o n t h a t itd o e s not a p p l y if d o e s not i n c l u d e a ll u n k n o w n q u a n t i t i e s g e r m a n e to the exp e r i m e n torp r o b l e m . Fori n s t a n c e , ind e s i g n orp r e d i c t i o n p r o b l e m s the unk n o w n f u t u r e o b s e r v a t i o n is o b v i o u s l y r e l e v a n t , a n d y e t is notn e c e s s a r i l y ap a r tof thep a r a m e t e r d e f i n i n gthe i s t r i b u t i o n oft h e o b s e r v a b l e X. Ar e l a t e d d i f f i c u l t yis t h a t , o f t e n , o n l y a p a r t of isr e a l l y ofi n t e r e s t , ther e m a i n d e r b e i n ga " n u i s a n c e " p a r a m e t e r . T h e s e i s s u e s are e x p l o r e d in t h i ss e c t i o n .

W e b e g i n bye x p a n d i n g the e f i n i t i o n of toi n c l u d e u n o b s e r v e d andn u i s a n c e v a r i a b l e s . D e f i n e

= (z ) = ( y , w ; , ) ,w h e r e z = ( y , w ) is the v a l u e of anu n o b s e r v e d v a r i a b l e Z,w i t h y b e i n g of int e r e s t and b e i n g a n u i s a n c e v a r i a b l e , and w h e r e = ( , n ) is thep a r a m e t e r


53/252

T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S 3 7

t h a t d e t e r m i n e s t h e d i s t r i b u t i o n s o f b o t h X a n d Z , w i t h b e i n g o f i n t e r e s t a n d b e i n g a n u i s a n c e p a r a m e t e r . ( W e w i l l p u r p o s e f u l l y r e m a i n v a g u e o n t h e d e f i n i t i o n o f " n u i s a n c e v a r i a b l e " a n d " n u i s a n c e p a r a m e t e r " ; f o r m a l d e f i n i t i o n s c o u l db e a t t e m p t e d a l o n g d e c i s i o n t h e o r e t i c l i n e s , b u t w o u l d t a k e u s t o o f a r a f i e l d . )T o i n d i c a t e t h a t e v i d e n c e a b o u t a n d y i s d e s i r e d f r o m E w e w i l l w r i t e

E V ? (E' X)f o r t h e e v i d e n c e a b o u t a n d y f r o m t h e o b s e r v a t i o n o f x i n an e x p e r i m e n t E .

T w o d i f f i c u l t i e s a r i s e i n a t t e m p t i n g t o a p p l y t h e L P i n t h i s m o r eg e n e r a l c o n t e x t . T h e f i r s t i s t h a t t h i s g e n e r a l i z e d i s n o l o n g e r j u s t t h ep a r a m e t e r d e f i n i n g t h e d i s t r i b u t i o n o f X . T h u s t h e d e f i n i t i o n i n ( 3 . 1 . 1 ) o fi ( ) a s t h e d e n s i t y o f X g i v e n m a y n o l o n g e r b e a s u i t a b l e d e f i n i t i o n . I n d e e d , i f Z i s c o n d i t i o n a l l y i n d e p e n d e n t o f X g i v e n , t h e n ( by t h e d e f i n i t i o no f c o n d i t i o n a l i n d e p e n d e n c e ) i t c a n b e s h o w n t h a t ( 3 . 1 . 1 ) b e c o m e s

I ( ) f ( x ) = f ( x ) ,x z , 01 w h i c h d o e s n o t e v e n i n v o l v e z . T h e s e c o n d d i f f i c u l t y i s t h a t t h e n u i s a n c ep a r a m e t e r , , w i l l a p p e a r i n t h i s l i k e l i h o o d f u n c t i o n e v e n t h o u g h i t i s n o to f i n t e r e s t .

T o r e s o l v e t h e

Documents

Berger the Likelihho Principle 1988 Book