Priors, Utilities, Elicitation & Pharmaceutical R&D

Andy GrieveStatistical Research Centre

Pfizer Global R&D

2

Outline

• Use of Bayesian Methods in Pharmaceutical R&D

• Three Prior Elicitation Examples• Acute toxicity – LD50• Sample Sizing & Confidence Intervals• Counting Tablets in Dosing Dogs

• Elicitation for Internal Company Decision Making – Portfolio Management

04/18/23 CS4NS 3

Are Bayesian Methods Acceptable in Drug Development?

Not Forbidden by Regulation

04/18/23 CS4NS 4

Extracts from Drug Regulations

E4 : Dose Response 1993

Agencies should be open to the use of various statistical and pharmacometric techniques such as Bayesian and population methods, modelling, and pharmacokinetic-pharmacodynamic approaches.

= International Conference on Harmonisation

04/18/23 CS4NS 5

Extracts from Drug Regulations

CPMP Biostatistical Guidelines 1994

.. the use of Bayesian or other approaches may be considered when the reasons for their use are clear and when the resulting conclusions are sufficiently robust to alternative assumptions

= European Medicines

Evaluation Agency

04/18/23 CS4NS 6

Extracts from Drug Regulations

E9 : Statistical Principles in Clinical Trials 1998

Essentially same as CPMP Guidelines

.. the use of Bayesian or other approaches may be considered when the reasons for their use are clear and when the resulting conclusions are sufficiently robust (to alternative assumptions)

LUKEWARM !!!

04/18/23 CS4NS 7

Why ?

Clinical_Trials List (1996)

• Background• Hair-thinning • Researcher Bias

“If I hadn’t believed it, I wouldn’t have seen it with my own eyes”

- Trust

04/18/23 CS4NS 8

• Feeling that use of “subjective priors may allow unscrupulous companies and/or statisticians to attempt to pull the wool over the regulators eyes.” (Greg Campbell – FDA Centre for Devices & Radiological Health)

• If it were that easy they are not very good and we probably need new regulators

Trust

04/18/23 CS4NS 9

• Stephen Senn - “nowhere is the discipline of statistics conducted with greater discipline than in the pharmaceutical industry”

• Nowhere will Bayesian statistics be conducted with more discipline than in the pharmaceutical industry

• Document

Trust

04/18/23 CS4NS 10

• Document• Where did the prior come from ?• Is it based on data? Is it subjective ?

• “to present a Bayesian analysis in which the company’s own prior beliefs are used to augment the trial data will in general not be acceptable to a regulatory agency” (O’Hagan & Stevens, 2001)

• “the frequentist approach is less assumption dependent and can provide the statistical strength of evidence required for a confirmatory trial that may be lacking in a more assumption dependent Bayesian approach” (Chi, Hung & O’Neill – Biopharmaceutical Report, Vol. 9, No.2, 2001)

• SENSITIVITY ANALYSIS

Trust

11

• We work in a Frequentist World

Remember Acceptance of Bayesian methods is Lukewarm

• We will be asked about false positive rates• We will be asked about the impact of multiple

looks at the data• We need to be calibrated

Trust

12

Assessing a Prior in Acute toxicity

13

Motivating Example

Dose (mg/kg)

# of Animals

# of Deaths

500 5 1

1000 5 2

2500 5 3

5000 5 2

Based on these data we wish to determine the LD50 to classify the drug according

to the following classification scheme (Swiss Poison Regs.)Toxicity

Class1 2 3 4 5

Range of LD50

(mg/kg)

< 5 5-50 50-500 500-2000 2000-5000

14

Model

• Data triplets • { di , ni , ri } : i=1,..,k

• Probabilities of response• i : i=1,…,k

• Logistic Model • log [ i / (1-i)] = + log(di)

• Median Lethal dose (LD50)• log(LD50) =

Probit Model : i =( + log(di))

15

Bayesian Solutions

• Likelihood Function

• Prior distribution - p() ( > 0 )

• Define : - log(LD50)

• Inference

ii

i

rnk

ii

ri dGdGXL

1

))log((1))log(()|,(

0

)|,()|( dXpXp

U

L

dXpXP UL )|()|(

16

Likelihood Contours - Motivating Example

Dose (mg/kg

)

# of Animal

s

# of Deaths

500 5 1

1000 5 2

2500 5 3

5000 5 2

17

Likelihood Function - Hypothetical Example

Normal AnalyticApproximation

Dose (mg/kg

)

# of Animal

s

# of Deaths

100 3 1

1000 3 2

18

Motivating Example

Dose (mg/kg)

# of Animals

# of Deaths

500 5 1

1000 5 2

2500 5 3

5000 5 2

Experienced toxicologists will know that they need to span the LD50

with the doses they choose.

The choice of doses contains information concerning the toxicologist’s beliefs about the likely value of the LD50.

19

Choice of Prior 1) Tsutakawa (1975) : logit

• Choose doses d1 & d2

s.t. P(d1<LD50<d2)=0.5

0 10

1

1

2

2 1• Implies 1 and 2 uniform over the half square

• p) : logit n.c.p. probit BN (truncated)

(Grieve , 1988)

• Implies knowledge of i : i ≠1,2

20

Choice of Prior 2) Tsutakawa (1975) - logit

• Choose doses d1 & d2

iii

lmllml

ml

ˆ)2(1

)1()1( 12

12

11

11

222111

• p) : logit n.c.p. probit BN (truncated)

(Grieve, 1988)

• Implies knowledge of i : i ≠1,2

• Assume n.c.p. for 1 and 2

1̂ 2̂• Specify modal responses probabilities and

21

Choice of Prior 3) Grieve (1988) - probit

Toxicity Class

1 2 3 4 5

Probability 0.05 0.15 0.40 0.30 0.05

2

2

0

0 ,

BN• Suppose p() is bivariate normal :

• Can the parameters be determined ? • Not uniquely !!!• The c.d.f. of –depends only on :

430

20

1 ,,, cccc• Implying any 4 probabilities are sufficient to determine c1,c2,c3 and c4

• Any one of the 5 parameters is also needed • Modal slope ? How about median ?• Feedback•

22

Determining a Prior for Sample Sizing

04/18/23 CS4NS 23

Sample Sizing CIs : Simon Day (Lancet, 1988)

2n patients :

(1-)% CI :

Width :

Acceptable Width =

x x s1 22, ,

x x t sn

x x t sn

1 2 1 22 2

,

w t sn

22

w0 nt s

w

8 2

02

04/18/23 CS4NS 24

Alternative Approach : Grieve - Lancet, 1989

Required :

Solve by search

P w w

P t sn

w

Ps w n

t

( )

0

0

2

202

2 2

1

22

1

81

2

04/18/23 CS4NS 25

Simon Day’s Example

Two Anti-HypertensivesDifference in Diastolic BP - 95% CI = 10 mm Hg, w0=10

Grieve - Lancet , 1989

1- n 32 37 39 41

n=32

04/18/23 CS4NS 26

Never be absolutely certain of Never be absolutely certain of anythinganything

Bertrand RussellBertrand Russell

A Bayesian approach is an A Bayesian approach is an unconditional approach accounting unconditional approach accounting for uncertainty in parametersfor uncertainty in parameters

04/18/23 CS4NS 27

Beal - Biometrics , 1989

“ “ A prior estimate of A prior estimate of …… is …… is needed. This clearly introduces needed. This clearly introduces some uncertainty regarding the some uncertainty regarding the required sample size “required sample size “

ConditionalityConditionality

04/18/23 CS4NS 28

Relation Between and n

7 8 9 10 11 12 13 14

n 21 27 33 39 46 54 62 71

Suppose we have some idea about the likely value of through a probabilitydistribution

04/18/23 CS4NS 29

Unconditional Approach

P w w P w w p d( ) ( | ) ( ) 0 02 2 2

2

04/18/23 CS4NS 30

Where do we get p(2) from ?

• Previous studies• Expert opinion - subjective ?

• Estimate of 2 : based on 0 d.f.

• Inverse-Gamma prior

s02

2220

2200

22002

0

0

2

22/)(

/

)(/

)]/(sexp[)/s()(p

04/18/23 CS4NS 31

Conditional Formula

Unconditional Formula – Unconditional Formula – Grieve(1991)Grieve(1991)

P w w Pw n

t( )

0

2 02

2 281

P w w P Fw n

t s( ) ,

0

02

2020 8

1

32

Elicitation of Inverse-Gamma

• Expert provides and s.t.

• Not enough information – assume upper and lower limits are (1-p0)/2 percentiles

• Solve directly or modify algorithm in Martz and Waller(1982 – Bayesian Reliability Analysis), Grieve (1987,1991)

2L

2U

02

2220

2200

22002

2

2 0

0

2

22pd

)(/

)]/(sexp[)/s()(p

U

L/)(

/

04/18/23 CS4NS 33

Illustrative Example

Probability ( 8 < < 13 ) = 0.8

Implies

0=14.66 , =95.55

20s

04/18/23 CS4NS 34

Relation Between P(w<w0) and n

n 51 52 53 54 55 56 57

P(w<w0) 0.873 0.882 0.892 0.899 0.906 0.912 0.918

In this example accounting for uncertaintyincreases the sample size by 40 %

35

Elaborating a Prior for Tablet Counting

36

Checking the Dosing of Dogs

• dogs dosed on mg/kg basis• adjusted weekly

• Example• Unit Dose : 36 mg/kg• Weight : 19.2 kg• Required dose : 691 mg

37

Pre-Manufactured Tablet Strengths

300 mg300 mg 25 mg25 mg 5 mg5 mg 0.5 mg

691 mg

2 23 3

4 - 5 0 -11 0 - 4 0 - 4

38

Dog Dosing

• Tablets placed in a gelatine capsule

• Are the correct number of tablets in the capsule ?

39

Possible Approaches• Do Nothing

• Hope - No : Inspection• Acceptance Sampling

• too few samples - 308 capsules/wk• checking creates errors

• Check Everything• checking creates errors

• Weigh Capsules & Contents

40

Tablet /Capsule Weights

Tablet Strength (mg)

Mean (g) St. Dev.

300 0.602 0.0036

25 0.298 0.0035

5 0.150 0.0013

0.5 0.075 0.0008

Capsules 0.701 0.0410

41

Statistical Model

• T tablet sizes• tablet weights are : • capsule weights are :

• Ni tablets of each size chosen

• Total weight w is distributed as

42

Hypothetical Example

2 X 300 mg = 600 mg3 X 25 mg = 75 mg3 X 5 mg = 15 mg2 x 0.5 mg = 1 mg

691 mg

• Given a total weight of 3.397g (simulated)• What can we say about the likely numbers of

tablets?

43

Dog Dosing - Solution (1)

• Co-primal Weights• 3 : 7 : 13 : 23 instead of 1 : 2 : 4 : 8

44

Dog Dosing - Solution (2)

• Co-primal Weights• 3 : 7 : 13 : 23 instead of 1 : 2 : 4 : 8

• Pre-Weighing of Capsules

45

Dog Dosing - Weights

Tablet Strength (mg)

CV (%)

300 0.6

25 1.2

5 0.9

0.5 1.1

Capsules 5.8

46

Solution (3)

• Co-primal Weights• 3 : 7 : 13 : 23 instead of 1 : 2 : 4 : 8

• Pre-Weighing of Capsules• Prior distribution

belief in ability to count to 5

greater than

belief in ability to count to 19

47

Elaborating a Prior Grieve et al (1994)• Suppose a technician tries to count to M

tablets of a given strength• A model of the process could be :

• The total of M tablets is “achieved” by M individual operations each attempting to count to 1

• An error can be made in either direction : xj=0,1 or 2

• The total count is : x1+x2+ …. + xM=N

48

Elaborating a Prior

• Suppose the probability distribution of results from a single count is given by :

x 0 1 2

P(xj=x) q r pwith p.g.f. - P(t)=q+rt+pt2

212121

1

1

2

22

0 0 2121

2

)!(!

!

)()()(

kkMkkMkkM

k

cM

k

NNN

tprqkkMkk

M

ptrtqtPtP

•Assuming independent counts the p.g.f. of N is :

Mcn]/)n[(M

)nM,max(k

knMk prq)!Mkn()!knM(!k

!M)nN(P

21

0

22

22

• Giving :

49

Elaborating a Prior

ADVANTAGES

• A prior distribution need not be elicited for every M

• Elaboration ensures consistency• If Mk=Mj+1 then P(Nk=Mk) < P(Nj=Mj)

DISADVANTAGES

• Need to elicit p,q (r=1-p-q)

• Assumptions

50

Feeding Back

MValues of r ( p=q=(1-r)/2)

0.9 0.95 0.99 0.995

0.999

2P(N=M) 0.81

50.904 0.98

00.99

00.99

8

P(N=M1) 0.090

0.048 0.010

0.005

0.001

4P(N=M) 0.68

00.821 0.96

10.98

00.99

6

P(N=M1) 0.147

0.086 0.019

0.010

0.002

6P(N=M) 0.58

10.750 0.94

20.97

10.99

4

P(N=M1) 0.183

0.117 0.029

0.015

0.003

8P(N=M) 0.50

70.689 0.92

40.96

10.99

2

P(N=M1) 0.204

0.141 0.037

0.019

0.004

51

Dog Dosing - Conclusions

• Such a scheme is practicable• Computations trivial• Pre-weighing essential• Prior distribution essential

• Perfect for robotification