39
What is wrong with What is said? Hasan Yazici, MD University of Istanbul

What is wrong with What is said ?

  • Upload
    kyoko

  • View
    30

  • Download
    4

Embed Size (px)

DESCRIPTION

What is wrong with What is said ?. Hasan Yazici , MD University of Istanbul. Vitamin-E in gout the Hypothesis. Several recent reports hint that vitamin E can lower serum uric acid levels (1-3). - PowerPoint PPT Presentation

Citation preview

Page 1: What is  wrong  with  What is  said ?

What is wrong with

What is said?

Hasan Yazici, MDUniversity of Istanbul

Page 2: What is  wrong  with  What is  said ?

Vitamin-E in goutthe Hypothesis

• Several recent reports hint that vitamin E can lower serum uric acid levels (1-3).

• These are either single case reports or uncontrolled, open studies in small groups of patients.

• Thus we tested the hypothesis to see whether vitamin E was effective treatment for gout in a 6 month, double blind, placebo controlled study.

Page 3: What is  wrong  with  What is  said ?

What is wrong?

A hypothesis should not be formulated as a question.

Page 4: What is  wrong  with  What is  said ?

Methods - 1• The patients were randomized into the

vitamin E (n = 60) and placebo (n= 60) groups.

• After randomization each patient signed a written informed consent.

• The patients were assessed monthly with serum uric acid, hematocrit, serum creatinine, cholesterol and testosterone levels.

• The number of gout attacks were also noted.

Page 5: What is  wrong  with  What is  said ?

What is wrong?• The randomization follows, not

precedes, the informed consent.

Page 6: What is  wrong  with  What is  said ?

Methods -2 • 30 patients had to leave the study for various

reasons. – 20 patients from the active drug group and 10 from

the placebo left the study before its completion (ns). – Thus the final analysis for drug efficiency was made

among 90 patients. • Among the 20 patients who dropped out from

the treatment group 3 moved to another city. • 15 complained of severe indigestion and no

information was available for the remaining 2. • Dyspepsia was the reason for withdrawal in 7

patients in the placebo group while headaches were the main reason for discontinuation of treatment in the remaining 3.

Page 7: What is  wrong  with  What is  said ?

What is wrong?• The intention to treat principle is

neglected.

Page 8: What is  wrong  with  What is  said ?

Intention to treatanother study

• 90 patients were initially randomized into 30 patients each of placebo, QXY 2mg qd and QXY 5mg qd groups.

• One patient in the QXY 5mg group developed pneumonia and had to be hospitalized 10 days after treatment started.

• The patient was withdrawn and, according to the protocol, another patient was recruited.

• Thus, the intention to treat analysis brought the number analyzed to 31 in the QXY 5mg group and the total number to 91.

Page 9: What is  wrong  with  What is  said ?

What is wrong?• How was randomization achieved

in the newcomer? • The newcomer inflates the

denominator when looking at harm.

Page 10: What is  wrong  with  What is  said ?

Methods-3• The allocation of 60 patients to the

active drug and the placeo arms ensured a 86% power to detect a difference rate of 25% in the uric acid levels between the 2 groups using a 2-sided Fisher’s exact test with α set at 0.05

Page 11: What is  wrong  with  What is  said ?

What is wrong?• In a power calculation it is not

enough to state the magnitude of change, the anticapated event rates should also be given.

• Ensured 86% power?

Page 12: What is  wrong  with  What is  said ?

Sample size calculations 4 components

I. Type I (alpha) error

II. Type II (beta) error

III. Event rate in the control group

IV. Event rate in the treatment group

Page 13: What is  wrong  with  What is  said ?

Power calculations for efficacy and harm

• The primary end point was the achievement of at least 25% improvement according to the ACR criteria (ACR20) at week 20.

• The sample size of 180 patients per group was chosen to ensure an adequate safety evaluation.

• The sample size also ensured that there was 90% power to detect a significant difference in the proportion of ACR20 responders between the treatment groups using a significance level of 0.05, assuming 20% and 42% of the patients in the control and the treatment groups respectively achieved ACR 20 responses.

Page 14: What is  wrong  with  What is  said ?

What is wrong?

• No basis for the assumptions related to harm

• Post hoc power calculations

Page 15: What is  wrong  with  What is  said ?

Powering for safety• A sample size of 300 patients each

in the study drug and the control groups was determined to demonstrate a specific adverse event rate of 1% or less with 95% confidence.

Page 16: What is  wrong  with  What is  said ?

What is wrong? The probability of an event not

happening does not give us information about the likelihood of events happening in the different arms of the study.

Page 17: What is  wrong  with  What is  said ?

The zero patient method

• If one screens n individuals and does not find an attribute y among this group;

• Then one can conclude that the frequency of y is less than 1/0.33n among the n individuals surveyed, with 95% confidence.

• This approximation is true for prevalances < 0.02 of the attribute surveyed.

H Yazici et al Rheumatology Oxford (2001)

Page 18: What is  wrong  with  What is  said ?

Efficacya subgroup analysis

• The uric acid levels decreased to 5.2 ± 1.5mg/dL from a baseline of 9.3 ± 2.3 mg/dL in the treatment and from 9.7 ± 1.9 mg/dL in the placebo groups.

• There were no differences in the lowering of uric acid between the 2 groups (p>0.05).

• There were also no differences in the number of gout attacks between the 2 groups of patients. However a subgroup analysis was also done.

• Among those patients in the treatment group who gave a history of more than 5 gouty attacks per year, as compared to those who had less than 3, the uric acid levels were significantly lower after treatment (p< 0.05).

Page 19: What is  wrong  with  What is  said ?

Reviewer comments

• The findings in the subgroup analysis, conducted among a small number of patients, should be interpreted with caution.

• Even though the authors found a statistically significant difference in efficacy the study was not primarily planned to assess this difference.

• This lessens the external validity of these results.

Page 20: What is  wrong  with  What is  said ?

What is wrong? The main problem with this

subgroup analysis is not that the subgroup is small. It is an effort in the direction of proving that the new drug at hand is effecacious.

Page 21: What is  wrong  with  What is  said ?

Harm another subgroup analysis

• Table 3 gives the rate of observed adverse events. Indigestion was rather frequent in either group; 15/45 patients in the treatment and 14/45 in the placebo groups.

• A subgroup analysis was also done after the patients who participated in the trial were questioned about a pre-trial history of indigestion. It turned out that 20/45 patients in the treatment and 18/45 in the placebo group had pre-trial indigestion.

• A further analysis among these patients revealed that 14/20 in the treatment and 2/18 in the placebo group (p< 0.03) with a history of pre-trial indigeston also reported indigestion during the trial.

Page 22: What is  wrong  with  What is  said ?

Reviewer comments • The findings in the subgroup analysis,

conducted among a small number of patients, should be interpreted with caution.

• Even though the authors found a statistically significant difference in harm the study was not primarily planned to assess this difference. This lessens the external validity of these results.

Page 23: What is  wrong  with  What is  said ?

What is wrong?

Again the main issue is not the size of the subgroup. This subgroup analysis looks at the possibility that this new drug might not be all that harmless. Thus the exercise is in the direction of falsification and therefore is justified.

Page 24: What is  wrong  with  What is  said ?

Dr. Pincus study• Recently, in a placebo controlled

withdrawal study, Dr. Pincus showed that small doses of prednisone (1- 4 mg/day) were significantly effective in the management of rheumatoid arthritis.

• The study was conducted among 31 patients

Page 25: What is  wrong  with  What is  said ?

Reviewer comments

This beneficial effect of prednisone described among a small number of patients should be interpreted with extreme caution even if the authors found a statistically significant difference. The number of patients studied, thus the study power, was simply too small.

Page 26: What is  wrong  with  What is  said ?

What is wrong?• The reviewer is partially right. • The issue of power applies only if we

are missing a more important outcome that would have been more evident in a larger group.

• External validity? The small number of patients might be quite different from the real life RA patients.

• However (!) this is not simply an issue of numbers but of patient selection.

Page 27: What is  wrong  with  What is  said ?

The coxib study• 6000 patients with osteoarthritis of the knee

were randomized to recieve either the new coxib (NCB) or the traditional (TCB).

• 40% of the NCB and 35% of the TCB patients found total pain relief.

• Hypertension was a problem in 2.5 % of the NCB and 4.5 % of the TCB patients.

• It was concluded:a. NCB decreased pain by 13% b. There was 45% less hypertension with NCB

Page 28: What is  wrong  with  What is  said ?

What is wrong?• Nothing. Needs more interpretation. • A NNT analysis will tell you that you

have to treat 10 patients to see the superiority of NCB over TCB in 1 patient.

• A NNH analysis will say that you have to treat 50 patients with TCB to harm 1 more patient with hypertension as compared to using NCB.

Page 29: What is  wrong  with  What is  said ?

NNT & NNH • Relative risk (RR): outcome ratio in the control

group/ outcome ratio in the control group

• Absolute risk reduction (ARR): difference in the ratios

• Relative risk reduction (RRR): ARR/outcome in the control group OR 1-RR

• NNH: 1/ARR (frequency of hypertension in the control group 0.045, frequency of hypertension in the study group 0.025, ARR= 0.020; NNT = 50.)

Page 30: What is  wrong  with  What is  said ?

A withdrawal study• Two previous double blind studies of colchicine

in BS had shown no superiroity of this agent over placebo in treating the oral ulcers in this condition.

• Recently a withdrawal study was done among those patients who claimed benefit from colchicine. They were randomized to contiune to receive the active drug or placebo.

• After 3 months those who stopped taking colchicine had significantly more ulcers (sign test, p = 0.02).

Page 31: What is  wrong  with  What is  said ?

Reviewer comments• This study is interesting but is of limited

use. A problem with all withdrawal trials, they seldom represent real life use.

• The authors used the “sign test” to analyze the differences in oral ulcers between the 2 groups. Is this a new test? I suspect it is not quite powerful. Why not use the more powerful tests of significance to show the real differences, if any?

Page 32: What is  wrong  with  What is  said ?

What is wrong? (I)• The main problem with withdrawal

studies is that they are not done often enough.

• They provide excellent information about possible a type II error in previous, traditional RCT’s.

• An important issue is that they do not give a fair picture of drug associated harm.

Page 33: What is  wrong  with  What is  said ?

What is wrong? (II)• The sign test is a time honored,

conservative tool.

• Significance in a conservative test gives more validity to the significant differences observed.

Page 34: What is  wrong  with  What is  said ?

The mighty “p”

Among the 96 patients allocated to the new medication there were 3 cases of myocardial infarction while the same was true for 2/94 patients allocated to placebo (p=0.86).

Page 35: What is  wrong  with  What is  said ?

What is wrong?• Better give it with the statistic

used (sign test, p = 0.02).

• Do not use it to mesmerize the reader.

Page 36: What is  wrong  with  What is  said ?

The Commandments

• Hypothesis as an affirmative statement • Intention to treat• Components of the power calculation• Post hoc power analyses• Powering for harm, no short cuts• Justifications for subgroup analyses• Small but significant• Meaning of treatment effects• Withdrawal studies to be cherished• The “mighty p”

Page 37: What is  wrong  with  What is  said ?

Summary•In a RCT, like in all scientific endeavor, all efforts aimed at proving the hypotheses are wrong.

•The aim is falcification.

Page 38: What is  wrong  with  What is  said ?

Vitamin-E in goutthe Conclusion

Vitamin E is an innocuous agent and with the possible additional benefit of an antihypertensive effect, as was also serendipitously noted in this study, such studies are surely warranted. We are about to start a study along those lines at our institution.

Page 39: What is  wrong  with  What is  said ?

What is wrong?

Do not preempt!