Effective Presentation of Study Results How are RCTs presented in abstracts & publications? and Some things to consider in your own presentations NCIC

Effective Presentation of Study Results

How are RCTs presented in abstracts & How are RCTs presented in abstracts & publications?publications?

andandSome things to consider in your own Some things to consider in your own

presentationspresentationsNCIC CTG New Investigators Course NCIC CTG New Investigators Course

October 2009October 2009

Christopher Booth MD FRCPCChristopher Booth MD FRCPC

Queen’s University Cancer Research Queen’s University Cancer Research InstituteInstitute

The Cancer Research Institute at Queen’s UniversityDivision of Cancer Care and Epidemiology

Outline: Part I

How are RCTs presented in abstracts & How are RCTs presented in abstracts & publications?publications?

• Evolution of endpoints and perception of benefit in oncology RCTs over time

• What results are clinically meaningful?• RCTs closed early for benefit• How are RCTs presented at conferences?


Outline: Part II

Issues to consider when presenting your Issues to consider when presenting your study results…study results…

• Audience• Preparation• Key messages• Fancy PowerPoint• Summary


Part I How are RCTs presented in How are RCTs presented in

abstracts & publications?abstracts & publications?


1. How have endpoints and perception of benefit evolved in oncology RCTs?

Study Design

• Overview of all RCTs systemic therapy in breast, NSCLC, colorectal cancer

• 1975-2004• 6 major journals: JCO, Cancer

Treatment Reports, JNCI, NEJM, Lancet, JAMA

• Data abstraction using standardized forms and methodology


Temporal Trends

1975-84 1985-94 1995-2004 P value (trend)

Total RCTs 47 (15%) 107 (33%) 167 (52%) <0.0001

Breast RCTs 19 (40%) 53 (50%) 81 (49%) 0.475

NSCLC RCTs 23 (49%) 29 (27%) 39 (23%) 0.002

CRC RCTs 5 (11%) 25 (23%) 47 (28%) 0.017

321 RCTS over three 321 RCTS over three decades decades involving >170 000 involving >170 000 patientspatients


Study Participants

1975-84 1985-94 1995-04 P (trend)

International 26% 28% 52% <0.0001

Co-op group 28% 56% 43% 0.661

Sample size 100 249 446 <0.0001

Accrual time 30 mo 41 mo 33 mo 0.93


Statistics

1975-84 1985-94 1995-04 P (trend)

1º endpoint

Time to event 39% 72% 78% <0.0001

RR 54% 25% 14% <0.0001

ITT analysis

Any 70% 87% 93% <0.0001

33% of RCTs in 1995-2004 did not clearly identify primary endpoint


Sponsorship

1975-84 1985-94 1995-04 P (trend)

Government 60% 62% 31% <0.0001

Industry 4% 23% 57% <0.0001


Effect Size and Conclusions

1975-1984 1985-1994 1995-2004

Median HR (95%CI) 1.4 (1.0-2.3) 1.2 (1.0-2.4) 1.2 (1.1-1.3)

P<0.05 for primary EP 23% 30% 42%

Strong endorsement 31% 39% 49%

1. Effect size stable over time2. Modern RCTs more likely to

have p<0.053. Modern authors more likely

to call their trial “positive” The Cancer Research Institute at Queen’s UniversityDivision of Cancer Care and Epidemiology


Key Findings1. Increase in number and size of RCTs2. More international trials, faster accrual3. Shift in primary endpoint from RR to

survival EPs 4. Major shift towards for-profit sponsorship5. Effect size has remained stable over time6. Authors of modern RCTs more likely to

endorse experimental arm7. For-profit sponsorship and p<0.05 are

independently associated with strong endorsement of experimental arm


2. Are these results clinically meaningful?Clinically Relevant Endpoints• Patients define a useful therapy as one

that…increases survival ORimproves QOL/reduces symptoms of

cancer• Reassuring shift from RR to survival

endpoints• Increasing use and recognition of PROs

Gemcitabine Pancreas TAX 327 HRPCThe Cancer Research Institute at Queen’s

UniversityDivision of Cancer Care and Epidemiology

Clinically Relevant Endpoints• Current standards for analyzing QOL

and symptom control need improvement

112 RCTs for advanced cancer 19% established a priori hypothesis 21% defined minimal differences in QOL scores that were clinically meaningful

• Increasing use of surrogate endpoints (DFS, PFS)

Joly et al Ann Oncol Joly et al Ann Oncol 20072007Sargent et al JCO Sargent et al JCO 20052005



Clinically Relevant Endpoints• Clinical benefit (CB) in trials of

pancreas cancercomposite improvement in pain, weight, performance status

• CB now widely (mis)used to describe PR/CR/SD

71 trials in JCO since 199728% used patient-centered definition72% referred to objective tumor measurements

Burris et al JCO 1997Burris et al JCO 1997Ohorodnyk et al ASCO Ohorodnyk et al ASCO 20092009


3. RCTs Closed Early for Benefit

Korn et al JCO Korn et al JCO 20092009Sargent JCO 2009Sargent JCO 2009


Korn et al JCO 2008

• 27 NCI co-operative group trials that were closed early for benefit

• Of the 18 trials with follow-up available, initial magnitudes of benefit were preserved in 17 (94%) trials

• “…the system is working” Dan Sargent JCO 2008


Booth, Meyer et al Under Review• Literature search identified 62 RCTCEB

– Primary endpoint not explicitly stated in 19%– ITT all randomized patients in only 66%

• Most trials open to accrual (45/62, 73%) at the time of closure.

• Formal IA performed in 56 (90%) trials– 75% (42/56) planned and 79% (44/56) reported

stopping rules. • Trials on average accrued 73% of the

planned sample size. • Follow-up reports for 18 (29%) RCTCEB

show that results and conclusions were maintained.

4. How are RCTs presented at conferences?


Trial Reporting: NFAs• 138 RCTs published 2000-2004

– 197 corresponding abstracts 1990-2004

• Results were stated or implied to be non-final analyses in 86 abstracts (44%)

• 124 abstracts (63%) discordant with article

• Conclusions substantively different in 17 (10%)


Meeting abstracts often include NFAs and are often discordant with mature publication

Bias in Oncology RCTs

Publication Bias• 510 RCTs presented at ASCO 1989-

199826% were not published within 5 years81% of trials with p<0.05 were published

compared to 68% of trials with p>0.05

• This should improve with mandatory trial registration

Krzyzanowska JAMA Krzyzanowska JAMA 20032003


Bias in Oncology RCTs

Sponsorship Bias• Multiple studies have demonstrated

that RCTs sponsored by industry are more likely to be “positive” than non-industry trialsReasons are likely complex and

multifactorial Djulbegovic Lancet Djulbegovic Lancet 20002000Booth JCO 2008Booth JCO 2008Peppercorn Cancer Peppercorn Cancer 20072007


ACS 2008 StatisticsACS 2008 Statistics

Part I: SUMMARY1. Major changes in cancer treatment/research

since 1970s

Patient outcomes have improved

RCT methodology and reporting are improvingTrials are larger, complex, stronger correlative

componentWhat constitutes a “positive trial” has changed

2. It is critical to keep in patient-centered outcomes in focus at every step of the drug development pathway

3. Be critical in the way you interpret results of RCTs in published form and at conferences


AcknowledgementsDrs. Ralph Meyer and Bill Mackillop

NCIC Clinical Trials GroupQueen’s University Cancer Research InstituteKingston, Ontario

Drs. Ian Tannock and Monika KrzyzanowskaPrincess Margaret Hospital, Toronto, OntarioThe Cancer Research Institute at Queen’s

UniversityDivision of Cancer Care and Epidemiology

Part II:Things to consider when

presenting your own study results


Topics for Discussion

1. Audience2. Preparation and timing3. Key messages4. Fancy PowerPoint (?)5. Summary6. Time for discussion


Documents

Effective Presentation of Study Results How are RCTs presented in abstracts & publications? and Some things to consider in your own presentations NCIC