When an article is published we post the peer reviewers ... · Management, Yale School of Public Health, and Center for Outcomes Research and Evaluation, Yale-New Haven Health, all

BMJ Open is committed to open peer review. As part of this commitment we make the peer review

history of every article we publish publicly available.

When an article is published we post the peer reviewers’ comments and the authors’ responses

online. We also post the versions of the paper that were used during peer review. These are the

versions that the peer review comments apply to.

The versions of the paper that follow are the versions that were submitted during the peer review

process. They are not the versions of record or the final published versions. They should not be cited

or distributed as the published version of this manuscript.

BMJ Open is an open access journal and the full, final, typeset and author-corrected version of

record of the manuscript is available on our site with no access controls, subscription charges or pay-

per-view fees (http://bmjopen.bmj.com).

If you have any questions on BMJ Open’s open peer review process please email

[email protected]

on Novem

ber 23, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2017-017917 on 5 Decem

ber 2017. Dow

nloaded from

http://bmjopen.bmj.com/

mailto:[email protected]


For peer review only

Measuring clinical trial transparency: An empirical analysis of newly approved drugs

Journal: BMJ Open

Manuscript ID bmjopen-2017-017917

Article Type: Research

Date Submitted by the Author: 26-May-2017

Complete List of Authors: Miller, Jennifer; NYU School of Medicine, Division of Medical Ethics, Department of Population Health; Bioethics International Wilenzick, Marc; International Aids Vaccine Initiative Ritcey, Nolan; Bioethics International; NYU School of Medicine Ross, Joseph; Yale University School of Medicine, Internal Medicine; Yale University Yale School of Public Health Mello, Michelle; Stanford University Stanford Law School, Department of Health Research and Policy; Stanford University School of Medicine

<b>Primary Subject Heading</b>:

Ethics

Secondary Subject Heading: Health policy, Medical publishing and peer review, Patient-centred medicine, Public health, Pharmacology and therapeutics

Keywords: AUDIT, CLINICAL PHARMACOLOGY, Health policy < HEALTH SERVICES ADMINISTRATION & MANAGEMENT, LAW (see Medical Law), MEDICAL ETHICS, PUBLIC HEALTH

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on N

ovember 23, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2017-017917 on 5 D

ecember 2017. D

ownloaded from



1

Title: Measuring clinical trial transparency: An empirical analysis of newly approved drugs

Authors: Jennifer E. Miller, PhD, Marc Wilenzick, Nolan Ritcey, PhD, Joseph S. Ross, MD,

MHS, Michelle M. Mello, PhD*

*From The Division of Medical Ethics and the Department of Population Health, NYU School

of Medicine, and Bioethics International, New York, NY (JM), International Aids Vaccine

Initiative New York, NY (MW), Bioethics International, New York, NY (NR), Section of

General Internal Medicine and Robert Wood Johnson Foundation Clinical Scholars Program,

Department of Medicine, Yale School of Medicine, Department of Health Policy and

Management, Yale School of Public Health, and Center for Outcomes Research and Evaluation,

Yale-New Haven Health, all New Haven, CT (JSR), Stanford Law School and the Department of

Health Research and Policy, Stanford University School of Medicine, Stanford, CA (MMM)

Corresponding author:

Dr. Jennifer Miller

Division of Medical Ethics

Department of Population Health

NYU School of Medicine

227 E. 30th St., Rm 723

New York, NY, 10016

[email protected]

Paper Word Count: 3614

Abstract Word Count: 295

Page 1 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



2

ABSTRACT

Objectives: We defined a series of clinical trial transparency metrics, on ethics and legal levels,

and applied them to large pharmaceutical companies and their 2014 FDA approved drugs.

Additionally, we tracked where trial information was disclosed to landscape use of corporate,

national, and international registries.

Design: Cross-sectional analysis of all clinical trials supporting 2014 FDA approved New Drug

Applications (NDAs), sponsored by large companies.

Data sources: Data from 44 sources, including [email protected], ClinicalTrials.gov, corporate

and international registries; PubMed, Google Scholar, EMBASE, corporate press releases, and

personal communications with drug manufacturers.

Outcome Measures: Trial registration, results reporting, clinical study report synopsis (CSR)

sharing, biomedical journal publication, and FDA Amendments Acts (FDAAA) compliance,

analyzed on the drug level.

Results: The FDA approved 19 new drugs, sponsored by 11 large companies, involving 553

trials (median of 24 trials per drug), in 2014. We analyzed 507 relevant trials. Per drug, a median

of 100% (IQR 86-100%) of patient trials were registered, 71% (IQR 57-100%) reported results

or shared a CSR, 80% (70-100%) were published, and 96% (80-100%) were publicly available in

some form. Per drug, a median of 100% (IQR 75-100%) of FDAAA applicable trials, were

compliant. Half of reviewed drugs had publicly disclosed results for all patient trials. One trial

was uniquely registered in a corporate registry, and not ClinicalTrials.gov; 0 trials were uniquely

registered in international registries.

Conclusions: Among large pharmaceutical companies and new drugs, clinical trial transparency

is high based on several measures, although opportunities for improvement remain. Best

practices are emerging; 2 of 11 companies disclosed all patient trials and complied with legal

disclosure requirements. Reaching consensus about standards, tracking adherence over time, and

celebrating progress can move the field forward toward a shared vision of transparency and what

it can achieve.

Trial registration: Not Applicable.

Page 2 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



3

Strengths and limitations of this study

• This analysis uniquely evaluates new drugs and large biotechnology and pharmaceutical

companies on several different clinical trial transparency benchmarks on the ethics and legal

levels—not merely the usual crude measure of whether companies have reported results for

the trials they registered on ClinicalTrials.gov.

• Our review is global; we searched every public trial registry, including international,

corporate, and patient registries, and provide an assessment on the completeness of

ClinicalTrials.gov and the value of allocating resources to link existing trial databases.

• A further innovation is rigorously validating all study measures and data with NDA holders

(pharmaceutical companies); we obtained NDA holders’ feedback on their results and

convened 18 companies to discuss the methods.

• We added a company ranking to accompany the drug rankings on clinical trial transparency

performance.

• This study captures static snapshots of clinical trial transparency for new drugs approved by

the FDA in 2014, at the time of FDA approval, 3 months post FDA approval, 6 months post

FDA approval, and 13 months post FDA approval. Subsequent trial disclosures are not

captured. We do not count abstracts submitted to scientific conferences as “publications”.

Page 3 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



4

INTRODUCTION

The importance of clinical trial transparency for supporting evidence-based patient care,

prescription guideline development, formulary decisions, public trust in research, and healthcare

innovation is now well established—yet it does not always occur.[1]-[6] Despite the

promulgation of multiple legal and guidance standards in the 20 years since Congress enacted the

first law requiring registration of certain trials,[7] transparency practices remain highly variable

across research sponsors like universities and drug companies.[3, 6, 8, 9] Moreover, studies that

measure the transparency of trials, drugs, drug manufactures, and research sponsors often use

markedly different transparency benchmarks and standards, yielding different findings and

progress reports. A research sponsor, trial, or drug may look transparent under one study and

opaque in another.

Efforts to establish clinical trial transparency standards and improve practices span

decades. In 1997, the United States (US) adopted the Food and Drug Administration

Modernization Act [7] requiring the registration of drug trials for serious or life-threatening

conditions. In 2007, the Food and Drug Administration Amendments Act (FDAAA) expanded

disclosure requirements to include trials for all types of health conditions and required that select

trial results be publicly reported for FDA regulated products.[10] In 2008, the World Medical

Association identified trial registration and reporting as an ethical obligation, for all-trials, in the

Declaration of Helsinki.[11] A bevy of other leading scientific and development organizations

have also endeavored to improve research transparency, including the Gates Foundation, World

Health Organization, and Wellcome Trust.[12-19] Most recently, the US Department of Health

and Human Services and National Institutes of Health (NIH) released policies and rules to

further expand and clarify the types of trials for which reporting is required.[20-22] The new

Page 4 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



5

NIH policy requires, for the first time, results reporting for all NIH sponsored trials, including

phase I trials conducted in healthy volunteers.[23]

These efforts have helped foster a culture of transparency in research, but they have also

introduced ambiguous, and at times conflicting, standards. To help harmonize benchmarks going

forward, assess the current state of transparency, as well as to identify and reform areas where

further improvement is needed, we defined a series of concrete transparency metrics and applied

them to drugs approved by the FDA in 2014 and their large company sponsors.

The transparency metrics include both US legal compliance measures—whether

companies report what FDAAA requires them to—and two expanded standards applying to

broader ranges of trials, one of which includes phase 1 trials in healthy volunteers, and one that

only looks at trials in patients. This paper and efforts are a continuation of efforts, called The

Good Pharma Scorecard, that began with benchmarking the transparency of drugs approved by

the FDA in 2012, sponsored by large drug companies, on a significantly narrower set of

standards.[24,30-32]

This analysis is innovative by uniquely evaluating new drugs and their sponsors on

several different benchmarks—not merely the usual crude measure of whether companies have

reported results for the trials they registered on ClinicalTrials.gov. Our review is global; we

searched every public trial registry, including international, corporate, and patient registries, and

provide an assessment on the completeness of ClinicalTrials.gov. A further innovation is

rigorously validating all study measures and data with NDA holders; we obtained NDA holders’

feedback on their results and convened 18 companies to discuss the methods. Lastly, we added a

company ranking to accompany the drug rankings.

Page 5 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



6

METHODS

Data

We used data from multiple (over 44) sources (details in Appendix 1), including

[email protected], a publicly accessible database containing records of FDA regulatory

decisions; 39 trial registries including ClinicalTrials.gov, the Aggregate Analysis of

ClinicalTrials.gov database (AACT), individual corporate registries, the World Health

Organization’s International Clinical Trials Registry Platform, which aggregates 16 country

registries, the Clinical Study Data Request Repository; journals indexed in PubMed, Google

Scholar, and EMBASE; and corporate press releases . Additional information was obtained

through personal communications with drug manufacturers. All databases were accessed several

times in January-August 2016.

Sampled Drugs and Companies

We examined clinical trials relating to New Molecular Entities (NMEs), and new

combination drugs containing at least one NME component, that were approved by the FDA in

2014.[25] For feasibility reasons and to highlight practices among the better-resourced sponsors,

we confined our analysis to New Drug Applications (NDA) that were sponsored by the 20

largest biotechnology and pharmaceutical companies, or their subsidiaries, as measured by 2014

market capitalization ranking.[26]

Sampled Clinical Trials

A list of every clinical trial included in the NDA was created by reviewing the 2014 FDA

approval packages for each of the 19 drugs. Basic characteristics of each trial were extracted

(details in Appendix 2). We excluded trials terminated without enrollment, expanded access

Page 6 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



7

trials, observational studies, and trials that were ongoing or not at least 1 year past their primary

completion date (PCD) by our study cutoff date of February 1, 2016.

From this “all-trials” sample we then selected two subsamples. The “trials in patients”

subsample was confined to trials conducted in patients (as opposed to healthy volunteers). The

“FDAAA trials” subsample was limited to trials subject to mandatory registration and results

reporting under FDAAA.

The legal requirements under FDAAA for reporting trial results apply only to “controlled

clinical investigations(s), other than a phase I clinical investigation” [27] of a drug that is the

subject of an approved NDA or for which an NDA would be required in order for the drug to be

legally marketed in the US. The requirements apply only if the trial began after September 27,

2007 or was ongoing as of December 26, 2007. Finally, the trial must meet one of the following

conditions: (1) at least one U.S. site, (2) conducted under an FDA investigational new drug

(IND) application, or (3) involve a drug, biologic, or device manufactured in U.S., or its

territories, and exported for research.[10, 27] Trials with unknown phases (n=7) were excluded

from the FDAAA subsample.

Data Collection Methods

Search terms to match trials in the registries included the trial’s organizational

identification number, product name, chemical name(s), number of participants, and other

characteristics captured from the FDA approval packages. We abstracted all available

characteristics on each trial from the registry (details in Appendix 2). We matched trials to

journal articles using a minimum of 3 trial characteristics, and searched the registries for links to

publications.

Page 7 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



8

For each trial, data were retrieved by at least 2 research assistants who received training

and worked independently. Discrepancies between coders’ findings were resolved by consensus.

Final datasets and findings were sent to drug companies for validation. Feedback from

companies was generally incorporated into findings if it could be validated through our public

sources.

Outcome Measures

Our first outcome measure examined whether trials were registered in any public registry,

including corporate and international registries. Second, we determined whether either trial

results or a CSR synopsis were provided in the registry. Third, we determined whether each trial

was published in a journal indexed by PubMed, Google Scholar, or EMBASE. Fourth, we

deemed each trial “publicly available” if it had results reported in a registry, a CSR synopsis

provided in a registry, or results published in a journal.

Results were considered available if received by a registry or published by February 1,

2016. This date was chosen to provide a generous period of time for reporting results: at least 1

year after FDA approval of the drug plus a 1-month grace period.

We also measured the availability of trial results at the time each drug was approved by

the FDA, 3 months after approval, and 6 months after approval to track reporting timelines.

Additionally, we tracked where trials were registered, reported, or had shared CSRs to get a

sense of the overall use of corporate, national, and international registries and the need to invest

in linking multiple data-bases together.

We applied these measures to 2 different samples of trials (Table 1): (1) all trials,

including trials enrolling healthy volunteers and patients, and (2) only trials enrolling patients.

The all-trials analysis evaluates companies against the World Medical Association [11]

Page 8 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



9

recommendation that all-trials be registered and have results disclosed to honor ethical

obligations to research subjects, both healthy volunteers and patient participants, as described in

the Declaration of Helsinki and Belmont Report. The Belmont Report defines research as, “an

activity designed… to develop or contribute to generalizable knowledge”, which generally

requires the dissemination of research results. Most companies that participated in a meeting

convened by our project in August 2016, expressed a commitment to disclosing results only for

trials enrolling patients, and codes of conduct from industry trade associations generally only

charge member companies with the obligation to disclose results from patient trials. Therefore,

we also analyzed transparency for those trials that enrolled patients with the disease of interest,

excluding trials of healthy volunteers and patients with renal impairment but not the indication

being studied. We also assessed the extent to which trials subject to FDAAA met that statute’s

transparency requirements. Setting cutoff points for this analysis was complex because there is a

disagreement among companies about what the statute requires. There is broad agreement that

FDAAA requires trial registration within 21 days after enrolling the first participant (we gave

sponsors a 7-day grace period to account for delayed postings, weekends, holidays, time zones).

However, 2 views exist about when results must be reported, both of which are plausible. One

interpretation, which we call the “trial completion date” view, is that results generally must be

reported within 12 months after a trial’s primary completion date, but may be delayed until 30

days after FDA approval if a company files a “certificate of delay” with the NIH [7]. The other is

that trial results are not due until 30 days after FDA approval of a new drug for an initial use

approval. We examined compliance with FDAAA among applicable trials using both

interpretations.

Page 9 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



10

Lastly, we ranked NDA sponsors by their overall clinical trial transparency. For

pharmaceutical companies with only one drug approved by the FDA in 2014, we averaged their

scores on (1) the trials-in-patients analysis, excluding trials in healthy volunteers, and (2) a

FDAAA compliance measure that counted a trial as compliant if it satisfied either interpretation

of the reporting requirements. For companies with multiple drugs approved, we pooled the trials

from all drugs and then calculated the percentages of trials satisfying each of the 2 metrics. In

rare cases, we excluded trials (n=12) from a particular company ranking if the NDA sponsor was

not the trial sponsor and therefore not responsible for publicly reporting trial results. If the

responsible party was a ranked company, and we could confirm they were indeed the responsible

party, we transferred the trials to their denominator and included them in their rankings.

Table 1: Transparency Benchmarks and Analyzed Clinical Trial Samples

Samples analyzed

(from successful NDAs)

Transparency Measure

All-trials

(including

those in

healthy

volunteers)

Trials in

patients

Trials

subject to

FDAAA

Registered in a public registry by 13 months

post FDA approval X X

Either trial results or a CSR synopsis provided

in a public registry by 13 months post FDA

approval

X X

Published in a journal indexed in PubMed,

Google Scholar, or EMBASE by 13 months

post FDA approval

X X

Results publicly available in some form

(results or CSR synopsis in registry, or journal

article) by 13 months post FDA approval

X X

Compliant with FDAAA– “trial completion

date” interpretation X

Compliant with FDAAA– “approval date”

interpretation X

Page 10 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



11

Statistical Analysis

Summary statistics (medians and interquartile ranges) were calculated to show how

commonly trials for each approved drug and drug company met the transparency benchmarks.

All data were collected and analyzed in Microsoft Excel v.15.18 (Redmond, Washington).

Validation

Datasets and results were sent to the NDA holders of each drug for validation (details in

Appendix 3). Nine of 11 evaluated companies participated in both the validation process and a

meeting to discuss study methods and findings, affording validation for 79% of drugs reviewed.

Validated results are presented below.

RESULTS

In 2014, the FDA approved 31 new molecular entities (NMEs) or new combination

drugs with at least one NME, 19 of which were sponsored by 11 of the 20 largest pharmaceutical

or biotechnology companies (Figure 1). A total of 553 trials (median of 24 trials per drug) were

included in the NDAs.

We analyzed 507 of these trials (median of 22 trials per drug), after excluding trials

that were not at least 1 year past their primary completion date by our study cutoff of February 1,

2016, trials terminated without enrollment, and expanded-access trials. These trials enrolled

133,428 participants, 93% of whom were patients and 7% of whom were healthy volunteers. A

median of 7 out of 22 trials, per drug, were conducted in patients.

Transparency Scores Based on Patient Trials

We first report results using the standard that trials conducted in patients should be

publicly available, excluding trials conducted in healthy volunteers (Table 2). A median of 100%

Page 11 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



12

(IQR 86-100%) of patient trials per drug were registered. A median of 71% (IQR 57-100%)

reported results or provided a CSR synopsis, and 80% (IQR 70-100%) were published. Overall,

results for a median of 96% (IQR 80-100%) of trials, per drug, were publicly available in some

form.

Ten of 19 drugs (53%) had at least one undisclosed trial conducted in patients. A median

of 46% of undisclosed trials were phase 1 trials in patients (IQR 0-100%). Six drugs (32%) had

at least one undisclosed Phase 2 or Phase 3 trial. At least 2,864 patients participated in trials

with undisclosed results.

Transparency Scores Based on All-Trials in an NDA, Including Healthy Volunteers

We next report results using the standard that all-trials in a successful NDA, including

those of healthy volunteers, should be publicly available (Table 2). A median of 53% (IQR 33-

85%) of all-trials per drug were registered, 24% (IQR 19-50%) reported results or shared a CSR

synopsis, and 60% (IQR 37-75%) were published. Overall, a median of 68% (IQR 40-82%) of

trial results per drug, were publicly available (reported, shared in a CSR, or published).

All 19 drugs had at least one publicly unavailable trial conducted in patients or healthy

volunteers. Most of these trials were phase 1 trials involving healthy volunteers (median of 100%

of trials, IQR 72-100%). At least 7,287 patients and healthy volunteers participated in trials with

undisclosed results.

Page 12 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



13

*Some trials for these drugs were sponsored by a company other than the NDA sponsor, including the below. +Full FDA approved indications are listed in Appendix 4.

• Xtoro: Alcon sponsored 3 trials; 2 had publicly disclosed results, 1 trial was not registered. Novartis states that the unregistered trial “was conducted before the Novartis position to

register all phase I trials in patients became applicable to Alcon. Alcon was purchased by Novartis in April 2011”.

• Esbriet: Intermune sponsored 2 trials; both had publicly disclosed results. Marnac sponsored 5 trials; 3 were publicly disclosed. Genentech merged with Intermune in 2014.

Intermune purchased the rights to sell Esbriet in the U.S. in 2007.

• Sivextro: Bayer/Trius sponsored 21 trials, 15 had public results. Bayer and Trius partnered to develop Sivextro (2011). Cubist acquired Trius (2013). Merck acquired Cubist (2014)

• Farxiga: Bristol Myers Squib (BMS) co-sponsored multiple trials with AstraZeneca.

• Xigduo XR: BMS co-sponsored 16 trials with AstraZeneca; 14 had publicly disclosed results.

• Invokamet: Mitsubishi Tanabe sponsored 4 trials; all had publicly disclosed results. Misubishi was Johnson & Johnson’s research partner in Japan.

Table 2: Transparency of Clinical Trials in Patients and all Clinical Trials Listed in New Drug Applications

Ethics Standard 1: All clinical trials Ethics Standard 2: Trials in patients (excluding healthy

volunteer trials)

Drug NDA Sponsor

Short

Indication+

# trials

analyzed %registered %reported %published

%publicly

available

# trials


%publicly

available

Xtoro Alcon/Novartis* Otitis Externa 4 50% 50% 0% 50% 3 67% 67% 0% 67%

Esbriet Genentech/Roche *

Pulmonary

Fibrosis 17 33% 24% 47% 47% 10 46% 40% 70% 70%

Harvoni Gilead Hepatitis C 60 53% 32% 37% 40% 31 92% 61% 68% 74%

Dalvance Durata/Allergan Skin Infection 22 18% 9% 68% 68% 8 50% 25% 75% 75%

Zydelig Gilead Leukemia 16 58% 19% 19% 25% 5 100% 60% 60% 80%

Sivextro Cubist/Merck* Skin Infection 21 90% 19% 67% 71% 7 100% 57% 71% 86%

Zontivity MerckSharpDohme

Thrombotic

Cardiovascular

Events 27 26% 19% 74% 78% 7 86% 71% 71% 86%

Farxiga Astrazeneca* Type 2 Diabetes 60 76% 33% 75% 82% 41 91% 46% 80% 88%

Xigduo

XR Astrazeneca* Type 2 Diabetes 25 85% 56% 64% 72% 20 100% 70% 80% 90%

Lynparza Astrazeneca Ovarian Cancer 24 96% 92% 75% 96% 24 96% 92% 75% 96%

Zykadia Novartis Lung Cancer 5 69% 20% 20% 20% 1 100% 100% 100% 100%

Belsomra MerckSharpDohme Insomnia 37 24% 22% 27% 32% 6 100% 100% 100% 100%

Cerdelga Genzyme/Sanofi

Gaucher

Disease 16 41% 19% 38% 38% 3 100% 100% 100% 100%

Viekira

Pak Abbvie Hepatitis C 59 35% 22% 58% 58% 15 100% 87% 100% 100%

Otezla Celgene

Arthritis;

Psoriasis 30 57% 47% 53% 63% 15 93% 93% 80% 100%

Jublia Dow/Valeant Onychomycosis 9 33% 0% 78% 78% 5 60% 0% 100% 100%

Movantik Astrazeneca Constipation 20 85% 85% 60% 85% 6 100% 100% 67% 100%

Invokamet Janssen / J&J* Type 2 Diabetes 40 88% 63% 85% 88% 22 100% 100% 100% 100%

Zerbaxa Cubist/Merck

Urinary &

Abdominal

Infections 15 50% 40% 93% 93% 6 100% 100% 100% 100%

Quartile 1 16 33% 19% 37% 40% 5 86% 57% 70% 80%

Quartile 3 37 85% 50% 75% 82% 20 100% 100% 100% 100%

Median 22 53% 24% 60% 68% 7 100% 71% 80% 96%

Page 13 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on November 23, 2020 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2017-017917 on 5 December 2017. Downloaded from



14

Compliance with FDAAA Requirements

A median of 25% (IQR 19-38%) of trials per drug (or 4 trials per drug) were subject to

mandatory disclosures under FDAAA. Applying first the “trial completion date” interpretation of

FDAAA’s requirements, a median of 71% (IQR 0-87%) of these trials per drug were FDAAA

compliant (Table 3). A median of 100% (IQR 100-100%) of these trials per drug were registered

on time and 71% (IQR 0-87%) reported results on time. One trial was registered a few days late,

but had timely results reporting. We counted the trial as FDAAA compliant because the

registration deadline fell close to a weekend, was conducted in Japan and co-sponsored by more

than one company (drug was manufactured in the US so subject to FDAAA). Of the 110,426

participants in trials covered by FDAAA, 66% were in noncompliant trials under this

interpretation.

Applying the “approval date” interpretation of the law, a median of 100% (IQR 75-

100%) of trials per drug were FDAAA compliant (Table 3). A median of 100% (IQR 100-100%)

were registered on time and 100% (IQR 75-100%) had results reported on time. A median of

97% (IQR 88-100%) of participants were in compliant trials.

Most companies (73%) filed at least one certificate of delay for each of their drugs.

However, a median of only 28% of FDAAA-applicable trials per drug had certificates of delay

filed.

Page 14 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



15

*Some trials for this drug were sponsored by a company other than the NDA sponsor, including the below. +Full FDA approved indications are listed in Appendix 4.

• Farxiga: AstraZeneca (AZ) reports that this drug approved by the FDA on 1/1/14 and acquired from BMS on 2/1/14. AZ states that “the transition of Farxiga studies to

AstraZeneca is still in progress. Results were to be posted before the transition to AstraZeneca, which did not occur within the timeline for newly approved products.”

AstraZeneca has now posted results for the 7 FDAAA-applicable studies that did not meet FDAAA requirements. 6/7 were published in the medical literature on or before 30

days post FDA approval; the other was published after the 30-day cutoff.

• Sivextro: Bayer / Trius sponsored 4 trials, 3/4 were FDAAA compliant under the “approval date” interpretation.

• Xigduo XR: BMS co-sponsored 10 trials with AstraZeneca, 9/10 were FDAAA compliant under the “approval date” interpretation.

• Invokamet: Mitsubishi Tanabe sponsored 2 trials, 1/2 was FDAAA compliant under the “approval date” interpretation.

• Xtoro: Alcon sponsored 2 trials, 2/2 were FDAAA compliant under the “approval date” interpretation.

• Esbriet: Intermune sponsored 2 trials, 2/2 were FDAAA compliant under the “approval date” interpretation.

Table 3: Compliance with FDAAA Legal Disclosure Requirements

"Trial Completion Date" Interpretation "Approval Date" Interpretation

Drug NDA Sponsor Indication

# trials

analyzed %registered %reported

% FDAAA

compliant

# trials


% FDAAA

compliant

Jublia Dow/Valeant Onychomycosis 2 100% 0% 0% 2 100% 0% 0%

Dalvance Durata/Allergan Skin Infection 2 50% 0% 0% 2 50% 100% 50%

Farxiga Astrazeneca* Type 2 Diabetes 17 100% 12% 12% 17 100% 59% 59%

Harvoni Gilead Hepatitis C 27 100% 59% 59% 27 100% 70% 70%

Sivextro Cubist/Merck* Skin Infection 4 100% 75% 75% 4 100% 75% 75%

Zydelig Gilead Leukemia 4 100% 75% 75% 4 100% 75% 75%

Xigduo XR Astrazeneca* Type 2 Diabetes 13 100% 23% 23% 13 100% 77% 77%

Otezla Celgene Psoriasis 11 100% 91% 91% 11 100% 91% 91%

Viekira Pak Abbvie Hepatitis C 13 100% 77% 77% 13 100% 92% 92%

Belsomra MerckSharpDohme Insomnia 4 100% 0% 0% 4 100% 100% 100%


Urinary & Abdominal

Infections 6 100% 0% 0% 6 100% 100% 100%


Thrombotic

Cardiovascular Events 3 100% 0% 0% 3 100% 100% 100%

Movantik Astrazeneca Constipation 6 100% 17% 17% 6 100% 100% 100%

Lynparza Astrazeneca Ovarian Cancer 7 100% 71% 71% 7 100% 100% 100%

Esbriet Genentech/Roche* Pulmonary Fibrosis 4 100% 75% 75% 4 100% 100% 100%

Invokamet Janssen / J&J* Type 2 Diabetes 15 100% 87% 87% 15 100% 100% 100%

Cerdelga Genzyme/Sanofi Gaucher Disease 3 100% 100% 100% 3 100% 100% 100%

Xtoro Alcon/Novartis* Otitis Externa 2 100% 100% 100% 2 100% 100% 100%

Zykadia Novartis Lung Cancer 1 100% 100% 100% 1 100% 100% 100%

Quartile 1 3 100% 0% 0% 3 100% 75% 75%

Quartile 3 13 100% 87% 87% 13 100% 100% 100%

Median 4 100% 71% 71% 4 100% 100% 100%

Page 15 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




16

Timing of Results Reporting

At the time of FDA approval, results for a median of 45% (IQR 20-52%) of all-trials per

drug are publicly available in some form (Table 4). At 3 months post-approval, 50% (IQR 28-

67%) of all-trials were publicly available. This median did not increase at 6 months post-

approval, remaining at 50% (IQR 33-73%).

For trials in patients, results for a median of 65% (IQR 50-71%) of trials were publicly

available at the time of FDA approval. At 3 months post-approval, the median was 80% (IQR

67-100%), 86% (IQR 67-100%) at 6 months.

Page 16 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



17

Table 4: Timing of Public Posting of Trial Results For New Drugs Approved by the FDA in 2014

All Clinical Trials in NDA Trials in Patients (excluding healthy volunteer trials) in NDA

At FDA approval By 3 months post-

approval 6 months At FDA approval 3 months 6 months

Drug

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data

available

Zydelig 19% 48% 19% 48% 19% 48% 60% 65% 60% 65% 60% 65%

Harvoni 25% 60% 28% 61% 33% 68% 45% 64% 52% 66% 61% 73%

Lynparza 50% 46% 58% 69% 67% 73% 50% 46% 58% 69% 67% 73%

Dalvance 50% 71% 50% 71% 50% 71% 63% 76% 63% 76% 63% 76%

Invokamet 45% 83% 65% 87% 73% 88% 82% 87% 95% 89% 95% 89%

Esbriet 47% 79% 47% 79% 47% 79% 70% 92% 70% 92% 70% 92%

Xigduo

XR 52% 84% 56% 92% 56% 92% 65% 85% 70% 93% 70% 93%

Farxiga 68% 66% 75% 94% 75% 94% 71% 66% 80% 95% 80% 95%

Xtoro 50% 95% 50% 95% 50% 95% 67% 97% 67% 97% 67% 97%

Otezla 43% 66% 60% 92% 60% 92% 73% 71% 93% 98% 93% 98%

Sivextro 57% 79% 67% 81% 71% 90% 71% 88% 71% 88% 86% 99%

Zontivity 63% 98% 74% 99% 74% 99% 71% 99% 86% 100% 86% 100%

Belsomra 8% 27% 24% 81% 24% 81% 33% 33% 100% 100% 100% 100%

Cerdelga 25% 24% 38% 56% 38% 56% 33% 12% 100% 100% 100% 100%

Jublia 67% 93% 67% 93% 78% 95% 80% 98% 80% 98% 100% 100%

Movantik 20% 48% 45% 84% 45% 84% 50% 57% 100% 100% 100% 100%

Viekira

Pak 15% 61% 22% 65% 29% 75% 60% 83% 87% 88% 100% 100%

Zerbaxa 27% 11% 93% 99% 93% 99% 17% 5% 100% 100% 100% 100%

Zykadia 20% 81% 20% 81% 20% 81% 100% 100% 100% 100% 100% 100%

Quartile 1 20% 48% 28% 69% 33% 73% 50% 57% 67% 88% 67% 89%

Quartile 3 52% 83% 67% 93% 73% 94% 71% 92% 100% 100% 100% 100%

Median 45% 66% 50% 81% 50% 84% 65% 76% 80% 95% 86% 98%

Page 17 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




18

Location of Registration, Reporting, and CSRs

Almost all registered trials (314 of 315) were registered in ClinicalTrials.gov. No trials

were uniquely registered or had summary results reported in international registries. Only one

trial was uniquely registered in a corporate registry and not in ClinicalTrials.gov, no summary

results were uniquely posted in corporate registries. While ClinicalTrials.gov contained

occasional links to CSRs, it was not comprehensive; 41 CSRs were uniquely posted on corporate

registries.

Company Rankings

Sanofi / Genzyme and Johnson & Johnson / Janssen achieved the highest overall clinical

trial transparency scores, tying for first place in the rankings and scoring 100% on the patient and

FDAAA trial metrics (Table 5). AbbVie (96%), Celgene (95%), Merck (93%) and Astra Zeneca

also scored at or above the industry median. Valeant scored lowest (50%).

Table 5: Companies’ Overall Clinical Trial Transparency Rankings for Drugs Approved in

2014†

Rank Company Transparency Score

#1 Johnson & Johnson / Janssen 100%

#1 Sanofi / Genzyme 100%

#3 Abbvie 96%

#4 Celgene 95%

#5 Merck 93%

#6 AstraZeneca 91%

#7 Roche 90%

#8 Novartis 88%

#9 Gilead 73%

#10 Allergan 63%

#11 Valeant 50%

Median 91%

† Based on the average of companies’ scores for (1) the trials-in-patients analysis and (2) FDAAA

compliance measure that counted a trial as compliant if it satisfied either interpretation of

the reporting requirements.

Page 18 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



19

DISCUSSION

This analysis of all known sources of publicly available information about clinical trials

found high levels of transparency among large pharmaceutical companies and newly approved

drugs for trials conducted in patients. Per drug, a median of 96% of trials in patients were

publicly available in some form, 100% were registered, 71% reported results or shared CSRs,

and 80% were published. Compliance with FDAAA requirements was also high (median of

100% per drug).

Our earlier work examined transparency levels on these measures for all-trials in a

successful NDA (including trials in healthy volunteers) for drugs approved in 2012 [24].

Juxtaposing the two studies’ findings shows little change in transparency levels for this metric.

The median proportion of all trials registered per drug was 57% in 2012 vs. 53% in 2014; the

median for reporting results, 20% vs. 24%; the median for publication, 56% vs. 60%; and the

median for overall availability, 65% vs. 68%. The lack of increase in the proportions registered

and with results reported is surprising because our 2014 methodology newly incorporated more

registries and included CSR synopses as satisfying the standard for posting results.

Among the 2014 drugs, the gap between transparency of results from all-trials and the

trials-in-patients is striking. The median proportion of trials available per drug was markedly

lower in the all-trials sample for trial registration (53% vs. 100%), reporting results or CSR

summaries (24% vs. 71%), publication (60% vs. 80%), and overall availability (68% vs. 96%).

Compliance with FDAAA, measured by the “trial completion date” interpretation, increased

from a median of 67% of covered trials per drug in 2012 to 71% in 2014.

There is disagreement about the value of disclosing information about trials in healthy

volunteers. Some pharmaceutical companies, pharmaceutical trade associations,[14] and

Page 19 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



20

commentators [29] have focused on trials in patients for several reasons. Historically, later-

phase trials have been considered to have the greatest public health relevance and salience for

prescription guidelines writers and drug formulary committees. Limiting disclosure to trials in

patients captures phase 1 trials for serious diseases, like most cancers, where the relevance of

early data to patient care is high. Arguably, any important safety signals that emerge in phase 1

trials in healthy volunteers resurface in phase 2 trials, so critical safety information likely does

reach the public. Lastly, trials in healthy volunteers are small, seldom controlled, and therefore

not generally powered to provide statistical significance.

Notwithstanding these arguments, there is value in making information about all-trials

available. NIH policy now requires it. Many companies already operate on this standard. It

would alleviate public concerns about whether useful information is being hidden, and speed

decision making based on safety signals. Additionally, disclosing phase I trials may help speed

innovation and save money, particularly for small biotechnology companies (by preventing them

from travelling down known dead-end pathways or empowering them to design better trials

based on the lessons learned from previous studies).

Whether transparency standards focus on all-trials or trials in patients, there is a need for

clearer, more harmonized metrics so that progress over time can be gauged and companies

receive a consistent message about what is important for them to do. At a minimum, FDA should

clarify which of the two interpretations of FDAAA is correct. The Final Rule may help in this

regard, but it is too early to confirm.

More broadly, we believe the metrics articulated in this study are useful, broadly

acceptable, and demonstrably workable to implement. In ongoing work, we will supplement

them with a metric for patient-level data sharing—the new frontier in clinical trial transparency.

Page 20 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



21

Our study has limitations. First, our decision not to count abstracts submitted to scientific

conferences as “publications” may be controversial. Second, some studies may be published after

our study cut-off of 13 months post FDA approval of a drug. Third, we attributed transparency

scores, on the drug level, generally to the company that submitted a drug’s NDA although a few

trials in the NDA were sponsored by other companies—typically, a company the NDA sponsor

acquired (see Appendix 5). NDA sponsors presumably had access to data from those trials in

order to file the NDA that included them; however, one company reported being unable to edit

the ClinicalTrials.gov entry created by the acquired company. This limitation only applied to the

drug evaluations, not the company rankings, as these trials were often excluded from the

Company rankings. Also, it is worth noting that some trials in an NDA are for different

indications than the approved indication. The FDA generally evaluates these trials as safety trials

for the approved indication.

CONCLUSION

Our study demonstrates that clinical trial transparency benchmarks and practices can

vary. On the drug level, about half of FDA approved drugs have publicly disclosed results for all

patient trials. On the company level, about 18% of large companies fully disclosed all patient

trial results and complied with FDAAA disclosure requirements. Per drug, among trials in

patients, a median of 100% of trials were registered and 96% had publicly available trial results,

in some form. Among large pharmaceutical companies, clinical trial transparency is high based

on many measures, although opportunities for improvement remain. Momentum for greater

clinical trial transparency will grow as we continue to experience its benefits. Reaching

Page 21 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



22

consensus about standards, tracking adherence over time, and celebrating progress can move the

field forward toward a shared vision of transparency and what it can achieve.

Acknowledgments

The authors thank the the Laura and John Arnold Foundation; pharmaceutical company

representatives who provided information critical to the preparation of this report; and Yael Bree,

Tamara Hardoby, Mindy Kresch, Rosa Macrito, and Luke Sleiter for research assistance. All

errors and conclusions are those of the authors only.

Funding and Competing Interests

This work was funded by a grant from the Laura and John Arnold Foundation. Dr. Ross receives

support through Yale University from Johnson and Johnson to develop methods of clinical trial

data sharing, from Medtronic, Inc. and the Food and Drug Administration (FDA) to develop

methods for post-market surveillance of medical devices, from the Food and Drug

Administration (FDA) to establish the Yale-Mayo Center for Excellence in Regulatory Science

and Innovation (CERSI), from the Blue Cross Blue Shield Association to better understand

medical technology evaluation, from the Centers of Medicare and Medicaid Services (CMS) to

develop and maintain performance measures that are used for public reporting, and from the

Laura and John Arnold Foundation to support the Collaboration on Research Integrity and

Transparency (CRIT) at Yale.

Data Sharing Statement

The authors will make these data publicly available in a data repository, like Dryad, and/or a

public website.

Page 22 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



23

Authorship Statement

J.M. conceived the study, supervised the collection and coding of data, led the data analysis, and

wrote the initial manuscript draft. N.R. contributed to data collection, coding, and analysis.

M.M.M., J.S.R., and M.W. provided guidance on the analytical approach, contributed to the

interpretation of results, and revised the manuscript for critical intellectual content.

References

1. Ross JS, Mulvey GK, Hines EM, et al. Trial publication after registration in

ClinicalTrials. Gov: a cross-sectional analysis. PLoS Med. 2009;6(9):e1000144.

2. Jones CW, Handler L, Crowell KE, et al. Non-publication of large randomized clinical

trials: cross sectional analysis. BMJ. 2013;347:f6104.

3. Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug

applications: a literature analysis. PLoS Med. 2008;5(9):e191.

4. Ross JS, Tse T, Zarin DA, et al. Publication of NIH funded trials registered in

ClinicalTrials. gov: cross sectional analysis. BMJ. 2012;344:d7292.

5. Dickersin K, Rennie D. Registering clinical trials. Jama. 2003;290(4):516-23.

6. Anderson ML, Chiswell K, Peterson ED, et al. Compliance with results reporting at

ClinicalTrials. gov. N Engl J Med. 2015;372(11):1031-9.

7. Food and Drug Administration. Food and Drug Administration Modernization Act

(FDAMA) of 1997.

8. Piller C, Law ignored, patients at risk. StatNews. 2015. Available from

https://www.statnews.com/2015/12/13/clinical-trials-investigation/ (Accessed May, 2017).

9. Chen R, Desai NR, Ross JS, et al. Publication and reporting of clinical trial results: cross

sectional analysis across academic medical centers. BMJ. 2016;352:i637.

10. US Food and Drug Administration. Food and Drug Administration Amendments Act of

2007. Public Law. 2007;110:85.

11. World Medical Association. World medical association declaration of Helsinki.

https://www.wma.net/what-we-do/medical-ethics/declaration-of-helsinki/ (Accessed May, 2017).

Page 23 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



24

12. De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from

the International Committee of Medical Journal Editors.

13. Lo B. Sharing clinical trial data: maximizing benefits, minimizing risk. Jama.

2015;313(8):793-4.

14. PhRMA EF. Principles for Responsible Clinical Trial Data Sharing. PhRMA; 2013 Jul

18. Available from http://www.phrma.org/press-release/joint-efpia-phrma-principles-for-

responsible-clinical-trial-data-sharing-become-effective-today (Accessed May, 2017).

15. Bonini S, Eichler HG, Wathion N, et al. Transparency and the European Medicines

Agency—sharing of clinical trial data. N Engl J Med. 2014;371(26):2452-5.

16. Moorthy VS, Karam G, Vannice KS, et al. Rationale for WHO's new position calling for

prompt reporting and public disclosure of interventional clinical trial results. PLoS Med.

2015;12(4):e1001819.

17. Bill and Melinda Gates Foundation. Bill & Melinda Gates Foundation Open Access

Policy. Available from: http://www.gatesfoundation.org/How-We-Work/General-

Information/Open-Access-Policy (Accessed May, 2017).

18. Wellcome Trust. Data sharing | Wellcome. Available from: https://wellcome.ac.uk/what-

we-do/topics/data-sharing (Accessed May, 2017).

19. Laura and John Arnold Foundation. Transparency. Laura and John Arnold Foundation.

2016. Available from: http://www.arnoldfoundation.org/initiative/research-integrity/guidelines-

for-investment-research/ (Accessed May, 2017).

20. National Institutes of Health. Clinical Trials Registration and Results Information

Submission. Final rule. Federal register. 2016;81(183):64981.

21. Zarin DA, Tse T, Sheehan J. The proposed rule for US clinical trial registration and

results submission. N Engl J Med. 2015;2015(372):174-80.

22. Zarin DA, Tse T, Williams RJ, et al. Trial reporting in ClinicalTrials. gov—the final rule.

N Engl J Med. 2016;375(20):1998-2004.

23. National Institutes of Health. NIH Policy on the Dissemination of NIH-Funded Clinical

Trial Information. 2016.

24. Miller JE, Korn D, Ross JS. Clinical trial registration, reporting, publication and FDAAA

compliance: a cross-sectional analysis and ranking of new drugs approved by the FDA in 2012.

BMJ Open. 2015;5(11):e009758.

25. http://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/HowDrugsare

DevelopedandApproved/DrugandBiologicApprovalReports/NDAandBLAApprovalReports/

Page 24 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



25

UCM435753.pdf

26. http://siblisresearch.com/data/market-caps-sp-100-us/ (Accessed May, 2017).

27. https://clinicaltrials.gov/ct2/manage-recs/fdaaa (Accessed May, 2017).

28. Krlezˇa-Jeriç K, Lemmens T. 7th revision of the Declaration of Helsinki: good news for

the transparency of clinical trials.

29. Doernberg SN, Wendler D. Ensuring Respect for Human Research Participants:

Institutional Review Boards and Sharing Results From Research. Jama. 2016;316(11):1149-50.

30. Miller, JE, Bioethical accreditation or rating needed to restore trust in pharma. Nature Medicine.

2013;19:261

31. Miller JE. How a clinical trial registry became a symbol of misinformation. Hastings

Cent Rep. 2013;43:11–12.

Page 25 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



Figure 1: Drugs included in transparency analyses

Novel New Drugs and New Combinations approved by FDA in 2014 filed under individual

New Drug Applications (NDA) or New Biologic License Application (BLA) (n=116)

NMEs (FDA Chemical Classification 1) and New Combinations

(FDA Chemical Classification 4) (n=43)

NMEs and New Combinations containing at least one NME,

hereinafter “Drugs” (n=31)

Included drugs sponsored by the 20 pharmaceutical and biotechnology companies

with the largest market capitalizations in 2014 (or their subsidiaries) (n=19)

Excluded New Combinations which did

not contain at least one NME (n=12)

Excluded NDA approvals for new active

ingredient (2), new dosage (21), new formulations (32), over-the-counter switch

(4), prior marketing without approved

NDA (2), and new indication (n=1);

excluded BLAs (n=11)

Center for Drug Evaluation and Research Drug and Biologic Calendar Year 2014 Approvals

(n=119)

Combined duplicates

(n=3)

Page 26 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



1

APPENDIX

to

Miller JE et al., “Measuring Clinical Trial Transparency: An Empirical Analysis of Newly

Approved Drugs”

Appendix 1: List of Analyzed Registries and Data Sources, Including World Health

Organization (WHO) Indexed, International, National, and Corporate Registries

WHO International Clinical Trials Registry Platform (ICTRP), which

includes the following: Location

1. Association of Clinical Trials Organizations (ACTO) Russia

2. Australian New Zealand Clinical Trials Registry (ANZCTR) Australia/New Zealand

3. Brazilian Clinical Trials Registry (ReBec) Brazil

4. Chinese Clinical Trial Registry (ChiCTR) China

5. Clinical Research Information Service (CRiS), Republic of Korea Korea

6. Clinical Trials.gov United States

7. Clinical Trials Registry - India (CTRI) India

8. Cuban Public Registry of Clinical Trials (RPCEC) Cuba

9. EU Clinical Trials Register (EU-CTR) European Union

10. German Clinical Trials Register (DRKS) (Affiliated registry: DRKS) Germany

11. International Standard Randomised Controlled Trial Number

(ISRCTN.org) Global

12. Iranian Registry of Clinical Trials (IRCT) Iran

13. Japan Primary Registries Network (JPRN) Japan

14. The Netherlands National Trial Register (NTR) Netherlands

15. Pan African Clinical Trial Registry (PACTR) Africa

16. Peruvian Registry of Clinical Trials (through PAHO) Peru

17. South African National Clinical Trial Register South Africa

18. Sri Lanka Clinical Trials Registry (SLCTR) Sri Lanka

19. Tanzania Clinical Trials Registry Tanzania

20. Thai Clinical Trials Registry (TCTR) Thailand

Other Registries, Databases, and

Websites Website

21. ClinicalTrials.gov www.Clinicaltrials.gov

22. EU Clinical Trials Database

(EudraCT) https://www.clinicaltrialsregister.eu/ctr-search/search

23. [email protected] [email protected] 24. Aggregate Analysis of

ClinicalTrials.gov database

(AACT) https://www.ctti-clinicaltrials.org/aact-database

25. PubMed

Page 27 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



2

26. Google Scholar

27. EMBASE

28. Corporate Press Releases

Corporate Registry/Website Link

29. Actavis

http://www.allergan.com/research-and-development/clinical-

trials/clinical-trial-results-sharing

30. AstraZeneca http://www.astrazenecaclinicaltrials.com/Submission/Search

31. Allergan www.allerganclinicaltrials.com/results

32. Amgen www.amgentrials.com/amgen/study.aspx

33. BMS / DCRI https://www.dcri.org/soar-data/

34. Celgene

http://www.celgene.com/research-development/clinical-

trials/celgene-sponsored-trials/

35. Genentech http://www.genentechclinicaltrials.com

36. Genzyme (Sanofi)

http://www.genzymeclinicalresearch.com/clinicaltrials/gzcr_p_

ot_ourtrials.asp

37. Merck http://www.merck.com/clinical-trials/search.html

38. Novartis https://www.novartisclinicaltrials.com

39. Roche http://www.roche-trials.com/searchFullText.action?drug=2 40. Clinical Study Data Request

(CSDR). Includes: Astellas,

Bayer, Boehringer Ingelheim,

Daiichi-Sankyo, Eisai, GSK,

Lilly, Novartis, Roche, Sanofi,

Takeda, UCB, ViiV http://www.clinicalstudydatarequest.com

41. Abbvie

http://www.abbvie.com/research-innovation/clinical-trials-data-

and-information-sharing/registration-of-protocols-and-results-

reporting.html

42. Janssen http://www.janssen.com/clinical-trials/transparency

43. Gilead http://www.gilead.com/research/clinical-trials

44. Pfizer

www.pfizer.com/research/research_clinical_trials/trial_res

ults

Page 28 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



3

Appendix 2: List of Trial Characteristics Abstracted from Registries and FDA Approval Packages

Characteristics: National Clinical Trial (NCT) number, title, recruitment status, whether study results were

reported and the date any results were first received, description of the treatment (e.g. dosage and comparators),

whether the trial was interventional, sponsors/collaborators, gender enrollment, age groups, phase, enrollment

numbers, funder, study type, study design, other IDs, registration date, start date, primary completion date (date

the last participant was examined and data for the primary outcome measure collected), primary outcome

measures, site locations, and any links to CSRs.

Page 29 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



4

Appendix 3. Details of Validation Process with Drug Companies

To ensure that our search process did not miss or mischaracterize any facts, NDA holders (generally the

manufacturers) of drugs reviewed in the study were sent a copy of their results and the supporting dataset and

asked to provide corrections and other feedback. The data generally went through 3 rounds of exchanges and

revisions with the companies. Companies were also invited to a 3-hour meeting to discuss study methods and

findings. Nine of the 11 companies participated in both the meeting and our drug data validation process,

affording data validation for 79% of drugs reviewed. Nine additional companies joined the 3-hour meeting.

After validation, the median number of trials per drug with publicly available results increased from

90% to 96% in the trials-in-patients sample and from 60% to 68% for all trials. FDAAA compliance increased

from 51% to 59% for the “trial completion date” interpretation and from 88% to 92% for the “approval date”

interpretation.

Scores changed primarily because the validation process located additional, publicly verifiable journal

publications (n=23) and identified typographical errors in FDA approval packages or ClinicalTrials.gov (n=17).

Additionally, results for 2 more trials were found in ClinicalTrials.gov (the registry had delayed the postings)

and 1 trial was removed from the FDAAA trials sample because the manufacturer attested that the drug was not

manufactured in the U.S.

Page 30 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



5

Appendix 4: List of FDA 2014 Approved Indications for Evaluated Drugs

Drug

Name

Short

Indication Indication Generic

Belsomra Insomnia

indicated for the treatment of insomnia characterized

by difficulties with sleep onset and/or sleep

maintenance

Dasabuvir Sodium;

Ombitasvir;

Paritaprevir;

Ritonavir

Cerdelga

Gaucher

Disease

indicated for the long-term treatment of adult

patients with Gaucher disease type 1 (GD1) who are

CYP2D6 extensive metabolizers (EMs), intermediate

metabolizers (IMs), or poor metabolizers (PMs) as

detected by an FDA-cleared test

Dalbavancin

Hydrochloride

Dalvance Skin Infection

indicated for the treatment of adult patients with

acute bacterial skin and skin structure infections

(ABSSSI), caused by susceptible isolates of the

following Gram-positive microorganisms:

Staphylococcus aureus (including methicillin-

susceptible and methicillin-resistant strains),

Streptococcus pyogenes, Streptococcus agalactiae,

Streptococcus dysgalactiae, Streptococcus anginosus

group (including S. anginosus, S. intermedius, S.

constellatus) and Enterococcus faecalis (vancomycin

susceptible strains)

Dapagliflozin

Propanediol

Esbriet

Pulmonary

Fibrosis

indicated for the treatment of idiopathic pulmonary

fibrosis (IPF) Olaparib

Farxiga

Type 2

Diabetes

indicated as an adjunct to diet and exercise to improve

glycemic control in adults with type 2 diabetes mellitus Naloxegol Oxalate

Harvoni Hepatitis C

indicated with or without ribavirin for the treatment

of patients with chronic hepatitis C virus (HCV)

genotype 1, 4, 5, or 6 infection

Dapagliflozin

Propanediol;

Metformin

Hydrochloride

Invokamet

Type 2

Diabetes


glycemic control in adults with type 2 diabetes mellitus

who are not adequately controlled on a regimen

containing metformin or canagliflozin, or in patients

who are already treated with both canagliflozin and

metformin Apremilast

Jublia Onychomycosis

indicated for the topical treatment of onychomycosis

of the toenail(s) due to Trichophyton rubrum and

Trichophyton mentagrophytes Efinaconazole

Page 31 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



6

Lynparza Ovarian Cancer

indicated as monotherapy in patients with

deleterious or suspected deleterious germline BRCA

mutated (as detected by an FDA-approved test)

advanced ovarian cancer who have been treated with

three or more prior lines of chemotherapy

Ledipasvir;

Sofosbuvir

Movantik Constipation

indicated for the treatment of opioid-induced

constipation (OIC) in adult patients with chronic non-

cancer pain Idelalisib

Otezla

Arthritis;

Psoriasis

indicated for the treatment of adult patients with

active psoriatic arthritis; indicated for the treatment

of patients with moderate to severe plaque psoriasis

who are candidates for phototherapy or systemic

therapy

Canaglifflozin;

Metformin

Hydrochloride

Sivextro Skin Infection

indicated for the treatment of acute bacterial skin

and skin structure infections (ABSSSI) caused by

susceptible isolates of the following Gram-positive

microorganisms: Staphylococcus aureus (including

methicillin-resistant [MRSA] and methicillin-

susceptible [MSSA] isolates), Streptococcus

pyogenes, Streptococcus agalactiae, Streptococcus

anginosus Group (including Streptococcus anginosus,

Streptococcus intermedius, and Streptococcus

constellatus), and Enterococcus faecalis Suvorexant

Viekira

Pak Hepatitis C

indicated for the treatment of patients with

genotype 1 chronic hepatitis C virus (HCV) infection

including those with compensated cirrhosis Tedizolid Phosphate

Xigduo XR Type 2 Diabetes

indicated as an adjunct to diet and exercise to

improve glycemic control in adults with type 2

diabetes mellitus when treatment with both

dapagliflozin and metformin is appropriate

Ceftolozane Sulfate;

Tazobactam Sodium

Xtoro Otitis Externa

indicated for the treatment of acute otitis externa

(AOE) with or without an otowick, caused by

susceptible strains of Pseudomonas aeruginosa and

Staphylococcus aureus in patients age 1 year and

older Vorapaxar Sulfate

Zerbaxa

Urinary &

Abdominal

Infections

indicated for the treatment of patients 18 years or

older with the following infections caused by

designated susceptible microorganisms: Complicated

Intra-abdominal Infections, used in combination with

metronidazole; Complicated Urinary Tract Infections,

including Pyelonephritis Finafloxacin

Zontivity

Thrombotic

Cardiovascular

Events

indicated for the reduction of thrombotic

cardiovascular events in patients with a history of

myocardial infarction (MI) or with peripheral arterial

disease (PAD). ZONTIVITY has been shown to reduce

the rate of a combined endpoint of cardiovascular

death, MI, stroke, and urgent coronary Ceritinib

Page 32 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



7

revascularization (UCR)

Zydelig Leukemia

indicated, in combination with rituximab, for the

treatment of patients with relapsed chronic

lymphocytic leukemia (CLL) for whom rituximab

alone would be considered appropriate therapy due

to other co-morbidities Pirfenidone

Zykadia Lung Cancer

indicated for the treatment of patients with

anaplastic lymphoma kinase (ALK)-positive

metastatic non-small cell lung cancer (NSCLC) who

have progressed on or are intolerant to crizotinib Eliglustat Tartrate

Page 33 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



8

Appendix 5: List of Key Trial and NDA Sponsors for Analyzed Drugs†

Drug --> NDA holder --> Registered Trial Sponsor

No.

Trials in

NDA Source

Belsomra

Merck, Sharp & Dohme Corp. 37 Drugs@FDA

Merck Sharp & Dohme Corp. 9 ClinicalTrials.gov

Not Registered 28 N/A

Cerdelga

Genzyme Corp/Sanofi 17 Drugs@FDA

Genzyme, a Sanofi Company|Sanofi 7 ClinicalTrials.gov


Dalvance

Durata Therapeutics Inc/ Allergan plc 22 Drugs@FDA

Durata Therapeutics Inc., an affiliate of Allergan plc 2 ClinicalTrials.gov

Pfizer 1 ClinicalTrials.gov

Vicuron Pharmaceuticals 1 ClinicalTrials.gov


Esbriet

Genentech / Roche Holding AG 21 Drugs@FDA

Genentech, Inc. 4 ClinicalTrials.gov

InterMune 2 ClinicalTrials.gov

Marnac, not registered 4

Personal Communication with

NDA holder

William Gahl, M.D.|National Human Genome

Research Institute (NHGRI)|National Institutes of

Health Clinical Center (CC) (Investigator Initiated 1

ClinicalTrials.gov, Personal

communication with NDA

holder

Investigator initiated 1


NDA holder


Farxiga

Astrazeneca 63 Drugs@FDA

AstraZeneca 17 ClinicalTrials.gov

AstraZeneca|Bristol-Myers Squibb 22 ClinicalTrials.gov

AstraZeneca|Bristol-Myers Squibb|The TIMI Study

Group|Hadassah Medical Organization 1 ClinicalTrials.gov

AstraZeneca|Parexel|Q2 solutions|PRA Health

Sciences|Covance Laboratories, Inc 1 ClinicalTrials.gov

AstraZeneca; Bristol-Myers Squibb co-sponsor 7


NDA holder


Harvoni

Gilead Sciences 68 Drugs@FDA

Page 34 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



9

Gilead Sciences 35 ClinicalTrials.gov

National Institute of Allergy and Infectious Diseases

(NIAID)|National Institutes of Health Clinical Center 1 ClinicalTrials.gov


Invokamet

Janssen / Johnson and Johnson 42 Drugs@FDA

Janssen Research & Development, LLC 16 ClinicalTrials.gov

Janssen Research & Development, LLC|The George

Institute for Global Health, Australia 1 ClinicalTrials.gov

Janssen Scientific Affairs, LLC 1 ClinicalTrials.gov

Janssen-Cilag International NV 1 ClinicalTrials.gov

Johnson & Johnson Pharmaceutical Research &

Development, L.L.C. 14 ClinicalTrials.gov

Mitsubishi Tanabe Pharma Corporation 4 ClinicalTrials.gov


Jublia

Dow Pharmaceutical Sciences / Valeant 9 Drugs@FDA

Dow Pharmaceutical Sciences 3 ClinicalTrials.gov


Lynparza



AstraZeneca|British Columbia Cancer Agency 1 ClinicalTrials.gov

AstraZeneca|European Network of Gynaecological

Oncology Trial Groups (ENGOT)|Myriad Genetic

Laboratories, Inc. 1 ClinicalTrials.gov

AstraZeneca|KuDOS Pharmaceuticals Limited 4 ClinicalTrials.gov

AstraZeneca|Myriad Genetic Laboratories, Inc. 1 ClinicalTrials.gov


Movantik



AstraZeneca|Nektar Therapeutics 1 ClinicalTrials.gov


Otezla

Celgene Corp 30 Drugs@FDA

Celgene Corporation 16 ClinicalTrials.gov


Sivextro

Cubist Pharmaceuticals / Merck and Co. 21 Drugs@FDA

Trius Therapeutics LLC 18 ClinicalTrials.gov

Trius Therapeutics LLC|Bayer 1 ClinicalTrials.gov

Page 35 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



10

Not Registered Bayer 2


NDA holder

Viekira Pak

AbbVie 65 Drugs@FDA

Abbott 3 ClinicalTrials.gov

AbbVie 7 ClinicalTrials.gov

AbbVie (prior sponsor, Abbott)|AbbVie 13 ClinicalTrials.gov


Xigduo XR




AstraZeneca|Bristol-Myers Squibb|The TIMI Study

Group|Hadassah Medical Organization 1 ClinicalTrials.gov


Xtoro

Alcon Research / Novartis 4 Drugs@FDA

Alcon Research 2 ClinicalTrials.gov


Zerbaxa


Cubist Pharmaceuticals 6 ClinicalTrials.gov


Zontivity



Merck Sharp & Dohme Corp.|Duke Clinical Research

Institute 1 ClinicalTrials.gov

Merck Sharp & Dohme Corp.|The Thrombolysis in

Myocardial Infarction Study (TIMI) Group 1 ClinicalTrials.gov

Merck Sharp & Dohme Corp.|The TIMI

(Thrombolysis in Myocardial Infarction) Study

Group|Duke Clinical Research Institute 1 ClinicalTrials.gov


Zydelig




Zykadia

Novartis Pharmaceuticals 13 Drugs@FDA

Novartis Pharmaceuticals|Novartis 9 ClinicalTrials.gov

Not Registered 4 N/A † Multiple parties can be involved in a drug’s development. This table lists the NDA sponsor and individual trial sponsors for each

drug we reviewed. Trial sponsors were identified from ClinicalTrials.gov, unless stated otherwise in the table.

Page 36 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



PRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 Checklist

Section/topic # Checklist item Reported on page #

TITLE

Title 1 Identify the report as a systematic review, meta-analysis, or both. 1

ABSTRACT

Structured summary 2 Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number.

2-3

INTRODUCTION

Rationale 3 Describe the rationale for the review in the context of what is already known. 4-5

Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).

4-5

METHODS

Protocol and registration 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number.

N/A

Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered,

language, publication status) used as criteria for eligibility, giving rationale. 6-7

Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.

6-7

Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.

27-29

Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable,

included in the meta-analysis). 6-7

Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.

7-8

Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.

8-9; 27-29

Risk of bias in individual studies

12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.

N/A

Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means). 8-10

Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done, including measures of consistency

(e.g., I2) for each meta-analysis.

8-10

Page 37 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 1 of 2


Risk of bias across studies 15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).

5

Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified.

N/A

RESULTS

Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.

11

Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations.

11-18

Risk of bias within studies 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12). N/A

Results of individual studies 20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot.

11-17

Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency. 18

Risk of bias across studies 22 Present results of any assessment of risk of bias across studies (see Item 15). N/A

Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]). N/A

DISCUSSION

Summary of evidence 24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers).

19-20

Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias).

21

Conclusions 26 Provide a general interpretation of the results in the context of other evidence, and implications for future research. 21

FUNDING

Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review.

22

From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097

For more information, visit: www.prisma-statement.org.

Page 2 of 2

Page 38 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Measuring clinical trial transparency: An empirical analysis of newly approved drugs and large pharmaceutical

companies

Journal: BMJ Open

Manuscript ID bmjopen-2017-017917.R1


Date Submitted by the Author: 14-Sep-2017

Complete List of Authors: Miller, Jennifer; NYU School of Medicine, Division of Medical Ethics, Department of Population Health; Bioethics International Wilenzick, Marc; International Aids Vaccine Initiative

Ritcey, Nolan; Bioethics International; NYU School of Medicine Ross, Joseph; Yale University School of Medicine, Internal Medicine; Yale University Yale School of Public Health Mello, Michelle; Stanford University Stanford Law School, Department of Health Research and Policy; Stanford University School of Medicine


Ethics


Keywords: CLINICAL PHARMACOLOGY, Health policy < HEALTH SERVICES ADMINISTRATION & MANAGEMENT, LAW (see Medical Law), MEDICAL ETHICS, PUBLIC HEALTH, bioethics


BMJ Open on N



jopen.bmj.com

/B

MJ O


ecember 2017. D

ownloaded from



1

Title: Measuring clinical trial transparency: An empirical analysis of newly approved drugs

and large pharmaceutical companies




of Medicine, and Bioethics International, New York, NY (JEM); International Aids Vaccine

Initiative New York, NY (MW); Bioethics International, New York, NY (NR); Section of




Yale-New Haven Health, all New Haven, CT (JSR); Stanford Law School and the Department of



Dr. Jennifer E. Miller




227 E. 30th St., Rm 723

New York, NY, 10016

[email protected]

Version: 9/5/17

Paper Word Count: 5,145


Page 1 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



2

ABSTRACT

Objectives: To define a series of clinical trial transparency measures and apply them to large

pharmaceutical and biotechnology companies and their 2014 FDA approved drugs.

Design: Cross-sectional descriptive analysis of all clinical trials supporting 2014 FDA approved

New Drug Applications (NDAs), for novel drugs sponsored by large companies.

Data sources: Data from over 45 sources, including [email protected], ClinicalTrials.gov,

corporate and international registries; PubMed, Google Scholar, EMBASE, corporate press

releases, and personal communications with drug manufacturers.




Results: The FDA approved 19 novel new drugs, sponsored by 11 large companies, involving

553 trials, in 2014. We analyzed 505 relevant trials. Per drug, a median of 100% (IQR 86-100%)

of trials in patients were registered, 71% (IQR 57-100%) reported results or shared a CSR

synopsis, 80% (70-100%) were published, and 96% (80-100%) were publicly available in some

form by 13 months after FDA approval. Disclosure rates were lower at FDA approval (65%) and

improved significantly by 6 months post FDA approval. Per drug, a median of 100% (IQR 75-

100%) of FDAAA-applicable trials were compliant. Half of reviewed drugs had publicly

disclosed results for all trials in patients in our sample. One trial was uniquely registered in a

corporate registry, and not ClinicalTrials.gov; 0 trials were uniquely registered in international

registries.


is high based on several standards, although opportunities for improvement remain.

Transparency is markedly higher for trials in patients than among all trials supporting drug

approval, including trials in healthy volunteers. Ongoing efforts to publicly track companies’

transparency records and recognize exemplary companies may encourage further progress.


Page 2 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



3



companies on several different clinical trial transparency standards on the ethics and legal


the trials they registered on ClinicalTrials.gov. This analysis is limited to drugs approved by

the FDA in 2014 for which the NDA holder was a large company. Subsequent and previous

publications evaluate more drugs and trial sponsors.

• Our review is global; we searched over 39 public trial registries, including international,

corporate, and patient registries, and we provide an assessment on the completeness of


• A further innovation is rigorously validating all study measures and data with NDA holders

(pharmaceutical companies); we obtained NDA holders’ feedback on their results and

convened 18 companies to discuss methods and root causes of transparency performance.

• We created a company ranking to accompany the drug rankings on clinical trial transparency

performance.

• We analyzed two different samples of trials: trials conducted in patients and all trials,

including those involving healthy volunteers.




captured.

• We do not count abstracts submitted to scientific conferences as “publications”.

• We included only those trials related to the indication(s) for which the new drug was initially

approved, in our legal compliance assessments.

Page 3 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



4

INTRODUCTION

Despite its importance in supporting evidence-based patient care, prescription guideline

development, formulary decisions, public trust in research, and healthcare innovation,

transparency around clinical trial results does not always occur.[1]-[7] Despite the promulgation

of multiple legal and guidance standards in the 20 years since Congress enacted the first law

requiring registration of certain trials,[8] transparency practices remain highly variable across

research sponsors like universities and drug companies.[4, 7, 9, 10] Moreover, studies that


markedly different transparency measures and standards, yielding different findings and progress

reports. A research sponsor, trial, or drug may look transparent in one study and opaque in

another.







Association identified trial registration and reporting as an ethical obligation, for all trials, in the






Page 4 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



5




introduced ambiguous, and at times conflicting, standards. To help harmonize standards going


further improvement is needed, we defined a series of concrete transparency measures and

applied them to drugs approved by the FDA in 2014 and their large company sponsors.

Our transparency standards include both measures of compliance with US legal

requirements—that is, whether companies report what FDAAA requires them to—and two

expanded standards applying to broader ranges of trials, one of which includes phase 1 trials in

healthy volunteers, and one that only looks at trials in patients. This paper and efforts are a

continuation of efforts, called The Good Pharma Scorecard, that began with benchmarking the

transparency of drugs approved by the FDA in 2012, sponsored by large drug companies, on a

significantly narrower set of measures.[25-27]


several different standards and measures—not merely the usual crude measure of whether

companies have reported results for the trials they registered on ClinicalTrials.gov. Our review is

global; we searched 39 public trial registries, including international, corporate, and patient

registries, and provide an assessment on the completeness of ClinicalTrials.gov. A further

innovation is rigorously validating all study measures and data with NDA holders; we obtained

NDA holders’ feedback on their results and convened 18 companies to discuss methods and root

causes of inferior and best practices. Lastly, we added a company ranking to accompany the drug

rankings.

Page 5 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



6

METHODS

Data













2014.[28] We confined our analysis to New Drug Applications (NDA) that were sponsored by

the 20 largest biotechnology and pharmaceutical companies, or their subsidiaries, as measured by

2014 market capitalization ranking.[29] We began with large companies to highlight practices

among those with the most resources available to deploy toward satisfying transparency

standards that are more comprehensive than those currently imposed by law. Future versions of

our scorecard expand the analysis to include all trial sponsors, including small and medium sized

companies.

Page 6 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



7






completion date (PCD) by our study cutoff date of February 1, 2016. PCD was defined as on

ClinicalTrials.gov. Observational studies (which constituted 5 of the 553 studies we reviewed)

were generally excluded because they were ongoing at the time of our study cut-off date.

Additionally, they are not covered under FDAAA legal requirements to report trial results.




reporting under FDAAA. Because our analyses examine both legal compliance with FDAAA

and satisfaction of a more aspirational standard, different samples of trials were required.








territories, and exported for research.[11, 30]

Page 7 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



8







publications.


and worked independently (details in Appendix 3). Discrepancies between coders’ findings were

resolved by consensus. Final datasets and findings were sent to drug companies for validation

(details in Appendix 4). Feedback from companies was generally incorporated into findings if it

could be validated through our public sources. For example, companies in some cases provided

a web link to a publication missed in our matching process.

The study did not undergo IRB review because it was not human subjects research.

Outcome Measures

Our first outcome measure examined whether the trials in our samples were registered in

any public registry, including corporate and international registries. Second, we determined

whether either trial results or a CSR synopsis were provided in the registry. Third, we

determined whether each trial was published in a journal indexed by PubMed, Google Scholar,

or EMBASE. Fourth, we deemed each trial “publicly available” if it had results reported in a

registry, a CSR synopsis provided in a registry, or results published in a journal.

Page 8 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



9












recommendation that all trials be registered and have results disclosed to honor ethical




requires the dissemination of research results. Most companies that participated in a meeting

convened by our project in August 2016, expressed a commitment to disclosing results only for

trials enrolling patients, and codes of conduct from industry trade associations generally only

charge member companies with the obligation to disclose results from trials in patients.

Therefore, we also applied our transparency standards to those trials that enrolled patients in the

intent-to-treat population. That analysis excluded patients with renal or hepatic impairment (who

did not have the condition being studied) and trials in healthy volunteers.

Page 9 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



10

We also assessed the extent to which trials subject to FDAAA met that statute’s








days after FDA approval if a company files a “certificate of delay” with the NIH.[8] The other is


approval. The Final Rule, a regulation released in September 2016 by the NIH and HHS clarifies

trial results reporting requirements, however, its effective date postdates the trials in our

sample.[21] We examined compliance with FDAAA among applicable trials using both

interpretations and used the “approval date” interpretation in calculating company rankings.




FDAAA compliance standard that counted a trial as compliant if it satisfied either interpretation


from all drugs and then calculated the percentages of trials satisfying each of the 2 standards. In



Page 10 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



11



Table 1: Transparency Measures and Analyzed Clinical Trial Samples

Samples analyzed



All trials

(including

those in

healthy

volunteers)

Trials in

patients

Trials

subject to

FDAAA*





approval

X X



post FDA approval

X X




X X




interpretation X

* Trials that FDAAA defines as being covered by its results reporting requirements.



commonly trials for each approved drug and drug company met the transparency measures. All

data were collected and analyzed in Microsoft Excel v.15.18 (Redmond, Washington).

Validation



Page 11 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



12

meeting to discuss study methods and findings (details in Appendix 5), affording validation for

79% of drugs reviewed. Validated results are presented below.

RESULTS







2016, trials terminated without enrollment, and expanded-access trials. Trials with unknown

phases (n=7) were excluded from the FDAAA subsample. A median of 7 trials per drug were

conducted in patients. Trials in patients accounted for 233 of 505 trials, but 93% of all trial

participants (124,664/133,428).

Transparency Scores Based on Trials in Patients

We first report results for the sample of trials in patients. A median of 100% (IQR 86-

100%) of trials in patients per drug were registered (Table 2). A median of 71% (IQR 57-100%)



form.

Ten of 19 drugs (53%) had at least one undisclosed trial conducted in patients. Six drugs

(32%) had at least one undisclosed Phase 2 or Phase 3 trial. At least 2,864 patients participated

in trials with undisclosed results.

Page 12 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



13

Transparency Scores Based on All Trials in an NDA, Including Healthy Volunteers

Transparency was lower using the standard that all trials in a successful NDA should be

publicly available, than it was for the trials-in-patients standard (Table 2). A median of 53%

(IQR 33-85%) of all trials per drug were registered, 24% (IQR 19-50%) reported results or

shared a CSR synopsis, and 60% (IQR 37-75%) were published. Overall, a median of 68% (IQR

40-84%) of trial results per drug, were publicly available (reported, shared in a CSR, or

published).





Page 13 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



14




• Esbriet: Intermune sponsored 2 trials; both had publicly disclosed results. Marnac sponsored 4 trials; 2 were publicly disclosed. Two trials were investigator initiated and were both

publicly available. Genentech merged with Intermune in 2014. Intermune purchased the rights to sell Esbriet in the U.S. in 2007.






Ethics Standard A: Trials in patients

(excluding healthy volunteer trials)

Ethics Standard B: All clinical trials

Drug NDA Sponsor

Short

Indication+

# trials


%publicly

available

# trials


%publicly

available



Pulmonary

Fibrosis 10 46% 40% 70% 70% 17 33% 24% 47% 47%






Thrombotic

Cardiovascular

Events 7 86% 71% 71% 86% 27 26% 19% 74% 78%

Xigduo







Gaucher

Disease 3 100% 100% 100% 100% 16 41% 19% 38% 38%

Viekira


Otezla Celgene

Arthritis;

Psoriasis 15 93% 93% 80% 100% 30 57% 47% 53% 63%





Urinary &

Abdominal

Infections 6 100% 100% 100% 100% 15 50% 40% 93% 93%

Quartile 1 5 86% 57% 70% 80% 16 33% 19% 37% 40%

Quartile 3 20 100% 100% 100% 100% 37 85% 50% 75% 84%

Median 7 100% 71% 80% 96% 22 53% 24% 60% 68%

Page 14 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




15






on time and 71% (IQR 0-87%) reported results on time. Of the 110,426 participants in trials

covered by FDAAA, 66% were in noncompliant trials under this interpretation.







filed.

Page 15 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



16












“Trial Completion Date” Interpretation “FDA Approval Date” Interpretation


# trials


% FDAAA

compliant

# trials


% FDAAA

compliant












Urinary & Abdominal

Infections 6 100% 0% 0% 6 100% 100% 100%


Thrombotic









Quartile 1 3 100% 0% 0% 3 100% 75% 75%

Quartile 3 13 100% 87% 87% 13 100% 100% 100%

Median 4 100% 71% 71% 4 100% 100% 100%

Page 16 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




17


At the time of FDA approval, results for a median of 45% (IQR 20-52%) of all trials per


67%) of all trials were publicly available. This median did not increase at 6 months post-


For trials in patients, transparency was achieved more quickly. Results for a median of

65% (IQR 50-73%) of trials were publicly available at the time of FDA approval. At 3 months

post-approval, the median was 85% (IQR 67-100%), 86% (IQR 67-100%) at 6 months.

Page 17 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



18

Table 4: Timing of Public Availability of Trial Results For New Drugs Approved by the FDA in 2014

Trials in Patients (excluding healthy volunteer trials) in NDA All Clinical Trials in NDA

At FDA approval 3 months post-approval 6 months post-approval At FDA approval 3 months post-approval 6 months post-

approval

Drug

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data

available

Zydelig 60% 65% 60% 65% 60% 65% 19% 48% 19% 48% 19% 48%

Harvoni 45% 64% 52% 66% 61% 73% 25% 60% 28% 61% 33% 68%

Lynparza 50% 46% 58% 69% 67% 73% 50% 46% 58% 69% 67% 73%

Dalvance 63% 76% 63% 76% 63% 76% 50% 71% 50% 71% 50% 71%

Invokamet 82% 87% 95% 89% 95% 89% 45% 83% 65% 87% 73% 88%

Esbriet 70% 92% 70% 92% 70% 92% 47% 79% 47% 79% 47% 79%

Xigduo

XR 65% 85% 70% 93% 70% 93% 52% 84% 56% 92% 56% 92%

Farxiga 74% 66% 85% 95% 85% 95% 71% 66% 78% 94% 78% 94%

Xtoro 67% 97% 67% 97% 67% 97% 50% 95% 50% 95% 50% 95%

Otezla 73% 71% 93% 98% 93% 98% 43% 66% 60% 92% 60% 92%

Sivextro 71% 88% 71% 88% 86% 99% 57% 79% 67% 81% 71% 90%

Zontivity 71% 99% 86% 100% 86% 100% 63% 98% 74% 99% 74% 99%

Belsomra 33% 33% 100% 100% 100% 100% 8% 27% 24% 81% 24% 81%

Cerdelga 33% 12% 100% 100% 100% 100% 25% 24% 38% 56% 38% 56%

Jublia 80% 98% 80% 98% 100% 100% 67% 93% 67% 93% 78% 95%

Movantik 50% 57% 100% 100% 100% 100% 20% 48% 45% 84% 45% 84%

Viekira

Pak 60% 83% 87% 88% 100% 100% 15% 61% 22% 65% 29% 75%

Zerbaxa 17% 5% 100% 100% 100% 100% 27% 11% 93% 99% 93% 99%

Zykadia 100% 100% 100% 100% 100% 100% 20% 81% 20% 81% 20% 81%

Quartile 1 50% 57% 67% 88% 67% 89% 20% 48% 28% 69% 33% 73%

Quartile 3 73% 92% 100% 100% 100% 100% 52% 83% 67% 93% 73% 94%

Median 65% 76% 85% 95% 86% 98% 45% 66% 50% 81% 50% 84%

Page 18 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




19







registries.

Company Rankings



FDAAA trial standards (Table 5). AbbVie (96%), Celgene (95%), Merck (93%) and Astra

Zeneca also scored at or above the industry median. Valeant scored lowest (50%).


2014†




#3 Abbvie 96%

#4 Celgene 95%

#5 Merck 93%

#6 AstraZeneca 91%

#7 Roche 90%

#8 Novartis 88%

#9 Gilead 73%

#10 Allergan 63%

#11 Valeant 50%

Median 91%




Page 19 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



20

DISCUSSION





and 80% were published, by 13 months after FDA approval. It takes about a year after FDA

approval for companies to publicize most (96%) of their trials in patients. At FDA approval 65%

of trials per drug are available and at 3 months after FDA approval, 80%. Compliance with

FDAAA requirements was high (median of 100% per drug).

The gap between transparency of results from the all-trials and trials-in-patients samples

is striking. The median proportion of trials available per drug was markedly lower in the all-trials

sample for trial registration (53% vs. 100%), reporting results or CSR summaries (24% vs. 71%),

publication (60% vs. 80%), and overall availability (68% vs. 96%). The insomnia drug,

Belsomra, is an extreme example of the difference: only 6 of 37 trials supporting the NDA were

in patients, and the percentage publicly available was 100% for trials in patients but 20% among

all trials (Table 2).

Our earlier work examined transparency levels on these measures for all trials in a


Juxtaposing the two studies’ findings shows little change in transparency levels for this standard.



median for overall availability, 65% vs. 68%. The lack of increase in the proportions of trials

registered and with results reported is surprising because our 2014 methodology newly

Page 20 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



21

incorporated more registries and included CSR synopses as satisfying the requirement for

posting results. Compliance with FDAAA, measured by the “trial completion date”

interpretation, increased from a median of 67% of covered trials per drug in 2012 to 71% in

2014.

There is disagreement about the value of disclosing information for trials in healthy


commentators [31-32] have focused on trials in patients for several reasons. Historically, later-

phase efficacy trials have been considered to have the greatest public health relevance and

salience for prescription guidelines writers and drug formulary committees. Limiting disclosure

to trials in patients captures phase 1 trials for serious diseases, like most cancers, where the

relevance of early data to patient care is high. Arguably, any important safety signals that emerge

in phase 1 trials in healthy volunteers resurface in phase 2 trials, so critical safety information

likely does reach the public. Lastly, trials in healthy volunteers are small, seldom controlled, and

therefore not generally powered to provide statistical significance.

Notwithstanding these arguments, there is value in making information about all trials




innovation and save money, particularly for small biotechnology companies, by preventing

others from travelling down known dead-end pathways or empowering them to design better

trials based on the lessons learned from previous studies.

Whether transparency analyses focus on all trials or trials in patients, there is a need for

clearer, more harmonized standards so that progress over time can be gauged and companies

Page 21 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



22


clarify which of the two interpretations of FDAAA is correct. The Final Rule is helpful in this

regard.

More broadly, we believe the measures articulated in this study are useful, broadly


them with a measure for patient-level data sharing—the new frontier in clinical trial

transparency.












for the approved indication. Our analysis was limited to large companies, to drugs approved by

the FDA in 2014, and to trials included in the relevant NDAs approved in 2014. Finally, our

company rankings are not adjusted for the volume of trials conducted. Some may object that this

disadvantages companies with a large number of trials, for whom compliance may be more

resource intensive.

Page 22 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



23

CONCLUSION

Our study shows that clinical trial transparency practices vary according to the standard

used to measure them. On the drug level, about half of FDA approved drugs have publicly

disclosed results for all trials in patients that were included in our sample. On the company level,

about 18% of large companies fully disclosed all such results and complied with FDAAA

disclosure requirements. Per drug, among trials in patients, a median of 100% of trials were

registered and 96% had publicly available trial results, in some form. Among large

pharmaceutical companies, clinical trial transparency is high based on many measures, although

opportunities for improvement remain.

Momentum for greater clinical trial transparency will grow as we continue to experience

its benefits. Legal requirements in FDAAA and NIH rules push the effort forward, along with

efforts by other organizations like the Gates Foundation, World Health Organization, and

Wellcome Trust, but reaching consensus on standards and monitoring and publicizing

companies’ adherence to emerging standards are also critical. Celebrating progress—and

identifying where it is not occurring as quickly as it could—can move the field forward toward a

shared vision of transparency and what it can achieve.

Acknowledgments

The authors thank the Laura and John Arnold Foundation; pharmaceutical company

representatives who validated their datasets; and Yael Bree (Baruch student, NYU and BEI

intern), Tamara Hardoby (NYU graduate and BEI researcher), Mindy Kresch (Baruch student,

NYU and BEI intern), Rosa Macrito (Columbia Student, BEI researcher), and Luke Sleiter

Page 23 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



24

(NYU graduate, NYU Researcher, BEI researcher) for research assistance. All errors and

conclusions are those of the authors only.













Study data will be posted at bioethicsinternational.org upon publication of this article and can

also be obtained from the lead author.






Page 24 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



25

References

1. Dickersin K, Rennie D. The evolution of trial registries and their use to assess the clinical

trial enterprise. Jama. 2012 May 2;307(17):1861-4.


ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med. 2009;6(9):e1000144.











(FDAMA) of 1997.






2007. Public Law. 2007;110:85.






2015;313(8):793-4.




Page 25 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



26





2015;12(4):e1001819.










Submission. Final rule. Federal Register. 2016;81(183):64981.




N Engl J Med. 2016;375(20):1998-2004.





BMJ Open. 2015;5(11):e009758.


2013;19:261.


Cent Rep. 2013;43:11–12.



UCM435753.pdf


Page 26 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



27






List of Figures:

Figure 1 (figure)

Caption: Drugs included in transparency analyses

Page 27 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



Figure 1: Drugs Included in transparency analysis

279x361mm (300 x 300 DPI)

Page 28 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



1

APPENDIX

to


Approved Drugs and Drug Companies”

CONTENTS:

1. Appendix 1: List of Analyzed Registries and Data Sources, Including World Health


2. Appendix 2: List of Trial Characteristics Abstracted from Registries and FDA

Approval Packages

3. Appendix 3. Additional Information About the Data Collection and Analysis

Methods

4. Appendix 4. Details of Validation Process with Drug Companies

5. Appendix 5. Information about the Pharmaceutical Executive Roundtable Meeting

6. Appendix 6: List of FDA 2014 Approved Indications for Evaluated Drugs

7. Appendix 7. List of Key Trial and NDA Sponsors for Analyzed Drugs
















(ISRCTN.org) Global










Page 29 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



2


Websites Website




23. [email protected] [email protected]

24. Aggregate Analysis of



25. PubMed

26. Google Scholar

27. EMBASE



29. Actavis







34. Celgene






ot_ourtrials.asp



39. Roche http://www.roche-trials.com/searchFullText.action?drug=2

40. Clinical Study Data Request






41. Abbvie



reporting.html



44. Pfizer www.pfizer.com/research/research_clinical_trials/trial_results

Page 30 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from

http://www.clinicaltrials.gov/

https://www.clinicaltrialsregister.eu/ctr-search/search


https://www.ctti-clinicaltrials.org/aact-database

http://www.allergan.com/research-and-development/clinical-trials/clinical-trial-results-sharing


http://www.astrazenecaclinicaltrials.com/Submission/Search

http://www.amgentrials.com/amgen/study.aspx


http://www.celgene.com/research-development/clinical-trials/celgene-sponsored-trials/


http://www.genentechclinicaltrials.com/

http://www.genzymeclinicalresearch.com/clinicaltrials/gzcr_p_ot_ourtrials.asp


http://www.merck.com/clinical-trials/search.html

https://www.novartisclinicaltrials.com/

http://www.roche-trials.com/searchFullText.action?drug=2

http://www.clinicalstudydatarequest.com/

http://www.abbvie.com/research-innovation/clinical-trials-data-and-information-sharing/registration-of-protocols-and-results-reporting.html



http://www.janssen.com/clinical-trials/transparency

http://www.gilead.com/research/clinical-trials

http://www.pfizer.com/research/research_clinical_trials/trial_results



3

Appendix 2: List of Trial Characteristics Abstracted from Registries and FDA Approval

Packages

Characteristics: National Clinical Trial (NCT) number, title, recruitment status, whether study

results were reported and the date any results were first received, description of the treatment

(e.g. dosage and comparators), whether the trial was interventional, sponsors/collaborators,

gender enrollment, age groups, phase, enrollment numbers, funder, study type, study design,

other IDs, registration date, start date, primary completion date (date the last participant was

examined and data for the primary outcome measure collected), primary outcome measures, site

locations, and any links to CSRs.

Page 31 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



4

Appendix 3. Additional Information About the Data Collection and Analysis Methods

Data were collected by a team of 5 undergraduate and graduate-level researchers and a PhD-level

research director, all of whom were supervised by the principal investigator, Dr. Miller.

Data collectors were trained to read FDA approval packages, use ClinicalTrials.gov and other

trial registries, and search PubMed using the search methodology described in Miller et al.,

2015.1 Validated data from drugs approved in 2012 and reported in that article were used as a

training dataset. Each data collector’s accuracy was tested using a sample from the 2012 data

before that person was permitted to move on to collect data for the 2014 sample.

Data for each drug within the sample were collected by at least 2 data collectors, working

independently and blinded to one another’s work. The researchers then met to validate each

other’s work and create one master spreadsheet for each drug. If there were discrepancies in the

data, they were resolved by consensus of the research group including the principal investigators,

sometimes using additional information elicited from the company. Data were recorded,

analyzed, and summarized using Microsoft Excel v.15. A list of the data fields extracted are

listed in Appendix 2. Many of the variables came directly from ClinicalTrials.gov and

Drugs@FDA. We used the NIH data dictionary to define terms.

Data were analyzed by the 2 most experienced data collectors and the PhD-level director, with

direction from the faculty investigators. In determining whether a trial met requirements for

registration, reporting, publication, public availability, FDAAA applicability, and FDAAA

compliance, each analyst reached his/her determination independently of the others.

Discrepancies in these determinations were resolved by consensus, with support from the

principal investigators. Determination of FDAAA applicability and compliance were made

using two primary resources, the Food and Drug Administration Amendments Act of 2007

(FDAAA) and guidance provided by NIH through clinicaltrials.gov information sections on

FDAAA.

1 Miller JE, Korn D, Ross JS. Clinical trial registration, reporting, publication and FDAAA compliance: a cross-sectional analysis and ranking of new drugs approved by the FDA in 2012. BMJ Open. 2015;5(11):e009758.

Page 32 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



5


To ensure our search process did not miss or mischaracterize facts, NDA holders

(generally the manufacturers) of drugs reviewed in the study were sent a copy of their results and

the supporting dataset and asked to provide corrections. Companies were also invited to a 3-hour

meeting to discuss study methods and findings. Nine of the 11 companies participated in both the

meeting and our drug data validation process, affording data validation for 79% of drugs

reviewed. Nine additional companies joined the 3-hour meeting.

Corrections were accepted if the research team could corroborate the information with public

sources, e.g., by visiting a website to which the company directed us. The scenarios below

typify the types of feedback we received:

1) A company indicates that results were reported for a trial on ClinicalTrials.gov, where we

marked it as not having results there. We recheck ClinicalTrials.gov, confirm that the

results are there, and revise our data. The records on ClinicalTrials.gov are constantly

being updated with new information, so occasionally companies submitted information

that had not yet appeared publicly at the time of our review.

2) A company clarifies whether FDAAA applies to a particular trial. One requirement for

FDAAA coverage turns on manufacturing data, which is difficult to come by through

public sources. FDAAA says that a trial must be conducted in the US, or have a drug

manufacturer in the US for export, to be covered. In several instances, companies

informed us that this requirement was not in fact satisfied, which caused us to correct an

initial judgment that the trial was FDAAA-applicable.

3) The company provides a web link to a publication, where we had found no publications.

Matching trial characteristics to publications can be challenging, especially for phase 1

trials with no registration record, so some publications were initially missed.

After validation, the median number of trials per drug with publicly available results

increased from 90% to 96% in the trials-in-patients sample and from 60% to 68% for all

trials. FDAAA compliance increased from 51% to 59% for the “trial completion date”

interpretation and from 88% to 92% for the “approval date” interpretation.

Scores changed primarily because the validation process located additional, publicly

verifiable journal publications (n=23) and identified typographical errors in FDA approval

packages or ClinicalTrials.gov (n=17). Additionally, results for 2 more trials were found in

ClinicalTrials.gov (the registry had delayed the postings) and 1 trial was removed from the

FDAAA trials sample because the manufacturer attested that the drug was not manufactured

in the U.S.

Page 33 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



6

Appendix 5. Information about the Pharmaceutical Executive Roundtable Meeting

On August 17, 2016 and March 10, 2017, the research team hosted Pharmaceutical Executive

Roundtables. The first was held at the NYU School of Medicine offices of the Alexandria Center

for Life Science in New York City. The second meeting was held at Ernst & Young in New

York City. Both meetings had more than 30 participants. The below institutions participated in

at least one of the two meetings:

• Representatives from AbbVie, AstraZeneca, Bayer, Biogen, Bristol-Myers Squibb,

Celgene, GlaxoSmithKline, Johnson & Johnson, Lilly, Novartis, Novo Nordisk, Pfizer,

Roche, Sanofi, and Syndax;

• Bioethics International research staff and Board of Directors members;

• Academic collaborators on the Scorecard project, based at Harvard University, Stanford

University School of Law, Yale School of Medicine, and NYU School of Medicine);

• Representatives from the Laura and John Arnold Foundation, Gates Foundation, and

Helmsley Foundation, and

• Representatives from Ernst and Young.

The pharmaceutical company representatives had various job titles, such as Vice President and

Head of U.S. Medical Affairs, Director of Clinical Trial Transparency, Chief Medical Officer,

Chief Executive Officer, and Global Head of Patient Affairs and Policy.

The 2016 Roundtable offered participants the opportunity to learn about the Good Pharma

Scorecard’s history, methods, and how to comply with its standards. Additionally, participants

could candidly discuss concerns with, and provide feedback on, the study team’s methodological

approach to data collection and analysis. Moreover, participants were encouraged to discuss

challenges and barriers to registering and reporting results of clinical trials and publishing the

results of trials in medical journals. Root causes of best and inferior practices were also

explored.

Each meeting lasted approximately 3 hours and included discussion about the importance of

disclosing phase 1 trial results and interpretations of FDAAA’s legal requirements for disclosure.

The principal changes to methods in the Scorecard made after the Roundtable were as follows:

• Expanding our search to include international and corporate registries (in addition to

registration at ClinicalTrials.gov); and

• Including the clinical study report synopsis as a measure of results reporting.

Page 34 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



7


Drug

Name

Short

Indication Indication Generic Name

Belsomra Insomnia

indicated for the treatment of insomnia characterized by

difficulties with sleep onset and/or sleep maintenance

Dasabuvir Sodium;

Ombitasvir;

Paritaprevir;

Ritonavir

Cerdelga Gaucher Disease

indicated for the long-term treatment of adult patients with

Gaucher disease type 1 (GD1) who are CYP2D6 extensive

metabolizers (EMs), intermediate metabolizers (IMs), or

poor metabolizers (PMs) as detected by an FDA-cleared test

Dalbavancin

Hydrochloride


indicated for the treatment of adult patients with acute

bacterial skin and skin structure infections (ABSSSI),

caused by susceptible isolates of the following Gram-

positive microorganisms: Staphylococcus aureus (including

methicillin-susceptible and methicillin-resistant strains),


Streptococcus dysgalactiae, Streptococcus anginosus group

(including S. anginosus, S. intermedius, S. constellatus) and

Enterococcus faecalis (vancomycin susceptible strains)

Dapagliflozin

Propanediol

Esbriet

Pulmonary

Fibrosis

indicated for the treatment of idiopathic pulmonary fibrosis

(IPF) Olaparib

Farxiga Type 2 Diabetes



Harvoni Hepatitis C

indicated with or without ribavirin for the treatment of

patients with chronic hepatitis C virus (HCV) genotype 1, 4,

5, or 6 infection

Dapagliflozin

Propanediol;

Metformin

Hydrochloride

Invokamet Type 2 Diabetes


glycemic control in adults with type 2 diabetes mellitus who

are not adequately controlled on a regimen containing

metformin or canagliflozin, or in patients who are already

treated with both canagliflozin and metformin Apremilast


indicated for the topical treatment of onychomycosis of the

toenail(s) due to Trichophyton rubrum and Trichophyton

mentagrophytes Efinaconazole


indicated as monotherapy in patients with deleterious or

suspected deleterious germline BRCA mutated (as detected

by an FDA-approved test) advanced ovarian cancer who

have been treated with three or more prior lines of

chemotherapy

Ledipasvir;

Sofosbuvir


indicated for the treatment of opioid-induced constipation

(OIC) in adult patients with chronic non-cancer pain Idelalisib

Otezla

Arthritis;

Psoriasis

indicated for the treatment of adult patients with active

psoriatic arthritis; indicated for the treatment of patients

with moderate to severe plaque psoriasis who are candidates

for phototherapy or systemic therapy

Canaglifflozin;

Metformin

Hydrochloride

Page 35 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



8


indicated for the treatment of acute bacterial skin and skin

structure infections (ABSSSI) caused by susceptible isolates

of the following Gram-positive microorganisms:

Staphylococcus aureus (including methicillin-resistant

[MRSA] and methicillin-susceptible [MSSA] isolates),


Streptococcus anginosus Group (including Streptococcus

anginosus, Streptococcus intermedius, and Streptococcus


Viekira

Pak Hepatitis C

indicated for the treatment of patients with genotype 1

chronic hepatitis C virus (HCV) infection including those

with compensated cirrhosis Tedizolid Phosphate




when treatment with both dapagliflozin and metformin is

appropriate


Tazobactam Sodium


indicated for the treatment of acute otitis externa (AOE)

with or without an otowick, caused by susceptible strains of

Pseudomonas aeruginosa and Staphylococcus aureus in

patients age 1 year and older Vorapaxar Sulfate

Zerbaxa

Urinary &

Abdominal

Infections

indicated for the treatment of patients 18 years or older

with the following infections caused by designated

susceptible microorganisms: Complicated Intra-abdominal

Infections, used in combination with



Zontivity

Thrombotic

Cardiovascular

Events

indicated for the reduction of thrombotic cardiovascular

events in patients with a history of myocardial infarction

(MI) or with peripheral arterial disease (PAD).

ZONTIVITY has been shown to reduce the rate of a

combined endpoint of cardiovascular death, MI, stroke, and

urgent coronary revascularization (UCR) Ceritinib

Zydelig Leukemia

indicated, in combination with rituximab, for the treatment

of patients with relapsed chronic lymphocytic leukemia

(CLL) for whom rituximab alone would be considered

appropriate therapy due to other co-morbidities Pirfenidone

Zykadia Lung Cancer

indicated for the treatment of patients with anaplastic

lymphoma kinase (ALK)-positive metastatic non-small cell

lung cancer (NSCLC) who have progressed on or are

intolerant to crizotinib Eliglustat Tartrate

Page 36 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



9



No. Trials in NDA Source

Belsomra




Cerdelga




Dalvance






Esbriet




Marnac, not registered 4 Personal Communication with NDA holder

William Gahl, M.D.|National Human Genome Research Institute (NHGRI)|National Institutes of Health Clinical Center (CC) (Investigator Initiated 1

ClinicalTrials.gov, Personal communication with NDA holder

Investigator initiated 1 Personal Communication with NDA holder


Farxiga




AstraZeneca|Bristol-Myers Squibb|The TIMI Study Group|Hadassah Medical Organization 1 ClinicalTrials.gov

AstraZeneca|Parexel|Q2 solutions|PRA Health Sciences|Covance Laboratories, Inc 1 ClinicalTrials.gov

AstraZeneca; Bristol-Myers Squibb co-sponsor 7 Personal Communication with

Page 37 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



10

NDA holder


Harvoni



National Institute of Allergy and Infectious Diseases (NIAID)|National Institutes of Health Clinical Center 1 ClinicalTrials.gov


Invokamet



Janssen Research & Development, LLC|The George Institute for Global Health, Australia 1 ClinicalTrials.gov



Johnson & Johnson Pharmaceutical Research & Development, L.L.C. 14 ClinicalTrials.gov



Jublia




Lynparza




AstraZeneca|European Network of Gynaecological Oncology Trial Groups (ENGOT)|Myriad Genetic Laboratories, Inc. 1 ClinicalTrials.gov




Movantik





Otezla

Page 38 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



11




Sivextro




Not Registered Bayer 2 Personal Communication with

NDA holder

Viekira Pak

AbbVie 65 Drugs@FDA





Xigduo XR






Xtoro




Zerbaxa




Zontivity



Merck Sharp & Dohme Corp.|Duke Clinical Research Institute 1 ClinicalTrials.gov

Merck Sharp & Dohme Corp.|The Thrombolysis in Myocardial Infarction Study (TIMI) Group 1 ClinicalTrials.gov

Merck Sharp & Dohme Corp.|The TIMI (Thrombolysis in Myocardial Infarction) Study Group|Duke Clinical Research Institute 1 ClinicalTrials.gov

Page 39 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



12


Zydelig




Zykadia



Not Registered 4 N/A † Multiple parties can be involved in a drug’s development. This table lists the NDA sponsor and individual trial

sponsors for each drug we reviewed. Trial sponsors were identified from ClinicalTrials.gov, unless stated otherwise in the table.

Page 40 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



13

Page 41 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from





TITLE


ABSTRACT


2-3

INTRODUCTION



4-5

METHODS


N/A




6-7


29-31




8-9


8-11; 29-31



N/A




9-10

Page 42 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 1 of 2



5


N/A

RESULTS


11, 28


11-18



12-19

Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency. N/A



DISCUSSION


20-22


22


FUNDING


23-24



Page 2 of 2

Page 43 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Measuring clinical trial transparency: An empirical analysis of newly approved drugs and large pharmaceutical

companies

Journal: BMJ Open

Manuscript ID bmjopen-2017-017917.R2


Date Submitted by the Author: 25-Oct-2017

Complete List of Authors: Miller, Jennifer; NYU School of Medicine, Division of Medical Ethics, Department of Population Health; Bioethics International Wilenzick, Marc; International Aids Vaccine Initiative

Ritcey, Nolan; Bioethics International; NYU School of Medicine Ross, Joseph; Yale University School of Medicine, Internal Medicine; Yale University Yale School of Public Health Mello, Michelle; Stanford University Stanford Law School, Department of Health Research and Policy; Stanford University School of Medicine


Ethics


Keywords: CLINICAL PHARMACOLOGY, Health policy < HEALTH SERVICES ADMINISTRATION & MANAGEMENT, LAW (see Medical Law), MEDICAL ETHICS, PUBLIC HEALTH, bioethics


BMJ Open on N



jopen.bmj.com

/B

MJ O


ecember 2017. D

ownloaded from



1

Title: Measuring clinical trial transparency: An empirical analysis of newly approved

drugs and large pharmaceutical companies




of Medicine, and Bioethics International, New York, NY (JEM); International Aids Vaccine

Initiative New York, NY (MW); Bioethics International, New York, NY (NR); Section of




Yale-New Haven Health, all New Haven, CT (JSR); Stanford Law School and the Department of



Dr. Jennifer E. Miller




227 E. 30th St., Rm 723

New York, NY, 10016

[email protected]

Version: 9/5/17

Paper Word Count: 5,236


Page 1 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



2

ABSTRACT

Objectives: To define a series of clinical trial transparency measures and apply them to large

pharmaceutical and biotechnology companies and their 2014 FDA approved drugs.

Design: Cross-sectional descriptive analysis of all clinical trials supporting 2014 FDA approved

New Drug Applications (NDAs), for novel drugs sponsored by large companies.

Data sources: Data from over 45 sources, including [email protected], ClinicalTrials.gov,

corporate and international registries; PubMed, Google Scholar, EMBASE, corporate press

releases, SEC filings, and personal communications with drug manufacturers.




Results: The FDA approved 19 novel new drugs, sponsored by 11 large companies, involving

553 trials, in 2014. We analyzed 505 relevant trials. Per drug, a median of 100% (IQR 86-100%)

of trials in patients were registered, 71% (IQR 57-100%) reported results or shared a CSR

synopsis, 80% (70-100%) were published, and 96% (80-100%) were publicly available in some

form by 13 months after FDA approval. Disclosure rates were lower at FDA approval (65%) and

improved significantly by 6 months post FDA approval. Per drug, a median of 100% (IQR 75-

100%) of FDAAA-applicable trials were compliant. Half of reviewed drugs had publicly

disclosed results for all trials in patients in our sample. One trial was uniquely registered in a

corporate registry, and not ClinicalTrials.gov; 0 trials were uniquely registered in international

registries.


is high based on several standards, although opportunities for improvement remain.

Transparency is markedly higher for trials in patients than among all trials supporting drug

approval, including trials in healthy volunteers. Ongoing efforts to publicly track companies’

transparency records and recognize exemplary companies may encourage further progress.


Page 2 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



3



companies on several different clinical trial transparency standards on the ethics and legal


the trials they registered on ClinicalTrials.gov. This analysis is limited to drugs approved by

the FDA in 2014 for which the NDA holder was a large company. Subsequent and previous

publications evaluate more drugs and trial sponsors.

• Our review is global; we searched over 39 public trial registries, including international,

corporate, and patient registries, and we provide an assessment on the completeness of


• A further innovation is rigorously validating study findings and data with NDA holders

(pharmaceutical companies).

• We created a company ranking to accompany the drug rankings on clinical trial transparency

performance.

• We analyzed two different samples of trials: trials conducted in patients and all trials,

including those involving healthy volunteers.




captured.

• We do not count abstracts submitted to scientific conferences as “publications”.

• We included only those trials related to the indication(s) for which the new drug was initially

approved, in our legal compliance assessments.

Page 3 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



4

INTRODUCTION

Despite its importance in supporting evidence-based patient care, prescription guideline

development, formulary decisions, public trust in research, and healthcare innovation,

transparency around clinical trial results does not always occur.[1]-[7] Despite the promulgation

of multiple legal and guidance standards in the 20 years since Congress enacted the first law

requiring registration of certain trials,[8] transparency practices remain highly variable across

research sponsors like universities and drug companies.[4, 7, 9, 10] Moreover, studies that


markedly different transparency measures and standards, yielding different findings and progress

reports. A research sponsor, trial, or drug may look transparent in one study and opaque in

another.







Association identified trial registration and reporting as an ethical obligation, for all trials, in the






Page 4 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



5




introduced ambiguous, and at times conflicting, standards. To help harmonize standards going


further improvement is needed, we defined a series of concrete transparency measures and

applied them to drugs approved by the FDA in 2014 and their large company sponsors.

Our transparency standards include both measures of compliance with US legal

requirements—that is, whether companies report what FDAAA requires them to—and two

expanded standards applying to broader ranges of trials, one of which includes phase 1 trials in

healthy volunteers, and one that only looks at trials in patients. This paper is a continuation of an

initiative, called The Good Pharma Scorecard, that began with benchmarking the transparency of

drugs approved by the FDA in 2012, sponsored by large drug companies, on a significantly

narrower set of measures.[25-27]


several different standards and measures—not merely the usual crude measure of whether

companies have reported results for the trials they registered on ClinicalTrials.gov. Our review is

global; we searched 39 public trial registries, including international, corporate, and patient

registries, and provide an assessment on the completeness of ClinicalTrials.gov. A further

innovation is rigorously validating all study measures and data with NDA holders

(pharmaceutical companies). Lastly, we added a company ranking to accompany the drug

rankings.

METHODS

Page 5 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



6

Data













2014.[28] We confined our analysis to New Drug Applications (NDA) that were sponsored by

the 20 largest biotechnology and pharmaceutical companies, or their subsidiaries, as measured by

2014 market capitalization ranking.[29] We began with large companies to highlight practices

among those with the most resources available to deploy toward satisfying transparency

standards that are more comprehensive than those currently imposed by law. Subsidiaries and

parent companies were identified by searching corporate websites, press releases and SEC filings

and via communications with companies during the data validation process. Future versions of

our scorecard will expand the analysis to include all trial sponsors, including small and medium

sized companies and public sponsors.

Page 6 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



7






completion date (PCD) by our study cutoff date of February 1, 2016. PCD was defined as on

ClinicalTrials.gov. Observational studies (which constituted 5 of the 553 studies we reviewed)

were generally excluded because they were ongoing at the time of our study cut-off date.

Additionally, they are not covered under FDAAA legal requirements to report trial results.




reporting under FDAAA. Because our analyses examine both legal compliance with FDAAA

and satisfaction of a more aspirational standard, different samples of trials were required.








territories, and exported for research.[11, 30]

Page 7 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



8







publications.


and worked independently (details in Appendix 3). Discrepancies between coders’ findings were

resolved by consensus. Final datasets and findings were sent to drug companies for validation

(details in Appendix 4). Feedback from companies was generally incorporated into findings if it

could be validated through our public sources. For example, companies in some cases provided

a web link to a publication missed in our matching process.

The study did not undergo IRB review because it was not human subjects research.

Outcome Measures

Our first outcome measure examined whether the trials in our samples were registered in

any public registry, including corporate and international registries. Second, we determined

whether either trial results or a CSR synopsis were provided in the registry. Third, we

determined whether each trial was published in a journal indexed by PubMed, Google Scholar,

or EMBASE. Fourth, we deemed each trial “publicly available” if it had results reported in a

registry, a CSR synopsis provided in a registry, or results published in a journal.

Page 8 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



9












recommendation that all trials be registered and have results disclosed to honor ethical




requires the dissemination of research results. In contrast, companies generally only commit to

disclosing results for trials enrolling patients, not healthy volunteers, as expressed in their trade

association codes of conduct. Therefore, we also applied our transparency standards to those

trials that enrolled patients in the intent-to-treat population. That analysis excluded patients with

renal or hepatic impairment (who did not have the condition being studied) and trials in healthy

volunteers.

We also assessed the extent to which trials subject to FDAAA met that statute’s


Page 9 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



10







days after FDA approval if a company files a “certificate of delay” with the NIH.[8] The other is


approval. The Final Rule, a regulation released in September 2016 by the NIH and HHS clarifies

that trial results must be reported for both approved and unapproved indications, however, its

effective date postdates the trials in our sample.[21] We examined compliance with FDAAA

among applicable trials using both interpretations and used the “approval date” interpretation in

calculating company rankings.




FDAAA compliance standard that counted a trial as compliant if it satisfied either interpretation


from all drugs and then calculated the percentages of trials satisfying each of the 2 standards. In





Page 10 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



11

Table 1: Transparency Measures and Analyzed Clinical Trial Samples

Samples analyzed



All trials

(including

those in

healthy

volunteers)

Trials in

patients

Trials

subject to

FDAAA*





approval

X X



post FDA approval

X X




X X




interpretation X

* Trials that FDAAA defines as being covered by its results reporting requirements.



commonly trials for each approved drug and drug company met the transparency measures. All

data were collected and analyzed in Microsoft Excel v.15.18 (Redmond, Washington).

Validation



meeting to discuss study methods and findings (details in Appendix 5), affording validation for

79% of drugs reviewed. Validated results are presented below.

Page 11 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



12

RESULTS







2016, trials terminated without enrollment, and expanded-access trials. Trials with unknown

phases (n=7) were excluded from the FDAAA subsample. A median of 7 trials per drug were

conducted in patients. Trials in patients accounted for 233 of 505 trials, but 93% of all trial

participants (124,664/133,428).

Transparency Scores Based on Trials in Patients

We first report results for the sample of trials in patients. A median of 100% (IQR 86-

100%) of trials in patients per drug were registered (Table 2). A median of 71% (IQR 57-100%)



form.

Ten of 19 drugs (53%) had at least one undisclosed trial conducted in patients. Six drugs

(32%) had at least one undisclosed Phase 2 or Phase 3 trial. At least 2,864 patients participated

in trials with undisclosed results.

Transparency Scores Based on All Trials in an NDA, Including Healthy Volunteers

Page 12 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



13

Transparency was lower using the standard that all trials in a successful NDA should be

publicly available, than it was for the trials-in-patients standard (Table 2). A median of 53%

(IQR 33-85%) of all trials per drug were registered, 24% (IQR 19-50%) reported results or

shared a CSR synopsis, and 60% (IQR 37-75%) were published. Overall, a median of 68% (IQR

40-84%) of trial results per drug, were publicly available (reported, shared in a CSR, or

published).





Page 13 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



14




• Esbriet: Intermune sponsored 2 trials; both had publicly disclosed results. Marnac sponsored 4 trials; 2 were publicly disclosed. Two trials were investigator initiated and were both

publicly available. Genentech merged with Intermune in 2014. Intermune purchased the rights to sell Esbriet in the U.S. in 2007.






Ethics Standard A: Trials in patients

(excluding healthy volunteer trials)

Ethics Standard B: All clinical trials

Drug NDA Sponsor

Short

Indication+

# trials


%publicly

available

# trials


%publicly

available



Pulmonary

Fibrosis 10 46% 40% 70% 70% 17 33% 24% 47% 47%






Thrombotic

Cardiovascular

Events 7 86% 71% 71% 86% 27 26% 19% 74% 78%

Xigduo







Gaucher

Disease 3 100% 100% 100% 100% 16 41% 19% 38% 38%

Viekira


Otezla Celgene

Arthritis;

Psoriasis 15 93% 93% 80% 100% 30 57% 47% 53% 63%





Urinary &

Abdominal

Infections 6 100% 100% 100% 100% 15 50% 40% 93% 93%

Quartile 1 5 86% 57% 70% 80% 16 33% 19% 37% 40%

Quartile 3 20 100% 100% 100% 100% 37 85% 50% 75% 84%

Median 7 100% 71% 80% 96% 22 53% 24% 60% 68%

Page 14 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




15






on time and 71% (IQR 0-87%) reported results on time. Of the 110,426 participants in trials

covered by FDAAA, 66% were in noncompliant trials under this interpretation.







filed.

Page 15 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



16












“Trial Completion Date” Interpretation “FDA Approval Date” Interpretation


# trials


% FDAAA

compliant

# trials


% FDAAA

compliant












Urinary & Abdominal

Infections 6 100% 0% 0% 6 100% 100% 100%


Thrombotic









Quartile 1 3 100% 0% 0% 3 100% 75% 75%

Quartile 3 13 100% 87% 87% 13 100% 100% 100%

Median 4 100% 71% 71% 4 100% 100% 100%

Page 16 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




17


At the time of FDA approval, results for a median of 45% (IQR 20-52%) of all trials per


67%) of all trials were publicly available. This median did not increase at 6 months post-


For trials in patients, transparency was achieved more quickly. Results for a median of

65% (IQR 50-73%) of trials were publicly available at the time of FDA approval. At 3 months

post-approval, the median was 85% (IQR 67-100%), 86% (IQR 67-100%) at 6 months.

Page 17 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



18

Table 4: Timing of Public Availability of Trial Results For New Drugs Approved by the FDA in 2014

Trials in Patients (excluding healthy volunteer trials) in NDA All Clinical Trials in NDA

At FDA approval 3 months post-approval 6 months post-approval At FDA approval 3 months post-approval 6 months post-

approval

Drug

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data available

%publicly

available

trials

%participant

data

available

Zydelig 60% 65% 60% 65% 60% 65% 19% 48% 19% 48% 19% 48%

Harvoni 45% 64% 52% 66% 61% 73% 25% 60% 28% 61% 33% 68%

Lynparza 50% 46% 58% 69% 67% 73% 50% 46% 58% 69% 67% 73%

Dalvance 63% 76% 63% 76% 63% 76% 50% 71% 50% 71% 50% 71%

Invokamet 82% 87% 95% 89% 95% 89% 45% 83% 65% 87% 73% 88%

Esbriet 70% 92% 70% 92% 70% 92% 47% 79% 47% 79% 47% 79%

Xigduo

XR 65% 85% 70% 93% 70% 93% 52% 84% 56% 92% 56% 92%

Farxiga 74% 66% 85% 95% 85% 95% 71% 66% 78% 94% 78% 94%

Xtoro 67% 97% 67% 97% 67% 97% 50% 95% 50% 95% 50% 95%

Otezla 73% 71% 93% 98% 93% 98% 43% 66% 60% 92% 60% 92%

Sivextro 71% 88% 71% 88% 86% 99% 57% 79% 67% 81% 71% 90%

Zontivity 71% 99% 86% 100% 86% 100% 63% 98% 74% 99% 74% 99%

Belsomra 33% 33% 100% 100% 100% 100% 8% 27% 24% 81% 24% 81%

Cerdelga 33% 12% 100% 100% 100% 100% 25% 24% 38% 56% 38% 56%

Jublia 80% 98% 80% 98% 100% 100% 67% 93% 67% 93% 78% 95%

Movantik 50% 57% 100% 100% 100% 100% 20% 48% 45% 84% 45% 84%

Viekira

Pak 60% 83% 87% 88% 100% 100% 15% 61% 22% 65% 29% 75%

Zerbaxa 17% 5% 100% 100% 100% 100% 27% 11% 93% 99% 93% 99%

Zykadia 100% 100% 100% 100% 100% 100% 20% 81% 20% 81% 20% 81%

Quartile 1 50% 57% 67% 88% 67% 89% 20% 48% 28% 69% 33% 73%

Quartile 3 73% 92% 100% 100% 100% 100% 52% 83% 67% 93% 73% 94%

Median 65% 76% 85% 95% 86% 98% 45% 66% 50% 81% 50% 84%

Page 18 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




19







registries.

Company Rankings



FDAAA trial standards (Table 5). AbbVie (96%), Celgene (95%), Merck (93%) and Astra

Zeneca also scored at or above the industry median. Valeant scored lowest (50%).


2014†




#3 Abbvie 96%

#4 Celgene 95%

#5 Merck 93%

#6 AstraZeneca 91%

#7 Roche 90%

#8 Novartis 88%

#9 Gilead 73%

#10 Allergan 63%

#11 Valeant 50%

Median 91%




Page 19 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



20

DISCUSSION





and 80% were published, by 13 months after FDA approval. It takes about a year after FDA

approval for companies to publicize most (96%) of their trials in patients. At FDA approval 65%

of trials per drug are available and at 3 months after FDA approval, 80%. Compliance with

FDAAA requirements was high (median of 100% per drug).

The gap between transparency of results from the all-trials and trials-in-patients samples

is striking. The median proportion of trials available per drug was markedly lower in the all-trials

sample for trial registration (53% vs. 100%), reporting results or CSR summaries (24% vs. 71%),

publication (60% vs. 80%), and overall availability (68% vs. 96%). The insomnia drug,

Belsomra, is an extreme example of the difference: only 6 of 37 trials supporting the NDA were

in patients, and the percentage publicly available was 100% for trials in patients but 20% among

all trials (Table 2).

Our earlier work examined transparency levels on these measures for all trials in a


Juxtaposing the two studies’ findings shows little change in transparency levels for this standard.



median for overall availability, 65% vs. 68%. The lack of increase in the proportions of trials

registered and with results reported is surprising because our 2014 methodology newly

Page 20 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



21

incorporated more registries and included CSR synopses as satisfying the requirement for

posting results. Compliance with FDAAA, measured by the “trial completion date”

interpretation, increased from a median of 67% of covered trials per drug in 2012 to 71% in

2014.

There is disagreement about the value of disclosing information for trials in healthy


commentators [31-32] have focused on trials in patients for several reasons. Historically, later-

phase efficacy trials have been considered to have the greatest public health relevance and

salience for prescription guidelines writers and drug formulary committees. Limiting disclosure

to trials in patients captures phase 1 trials for serious diseases, like most cancers, where the

relevance of early data to patient care is high. Arguably, any important safety signals that emerge

in phase 1 trials in healthy volunteers resurface in phase 2 trials, so critical safety information

likely does reach the public. Lastly, trials in healthy volunteers are small, seldom controlled, and

therefore not generally powered to provide statistical significance.

Notwithstanding these arguments, there is value in making information about all trials




innovation and save money, particularly for small biotechnology companies, by preventing

others from travelling down known dead-end pathways or empowering them to design better

trials based on the lessons learned from previous studies.

Whether transparency analyses focus on all trials or trials in patients, there is a need for

clearer, more harmonized standards so that progress over time can be gauged and companies

Page 21 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



22


clarify which of the two interpretations of FDAAA was correct. The Final Rule is helpful in this

regard in that it newly requires the reporting of trial results for both approved and unapproved

indications.

More broadly, we believe the measures articulated in this study are useful, broadly


them with a measure for patient-level data sharing—the new frontier in clinical trial

transparency.












for the approved indication. Our analysis was limited to large companies, to drugs approved by

the FDA in 2014, and to trials included in the relevant NDAs approved in 2014. Finally, our

company rankings are not adjusted for the volume of trials conducted. Some may object that this

Page 22 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



23

disadvantages companies with a large number of trials, for whom compliance may be more

resource intensive.

CONCLUSION

Our study shows that clinical trial transparency practices vary according to the standard

used to measure them. On the drug level, about half of FDA approved drugs have publicly

disclosed results for all trials in patients that were included in our sample. On the company level,

about 18% of large companies fully disclosed all such results and complied with FDAAA

disclosure requirements. Per drug, among trials in patients, a median of 100% of trials were

registered and 96% had publicly available trial results, in some form. Among large

pharmaceutical companies, clinical trial transparency is high based on many measures, although

opportunities for improvement remain.

Momentum for greater clinical trial transparency will grow as we continue to experience

its benefits. Legal requirements in FDAAA and NIH rules push the effort forward, along with

efforts by other organizations like the Gates Foundation, World Health Organization, and

Wellcome Trust, but reaching consensus on standards and monitoring and publicizing

companies’ adherence to emerging standards are also critical. Celebrating progress—and

identifying where it is not occurring as quickly as it could—can move the field forward toward a

shared vision of transparency and what it can achieve.

Acknowledgments

The authors thank the Laura and John Arnold Foundation; pharmaceutical company

representatives who validated their datasets; and Yael Bree (Baruch student, NYU and BEI

intern), Tamara Hardoby (NYU graduate and BEI researcher), Mindy Kresch (Baruch student,

Page 23 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



24

NYU and BEI intern), Rosa Macrito (Columbia Student, BEI researcher), and Luke Sleiter

(NYU graduate, NYU Researcher, BEI researcher) for research assistance. All errors and

conclusions are those of the authors only.













Study data will be posted at bioethicsinternational.org upon publication of this article and can

also be obtained from the lead author.






Page 24 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



25

References

1. Dickersin K, Rennie D. The evolution of trial registries and their use to assess the clinical

trial enterprise. Jama. 2012 May 2;307(17):1861-4.


ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med. 2009;6(9):e1000144.











(FDAMA) of 1997.






2007. Public Law. 2007;110:85.






2015;313(8):793-4.

Page 25 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



26








2015;12(4):e1001819.










Submission. Final rule. Federal Register. 2016;81(183):64981.




N Engl J Med. 2016;375(20):1998-2004.





BMJ Open. 2015;5(11):e009758.


2013;19:261.


Cent Rep. 2013;43:11–12.



UCM435753.pdf

Page 26 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



27







List of Figures:

Figure 1 (figure)

Caption: Drugs included in transparency analyses

Page 27 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



Figure 1: Drugs Included in transparency analysis

279x361mm (300 x 300 DPI)

Page 28 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



1

APPENDIX

to


Approved Drugs and Drug Companies”

CONTENTS:

1. Appendix 1: List of Analyzed Registries and Data Sources, Including World Health


2. Appendix 2: List of Trial Characteristics Abstracted from Registries and FDA

Approval Packages

3. Appendix 3. Additional Information About the Data Collection and Analysis

Methods

4. Appendix 4. Details of Validation Process with Drug Companies

5. Appendix 5. Information about the Pharmaceutical Executive Roundtable Meeting

6. Appendix 6: List of FDA 2014 Approved Indications for Evaluated Drugs

7. Appendix 7. List of Key Trial and NDA Sponsors for Analyzed Drugs
















(ISRCTN.org) Global










Page 29 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



2


Websites Website




23. [email protected] [email protected]

24. Aggregate Analysis of



25. PubMed

26. Google Scholar

27. EMBASE



29. Actavis







34. Celgene






ot_ourtrials.asp



39. Roche http://www.roche-trials.com/searchFullText.action?drug=2

40. Clinical Study Data Request






41. Abbvie



reporting.html



44. Pfizer www.pfizer.com/research/research_clinical_trials/trial_results

Page 30 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from

http://www.clinicaltrials.gov/

https://www.clinicaltrialsregister.eu/ctr-search/search


https://www.ctti-clinicaltrials.org/aact-database



http://www.astrazenecaclinicaltrials.com/Submission/Search





http://www.genentechclinicaltrials.com/



http://www.merck.com/clinical-trials/search.html

https://www.novartisclinicaltrials.com/

http://www.roche-trials.com/searchFullText.action?drug=2

http://www.clinicalstudydatarequest.com/




http://www.janssen.com/clinical-trials/transparency

http://www.gilead.com/research/clinical-trials

http://www.pfizer.com/research/research_clinical_trials/trial_results



3

Appendix 2: List of Trial Characteristics Abstracted from Registries and FDA Approval

Packages

Characteristics: National Clinical Trial (NCT) number, title, recruitment status, whether study

results were reported and the date any results were first received, description of the treatment

(e.g. dosage and comparators), whether the trial was interventional, sponsors/collaborators,

gender enrollment, age groups, phase, enrollment numbers, funder, study type, study design,

other IDs, registration date, start date, primary completion date (date the last participant was

examined and data for the primary outcome measure collected), primary outcome measures, site

locations, and any links to CSRs.

Page 31 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



4

Appendix 3. Additional Information About the Data Collection and Analysis Methods

Data were collected by a team of 5 undergraduate and graduate-level researchers and a PhD-level

research director, all of whom were supervised by the principal investigator, Dr. Miller.

Data collectors were trained to read FDA approval packages, use ClinicalTrials.gov and other

trial registries, and search PubMed using the search methodology described in Miller et al.,

2015.1 Validated data from drugs approved in 2012 and reported in that article were used as a

training dataset. Each data collector’s accuracy was tested using a sample from the 2012 data

before that person was permitted to move on to collect data for the 2014 sample.

Data for each drug within the sample were collected by at least 2 data collectors, working

independently and blinded to one another’s work. The researchers then met to validate each

other’s work and create one master spreadsheet for each drug. If there were discrepancies in the

data, they were resolved by consensus of the research group including the principal investigators,

sometimes using additional information elicited from the company. Data were recorded,

analyzed, and summarized using Microsoft Excel v.15. A list of the data fields extracted are

listed in Appendix 2. Many of the variables came directly from ClinicalTrials.gov and

Drugs@FDA. We used the NIH data dictionary to define terms.

Data were analyzed by the 2 most experienced data collectors and the PhD-level director, with

direction from the faculty investigators. In determining whether a trial met requirements for

registration, reporting, publication, public availability, FDAAA applicability, and FDAAA

compliance, each analyst reached his/her determination independently of the others.

Discrepancies in these determinations were resolved by consensus, with support from the

principal investigators. Determination of FDAAA applicability and compliance were made

using two primary resources, the Food and Drug Administration Amendments Act of 2007

(FDAAA) and guidance provided by NIH through clinicaltrials.gov information sections on

FDAAA.

1 Miller JE, Korn D, Ross JS. Clinical trial registration, reporting, publication and FDAAA compliance: a cross-sectional analysis and ranking of new drugs approved by the FDA in 2012. BMJ Open. 2015;5(11):e009758.

Page 32 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



5


To ensure our search process did not miss or mischaracterize facts, NDA holders

(generally the manufacturers) of drugs reviewed in the study were sent a copy of their results and

the supporting dataset and asked to provide corrections. Companies were also invited to a 3-hour

meeting to discuss study methods and findings. Nine of the 11 companies participated in both the

meeting and our drug data validation process, affording data validation for 79% of drugs

reviewed. Nine additional companies joined the 3-hour meeting.

Corrections were accepted if the research team could corroborate the information with public

sources, e.g., by visiting a website to which the company directed us. The scenarios below

typify the types of feedback we received:

1) A company indicates that results were reported for a trial on ClinicalTrials.gov, where we

marked it as not having results there. We recheck ClinicalTrials.gov, confirm that the

results are there, and revise our data. The records on ClinicalTrials.gov are constantly

being updated with new information, so occasionally companies submitted information

that had not yet appeared publicly at the time of our review.

2) A company clarifies whether FDAAA applies to a particular trial. One requirement for

FDAAA coverage turns on manufacturing data, which is difficult to come by through

public sources. FDAAA says that a trial must be conducted in the US, or have a drug

manufacturer in the US for export, to be covered. In several instances, companies

informed us that this requirement was not in fact satisfied, which caused us to correct an

initial judgment that the trial was FDAAA-applicable.

3) The company provides a web link to a publication, where we had found no publications.

Matching trial characteristics to publications can be challenging, especially for phase 1

trials with no registration record, so some publications were initially missed.

After validation, the median number of trials per drug with publicly available results

increased from 90% to 96% in the trials-in-patients sample and from 60% to 68% for all

trials. FDAAA compliance increased from 51% to 59% for the “trial completion date”

interpretation and from 88% to 92% for the “approval date” interpretation.

Scores changed primarily because the validation process located additional, publicly

verifiable journal publications (n=23) and identified typographical errors in FDA approval

packages or ClinicalTrials.gov (n=17). Additionally, results for 2 more trials were found in

ClinicalTrials.gov (the registry had delayed the postings) and 1 trial was removed from the

FDAAA trials sample because the manufacturer attested that the drug was not manufactured

in the U.S.

Page 33 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



6

Appendix 5. Information about the Pharmaceutical Executive Roundtable Meeting

On August 17, 2016 and March 10, 2017, the research team hosted Pharmaceutical Executive

Roundtables. The first was held at the NYU School of Medicine offices of the Alexandria Center

for Life Science in New York City. The second meeting was held at Ernst & Young in New

York City. Both meetings had more than 30 participants. The below institutions participated in

at least one of the two meetings:

• Representatives from AbbVie, AstraZeneca, Bayer, Biogen, Bristol-Myers Squibb,

Celgene, GlaxoSmithKline, Johnson & Johnson, Lilly, Novartis, Novo Nordisk, Pfizer,

Roche, Sanofi, and Syndax;

• Bioethics International research staff and Board of Directors members;

• Academic collaborators on the Scorecard project, based at Harvard University, Stanford

University School of Law, Yale School of Medicine, and NYU School of Medicine);

• Representatives from the Laura and John Arnold Foundation, Gates Foundation, and

Helmsley Foundation, and

• Representatives from Ernst and Young.

The pharmaceutical company representatives had various job titles, such as Vice President and

Head of U.S. Medical Affairs, Director of Clinical Trial Transparency, Chief Medical Officer,

Chief Executive Officer, and Global Head of Patient Affairs and Policy.

The 2016 Roundtable offered participants the opportunity to learn about the Good Pharma

Scorecard’s history, methods, and how to comply with its standards. Additionally, participants

could candidly discuss concerns with, and provide feedback on, the study team’s methodological

approach to data collection and analysis. Moreover, participants were encouraged to discuss

challenges and barriers to registering and reporting results of clinical trials and publishing the

results of trials in medical journals. Root causes of best and inferior practices were also

explored.

Each meeting lasted approximately 3 hours and included discussion about the importance of

disclosing phase 1 trial results and interpretations of FDAAA’s legal requirements for disclosure.

The principal changes to methods in the Scorecard made after the Roundtable were as follows:

• Expanding our search to include international and corporate registries (in addition to

registration at ClinicalTrials.gov); and

• Including the clinical study report synopsis as a measure of results reporting.

Page 34 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



7


Drug

Name

Short

Indication Indication Generic Name

Belsomra Insomnia

indicated for the treatment of insomnia characterized by

difficulties with sleep onset and/or sleep maintenance

Dasabuvir Sodium;

Ombitasvir;

Paritaprevir;

Ritonavir

Cerdelga Gaucher Disease

indicated for the long-term treatment of adult patients with

Gaucher disease type 1 (GD1) who are CYP2D6 extensive

metabolizers (EMs), intermediate metabolizers (IMs), or

poor metabolizers (PMs) as detected by an FDA-cleared test

Dalbavancin

Hydrochloride


indicated for the treatment of adult patients with acute

bacterial skin and skin structure infections (ABSSSI),

caused by susceptible isolates of the following Gram-

positive microorganisms: Staphylococcus aureus (including

methicillin-susceptible and methicillin-resistant strains),


Streptococcus dysgalactiae, Streptococcus anginosus group

(including S. anginosus, S. intermedius, S. constellatus) and

Enterococcus faecalis (vancomycin susceptible strains)

Dapagliflozin

Propanediol

Esbriet

Pulmonary

Fibrosis

indicated for the treatment of idiopathic pulmonary fibrosis

(IPF) Olaparib

Farxiga Type 2 Diabetes



Harvoni Hepatitis C

indicated with or without ribavirin for the treatment of

patients with chronic hepatitis C virus (HCV) genotype 1, 4,

5, or 6 infection

Dapagliflozin

Propanediol;

Metformin

Hydrochloride

Invokamet Type 2 Diabetes


glycemic control in adults with type 2 diabetes mellitus who

are not adequately controlled on a regimen containing

metformin or canagliflozin, or in patients who are already

treated with both canagliflozin and metformin Apremilast


indicated for the topical treatment of onychomycosis of the

toenail(s) due to Trichophyton rubrum and Trichophyton

mentagrophytes Efinaconazole


indicated as monotherapy in patients with deleterious or

suspected deleterious germline BRCA mutated (as detected

by an FDA-approved test) advanced ovarian cancer who

have been treated with three or more prior lines of

chemotherapy

Ledipasvir;

Sofosbuvir


indicated for the treatment of opioid-induced constipation

(OIC) in adult patients with chronic non-cancer pain Idelalisib

Otezla

Arthritis;

Psoriasis

indicated for the treatment of adult patients with active

psoriatic arthritis; indicated for the treatment of patients

with moderate to severe plaque psoriasis who are candidates

for phototherapy or systemic therapy

Canaglifflozin;

Metformin

Hydrochloride

Page 35 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



8


indicated for the treatment of acute bacterial skin and skin

structure infections (ABSSSI) caused by susceptible isolates

of the following Gram-positive microorganisms:

Staphylococcus aureus (including methicillin-resistant

[MRSA] and methicillin-susceptible [MSSA] isolates),


Streptococcus anginosus Group (including Streptococcus

anginosus, Streptococcus intermedius, and Streptococcus


Viekira

Pak Hepatitis C

indicated for the treatment of patients with genotype 1

chronic hepatitis C virus (HCV) infection including those

with compensated cirrhosis Tedizolid Phosphate




when treatment with both dapagliflozin and metformin is

appropriate


Tazobactam Sodium


indicated for the treatment of acute otitis externa (AOE)

with or without an otowick, caused by susceptible strains of

Pseudomonas aeruginosa and Staphylococcus aureus in

patients age 1 year and older Vorapaxar Sulfate

Zerbaxa

Urinary &

Abdominal

Infections

indicated for the treatment of patients 18 years or older

with the following infections caused by designated

susceptible microorganisms: Complicated Intra-abdominal

Infections, used in combination with



Zontivity

Thrombotic

Cardiovascular

Events

indicated for the reduction of thrombotic cardiovascular

events in patients with a history of myocardial infarction

(MI) or with peripheral arterial disease (PAD).

ZONTIVITY has been shown to reduce the rate of a

combined endpoint of cardiovascular death, MI, stroke, and

urgent coronary revascularization (UCR) Ceritinib

Zydelig Leukemia

indicated, in combination with rituximab, for the treatment

of patients with relapsed chronic lymphocytic leukemia

(CLL) for whom rituximab alone would be considered

appropriate therapy due to other co-morbidities Pirfenidone

Zykadia Lung Cancer

indicated for the treatment of patients with anaplastic

lymphoma kinase (ALK)-positive metastatic non-small cell

lung cancer (NSCLC) who have progressed on or are

intolerant to crizotinib Eliglustat Tartrate

Page 36 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



9



No. Trials in NDA Source

Belsomra




Cerdelga




Dalvance






Esbriet




Marnac, not registered 4 Personal Communication with NDA holder

William Gahl, M.D.|National Human Genome Research Institute (NHGRI)|National Institutes of Health Clinical Center (CC) (Investigator Initiated 1

ClinicalTrials.gov, Personal communication with NDA holder

Investigator initiated 1 Personal Communication with NDA holder


Farxiga





AstraZeneca|Parexel|Q2 solutions|PRA Health Sciences|Covance Laboratories, Inc 1 ClinicalTrials.gov

AstraZeneca; Bristol-Myers Squibb co-sponsor 7 Personal Communication with

Page 37 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



10

NDA holder


Harvoni



National Institute of Allergy and Infectious Diseases (NIAID)|National Institutes of Health Clinical Center 1 ClinicalTrials.gov


Invokamet



Janssen Research & Development, LLC|The George Institute for Global Health, Australia 1 ClinicalTrials.gov



Johnson & Johnson Pharmaceutical Research & Development, L.L.C. 14 ClinicalTrials.gov



Jublia




Lynparza




AstraZeneca|European Network of Gynaecological Oncology Trial Groups (ENGOT)|Myriad Genetic Laboratories, Inc. 1 ClinicalTrials.gov




Movantik





Otezla

Page 38 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



11




Sivextro




Not Registered Bayer 2 Personal Communication with

NDA holder

Viekira Pak

AbbVie 65 Drugs@FDA





Xigduo XR






Xtoro




Zerbaxa




Zontivity



Merck Sharp & Dohme Corp.|Duke Clinical Research Institute 1 ClinicalTrials.gov

Merck Sharp & Dohme Corp.|The Thrombolysis in Myocardial Infarction Study (TIMI) Group 1 ClinicalTrials.gov

Merck Sharp & Dohme Corp.|The TIMI (Thrombolysis in Myocardial Infarction) Study Group|Duke Clinical Research Institute 1 ClinicalTrials.gov

Page 39 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



12


Zydelig




Zykadia



Not Registered 4 N/A † Multiple parties can be involved in a drug’s development. This table lists the NDA sponsor and individual trial

sponsors for each drug we reviewed. Trial sponsors were identified from ClinicalTrials.gov, unless stated otherwise in the table.

Page 40 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from



13

Page 41 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Novem


http://bmjopen.bm

j.com/

BM



ber 2017. Dow

nloaded from





TITLE


ABSTRACT


2-3

INTRODUCTION



4-5

METHODS


N/A




6-7


29-31




8-9


8-11; 29-31



N/A




9-10

Page 42 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 1 of 2



5


N/A

RESULTS


11, 28


11-18



12-19

Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency. N/A



DISCUSSION


20-22


22


FUNDING


23-24



Page 2 of 2

Page 43 of 43


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



Documents

When an article is published we post the peer reviewers ... · Management, Yale School of Public Health, and Center for Outcomes Research and Evaluation, Yale-New Haven Health, all