86
Beware of Registries for their Biases Hasan Yazici University of Istanbul

Beware of Registries for their Biases Hasan Yazici University of Istanbul

Embed Size (px)

Citation preview

Page 1: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Beware of Registries for their Biases

Hasan YaziciUniversity of Istanbul

Page 2: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Disclosures

• Pfizer (Turkey) – travel support & speaker’s fees

• Merck (Turkey) – travel support & speaker’s fees

Page 3: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Beware of Observational Studies based on Registries

for their Biases

Hasan YaziciUniversity of Istanbul

Page 4: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Plan • For and against observational studies• Few historical notes• A summary of important biases in

observational studies based on registries and administrative data bases

• The bias of determining cancer incidence in registries - a recent example

• Some naive arithmetic • What to do? • In brief

Page 5: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Observational StudiesAdvantages

• Real life data • Relatively cheap • Potential to observe multiple outcomes• Ability to assess cause – effect relations• Long term observations

Page 6: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Observational StudiesProblems

• Many are cross sectional or retrospective.• Selection bias including completeness of

recruitment• Confounders• Uniformity of assessment• Control groups

Page 7: Beware of Registries for their Biases Hasan Yazici University of Istanbul

RCTs vs Observational studies

• Efficacy : RCT superior

• Harm: Observational study superior

• Biomarkers: Observational study superior

JP Vandenbroucke BMJ, 2011

Page 8: Beware of Registries for their Biases Hasan Yazici University of Istanbul

RCTs vs Observational studies

• Efficacy : RCT superior ?

• Harm: Observational study superior

• Biomarkers: Observational study superior

JP Vandenbroucke BMJ, 2011

Page 9: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 10: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 11: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The DES* Drama• DES began to be used for threatened abortion (1949)• Efficacy could not be shown in a double blind, placebo controlled study at

University of Chicago (1958) • An “epidemic” of vaginal cancer among girls in Boston (1971) • Among 8 patients with vaginal cancer 7 were daughters of mothers who

had used DES during pregnancy. This contrasted with mothers of 32 healthy girls (born in the same hospital within a day or two) among whom there were no DES users.

* diethylstilbestrol

Page 12: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 13: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The DES* Drama• DES began to be used for threatened abortion (1949)• Efficacy could not be shown in a double blind, placebo controlled study at

University of Chicago (1958) • An “Epidemic” of vaginal cancer among girls in Boston (1971) • Among 8 patients with vaginal cancer 7 were daughters of mothers who

had used DES during pregnancy. This contrasted with mothers of 32 healthy girls (born in the same hospital within a day or two) among whom there were no DES users.

• FDA banned DES in pregnancy (1971). • A study (Mayo Clinic) in mid 70’s among 800 young women with a mother

who had used DES during pregnancy did not reveal any cases of vaginal cancer.

• No surprise. Since frequency of vaginal cancer: <1/1000 users. • A further case control study from NY State Tumor Registry confirmed the

association. * diethylstilbestrol

Page 14: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The Estrone Drama

• “Like a gallant knight (the author of Feminine Forever) has come to rescue his fair lady not at the time of her bloom and flowering but in her desparing years; at a time of life when the preservation and prolongation of her femaleness are so paramount…By throwing down his gauntlet, he challanges the reluctant physician to follow him in providing the hormones that may allow for a smoother transition to the menopausal years ahead. Women will be emancipated only when the shackles of hormonal deprevation are loosed.”

in Investigating Disease Patterns, Stolley&Lasky, 1995

Page 15: Beware of Registries for their Biases Hasan Yazici University of Istanbul

in Investigating Disease Patterns, Stolley&Lasky, 1995

Page 16: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 17: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 18: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 19: Beware of Registries for their Biases Hasan Yazici University of Istanbul

....I think this is rather misleading. Of the 26 lymphomas they refer to in the main text, 14 occurred within 2 months of TNF antagonist use. Thus, a more realistic comparator would be a deduced (from the annual rates) 2-month incidence of lymphoma in the general population....

H Yazici Arthritis Rheum 2003

Page 20: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Time to Neoplasia

Based on T Bongartz et al. JAMA 2006

Page 21: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The Wandering Comparison of Risk(Lymphoma)

M. Hudson & S. Suissa Arthritis Care and Res, 2010

Page 22: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The Wandering Comparison of Risk(Lymphoma)

M. Hudson & S. Suissa Arthritis Care and Res, 2010

Page 23: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 24: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The Wandering Comparison of Risk

(Infections)

Page 25: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The Wandering Comparison of Risk

(Infections)

Page 26: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Channeling Bias(in an administrative data base)

Page 27: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Channeling Bias(in an administrative data base)

Page 28: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Channeling Bias(in an administrative data base)

Page 29: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The Immortal Time Bias

• ..arises in a cohort study where an outcome can hinder, totally or partially, the realization of the exposure.

Page 30: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The Immortal Time Bias

• Hydroxychloroquine decreases cancer in SLE by 85%

• G. Ruiz Irastorza et al. Ann Rheum Dis, 2007

• Statins decrease lung cancer by 45%.

• V. Khuarana et al. Chest, 2007

• LE Levesque et al. Br Med J, 2010

Page 31: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 32: Beware of Registries for their Biases Hasan Yazici University of Istanbul

References • Eur Respir J (4)• Arch Intern Med (2)• Am J Med (2)• Am J Respir Crit Care Med (2)• Lancet (2)• JAMA (1)• Am J Respir Med (1)• J Allergy Clinical Immunol (1) • Ann Allergy Asthma Immunol (1)• Thorax (1)• Pediatrics (1)• Ann Pharmacother

(1) • Diabet Med (1)

Page 33: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 34: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 35: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Misclassified Immortal Time

• The authors study a complete registry of SLE patients between 2 time points.

• The exposed group consists of patients who had ever used hydroxychlorquine.

• The non – exposed group consists of patients who have never used hydroxychloroquine.

• The authors find significantly less cancers in the “exposed = ever used” group.

Page 36: Beware of Registries for their Biases Hasan Yazici University of Istanbul

OR for death =

Odds of death in the exposed

Odds of death in the non-exposed

Page 37: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 38: Beware of Registries for their Biases Hasan Yazici University of Istanbul

What is wrong?

• In the numerator those patients who had died due to malignancy could not have been prescribed hydroxychloroqine.

• Thus the duration of follow up time that can lead to malignancy in the numerator is actually shorter and this decreases the odds for a malignancy in the numerator making it lower than what is said.

Page 39: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The RDPR Study• We hypothesize that topical antifungals (TA) decrease mortality. • In the Registryville Drug Prescriptions Registry (RDPR) between January 1, 2009 and

ending on March 31st, 2009 we identify 1000 patients with an ever prescription for TA. • We follow all patients fom the time of prescription to December 31st 2010, for

emerging death. • We use the RMR (Registryville Mortality Registry) to confirm the deaths.• From RDPR we also randomize a 1000 sample of age, gender and practically everything

else matched individuals AND also follow them for the same outcome up to December 31st. 2010.

• Exposed group: patients with a prescription for a TA• Non- exposed group: patients with other prescriptions • Outcome: death

Page 40: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The RDPR Study

• At the end of the study we compare:

OR (exposed)/OR (non-exposed) • 30 people (30/732 pt. yrs.) in the exposed

group; • 60 people (60/950 pt. yrs.) in the non-

exposed group have dies. • TA significantly lessens mortality • OR=0.50; p= 0.009

Page 41: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Excluded Immortal Time

• We correctly exclude from the numerator the follow up of those patients before they were prescribed the exposure drug. This is the immortal time and, again, deaths can only happen after the exposure (the prescription).

Page 42: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Excluded Immortal Time

• We correctly exclude from the numerator the follow up of those patients before they were prescribed the exposure drug. This is the immortal time and, again, deaths can only happen after the exposure (the prescription).

• However this is not enough. This time period should be added to the denominator.

Page 43: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed

Page 44: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed

Entry

Page 45: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed

Calendar AgeBirth dateDisease onsetDiagnosisRegistration in a clinic Registration in a database

Entry

Page 46: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed

Calendar AgeBirth dateDisease onsetDiagnosisRegistration in a clinic Registration in a database

Entry

Exposureie prescription

Page 47: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed

Calendar AgeBirth dateDisease onsetDiagnosisRegistration in a clinic Registration in a database

Entry

Exposureie prescription Observation ends

Page 48: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed Unexposed

Calendar AgeBirth dateDisease onsetDiagnosisRegistration in a clinic Registration in a database

Entry

Exposureie prescription

Observation ends

Page 49: Beware of Registries for their Biases Hasan Yazici University of Istanbul

ExposedUnexposed

Calendar AgeBirth dateDisease onsetDiagnosisRegistration in a clinic Registration in a database

Entry

Exposureie prescription

Observation ends

Entry

Observation ends

Page 50: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed Unexposed

Calendar AgeBirth dateDisease onsetDiagnosisRegistration in a clinic Registration in a database

Entry

Exposureie prescription

Observation ends

immortal time

EntryObservation ends

Page 51: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed Unexposed

Calendar AgeBirth dateDisease onsetDiagnosisRegistration in a clinic Registration in a database

Entry

Exposureie prescription

Observation ends

immortal time

EntryObservation ends

Page 52: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed Unexposed

Calendar AgeBirth dateDisease onsetDiagnosisRegistration in a clinic Registration in a database

Entry

Exposureie prescription

Observation ends

immortal time

Entry

Observation should end

Page 53: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Exposed Unexposed

Calendar AgeBirth dateDisease onsetDiagnosisRegistration in a clinic Registration in a database

Entry

Exposureie prescription

Observation ends

immortal time

Entry

Observation should end

outcome free!

Page 54: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Remedy (s)

• There might be none especially when we are unclear about the entry, hence the observation time.

• Always include the “immortal time” to the duration of follow up in the non-exposed.

Page 55: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The RDPR Study

• At the end of the study we compare:

OR (exposed)/OR (non-exposed)

• 30 people (30/732 pt. yrs.) in the exposed

group; • 60 people (60/950 pt. yrs.) in the non-

exposed group have died. • TA significantly lessens mortality • OR=0.50; p= 0.009

Page 56: Beware of Registries for their Biases Hasan Yazici University of Istanbul

The RDPR Study

• At the end of the study we compare:

OR (exposed)/OR (non-exposed)

• 30 people (30/732 pt. yrs.) in the exposed

group; • 60 people (60/1200 pt. yrs.) in the non-

exposed group have died. • TA does nothing to the mortality, OR=0.98

Page 57: Beware of Registries for their Biases Hasan Yazici University of Istanbul

S Suissa Pharmacoepidemiol Drug Saf, 2007

Page 58: Beware of Registries for their Biases Hasan Yazici University of Istanbul

S Suissa Pharmacoepidemiol Drug Saf, 2007

… which includes adding the immortal time to the none- exposed arms in either type of immortal time bias - HY

Page 59: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 60: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 61: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 62: Beware of Registries for their Biases Hasan Yazici University of Istanbul

H Yazici et al Arthritis Rheum , 2011

Page 63: Beware of Registries for their Biases Hasan Yazici University of Istanbul

A Possible Source of Error in the Method of Cancer Risk Estimation in Patients with Rheumatoid Arthritis •We surveyed the PubMed between years 2001-

2011•Search terms “registry” “cancer” “rheumatoid

arthritis”•Papers that reported comparative incidence in at

least 2 time points were included•Change over time was assessed

H Yazici et al. ACR, 2011

Page 64: Beware of Registries for their Biases Hasan Yazici University of Istanbul

A Possible Source of Error in the Method of Cancer Risk Estimation

in Patients with Rheumatoid Arthritis

•1274 articles•36 articles retrieved•6 reported incidence comparison at

multiple timepoints•The last SIR was less than the initial by 40

to 99% H Yazici et al. ACR, 2011

Page 65: Beware of Registries for their Biases Hasan Yazici University of Istanbul

A Possible Source of Error in the Method of Cancer Risk Estimation

in Patients with Rheumatoid Arthritis

H Yazici et al. ACR, 2011

* initial year excluded

Page 66: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Some naive arithmetic

Page 67: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

Given: • There are no biologic associations between RA and cancer • Incidence of RA: 1/1000/year• Incidence of cancer: 5/1000/year• These incidences remain same over time.• 1000 new cases of RA enter the registry every year. • 2/5 of all cancer patients die at the end 1 year.• Of the 2/5 who die 1 dies before and the other dies after developing RA.• Another 1/10 of the cancer patients die, in a logarithmically decreasing

fashion, at the end of 5 years. * • RA does not kill if not associated with cancer.• All other causes of death are ignored.

* This group is ignored for the purposes of this presentation. Its inclusion would only support our hypothesis.

Page 68: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000

Page 69: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000

Page 70: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000

1/5 cancer patients die beforehaving the chance to develop RA.

Page 71: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000• O/E: 0.80

1/5 cancer patients die beforehaving the chance to develop RA.

Page 72: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000• O/E: 0.80 • 2. year:• Expected: 15/2000

1/5 cancer patients die beforehaving the chance to develop RA.

Page 73: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000• O/E: 0.80 • 2. year:• Expected: 15/2000

1/5 cancer patients die beforehaving the chance to develop RA.

5 cancer patients are from the 1.year &10 from the 2. year, from the total of 2000 patients

Page 74: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000• O/E: 0.80 • 2. year:• Expected: 15/2000• Observed: 11/2000

1/5 cancer patients die beforehaving the chance to develop RA.

5 cancer patients are from the 1.year &10 from the total of 2000 patients

Page 75: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000• O/E: 0.80 • 2. year:• Expected: 15/2000• Observed: 11/2000

1/5 cancer patients die beforehaving the chance to develop RA.

5 cancer patients are from the 1.year &10 from the total of 2000 patients

3 cancer patients cases are from the 1. year. We expect 10 incident cancer cases in the 2. year. 2/10 of these die before they get the chance to develop RA. The remaining 8 new cases plus the 3 from the 1. year make up 11 prevalent cases at the end of 2 years.

Page 76: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000• O/E: 0.80 • 2. year:• Expected: 15/2000• Observed: 11/2000• O/E: 0.73

1/5 cancer patients die beforehaving the chance to develop RA.

5 prevalent cases are from the 1.year &10 from the total of 2000 patients

3 cancer patients cases are from the 1. year. We expect 10 incident cancer cases in the 2. year. 2/10 of these die before they get the chance to develop RA. The remaining 8 new cases plus the 3 from the 1. year make up 11 prevalent cases at the end of 2 years.

Page 77: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000• O/E: 0.80 • 2. year:• Expected: 15/2000• Observed: 11/2000• O/E: 0.73• 3. year:• Expected: 30/2000

1/5 cancer patients die beforehaving the chance to develop RA.

5 cancer patients cases are from the 1.year &10 from the total of 2000 patients

3 cancer patients cases are from the 1. year. We expect 10 incident cancer cases in the 2. year. 2/10 of these die before they get the chance to develop RA. The remaining 8 new cases plus the 3 from the 1. year make up 11 prevalent cases at the end of 2 years.

Page 78: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000• O/E: 0.80 • 2. year:• Expected: 15/2000• Observed: 11/2000• O/E: 0.73• 3. year:• Expected: 30/2000• Observed: 15/3000

1/5 cancer patients die beforehaving the chance to develop RA.

5 cancer patients are from the 1.year &10 from the total of 2000 patients

3 cancer patients cases are from the 1. year. We expect 10 incident cancer cases in the 2. year. 2/10 of these die before they get the chance to develop RA. The remaining 8 new cases plus the 3 from the 1. year make up 11 prevalent cases at the end of 2 years.

Page 79: Beware of Registries for their Biases Hasan Yazici University of Istanbul

Cumulative Cancer Prevalence in a Hypothetical RA Registry of Incident Cases of RA

• 1. year:• Expected: 5/1000• Observed: 4/1000• O/E: 0.80 • 2. year:• Expected: 15/2000• Observed: 11/2000• O/E: 0.73• 3. year:• Expected: 30/2000• Observed: 15/3000• O/E: 0.50

1/5 cancer patients die beforehaving the chance to develop RA.

5 cancer patients are from the 1.year &10 from the total of 2000 patients

3 cancer patients are from the 1. year. We expect 10 incident cancer cases in the 2. year. 2/10 of these die before they get the chance to develop RA. The remaining 8 new cases plus the 3 from the 1. year make up 11 prevalent cases at the end of 2 years.

Page 80: Beware of Registries for their Biases Hasan Yazici University of Istanbul

What to do?

• Be careful when comparing the frequency of a condition in a registry with the frequency of the same condition in the population from which that registry comes from; because of the selection bias.

• Give more importance to comparing like with like- attention to observation times, propensity scoring etc.

• Also give importance to comparing like with alike as we try to falcify our hypothesis – control groups with other diseases/conditions

• Proper case control studies starting from the hypothesized outcome – i.e looking for RA among neoplasms

Page 81: Beware of Registries for their Biases Hasan Yazici University of Istanbul

CR Meier et al. JAMA, 2000

Page 82: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 83: Beware of Registries for their Biases Hasan Yazici University of Istanbul

In brief

• Caveat lector !

Page 84: Beware of Registries for their Biases Hasan Yazici University of Istanbul
Page 85: Beware of Registries for their Biases Hasan Yazici University of Istanbul

In brief

• Caveat lector !

• “Aye, there is the rub !”

Page 86: Beware of Registries for their Biases Hasan Yazici University of Istanbul

In brief

• Caveat lector !

• “Aye, there is the rub!”

• Do not go for a grand slam with 10 points at hand !