Responsibly turning epidemiological findings into

public health recommendations

19th Research Postgraduate Symposium

University of Hong Kong Li Ka Shing School of Medicine

10 December 2014

James C. Coyne, Ph.D.Department of Health Psychology

University of Groningen, University Medical Center Groningen (UMCG), Groningen, the

Netherlandsjcoynester@gmail.com

Academics are under enormous pressure to publish in the highest impact journals

possible.

Yet, doing excellent science is not enough to ensure that

manuscripts are even sent out for review.

Editors of high impact journals seek papers that will be newsworthy and garner media attention. Why?

Journal impact factor (JIF) is calculated from the number of citations in the first two years after publication.

Immediate media attention is a well-established means of getting early citations.

High impact journals require authors play to the media.

The real work begins after publication. Use social media, listserves, and email to those who might act on your research.

--Richard Smith, Former Editor of BMJ, author of What's Wrong With Medical Journals]

Journalists/Media

Do not work for medical and scientific researchers.

Interested in stories that will attract attention.

Not primarily concerned with transmission of accurate information or effects on public health and well-being.

Pressures on Journalists

“I’m in competition with literally hundreds of stories every day, political and economic stories of compelling interest…we have to almost overstate, we have to come as close as we came within the boundaries of truth to dramatic, compelling statement. A weak statement will go no place.”

—-Journalist interviewed for JA Winsten, Science and media: the boundaries of truth.

Newspapers were more likely to cover observational studies and less likely to cover RCTs than high impact journals. Additionally, when the media does cover observational studies, they select articles of inferior quality. Newspapers preferentially cover medical research with weaker methodology.

Results of bad practices

Medical and public health effects include

Overdiagnosis.Overtreatment.Overreaction to emerging health issues.

Neglect of less dramatic measures that would more reliably improve medical and public health outcomes.

“It can be proven that most claimed research findings are false”

Ioannidis, JPA. (2005). Why most published research findings are false. PLOS Medicine 2: 696-701.

Ioannidis, JPA. (2005). Contradicted and initially stronger effects in highly cited clinical research. JAMA 294: 218-228.

Young, NS., Ioannidis, JPA. et al. (2008). Why Current Publication Practices May Distort Science. PLOS Medicine 5: 1418-1422.

I wear two hats

Teaching, coaching Groningen faculty to get their papers into high impact journals and grants funded.

Challenging exaggerated findings in the literature.

Of necessity, researchers must self-promote and market their manuscripts and grants to international journals and granting agencies.

Must craft messages emphasizing their work’s innovation and significance.

Goes against Dutch culture’s encouragement of modesty and not standing out.

Resist the temptation to hype and spin.

Recognize and resist pressures from institutions and journals to exaggerate importance of your findings.

Journalists are important allies, but are much more interested in getting attention for themselves than in accurately portraying your work.

Hype (defined)

“To publicize or promote, especially by extravagant, inflated, misleading, and deceitful claims.”

A story with a good outcome

Correcting the American response to Ebola

Controlling Ebola in the US requires Thorough isolation of symptomatic victims. Rigorous attention to personal protective equipment.Protocols for health care workers.

Containing Ebola in Africa requires international healthcare workers Providing treatment, organizing treatment centers, safe burials, and community mobilization.Returning to their own countries, whether infected or not.

Overreaction

Corrective Actions

Avoided sensationalism. Recognized fear a greater threat to public health

and well-being than Ebola itself. Placed actual small number of deaths and low

risk in US in perspective. Constant, honest, humble risk communication. Avoided being overconfident, accepted

inevitability of at least a small number of deaths.

A story not yet with a good outcome

BMJ article spins epidemiological to fit direction of

government policy to reduce obesity.

“Exposure to takeaway food outlets…was associated with marginally higher consumption of takeaway food, greater body mass index, and greater odds of obesity.

“Government strategies to promote healthier diets through planning restrictions for takeaway food could be most effective if focused around the workplace.”

Hype and nontransparency

Abstract exaggerated findings that were presented in results.

Results did not show ‘fast food outlets’ more associated with obesity than other factors.

Author did not provide table of basic associations.

Poor and selective control of confounds. Discussion ignored other studies showing

restricting fast food outlets decreases obesity.

Later authors of BMJ article had described how they could create illusory ‘obesogenic* realities’ with arbitrary methodological and statistical decisions and selective reporting.

*appearance that particular environmental factors cause obesity.

What was ignored

Increasing bike paths, parks for exercise as a an alternative.

Fast food outlets are in areas where residents cannot easily shop, store or prepare own meals or have time to do so.

Fast food outlets may be only source of vegetables in food deserts.

Need to ask….

Would paper have been accepted by BMJ if the findings had not been distorted or if basic statistics had been available to reviewers?

What you can do

Become informed about reporting standards and standards for talking to media.

Educate your trainees and readers.

Take part in post-publication review and challenge bad reporting practices.

Get involved in PubMed Commons.

PubMed Commons is a system that enables researchers to share their opinions about scientific publications.

Researchers can comment on any publication indexed by PubMed, and read the comments of others.

Join the Pubmed Commons Revolution

No longer will a few people grant an irrevocable judgment of “must stand

because it is peer reviewed.”

You can take post-publication peer review out of the hands of editors.

One of many free resources

http://tinyurl.com/pquj95k

http://www.testingtreatments.org/tt-main-text/the-book/download-the-book/

Summary

Researchers must

Emphasize significance and relevance of their work to biomedical and public health problems to get published.

Promote their work in press releases and social media.

Face pressures to exaggerate and distort their findings.

Summary (continued)

But researchers should practice responsible reporting.

Choose appropriate language (association, not causality).Recognize arbitrariness and limits of statistical controls to establish causality.Be transparent in reporting basic associations and rationale for multivariate analyses.Honestly, humbly remind audience of the limits of their findings.

Thank you

Follow me on Twitter

@CoyneoftheRealm

Blogging at PLOS Mind the Brain