Statistical presentation in international scientific publications 2. A symptomatic review Malcolm Campbell Lecturer in Statistics, School of Nursing, Midwifery

Statistical presentation in international scientific publications 2. A symptomatic review

Malcolm CampbellLecturer in Statistics, School of Nursing, Midwifery &

Social Work, The University of Manchester

Statistical Editor, Health & Social Care in the Community

26 March 2008 Statistical presentation - 2. A symptomatic review 2

2. A symptomatic review Contents

• 2.1 Statistical reporting

• 2.2 General (but not definitive) advice

• 2.3 The structure of a paper

• 2.4 Statistical errors in submitted papers

• 2.5 What should we do?


2.1 Statistical reportingPresenting statistics in scientific papers

• A scientific research paper should contain sufficient information to allow readers to

1. assess the authors’ aims, material, methods, findings and interpretation

2. decide whether their conclusions are justified

3. apply the authors’ conclusions to their own practice

4. repeat the study in their own setting, if required

• Quantitative research papers show (or hide?) information statistically– numbers, estimates, statistics, test results, p-values…

– but the reader still needs to be able to do 1 to 4


Statistical reporting in research papersLies, damned lies and submissions to journals

• Papers submitted to healthcare journals have a generally poor standard of statistical reporting– many papers do not have the right structure

– occasionally the study design is at fault

– most authors choose the correct methods to analyse their data but fall down on the description of the methods and the presentation of the results

– others present sequences of p-values without explanation, inviting the reader to “guess the test”

– many brush non-significant p-values out of the way

– tables of findings are often inconsistent, too complicated or depend on the text for their explanation


2.2 General (but not definitive) adviceConventions, recommendations and suggestions

• These presentations give general advice on how to present statistical findings in papers– there are common conventions, style and sense

– some recommendations on good practice

• There are few absolute rules– different journals (and editors) have different styles

• always read the journal’s “Instructions to Authors”

– it may be necessary to bend the rules occasionally

– accept that research in healthcare may not be perfect!

• Important bottom line: readability


The Good, the Bad and the UglyExamples of good, bad and ugly practice

• We will look at examples of all three in published papers

– “The Good” are ideals

– “The Bad” may be slightly wrong

– but “The Ugly” should have been corrected before printing


2.3 The structure of a paperQuantitative papers are usually laid out as follows

• Title: What was the study about, in a line?

• Authors: Who “performed” the study?

• Abstract: What was the study about, in a paragraph?

• Introduction: Why was the study carried out?

• Methods: How was the study carried out?

• Results: What did the study find?

• Discussion: What do the findings mean?

• Acknowledgements: Who should be thanked?

• References: What essential references were used?


IMRaD for healthcare papersThe standard structure for research papers (ICJME, 2005) • Many (but not all) healthcare journals adopt this

structure for main text of research papers– Introduction, Methods, Results and Discussion

– these sections may be broken down into subsections

• It reflects the process of scientific investigation– why the study was done, how it was done, what was

found, what it means

• though not always the order in which things were done!

– use IMRaD even when planning for a journal that uses other sections to get overall structure right

• should be easy to make minor changes to the structure before submission


Structures in selected journalsSections used by healthcare journals (* = structured Abstract)

BMJ HSCC Midwifery J Adv. Nursing

J Clinical Nursing

Nurse Educ Today

Abstract* Abstract Abstract* Abstract* Abstract* Abstract

Introduction Introduction Introduction Introduction/

Background

Introduction/

Background

Introduction/

Lit review

Lit review Background

Methods Methods Methods The study Methods Methods

Results Results Findings Results Results Results

Discussion Discussion Discussion Discussion Discussion Discussion

Conclusions Conclusions Conclusions

Acknowledg Acknowledg Acknowledg Acknowledg

Refs (Vanc) Refs (Harv) Refs (Harv) Refs (Harv) Refs (Harv) Refs (Harv)


Common faults with authors’ structureHow not to lay out a quantitative paper (eg Hall et al, 1998)

• The Introduction should not include any findings or conclusions

• The Methods should not include any findings– information not known before the data were collected

should ideally be in the Results

• The Results should not repeat detailed findings already summarised in tables

• The Discussion should not introduce findings not already in the Results

• The References should not be in the style of another journal (ie not the BMJ!)


Methods or Results?What goes where (eg ICJME, 2007)

• International Committee of Medical Journal Editors (2007) at http://www.icmje.org/ :– “The Methods section should include only information that

was available at the time the plan or protocol for the study was written; all information obtained during the conduct of the study belongs in the Results section. ”

• I would expect the Results to start with – number of participants, participation rate, details of

characteristics of the participants…

• Sometimes better for readability to include some “results” in the Methods

• Not all papers or journals follow this!


The BadLack of structure confusing

• Papanikolaou et al (2003)– Pressure ulcer risk assessment: application of logistic

analysis, J Advanced Nursing 44(2), 128-136

• title misleading – they meant logistic regression

• does not follow IMRaD structure

– characteristics of participants in Methods section

– analysis of variables used for logistic regression with percentage breakdown per group also in Methods section

– Results starts with the main analysis, logistic regression

• [also no sample size calculation (only 25 patients with pressure ulcers v 473 without)

• inconsistent percentages and p-values of “0.00”]


2.4 Statistical errors in submitted papers100 consecutive non-RCT peer reviews+ (Altman, 1998)

Yes Unclear No

Objective clear? 83 6 11

Appropriate study design? 72 25 3

Source of subjects? * 83 6 10

Sample size calculation? * 0 0 63

Satisfactory response rate? * 49 23 2

Methods described adequately? 47 0 53

Statistical analyses appropriate? 41 37 22

Statistical presentation satisfactory? 14 0 86

Confidence intervals given? 51 0 41

(CIs given inappropriately) (+8)

Conclusions justified? 40 49 11

Paper statistically acceptable? 4 0 96

If not, could it become acceptable? 89 6 1

* not always applicable + for BMJ 1991-1993


2.5 What should we do? Never fear - help is at hand! – Three sources of advice

1. ICMJE uniform requirements for manuscripts– International Committee of Medical Journal Editors

(2007) at http://www.icmje.org/

• eg use double spacing, pages should be numbered, start each section/table/figure on a new page

2. Journal’s instructions to authors– please read them!

3. Published reporting guidelines for different study designs– eg CONSORT, STARD, TREND, STROBE…


Guidelines for statistical reportingStandardised reporting guidelines

• There are published guidelines for reporting certain types of study designs– cover the type of information to be reported and in what

order/sections of the paper

– but not the finer details of numerical reporting

• refer to journals’ guidelines to authors

• and more later!

• Go to the EQUATOR Network for more details– a new initiative, Enhancing the QUAlity and

Transparency Of health Research Network

– http://www.equator-network.org/


The CONSORT statement (2001)Checklist/flow diagram for reporting randomised trials

• CONsolidated Standards Of Reporting Trials – for reporting (parallel) randomised controlled trials

– the statement – eg Moher et al (2001)

– explanation and elaboration – eg Altman et al (2001)

– http://www.consort-statement.org/

• download both documents and templates of checklist and flow diagram


The STARD statement (2003) Guidelines for studies of diagnostic accuracy

• STAndards for Reporting of Diagnostic Accuracy – for studies comparing outcomes from new diagnostic

test against reference standard

– the statement – Bossuyt et al (2003a)

• CONSORT adapted for studies of diagnostic accuracy

– explanation and elaboration – Bossuyt et al (2003b)

– http://www.stard-statement.org/

• download both documents and templates of checklist and flow diagram


The TREND statement (2004)Checklist for reporting non-randomised trials

• Transparent Reporting of Evaluations with Non-randomized Designs – for reporting comparative trials without randomisation

– the statement – Des Jarlais et al (2004)

• CONSORT + theoretical basis of intervention, tests of baseline equivalence and corresponding adjustments of main analyses

– http://www.trend-statement.org/

• download the statement


The STROBE statement (2007)Checklist/flow diagram for reporting observational studies

• STrengthening the Reporting of OBservational studies in Epidemiology– reporting of cohort studies, case-control studies and

cross-sectional studies (surveys)

– the statement – eg von Elm et al (2007)• CONSORT + sources of potential bias + outcomes

related to type of design + potential confounders and corresponding adjustments of main analyses

– explanation and elaboration – eg Vandenbroucke et al (2007)

– http://www.strobe-statement.org/• download both documents and templates of checklist

and flow diagram


Other guidelinesFor other kinds of studies

• Extensions to CONSORT for other kinds of randomised trials – cluster RCTs – Campbell et al (2004) [no relation!]

– non-inferiority/equivalence trials – Piaggio et al (2006)

– herbal medicinal interventions – Gagnier et al (2006)

– (coming) non-pharmaceutical interventions

– http://www.consort-statement.org/?o=1044

• download all documents

Documents

Statistical presentation in international scientific publications 2. A symptomatic review Malcolm Campbell Lecturer in Statistics, School of Nursing, Midwifery