What to choose: internal reporting guidelines or external reporting guidelines? A step closer to answering this dilemma May 2012Class 8 Introdução à Medicina

What to choose: internal reporting guidelines or external reporting guidelines?

A step closer to answering this dilemma

May 2012 Class 8

Introdução à Medicina II

Class 8 2

SUMMARY

• Introduction

• Justification

• Research question

• Aims

• Methods

• Results

• Discussion

• Conclusions

• Acknowledges

• References

What to choose: internal reporting guidelines or external reporting guidelines?A step closer to answering this dilemma

May 2012

Class 8 3

INTRODUCTION

May 2012

SCIENTIFIC KNOWLED

GE DIVULGATI

ON

Scientific journals

Books

Conferences

Governmental reports

Technical reports

Flyers

Online versions


Class 8 4

INTRODUCTION

May 2012

Scientific articles Scientific journals

First scientific journal First medical journal

Philosophical Transactions of the Royal Society1665

New England Journal of Medicine1812

published in


Class 8 5

INTRODUCTION

May 2012

Scientific articlesContain…

• Motivation of the author• Experimental design and execution• Meaning of the results• Conclusions and recommendations

Require…• Clarity• Concision• Organization• Objectivity

• Better comprehension• More effective way of using its results

To accomplish


Class 8 6

INTRODUCTION

May 2012

Impact of scientific journals

Impact Factor (IF) Average number of times an article published in a given journal was cited in the last two years

Comparison between different scientific areas can induce biased conclusions

Journal Citation Report ISI Web of Knowledge


Impact ≠ Quality

Class 8 7

INTRODUCTION

May 2012

Impact of scientific articles

Number of citations

Possible biases:• Size of the articles• IF of the journal in which they are published

ISI Web of Knowledge

Presented in the end of the academic work, bibliographies or specific indexes


Impact ≠ Quality

Class 8 8

INTRODUCTION

May 2012

GuidelinesInformation for the authors on how they must write their manuscript

Scientific Journals

No presentation of guidelines

Presentation of guidelines

(recommendation or obligation)

Editing Guidelines

Reporting Guidelines

External

Uniform RequirementsSTREGA SQUIRE STROBE

CONSORT STARD PRISMAGNOSISTRENDORIONCOREQ

QUORUM REMARKMOOSE

Internal


Class 8 9

INTRODUCTION

May 2012

GuidelinesUniform Requirementsfor Manuscripts Submitted to Biomedical Journals

General Reporting Guidelines

Vancouver Group(Vancouver, British Columbia, 1978)

International Committee of Medical Journal Editors (ICMJE)

First statements: 1979Last update: 2010


INTRODUCTION

May 2012 10Class 8

GuidelinesRefering to a particular type of study and to other specific aspects of science publication


GNOSIS Guidelines for neuro-oncology: Standards for investigational studies

CONSORT Consolidated Standards of Reporting Trials

COREQ Consolidated criteria for reporting qualitative research

MOOSE Meta-analysis of Observational Studies in Epidemiology

PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses

QUORUM Quality of Reporting of Meta-analyses of Randomized Controlled Trials

REMARK REporting recommendations for tumour MARKer prognostic studies

SQUIRE Standards for Quality Improvement Reporting Excellence

STARD Standards for the Reporting of Diagnostic Accuracy Studies

STREGA Strengthening the Reporting of Genetic Association studies

STROBE Strengthening the Reporting of Observational studies in Epidemiology

TREND Transparent Reporting of Evaluations with Nonrandomized Designs

JUSTIFICATION

December 2011 11Class 8

Scientific articles - Relevance for doctors:


Medical care is improved by new discoveries and studies that promote a better life perspective for patients.

Breakthroughs are published in scientific articles.

In order to achieve a better treatment for patients and a good management of resources it is imperative to divulgate the knowledge

in the clearest way possible Scientific articles highly effective.May 2012 Class 8 11

Class 8 12

Depends on…• Sientific content• Way that it is written

May 2012

JUSTIFICATION

The quality of a scientific articleInfluences…

• Interest of scientific community• Diffusion and applicability of the results

PROBLEM:It is not known which type of reporting guidelines leads to a

better diffusion of the scientific breakthroughs.


Highest impact of the articles published

Class 8 13

RESEARCH QUESTION

May 2012

Is there any association between the use of internal reporting guidelines or/and external reporting guidelines and the impact of scientific articles published in Journals of General and Internal

Medicine?


Class 8 14

AIMS

May 2012

Central aim:Evaluate the association between the use of internal or external reporting guidelines and the impact of scientific articles published in General and Internal Medicine Journals.

Specific aims:

• Determine the type of reporting guidelines recommended by each selected scientific journal of General and Internal Medicine.

• Measure the impact of different scientific papers published in 2008 in journals with reporting guidelines.

• Compare the impact of scientific papers published in journals that present internal reporting guidelines to the impact of the articles published in journals which recommend external reporting guidelines.

• Compare the advantages of the use of external reporting guidelines over the use of internal reporting guidelines.


Class 8 15

METHODS

May 2012


Type of study:

Observational studyTransversal studyAnalytic study

Class 8 16

METHODS

May 2012


Tasks Class 8 Class 21

Colect journals of General and Internal Medicine X X

Apply exclusion/inclusion criteria X (distribution)

X (collection)

Colect Impact Factors from 2008, 2009 and 2010 X (distribution)

X (collection)

Analyse the reporting guidelines X (distribution)

X (collection)

Colect articles from 2008 and their number of citations X

Analyse journals by group X

Analyse 6 articles from 2008 for each group X

Calculate impact of articles from 2008 and make statistical analyses X

Class 8 17May 2012

Search in ISI Web of Knowledge for journals of General and Internal Medicine.1

http:// www.isiknowledge.com

METHODS


Class 8 18May 2012

From all the journals found, distribute its primary analysis by the students of the two classes to apply some criteria and collect their Impact Factor of 2008, 2009 and 2010.2

Inclusion Criteria • Journals which have an available web site.

Exclusion Criteria • Websites not written in Portuguese, English or Spanish.

METHODSWhat to choose: internal reporting guidelines or external reporting guidelines?A step closer to answering this dilemma

May 2012 19Class 8

Compare the analysis of the journals made by the two reviewers (and third reviewer, when necessary) and select the journals for the study.

3

METHODS


• Some statistic analysis using SPSS was done to clarify the motives of exclusion of some journals and to verify how many journals were included in the study.

Class 8 20May 2012

From the included journals, select those with reporting guidelines and verify its type.4

Go to the website of the selected journalsRead the section of the instructions for authorsVerify if any guidelines are presented and analyse its typeFill in a database (SPSS) with the information colected 3 reviewersSelect journals that present some type of reporting guidelines

METHODS


Class 8 21May 2012

Search for articles of the selected journals which were published in 2008.5

This information was collected on ISI Web of Knowledge

METHODS


Class 8 22May 2012

Export information about the selected articles to SPSS.6

Title of the articleAuthorsNumber of citationsJournal in which it

was published

METHODS


ISI

NotebookOpenOffice Calc

Microsoft Excel

SPSS

Class 8 23May 2012

Calculate the impact for each article.7

2)2010()2009( IFIF

CNImpact of each article

CN – Citation NumberIF – Impact Factor of the journal

METHODS


May 2012 24Class 8

Verify if there is an association between the impact of the articles and the type of reporting guidelines used by the journals.8

METHODS


Separate articles into 3 groups:

Articles published in journals using internal reporting guidelines Articles published in journals using external reporting guidelines

(specifying what type) Articles published in journals using both types of reporting guidelines

Apply some statistc analysis to answer the question of our work, starting for calculating the median impact of the articles for each group.

RESULTS

May 201225


ISI Web of Knownledge Search – 153 journals

Appliance of inclusion and exclusion criteria

133

Analysis of the instructions for authors

Included – 12347 – internal reporting guidelines19 – external reporting guidelines57 – both types of reporting guidelines

Exluded - 20• 16 not available website• 4 unreadable language

Exluded – 10• No presentation of

reporting guidelines

57 – Uniform Requirements27 – CONSORT15 – STARD14 – STROBE12 – PRISMA38 – Other type

RESULTS

May 2012 26Class 8


Advantages and disadvantages of internal and external reporting guidelines found during the analysis of the instructions for authors:

Internal reporting guidelines External reporting guidelines

Possibility of highlighting specific points that the journal wants the author to focus on.

Normally presented on the form of checklists (except Uniform Requirements):• Less exhausting• Easier to interpretateHigh level of detail of Uniform Requirements

Necessity to read with a high level of attention due to the specific details that change from journal to journal.

Loss of individuality of the journal.

+

-

RESULTS

May 2012 27Class 8


IF: median (interquartile range)

2008 2009 2010

Website

Not available (n= 12) 0,8 (0,5) 0,9 (0,7) 0,9 (0,6)

Available (n= 90) 1,7 (1,8) 1,5 (1,8) 1,6 (1,9)

English language (n= 88) 1,7 (1,8) 1,5 (1,8) 1,6 (2,0)

Not English language (n=12) 0,7 (-) 0,8 (-) 1,0 (-)

Reporting guidelines

Internal (n= 32) 1,5 (1,9) 1,4 (1,6) 1,4 (2,2)

External (n= 12) 1,2 (2,3) 1,4 (1,9) 1,2 (1,9)

Both (n= 40) 1,9 (2,5) 2,0 (2,9) 2,0 (2,5)

None (n=6) 1,5 (2,0) 1,6 (1,9) 1,6 (2,1)

Table 1: Median and interquartile range of the impact factors of 2008, 2009 and 2010 for many categories concerned to the website and the type of reporting guidelines presented by the journal (missing = 13).

Class 8 28

RESULTS

May 2012


Internal reporting guidelines

External reporting guidelines

Both types of guidelines

Articles collected

10132

6796

15367

31695

Graphic 1: Number of articles collected in the total and subdivided by different categories: presentation , by the journal in which the articles were published, of external reporting guidelines, internal reporting guidelines or both types of reporting guidelines.

Class 8 29

RESULTS

May 2012


Graphic 2: Percentage of articles collected which presented an impact equal to zero or different than zero. 3% of the articles did not present impact, because one or more of the values needed to calculated it were missing.

52%

3%

45%

Impact of the articles

0Missing> 0

Class 8 30

RESULTS

May 2012


Graphic 2: Percentage of articles from journals that use internal reporting guidelines, external reporting guidelines or both types of reporting guidelines, within the group of articles with an impact equal to zero..

21%

31%

48%

Impact of the articles equal to zero

InternalExternalBoth

Because of the values found, the percentage of articles within groups with an impact equal to zero was calculated:

Class 8 31

RESULTS

May 2012


Table 2: Median and interquartile range of the impact of the articles selected subdivided in groups by the type of reporting guidelines presented and including articles with a zer impact.


Internal External Both

Impact of the articles: median(interquartile range)

0,00 (1,93) 0,00 (1,19) 0,07 (1,74)

Including the articles with zero impact:

Excluding the articles with zero impact:


Internal External Both

Impact of the articles: median(interquartile range)

2,22 (3,67) 3,19 (1,46) 1,43 (3,28)

Table 3: Median and interquartile range of the impact of the articles selected subdivided in groups by the type of reporting guidelines presented and excluding the articles with a zero impact.

DISCUSSION

May 2012 32Class 8


Median in journals with available website

Median in journals with no available website>

Median in journals with both types of


Median in journals with no


Comparison between medians of the impact factors for each year

Median in journals with

external reporting guidelines

Median in journals with

internal reporting guidelines

> > >

If the website of a journal is not available, it will be more difficult to access its contents. So, as expected, the impact factor of the journals with an available website is higher.

Journals with both types of reporting guidelines assume an identity, but they also follow a pattern, which may be the reason why their impact factor is the higher. Journals which only follow external reporting guidelines may loose creativity and, therefore, the interest aroused in the scientific community is lower.

DISCUSSION

May 2012 33Class 8


The majority of the articles has an impact factor equal to zero.

Reasons why this may occur:

The current popularity of the particular topics referred to in the articles is very low.The topics covered by the articles are too restricted.Incorrect use of references: not mentioning an article when it was used in the study.Among others.

DISCUSSION

May 2012 34Class 8


Majority of articles with an impact equal to zero uses

both types of reporting guidelines (48%).

Percentage of articles with an impact equal to zero that use internal reporting guidelines

(21%)

Most articles come from journals using both types of reporting guidelines, so it is expected that if there is no difference between the use of any group of reporting guidelines, the percentage of articles with a citation number equal to zero would be higher in this group, as verified.

Percentage of articles with an impact equal to zero that use external reporting guidelines

(31%)<

On the other hand:

On one hand:

Why? Note that there are more articles from journals using internal reporting guidelines rather than external ones.

DISCUSSION

May 2012 35Class 8


Internal reporting guidelines x External reporting guidelines1• The median of the impact of all the articles of this two categories is the same and it is equal

to zero. So, we can not conclude if there are difference in choosing between one type or another.

• But: The percentage of articles with zero citations is higher in journals that recommend the use of external reporting guidelines

Both types x The other 2 groups2•The median of the impact of all the articles is higher in journals which recommend the use of both types of reporting guidelines. •But: This difference is not very high and the percentage of articles with zero citation is highest in journals that recommending both types of reporting guidelines.

Excluding the articles with an impact equal to zero3•Excluding those articles, the medians change. In this case, the best option would be to choose external reporting guidelines and the worst option would be to choose both types of reporting guidelines. •But: As this scenario is induced, it should not be taken into consideration.

DISCUSSION

May 2012 36Class 8


Possible motives for the non existence of differences:

Incorrect division of the journals through the groups according to type of reporting guidelines presented in the website: the inherent subjectivity of the division may have led to some errors.

Existence of financial interests that led to the non citation of the articles.

Great amount of articles existing nowadays.

Difficulties on the publication of the articles leads to the non accounting of the citation included in the unpublished articles.

CONCLUSION

May 2012 37Class 8


The combination between both types of reporting guidelines should be an option to take into deeper consideration, taking into account the small difference found between the median of the impacts of the articles published in journals that follow

this reporting guidelines, when comparing to those from journals that present external or internal reporting guidelines.

Within the articles with zero impact, there are comparatively fewer articles that were published in journals that present

internal reporting guidelines.

BUT:

CONCLUSION

May 2012 38Class 8


There was found no strong evidence that a certain group of reporting guidelines is the best choice to achieve a higher impact

of the articles published. Therefore, it is up to the journals of General and Internal Medicine to decide which kind of reporting guidelines to use, since all have advantages and disadvantages

that lead to no significant difference among groups.

A similar study could be done to other areas because, depending on the areas, the results found may be completely different. In fact, the same analysis applied in other subjects may lead to the discovery of unexpected results and to a change on the process of expanding the knowledge.

Class 8 39

ACKNOWLEGDES

March 2012

We thank our Professor Pedro Pereira for his support throughout the whole year. Moreover, we feel our acknowledgements shouldalso be directed to Professor Altamiro Pereira, whose critics allowed us to improve. Finally, we would like to thank to class 21 for their effort on delivering all the data needed on time.


Class 8 40

Jacinto T. Informação e conhecimento biomédico, pesquisa bibliográfica, bibliotecas digitais [Biomedical information and knowledge, bibliographic research, digital libraries] [slide]. Porto (Portugal): Faculdade de Medicina da Universidade do Porto (PT), Departamento de Ciências da Informação e da Decisão em Saúde; 2011. Portuguese, English.

Nahas FX, Ferreira LM. A arte de redigir um trabalho científico [periódico na Internet] 2005;20 Suppl. 2:17-8. Disponível em URL:

http://www.scielo.br/acb Nte AR, Awi DD. Writing a scientific paper: getting to the basics. Niger J. Med (2007) Jul-Sep;16(3):212-8. Jacinto T, Morais A, Fonseca JA. How to write a scientific paper - Searching and Managing Biomedical information. Rev Port

Pneumol. 2011;17:190-194. Samil LR. Writing and publication of a medical article. Vandenbroucke J P. STREGA, STROBE, STARD, SQUIRE, MOOSE, PRISMA, GNOSIS, TREND, ORION, COREQ, QUOROM, REMARK and

CONSORT: for whom does the guideline toll?. Clin Epidemiol. 2009;62:594-596. International Committee of Medical Journal Editors. ICMJE, International Committee of Medical Journal Editors [homepage on the

Internet]. [place unknown]: International Committee of Medical Journal Editors; 2009 [cited 2011 Nov 11]. Available from: http://www.icmje.org/

REFERENCES

March 2012


Class 8 41

Rodrigues P. Artigos científicos, aspectos e orientações base [Scientific papers, guidelines and basic aspects] [slide]. Porto (Portugal): Faculdade de Medicina da Universidade do Porto (PT), Departamento de Ciências da Informação e da Decisão em Saúde; 2011. Portuguese, English.

CONSORT Group. CONSORT, TRANSPARENT REPORTING OF TRIALS [homepage on the Internet]. [place unknown]: Minervation;

[reviewed 2011 May 16; cited 2011 Nov 11]. Available from: http://www.consort-statement.org/ STROBE Group. STROBE Statement, Strengthening the Reporting of Observational studies in Epidemiology [homepage on the

Internet]. [Bern (Switzerland)]: ISPM – University of Bern; 2009 [cited 2011 Nov 11]. Available from: http://www.strobe-statement.org/

SQUIRE Group. SQUIRE, Standards for Quality Improvement Reporting Excellence [homepage on the Internet]. [Lebanon (NH)]:

[Publisher unknown]; [cited 2011 Dec 1]. Available from: http://squire-statement.org/ STARD Group. STARD Statement, STAndards for the Reporting of Diagnostic accuracy studies [homepage on the Internet].

[Amsterdam (The Netherlands)]: [Publisher unknown]; [reviewed 2008 Apr 22; cited 2011 Nov 11]. Available from: http://www.stard-statement.org/

Stefánsson E, Kivelä T. The ingredients of a good paper. Acta Ophthalmol. 2010; 88:619-621

REFERENCES

March 2012


Documents

What to choose: internal reporting guidelines or external reporting guidelines? A step closer to answering this dilemma May 2012Class 8 Introdução à Medicina