Statistical interpretation of studies among doctors and

medical studentsAZEVEDO André; CARDOSO Ana; FIGUEIREDO Carlos; GONÇALVES Francisco; GONÇALVES Raquel; MOITA João; MONTEIRO Renata; NOGUEIRA Miguel; PINTO Sérgio; RODRIGUES Daniel; RODRIGUES Marta; VIEIRA Joana; TEIXEIRA Ana

Advisors:Ricardo Filipe Sousa SantosAlfredo Mendes Castro

May 2012

Introdução à Medicina II

2

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

Medicine came to be as man sought cures for the diseases and remedies for the injuries that affected

human kind.

Fig.1

Evolutionpast prese

nt

3

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

Primitive man looked on disease as a curse cast on him by an evil spirit; his treatment consisted of driving the demon that possessed him[1].

Fig.2

[1] Claridge, J.; Fabian, T. History and Development of Evidence-based Medicine. World Journal of Surgery; 2005; 29, 547-553

Evolutionpast prese

nt

4

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

Observation of disease, its symptoms and treatments, were first found in written documents.

Fig.3

Evolutionpast prese

nt

5

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

However,

were the main foundation for medical decision making[2].

Expert opinion Experience Authoritarian judgment

[2] Dahm, P.; Sur, R.: History of Evidence-based medicine. Indian Journal of Urology; 2011; 27; 487-489

Evolutionpast prese

nt

6

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

Progressively, Medicine became to analytically study and observe the phenomenon of disease.

Textbooks and journals began to become more prominent[1].

Fig.4 Fig.5

[1] Claridge, J.; Fabian, T. History and Development of Evidence-based Medicine. World Journal of Surgery; 2005; 29, 547-553

Evolutionpast prese

nt

7

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

The use of scientific methodology and statistical analysis in biomedical research combined as the roots of Evidence-Based Medicine.

Biostatistics is an applied science for biology, medical and health sciences and it plays an increasingly important role

in the production of medical knowledge and in the clinical practice [3].

Informatics explosion with online journals and large databases[1].

[1] Claridge, J.; Fabian, T. History and Development of Evidence-based Medicine. World Journal of Surgery; 2005; 29, 547-553[3] Sahai H, Ojeda MM. Teaching Biostatistics to Medical Students and Professionals: Problems and Solutions. Int J of Math Educ in Sci % Technol. 1999; 30(2):187–196.

Fig.6 Database scheme.

Evolutionpast prese

nt

8

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

doctors must have a minimum grasp of biostatistic concepts to fully understand biomedical literature.

doctors need to stay up to date with the literature that drives clinical practice.

Informatics explosion with online journals and large databases[3]

[3] Sahai H, Ojeda MM. Teaching Biostatistics to Medical Students and Professionals: Problems and Solutions. Int J of Math Educ in Sci % Technol. 1999; 30(2):187–196.

9

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

Accounting for this problems, many researchers have asked the question of whether or not

medical do possess said knowledge.

10

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

Lack of knowledge to adequately interpret and evaluate results in medical research articles.

United States of America[5]

United Kingdom

Israel[4]

There was a seeming uniformity to the conclusions exposed on the reviewed literature.

[4] Cahan A, Gilon D, Manor O, Paltiel O. Probabilistic reasoning and clinical decision-making: do doctors overestimate diagnostic probabilities? QJM : monthly journal of the Association of Physicians. 2003;96(10):763-9. Epub 2003/09/23.[5] Windish DM, Huot SJ, Green ML. Medicine residents' understanding of the biostatistics and results in the medical literature. JAMA : the journal of the American Medical Association. 2007;298(9):1010-22. Epub 2007/09/06.

11

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

Will the case in study reflect the same results?

Portugal

Given that no other literature was found regarding statistical knowledge of Portuguese physicians (either in general or in the Oporto region), our work was designed to analyze this parameter.

12

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

Acknowledgments

Do doctors and medical students from Portugal have the necessary knowledge to adequately

interpret biostatistical information in scientific articles?

13

Aims

Introduction

Methods

Results

Discussion

Conclusion

References

Acknowledgments

To assess physicians and medical students’

knowledge of biostatistics

To compare participants confidence in evaluating

statistical information with their actual knowledge of

statistical concepts

To evaluate participants attitude towards

biostatistics

To assess the existence of correlations between

academic qualifications and participants biostatistical

knowledge

14

STUDY DESIGN

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

Type of Study

Cross sectional study

Target population

Physicians and medical

students from a Portuguese University Hospital

Unit of analysis

The individual

15

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

STUDY PARTICIPANTS

4 classes of the 5th year of MIMED

(“Mestrado Integrado em

Medicina” at FMUP) were randomly

selected out of a total of 8, using a random number

generator.

7 departments out of 43 at HSJ

(“Hospital de São João”) were

randomly selected. Only 5 colaborated.

16

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

STUDY PARTICIPANTS - INCLUSION AND EXCLUSION CRITERIA

Students: - Participants must belong to Hospital's students

community;- Participants must be studying at the Hospital at the

moment - Participant must be matriculated at the 5th year;- Participants mustn't have participated in the pilot survey.

Physicians:- Participants must integrate the medical population of the

Hospital.

Inclusion criteria

17

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

Students:

- Individuals studying at the hospital under mobility projects:

- The students studying abroad under mobility projects – http://erasmusfmup.blogspot.com

STUDY PARTICIPANTS - INCLUSION AND EXCLUSION CRITERIA

Exclusion criteria

18

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

Pilot surveyDevelopme

nt of the questionna

ire

1 2

Modifications based on pilot survey’s results

3

Meeting with head

of departments and 5 th

year students

4

Questionnaire delivery

5

19

Personal contact with the departments and students

Data collection

Beginning of data insertion

Running of syntax

1st contact with the departments and students

(via e-mail)

Questionnaire delivery

CALENDAR

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

20

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

1st: Questions regarding attitude towards statistics.

2 nd: Confidence questions about interpreting and assessing statistical

concepts.

3 rd: Biostatistical knowledge test that assesses understanding of: • statistical methods; • study design;• interpretation of study results.

4th: Questions about the academic and professional formation of the

interrogated person.

DATA COLLECTION METHODS:QUESTIONNAIRE

21

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

1st: 4 questions 2nd: 3 questions

DATA COLLECTION METHODS:QUESTIONNAIRE

22

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

3rd: 9 questions 4th: 7 questions

DATA COLLECTION METHODS:QUESTIONNAIRE

23

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

Attitudes will be rated on a 5-point Likert Scale;

Confidence questions will be rated using a 5-point scale;

Questions that aim to access participants knowledge of statistical methods will be classified as correct or incorrect. Missing values will be counted as incorrect responses.

VARIABLES DESCRIPTION

24

STATISTICAL ANALYSIS

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

Data (from the questionnaire) was processed using Statistical Package for Social Sciences (SPSS)

Several paramaters were analysed for the general population and in separate for students and physicians

Attitudes and confidence were quantified in a 1 to 5 scale and 4 and 5 were considered “Agree” and “Confident”,

respectively. Analysis used frequency tables

25

STATISTICAL ANALYSIS

Methods

Introduction

Aims

Results

Discussion

Conclusion

References

Acknowledgments

The percentage of each group that answered correctly at each question was calculated

The mean of correct answers were compared between different demographic groups, using T-test (results were

considered statistically relevant for p < 0,05)

Tables were constructed in order to present the results of the analysis

26

Results

Introduction

Aims

Methods

Discussion

Conclusion

References

Acknowledgments

Table 1. Percentages of Correct Answers for the Knowledge-Based Questions

  Doctors (n=35) Students (n=52)

  Mean correct (%) Mean Correct (%)Interpreting Results in a Kaplan-

Meyer Analysis 17,1 5,8

Identifying a continuous variable 60 78,8

Identifying an ordinal variable 40 51,9

Identifying a nominative variable 51,4 69,2

Determining which test is most specific 42,9 67,3

Interpreting the meaning of P<0.05 2,9 9,6

Understanding the purpose of double-blindness 74,3 92,3

Interpreting standard deviation 31,4 46,2

Understanding the definion of bias 71,4 84,6

Recognizing a case and control study 48,6 28,8

Interpreting CI 95% and statistical significance 14,3 11,5

27

Results

Introduction

Aims

Methods

Discussion

Conclusion

References

Acknowledgments

Table 2. Characteristics of the Participants (percentage).

     

Student (n=52)

Doctor (n=35)

    % %Total   59,8 40,2

Academic Degree 

Student 100 ///Licenciate - 71,4

Master - 14,3Doctorate - 8,6

Is a specialised physician

   54,3

Years since completing Licenciature

Less then one 77 0One to three 78,8 25,7Four to ten 86,5 25,7

Eleven to twenty 13,5 11,4More then twenty

one 0 31,4

Trained in Biostatistics   61,5 54,3

Trained in EBM (Evidence Based

Medicine)  46,2 31,4

Reads Scientific Literature frequently   50 85,7

 Has had oportunity to work in research

projects  30,8 65,7

28

Results

Introduction

Aims

Methods

Discussion

Conclusion

References

Acknowledgments

Table 3. Atittude and confidence results by General, Physicians and Students' characteristics (percentage)

  Doctors (n=35) %*

Students (n=52) %*

1 -"I have a desire to learn more about biostatistics." 85,7 50

2- "I can understand almost all statistical terms encountered in scientific literature." 17,1 34,6

3 -"Because it is easy to lie with statistics, I don't trust it." 17,1 1,9

4 -"I normally use statistics to form opinions or make clinical decisions." 57,1 55,8

5 -Interpreting a studies results with a statistical method  17,1 38,5

6 -Evaluating whether the correct statistical procedure was used in a certain study  2,9 11,5

 7- Interpreting the value of P for a given result 

 25,7 59,6

29

Results

Introduction

Aims

Methods

Discussion

Conclusion

References

Acknowledgments

Table 4. Knowledge Scores by General, Physicians and Students' characteristics

  Students (n=52) Doctors (n=35)

Characteristic Mean correct (%)

P value

*Mean

correct (%)P

value *

Has had previous training in biostatistics 52,8 0,133 45,5 0,573Has had previous training in evidence

based medicine 48,7 0,645 40,5 0,493

Regularly reads medical journals 49,3 0,875 45,2 0,207Has previous participated in scientific

research work 54,5   45,5 0,462

The amount of years passed since medical school graduation is... … <10 47,0

0,112 … >10 35,3Is a specialist 42,6 0,678

Total 49,7   41,3  

30

  Doctors (n=35) %*

Students (n=52) %*

1 -"I have a desire to learn more about biostatistics." 85,7 50

2- "I can understand almost all statistical terms encountered in scientific literature." 17,1 34,6

3 -"Because it is easy to lie with statistics, I don't trust it." 17,1 1,9

4 -"I normally use statistics to form opinions or make clinical decisions." 57,1 55,8

5 -Interpreting a studies results with a statistical method  17,1 38,5

6 -Evaluating whether the correct statistical procedure was used in a certain study  2,9 11,5

 7- Interpreting the value of P for a given result 

 25,7 59,6

Discussion

Introduction

Aims

Methods

Results

Conclusion

References

Acknowledgments

Table 3. Atittude and confidence results by General, Physicians and Students' characteristics (percentage)

ATITTUDDE/ CONFIDENCE

31

     

Student (n=52) Doctor (n=35)

    % %Total   59,8 40,2

Academic Degree 

Student 100 ///Licenciate - 71,4

Master - 14,3Doctorate - 8,6

Is a specialised physician

   54,3

Years since completing Licenciature

Less then one 77 0One to three 78,8 25,7Four to ten 86,5 25,7

Eleven to twenty 13,5 11,4

More then twenty one 0 31,4

Trained in Biostatistics   61,5 54,3

Trained in EBM (Evidence Based

Medicine)  46,2 31,4

Reads Scientific Literature frequently   50 85,7

 Has had oportunity to work in research

projects  30,8 65,7

Discussion

Introduction

Aims

Methods

Results

Conclusion

References

Acknowledgments

Table 2. Characteristics of the Participants (percentage)

DEMOGRAPHICS

32

  Doctors (n=35) Students (n=52)

  Mean correct (%) Mean Correct (%)

Interpreting Results in a Kaplan-Meyer Analysis 17,1 5,8

Identifying a continuous variable 60 78,8

Identifying an ordinal variable 40 51,9

Identifying a nominative variable 51,4 69,2

Determining which test is most specific 42,9 67,3

Interpreting the meaning of P<0.05 2,9 9,6

Understanding the purpose of double-blindness 74,3 92,3

Interpreting standard deviation 31,4 46,2

Understanding the definion of bias 71,4 84,6

Recognizing a case and control study 48,6 28,8

Interpreting CI 95% and statistical significance 14,3 11,5

Discussion

Introduction

Aims

Methods

Results

Conclusion

References

Acknowledgments

Table 1. Percentages of Correct Answers for the Knowledge-Based Questions

TEST

33

Discussion

Introduction

Aims

Methods

Results

Conclusion

References

Acknowledgments

Limitation Explanation

Low answer rate (87 out of 208)

It caused serious problems in results’ interpretation because it was not possible to extract large

amounts of information observable by statistical data.

Difficulties in contact with the heads of department

Caused problems in collecting surveys of physicians.

Existence of some forced conclusions

With some lack of information, it becomes possible to have

conclusions that are not wrong but are less observable as expected.

STUDY LIMITATIONS

34

Most of physicians and students did not know how to interpret the P-value (93,1%), in spite of revealing some confidence in this matter(46%).

Though almost 2/3 of the inquired had reading habits, half of them lacked confidence in understanding the results of a statistical method used in a research.

64,4% of the general population showed interest in learning more about biostatistics.

Conclusion

Introduction

Aims

Methods

Results

Discussion

References

Acknowledgments

35

1. Claridge, J.; Fabian, T. History and Development of Evidence-based Medicine. World Journal of Surgery; 2005; 29, 547-553

2. Dahm, P.; Sur, R.: History of Evidence-based medicine. Indian Journal of Urology; 2011; 27; 487-489

3. Sahai H, Ojeda MM. Teaching Biostatistics to Medical Students and Professionals: Problems and Solutions. Int J of Math Educ in Sci % Technol. 1999; 30(2):187–196.

4. Cahan A, Gilon D, Manor O, Paltiel O. Probabilistic reasoning and clinical decision-making: do doctors overestimate diagnostic probabilities? QJM : monthly journal of the Association of Physicians. 2003;96(10):763-9. Epub 2003/09/23.

5. Windish DM, Huot SJ, Green ML. Medicine residents' understanding of the biostatistics and results in the medical literature. JAMA : the journal of the American Medical Association. 2007;298(9):1010-22. Epub 2007/09/06.

References

Introduction

Aims

Methods

Results

Discussion

Conclusion

Acknowledgments

36

Acknowledgments

Introduction

Aims

Methods

Results

Discussion

Conclusion

References

We gratefully thank our advisors Ricardo Santos and Alfredo Castro for their support that helped us completing the task we have been assigned. We also direct our acknowledgements to the head of departments and students that cooperated with us. And finally, a special gratitude to Prof. Altamiro Pereira  for his constructive criticism.