1

Can we do more trustworthy science?

Mahmood Ibrahim Alsaydia

MSc. Data Telecommunications & Networks

University of Salford / UK

Mahmood.alsaydia@gmail.com

Abstract

Controversially, the published research start increases mostly in a conflicting way as the recipient

start became confused, and strongly arguing whether the published researches must be trusted or

not. And that debates or arguing has been moved to the researchers and scientists their selves. It

has been found that the collected data which presents the input of the research has a critical effect

on the finding validity. And these collected data can be affected by the normal factors of the

scientific research. Hence, it should be taken into consideration that data must be valid, varied and

up to date.

Introduction

Datum is very essential, sensitive and critical key in research. In terms of its quality, feasibility

and whether it is up to date or not, where premises and inductions which researchers tend to discuss

and analyse in their research are significantly depending on it. In addition to the fact that

availability and emerge of new data might change research findings from the wrong side to the

right side and vice versa. Therefore, it is very important to obviate the personal bias in collecting

and analysing evidences and data to present reliable and objective research, and the most

significant, with true findings. Where, in recent years, trust in many scientific areas has come under

scrutiny and the trustworthiness of research findings has been questioned in many domains of

science. (Stroebe, Postmes, & Spears, 2012). As nowadays the most worrisome and questionable

question is, do we trust in science? Anymore? Where most of the published research findings are

2

regarded to be false rather than true, according to John P. A. Ioannidis in his research " Why most

published research findings are false". (Ioannidis, August 2005)

Here questions must be asked, are John P. A. Ioannidis research findings true or false? Is the false

findings new issue to the science? Does that make any research with false or wrong finding useless,

vain and untrusted? Where, according to his study, which took about 10 years of individual teams

work in varying fields, he found that about 85% of published research findings are false, and that

lead to an assumption that his findings located within the remaining 15%. Likewise, he said,

"negative research is also very useful. Negative is actually a misnomer, and the misinterpretation

is widespread".

For the knowledge of how more reasonable, acceptable and trusted research can be achieved. The

factors that have been mentioned above need to be discussed. To be involved in such a debate, or

at least to make the scientists' debates about this subject understandable, if not good then at least,

brief background about the scientists' philosophy of the science and research methods need to be

exist in our personal knowledge. Also, what is the meaning of each factor or key must be known.

Thereby, a quick historical round to the history of the philosophy of science will be presented in

the next section.

History

The philosophy of science has its root in history and in sometimes in nineteenth century it came

out as an autonomous discipline. Where in that era fundamental questions like "What is science

and what is not? How it can be achieved? Were very controversial. And remarkable scientists and

philosophers like, Bacon, Copernicus, Kepler and even Galileo involved in this debate and later

on they developed thoughts which Subsequently considered as the basis of research methods.

Even in the earlier centuries like in the 17th century, the culture of Newton, which todays’

scientists consider it as the stable scientific age, where most of theories and inventions emerged at

that age. That age's scientists tried and they do underlined their thoughts and ways to practice

science in order to inheriting these ways to the following ages' researchers in a try to simplify the

path to one who might come after and want to examine their theories or innovate in their inventions

and conceptions. Hence, different schools of thoughts have been found. However, it is obvious

that until now the debate still stands and expands in some way, regarding to the differences in the

affecting factors of that age and nowadays science, for examples: data and premises, the political

and social effects, the technology and so on.

3

Scientific methods

Before plunging into the depth of this massive sea, answering the question ' What the scientific

method is?' Need to be clarified.

"Scientific methods has been defined by scientists as a body of techniques for

investigating phenomena, acquiring new knowledge, or correcting and integrating previous

knowledge". (Newton & Koyre, 1972). While The Oxford English Dictionary defines the scientific

method as "a method or procedure that has characterized natural science since the 17th century,

consisting in systematic observation, measurement, and experiment, and the formulation, testing,

and modification of hypotheses". (the Oxford English Dictionary definition for "scientific).

Thus, it is recipes, rules and a series of methods need to be followed to guarantee as far as possible

the results we want them to be. And it can be considered as an extension of the common sense.

But here, it must be taken into consideration if we want the results as we want them to be, then

does not that make everyone research based on bias to what he believes or want the result to be.

Obviously, from the definition, there as limitations and factors that usually affect scientific

research as it is related directly to the scientific methods.

Limitations of the Scientific Method

Falsifiability

Science cannot approve everything. For example, the experiment that can test ideas about God

and other supernatural beings do not exist, and this is the reason of disability to confirm or deny

these things. Yes, some intelligent design trying to interpret such beliefs, yet they cannot be

approved scientifically. Hence, the idea of science is to provide a better understanding of such

things, through refuting the previous hypotheses.

"Another restriction of scientific methods is that there is no absolute right or absolute wrong when

it comes to make judgements about goodness or badness of a certain thing. For example, it cannot

be said that global warming is harmful to the humanity in one hundred percent, but only the causes

and consequences of this phenomena can be studied objectively. Additionally, enquiries about

some aspects like morality cannot been answered by science, where, scientific results keen to be

far from scope of cultural, religious and social influences". (kopper, 1957)

4

But in the other hand, that does not mean that scientists or researchers should ignore all the

religious, social and humanity morals when they probe and apply their tests to approve theories

and claims especially in related fields, like medical, social, etc.

Criticism of the term

"The distinction between science and non-science is neither possible nor desirable. And that is

because the ability of the theories and hypothesis to accept new data (evolution of science). Also,

specific standards might be applicable to a specific field of science and not applicable to another".

(Gauch, 2003).

Factors affecting on research findings

These factors, which will be discussed below can be considered as an extension to the limitation

of science points that have been discussed above, but their influence extends relatively to the

findings in any research, and here the objectivity of science in general and more precisely the

research itself will be at stake.

Biases

Bias can be defined as the tendency in perceptions or in particular process or outcomes, often

accompanied by a refusal of considering the potential benefits of alternative views. People may

develop bias toward or against an individual, ethnic group, nation, religion, political parties,

ideologies and theoretical models within the academic fields. Bias can come in many forms, and

it is often considered synonymous with prejudice or intolerance. In other words, people expect to

see what they tend to belief.

"From the scientific research perspective, bias can be defined as a systematic error which can be

introduced into any stage of research whether it was in collecting data or the framework or testing

or by selecting or preferring one result over others even if it was wrong". (Pannucci & Wilkins, 2010).

5

Thereby, bias can be considered as a part of the human being's nature, therefore, as long as

researchers and scientists are human beings, it is hardly to avoid the fact that researcher commonly

be influenced by their previous belief or what they want to see in their work, and that might lead

to wrong or false findings. Thus, even if the right question has been asked at the beginning of the

research, the researcher, in this case, might find himself far from what has been planned to achieve.

And just from here, a concern about where and what did the researcher depend on in his work,

where did he look and dig (references and data) to approve his claim or to answer and support his

inquiry takes a place.

However, and despite of the possibility of testing the hypothesis in controlled experiments is vary

from one field to another. Bias, to some extent, and the other potential harmful effects can be

decreased by controlling the experiments and increasing the reproducibility percentage, as what

will be discussed later.

Corollaries

Some of the most interested corollaries that usually affect findings of any research can be

illustrated as in the points below:

1. The smaller the studies conducted in a scientific field, the less likely the research

findings are to be true.

Considering other factors are equal, then research findings are more likely to be true in

scientific fields that take large studies, and large samples. For example, when we compare

between randomized controlled trials in cardiology which usually takes several thousand

subjects randomized, with scientific fields with small studies and samples such as most

research of molecular predictors which its sample sizes may be 100-fold smaller.

Noticeably seen that the former findings keen to be much more true than the latter.

(Ioannidis, 2005).

2. The smaller the effect sizes in a scientific field, the less likely the research findings

are to be true.

Percentage of true findings are also related to the effect size. For example, the impact of

smoking on cancer is more likely to be true in comparison with the impact of genetics on

cancer.

3. The greater the number and the lesser the selection of tested relationships in a

scientific field, the less likely the research findings are to be true.

Research findings are more likely true in confirmatory designs, such as large phase III

randomized controlled trials, or meta-analyses thereof, than in hypothesis-generating

experiments.

6

4. The greater the flexibility in designs, definitions, outcomes, and analytical models in

a scientific field, the less likely the research findings are to be true. Where the ideas

and analysing the inductions will be scattered over more than a specific area of interest.

5. The greater the financial and other interests and prejudices in a scientific field, the

less likely the research findings are to be true. Conflicts of interest and prejudice may

increase bias. But prejudice may not necessarily have financial roots as what has been

mentioned previously. It might become as qualifications for promotion or tenure to be

given to researchers. Such nonfinancial conflicts may also lead to distorted reported results

and interpretations. "Prestigious investigators may suppress via the peer review process

the appearance and dissemination of findings that refute their findings, thus condemning

their field to perpetuate false dogma. Empirical evidence on expert opinion shows that it

is extremely unreliable". (Brock & Diggs, n.d., p184) (Ioannidis, 2005)

6. The hotter a scientific field (with more scientific teams involved), the less likely the

research findings are to be true. And that means in another one word 'competition'. And

in this precise point the objectivity of the researchers will be considered as the mirror of

the science objectivity.

More trustworthy science is needed

Despite of wrong and false finding are not new to science, the need to do more trustworthy

science still stand and leads to a wide debate. Most of the false finding research appear as a cause

of biases, corollaries effect, wrong inductions and misleading by data and premises, and that what

has been tried to be summarized above, where these factors considered to be the most common

factors which usually affect the research authenticity and the truth of its findings. Obviously, bias

has a noticeable effect on the research findings but it cannot be presented as a purely human being's

nature, but also, especially in specific fields like medical, economic and socio-political research,

personal benefits and political gains usually take place in the affection on the researcher when his

research is relevant to or can give chances to that opportunistic gains and benefits. Also the social

mores have its own share of the affection on research findings.

Ioannidis adds "Unfortunately, in some areas, the prevailing mentality until now keen to focus on

isolated discoveries by single teams and interpret research experiments in isolation. An increasing

number of questions have at least one study claiming a research finding, and this receives unilateral

attention. The probability that at least one study, among several done on the same question, claims

a statistically significant research finding is easy to estimate”. (Ioannidis, August 2005).

7

And that usually happens because the researcher's goal is to come out with different true findings

from the other researchers' findings which might become a discovery, and that might lead him to

search in the wrong corner. Hence, inductions of such a research will be deceptive and that

commonly lead to wrong deductions. Thereby, it is more trustworthy to look at the non-isolated

research which are mostly repeated and probe the same or similar question. Where there is a widely

believe that reproducible research can present more likely true findings and even sometimes the

outcomes might be a true discovery. And that is what the history of science filled with examples

of researchers arguing over the reproducibility of a published result and then stumbling onto a

completely new discovery.

Hence, Reproducibility of research is a very important factor can lead to obviate false findings in

research, where scientists believe that as much as reproducibility percentage is higher in a research,

findings are most likely feasible and reliable. Unfortunately, a misunderstanding between

replicability and reproducibility concepts is existing. Where, the former tend to be more influenced

by factors that have been discussed above, for example bias, whether it was personal bias or by

framework or bias in data, and so on. While the latter is more immune to such an influence,

presenting more objective science.

Therefore, it is very important not to conflate reproducibility, which is essential in science, with

the narrower idea of replicability which Drummond says "is not worth having. One should replicate

the result not the experiment, Drummond writes. A focus on exact replication of experiments

would cause a great deal of wasted effort, because researchers would spend time and money

recreating specific experimental details, applying exactly the same data in the same framework,

that are not broadly relevant. On the other hand, reproducible result, is one that holds up even

when you change the details". (Drummond, n.d.).

And because the fact that researchers rarely publish an attempt to replicate someone else's

experiment, because what has been discussed previously, there is no absolute knowledge about

what fraction of published studies are not reproducible. "However, a wide believe that there is

much greater risk of having a low reproducibility rate and failing to discover it. Science can only

self-correct if there is awareness of what needs correcting”. (Errington et al., 2014). Hence, "if

reproducibility is much lower than expected, then the continuity of innovations and inventions will

be in critical situation due to the lack of the evidence and data reliability". (Forscher, 1963).

As a result of that, a Project called reproducibility project has been presented, which is the first

systematic attempt to reproduce experiments from some of the most influential papers in the

literature. And that may show how much the scientific literature is not to be trusted or it could

show that researchers’ fears from this issue are overblown.

Also, it is necessary to take into consideration that false findings in research might be a temporarily

false where the question or the claim of the researcher who came out with "false findings" might

8

be approved later because the emerge of new data usually approve or adjust theories. And that is

why the false findings' research are not new to science.

Hence, "Ioannidis has a theory based on data that these incorrect or 'false' investigations are due

to bias and random error, and there is plenty of both. This may or may not turn out to be the case.

This theory is, exactly like all of the studies that Ioannidis investigates in his paper, a hypothesis

based on some data. When there is new data his theory may prove to be robust, or he may too be

disappointed and confused when new data shows that his findings were false". (McLain, 2013).

Now, referring to the all factors that have been mentioned above. Noticeably, they are not isolated

from each other, but they might even have connected in more than one way. Thereby, the best way

to presents more valuable and trustworthy science is by attempting to isolate these factors and then

obliterate and finish their effects. Only in this case the objectivity of research will increase and

that more likely means a more trustworthy science to present.

It is necessary to notice that the speech here is about (more trustworthy science), and that means

trustworthiness in science is already exist to some extent. Otherwise, how all this advance

technology and innovations became reality?

SWOT & PEST analysis

Through the use of SWOT analysis, the affection of the factors that have been mentioned above

on a research can be clearly seen. For example, let’s classify the factors above in SWOT matrix

and apply it on the research that has been presented by John P. A. Ioannidis, as below.

Strengths:

1. Reproducibility

2. Lack in Framework bias

Weakness:

1. Old data and research been

used

2. not up to date due to the long-

time of work

3. Insisting on the highest

percentage of research are false

9

Obviously, from the SWOT analysis, some of the factors that the research (why Most Published

Research Findings Are False) focus on them as factors lead to false findings, it has been affected

by them. Where, despite of the reproducibility that has been presented in the research through

probing many of the repeated and isolated research in the test, the weakness came through the type

and the date of the data that has been chosen and used, in addition to threats of personal bias and

bias in collecting data via using a specific field research in the majority and not enough research

from other fields.

Noticeably, the debate here is not about if there are false research findings in published research

or not, but about if it is the majority or the minority. Also by taking the affection of changes in the

PEST, which have been discussed to some degree previously, through a decade. Rules of

publishing research has been changed where it becomes easier because the internet era, and that

support the claim of what the research discussed. In contrast, the availability of usable data become

easier to get for the same reason. While socially, people became more aware about what they read

and when they look to read and that’s related in many ways to the technology factors.

On the other hand, economics and political interventions tend to increase biases in the research

and these have been increased noticeably in the last few years via privatizations and companies’

control. But that might apply to a specific filed more than others, for example, technology field is

less effective than the medical fields regarding to the last two factors. So it can be considered as a

matter of balance between these PEST impacts more than illuminate these affections which in

somehow seems impossible.

Opportunities:

1. Presenting a strong evidence

through the 10 years’ work on

many research

Threats:

1. New Data can refutes the

claims

2. Personal bias

3. bias in collecting evidences

and data (focus on specific field

more than others)

10

Conclusion

To sum up, it is hardly to avoid the impact of all factors above, which usually lead to false

findings or untrusted research, on the absolute manner. But as much as their percentages become

less, the trustworthiness and the objectivity of a research rise up. Obviously, the scientists debate

is not about if the science is not more trusted or important, as it still and always will be as it was

used to be, but about how it can be enhanced, kept immune to things that might ruin it and pushed

to the limitation of perfectness as much as possible. Remarkably, absence of personal bias in all

research's levels, reproducibility and accuracy regarding to data collecting and analysing can

effectively gain the opportunity to achieve the trustworthiness in terms of the reliability and

validity of findings. Hence, these three keys need to be taken into consideration when a project or

research takes place. And these keys can be achieved by asking the right question in terms of the

subject that researchers are tending to probe, and knowing how to support the answer of that

question with the right evidence by applying the right framework and searching in the right places

in terms of references, to guarantee the reproducibility rather than replicability, the objectivity

rather than bias and true rather than false findings.

References

Brock, K., & Diggs, G. The hunter-gatherer within.

Drummond, C. Replicability is not Reproducibility: Nor is it Good Science (1st ed.). Ottawa,

Ontario, Canada: Institute for Information Technology National Research Council Canada.

Retrieved from http://cogprints.org/7691/7/ICMLws09.pdf

Errington, T., Iorns, E., Gunn, W., Tan, F., Lomax, J., & Nosek, B. (2014). An open investigation of

the reproducibility of cancer biology research. Elife, 3. http://dx.doi.org/10.7554/elife.04333

Foerster, N. (1957). American poetry and prose. Boston: Houghton Mifflin.

Gauch, H. (2003). Scientific method in practice. New York: Cambridge University

Press.Feyerabend, P. (1975). Against method. London: NLB.

Ioannidis, J. (2005). Why Most Published Research Findings Are False (1st ed.). UK: PLoS Medicine. Retrieved from http://robotics.cs.tamu.edu/RSS2015NegativeResults/pmed.0020124.pdf

Klemke, E., Hollinger, R., Rudge, D., & Kline, A. (1998). Introductory readings in the philosophy of

science. Amherst, N.Y.: Prometheus Books.

kopper, k. (1957). CONJECTURES AND REFUTATIONS (1st ed.). Retrieved from

http://philosophyfaculty.ucsd.edu/faculty/rarneson/Courses/popperphil1.pdf

11

McLain, S. (2013). Not breaking news: many scientific studies are ultimately proved wrong!. the

Guardian. Retrieved 9 December 2015, from http://www.theguardian.com/science/occams-

corner/2013/sep/17/scientific-studies-wrong

Newton, Isaac (1999) [1687, 1713, 1726], Philosophiae Naturalis Principia Mathematica,

University of California Press, ISBN 0-520-08817-4, Third edition. From I. Bernard Cohen and

Anne Whitman's 1999 translation, 974 pages.

Stroebe, W., Postmes, T., & Spears, R. (2012). Scientific misconduct and the myth of self-correction in science. Perspectives on Psychological Science.

Pannucci, C., & Wilkins, E. (2010). Identifying and Avoiding Bias in Research. Plastic And Reconstructive Surgery, 126(2), 619-625. http://dx.doi.org/10.1097/prs.0b013e3181de24bc

Wikipedia,. (2015). Bias. Retrieved 9 December 2015, from https://en.wikipedia.org/wiki/Bias