Basic research crisis? Many results cannot be replicated
Many scientific studies cannot be replicated and this is symptomatic of a wider crisis in basic research, say scientists.
When scientists write up their results in a scientific article, they should describe their methods very precisely so that other scientists can replicate them.
And in research, it’s not enough to test something once. A study should preferably be replicated many times by independent groups of scientists before they can be confident that their results are correct. This process is called reproduction.
Reproduction eliminates erroneous results or bad science. It also ensures that scientists do not waste time and money pursuing flawed research, and reassures others that incorrect conclusions are filtered out of the system.
Basic Research
ScienceNordic takes you into the engine room of basic research to find out:
- What is it?
- Who does it?
- And how does it benefit us?
Read More: What is basic research?
But far too much of today’s published research cannot be reproduced. And it is a serious problem, say scientists.
The results of a Nature survey in 2016: “Is there a reproducibility crisis in research?” (Video: Nature Publishing Group).
One-time results are widespread
A survey published in Nature in 2016 asked researchers “is there a reproducibility crisis in research?”
Over half of 1,576 respondents, 52 per cent, said yes.
More than 70 per cent had tried to reproduce a colleagues experiment without success, and over half had tried, and failed, to reproduce their own experiments.
It is also easy to find examples of studies that could not be reproduced. In 2015, a group of researchers showed that only 36 per cent of studies from three respected psychology journals could be reproduced.
Only 47 per cent of the reproduced studies reported the same results as the originals study.
A similar study in 2012 showed that just six out of 53 landmark studies in cancer could be reproduced.
Read More articles in the Basic Research Theme on Science Nordic
Mistakes are reinforced
Lack of reproducibility is especially problematic because researchers today depend on their research being cited by other scientists, says Associate Professor David Budtz Pedersen from the Department of Communication, Aalborg University, Denmark.
“Today, we’re mutually dependent other groups' research, because we cite each other and build upon each other’s work. In this way, a single mistake can multiply and create a cascade of bad research, that is very hard to detect and takes long time to correct,” he says.
The ideals have changed over time, says psychologist Dorthe Berntsen from the Department of Psychology and Behavioural Sciences at Aarhus University, Denmark.
“When I started in science, the ideal was to conduct three to four experiments that could show the same effect in different ways. You replicated your experiments and then presented the three or four studies together in one article and submitted it to a journal,” says Berntsen.
“Later on, there was a change so that one or perhaps two studies was enough, but it had to be be a novel and exciting discovery, which should be really entertaining and fascinating to a wider public. This is the focus that you have today in Nature and Science, for example,” she says, but adds that there is also a movement in the opposite direction.
For example, the Association for Psychological Science recently launched a number of quality assurance initiatives, such as the Registered Replication Reports initiative, which aims to encourage replication studies.
Read More: What is basic research?
Lack of reproduction is not always a problematic
But just because a study cannot be reproduced, it does not necessarily mean that the research is bad, says Associate Professor Mikkel Johansen from the Department of Science Education at the University of Copenhagen, Denmark.
While he agrees that scientists should try to replicate results as often as possible, it is not always bad science or questionable research practice that means that a result cannot be reproduced.
“We should really take care when we talk about reproducibility in empirical sciences, because what does it mean exactly to reproduce an experiment? Reproduction is built partially on the false premise that scientific experiments are a well-defined and standardised machine that always spit out the same result from the same input,” says Johansen.
Read More: New minimum basic funding requirements in Norway
Genuine basic research is difficult to reproduce
We should be especially cautious of ringing the alarm when we fail to reproduce basic research, says Johansen.
“There’s clearly some cases where it’s reasonable to require reproduction, because we understand the context really well and the experiment can be very precisely described. For example, medical research. But when it comes to basic research, there are some places where you don’t know which conditions influence [the outcome]. So when you reproduce [the experiment] you may inadvertently change certain crucial details,” he says.
“A lack of reproducibility may also indicate that we don’t understand parts of the experiment well enough yet and so it’s difficult to reproduce all of the relevant parts, because we don’t know what they are, and then we change a couple of parameters that we thought didn’t matter.”
---------------
Read the Danish version of this article on Videnskab.dk
Translated by: Catherine Jex
