No publication bias in global climate change research

Rarely do we encounter a scientific fact that stirs widespread debate and distrust as does the science of climate change.

Despite scientific consensus among climate specialists around the globe about a theory that is supported by an ostensible mountain of facts from physical, natural, and cultural sciences, the climate change debate continues to be perpetrated by politicians, industrialists, academics, and arm-chair scientists.

For example, 61 per cent of politicians and directors from Swedish municipalities claim uncertainty about whether climate change is real, while another nine per cent reject climate change altogether.

When governments reject science, the rest of us are put at risk. By refusing to accept the facts and potential ramifications of climate change, as a society, we stand to delay or overlook actions that are urgently needed to reduce our impact on the environment and adapt our cities and farmlands for a future climate that will be different than what we experience today.

Read More: Climate Change theme on ScienceNordic

Climategate gave wind to the sceptics

Much of the intense scepticism about climate change science began in 2009, when thousands of emails and data files were stolen from the Climate Research Unit (CRU) at the University of East Anglia, UK, and later exposed under the guise of a purported conspiracy to alter facts.

The allegations claimed that scientists had only publicised results in support of their theory—that climate change is driven by human activities. Other facts, that may negate this claim, were said to have been hidden.

Selective reporting is indeed a serious issue in the scientific community, especially when it comes to theory building as theories require consideration of all available facts. Is it possible that the theory of climate change is based on a biased selection of facts?

Me and my team from Lund University in Sweden and Stantec Consulting in Canada decided to find out.

Read More: What makes the climate change? Part one

Publication bias in the medical sciences

But what exactly is publication bias? If researchers only publish results that confirm their specific view or previous expectations or hopes, then the bulk of results in this research field will be skewed towards that established belief.

For example, if a researcher is developing a medical drug to treat a disease, then all results of the clinical trial should be made public for the benefit of other researchers seeking the same cure.

We know that positive and statistically significant results are more likely to be published than non-results by authors, which poses a risk to medical sciences as failed experiments that are not reported may lead other researchers to waste precious funds pursuing dead-ends. Moreover, if only positive results are published, the consensus amongst readers will be that the drug is potentially more effective than it truly is.

Read More: Scientists: There is too much focus on positive results

A scientific theory that considers all available facts will be the most resilient, as consensus will be formed on results that both support and challenge personal beliefs.

Fortunately, there are established methods in numerical ecology and statistics that allow us to detect when non-significant results are missing from a field of research.

One such method is the “Fail-safe N” (or sometimes called “the file-drawer problem”). This refers to the practice of only publishing positive results but filing away studies with negative or non-confirmative results.

Statistically we can calculate the fail-safe N, that estimates how many negative studies would be required to make the statistical effect insignificant. This means that if publication bias was occurring in climate change science, we can detect it.

Read More: Basic research crisis? Many results cannot be replicated

No evidence of publication bias

My team and I analysed more than 1,100 published results from the field of climate change science and found no evidence of under-reporting or missing results—even results that were not statistically significant or showing no positive effects were reported.

However, the study revealed stylistic biases in how articles are written: the largest, most prominent effects (as they relate to climate change) were reported in the upfront summary sections (also called the abstract) where they are most readily seen by readers, whereas the lesser effects and those that were not significant tended to be buried within the technical results sections where relatively few readers are likely to see them.

Stylistic biases are less concerning than a systematic tendency to under-report non-significant effects, assuming researchers read entire reports before formulating theories. However, most audiences, especially non-scientific ones, are more likely to read article abstracts or summary paragraphs only, without perusing technical results.

The onus to effectively communicate science does not fall entirely on the reader; rather, it is the responsibility of scientists and editors to remain vigilant, to understand how biases may pervade their work, and to be proactive about communicating science to non-technical audiences in transparent and un-biased ways.

Read More: Crisis in basic research: scientists publish too much

Climate science is built on a solid foundation

It is important to stress that we are not climate scientists. Rather, in this instance, we functioned as scientists holding climate scientists to account and tested to see if the reporting practices by the climate research community is sound.

Although climate change scientists tend to highlight their most interesting results in the abstract of their articles, a truth that is unlikely unique to climate science, we can be confident that the theory of climate change is built on a solid foundation of science that gives credence to positive, neutral, and negative experimental results.

In scientific terms, we reject the accusation made by climate change skeptics and can confirm that there is no publication bias in climate change research.

The study was recently published in the journal Climatic Change.

Watch climate scientists at work in the lab and see how they investigate past climate change using ice cores. (Video: ScienceNordic)

---------------

Read this article in Danish on ForskerZonen, part of Videnskab.dk. This article is also published on The Conversation. 

Scientific links

• No evidence of publication bias in climate change science. Climatic Change (2017) DOI: 10.1007/s10584-016-1880-1
• Fanelli D (2012) Negative results are disappearing from most disciplines and countries. Scientometrics 90:891–904. doi:10.1007/s11192-011-0494-7
• Fanelli D (2013) Positive results receive more citations, but only in some disciplines. Scientometrics 94:701–709. doi:10.1007/s11192-012-0757-y

External links

• Johan Hollander
• Christian Harlos
• Tim C. Edgell

Related content

Basic research crisis? Many results cannot be replicated

Many scientific studies cannot be replicated and this is symptomatic of a wider crisis in basic research, say scientists.

Crisis in basic research: scientists publish too much

Scientists slice their results up into as many publications as they can, and the result is that a large portion of research output is unnecessary, say scientists.

Positive publication bias in cancer screening trials

There are cases where major cancer screenings damage people’s health, but these cases are only rarely featured in the scientific evaluation of the screening programmes, according to a new literature review.