Publication Bias

Where does publication bias come from? It seems that, in medical literature, bias towards publishing striking results originates from journal editors.

Written and verified by the psychologist Paula Villasante.

Last update: 21 December, 2022

In statistics, a bias is a systematic error that can occur in studies or experiments. When there’s a bias present, some responses are selected or favored over others. Interestingly, this bias occurs in a large part of investigations to a greater or lesser extent. So, what does publication bias consist of and what are the reasons behind it?

The most fundamental thing is to understand this concept in order to avoid, minimize or correct it. Biases can occur at any stage of the research process. In particular, those are the phases of planning, executing, analyzing, presenting results, and publishing.

Therefore, publication bias is the tendency to publish research that only shows positive results.

Positive results are those when researchers prefer some findings over others. These findings tend to be statistically significant like correlations or differences.

On the other hand, with this bias, researchers don’t publish negative results. As you can imagine, negative results are the opposite of positive results, so there are no statistically significant correlations or differences.

Another form of publication bias is when researchers publish results or articles that are more convenient or popular. Therefore, it would be more likely that a journal’s editor publishes the author’s article.

In the middle of the last century, Sterling first introduced the concept of publication bias. He pointed out that journals are more likely to publish studies with statistically significant results. Conversely, journals tend to not publish studies with insignificant results.

Evidence of publication bias

As we’ve mentioned, one of the first authors that pointed out publication bias was Sterling. Thus, this author reviewed all articles in four journals during one year (from 1955 to 1956). Afterward, he found that when the researchers performed significance tests, 97% of the articles rejected the null hypothesis.

 According to the results of the investigation, published studies outnumber unpublished studies with a ratio between 128:1 and 1:1. Most studies have a ratio between 10:1 and 1:1.

Factors that influence bias

According to Maria Carmen Rosa Garrido, there are currently several factors that influence publication bias:

• First, the author’s decision not to publish results that are not statistically significant in their study. 
• Second, the journal editor’s rejection or avoidance of publishing studies with negative results. This happens even if the methodological quality is enough to verify the reliability of the results.
• Third, the exclusion of these studies in the bibliographic searches of other researchers.

How to eliminate or prevent publication bias

In order to prevent or eliminate publication bias, some authors propose:

• Eliminating hypothesis tests
• Prior review by peers
• Stop publishing studies with inadequate sample sizes
• Developing a more positive attitude towards non-significant results
• Improve peer review and publication processes

Evaluating the bias

Many statistical procedures have been developed to evaluate whether a sample of studies is biased and to evaluate the impact of that bias.

Most are based on the assumption that, for any given substantial area, studies with small sample sizes should produce a relatively wide range of effect sizes. Meanwhile, studies with large sample sizes should produce an effect similar to that of the population’s size.

Journal editors and publication bias

Where does publication bias come from? It seems that, in medical literature, bias towards publishing striking results originates from journal editors.

This suspicion has some basis. In 1980, the British Medical Journal stated that their ideal article would describe “outcomes that affect clinical practice … and findings that improve prognosis or simplify the treatment of common diseases“. Thus, they accidentally alluded to the publication bias.

The most effective measure to prevent publication bias is registering all clinical trials. So, in order to move forward, the goal should be prevention.

In conclusion, it’s important to emphasize the importance of this bias in the scientific community. When using biased evidence, the decisions you make may not be the most optimal. So, in the medical community, this could lead to inappropriate prescriptions that may not be the best for patients or the scientific community in general.