Getting Positive Feedback About Publishing Negative Results

25 November 2013 by Matt Shipman, posted in Uncategorized

Image: John Crawford, National Cancer Institute

Editor’s Note: This is a guest post from Eva Amsen, outreach director for the open-access journal F1000Research. Her post discusses F1000Research’s efforts to publish more papers on negative results. For more on negative results, see a series of posts published on Communication Breakdown earlier this year.

In 2012 Ben Goldacre gave a compelling talk at the annual TEDMED conference, in which he told several stories about publication bias in medicine. In one particular example, he mentioned the case of the antidepressant Reboxetine. According to the published academic literature, this drug was at least as good as other antidepressants. But in reality, far more studies had been conducted that showed that the drug didn’t work well at all! Only the few that showed a positive effect had been published.

I often show a few minutes* of that video in presentations when I visit universities in my capacity as outreach director for F1000Research, and then I talk about the importance of publishing ALL your results, not just those that neatly match your hypothesis.

Eva Amsen (Image courtesy of Amsen.)

F1000Research is a relatively new open access journal, and we do publish negative results –probably better termed “null results”, which is the term I’ll use in the rest of this post. However, so many journals don’t publish null results that researchers often assume that no journal does. To make people aware that we do accept these papers, we ran a campaign from mid-May to the end of September, during which time all papers with null results would not be charged the regular article processing fee. It was an incentive to entice people to publish work they always thought was unpublishable.

Word spread quickly, the campaign was supported by Ben Goldacre as well as by members of our Editorial Board and Advisory Panel, and we received a lot of praise from the research community. During the campaign, about one in four of our total submissions were null results papers. That actually isn’t very much if you consider that a lot of people have a backlog of unpublished null results, but for a few weeks our submission inbox was a far more realistic representation of actual research output than usually arrives at journal offices.

For a lot of people though, writing up these results – especially if they weren’t recent – still took time away from their other work, and they didn’t find time to send us a manuscript during the period when publication was free. We are always open to receiving null results papers, though, and are still receiving them regularly.

Some of the papers we received were rejected by other journals simply for covering null results. We don’t think that’s a reason not to publish. The papers we published as a result of our campaign have received constructive and positive comments from referees, and many of the papers have already passed peer review. (The rest of them are still under review or in revision: F1000Research uses a transparent post-publication peer review model where articles go through formal invited peer review after they are published online.)

The most popular null results paper (by number of views and downloads) that we published as a result of the campaign is “Regulation of CYP3A genes by glucocorticoids in human lung cells.” In this study, researchers at the University of Utah tried to find out why steroid inhalation, the most common asthma treatment, does not work well for all asthma patients. They didn’t find a full explanation, but they were able to figure out some pieces of the puzzle that will ultimately help other researchers. It’s definitely work that was worth publishing.

During the campaign we also ran a Twitter theme week, using the hashtag #negativeresults, which sparked a lot of discussion. Many people who joined the Twitter conversation still believed they couldn’t publish null results, so we quickly set them straight. My personal favourite thing to come out of this was probably Nik Papageorgiou’s cartoon.

As a result of the null results campaign, we are now widely known as a journal that doesn’t discriminate against papers without positive results. However, this is not the only feature of F1000Research, so we’re currently focusing on some of our other aspects, such as the speed offered by our post-publication peer review model. We recently held another Twitter discussion, this time about scooping in science, and we’re involved in several collaborations around data sharing, reproducibility and open peer review.

Even though they’re not our key focus at the moment, discussions about null results still often come up. I recently attended a meeting in the Netherlands run by ZonMw, a funder that has made funds available for postdocs to write up null results from animal research. They invited us to give a workshop about publishing null results and we heard from animal researchers what their main concerns were: many felt pressured to only focus on publishing positive results, even with the knowledge that there was funding available to write up their null results.

It will be a while before the culture of scientific publishing and the reward system attached to it enables a fair balance in the publication of all research results, but judging from the feedback we’ve had over the past months, it’s a change that both researchers and funders would welcome – and we hope more publishers will join us in giving a home to null results.

(*From 5:18 to 8:18, if you’re curious.)


One Response to “Getting Positive Feedback About Publishing Negative Results”

Leave a Reply


− five = 0