What’s up with peer review?
With more and more scientific papers being published, the standard of the peer review process as gatekeeper of research quality is falling. Peer review is supposed to be a thorough, unbiased evaluation of scientific papers for the quality, significance and originality of their research. For a researcher, getting through the peer-review process can be likened to becoming a seal of approval for their research. However, the strong competition in many fields and the increasing pressure to publish papers (publish or perish) often lead to biased or sloppy reviews. Researchers complain that the mounting pressure to publish and allocate research funds leaves little time to review. As a result, many cases of scientific misconduct are detected only after a paper has been published (see the saga on STAP cells). What can be done to overcome the new challenges of peer review, and what new peer review models are arising?
The session “What’s up with peer review?” at ESOF14, organized by Julia Wilson from the UK charity Sense About Science, aimed at discussing these problems. Many scientists found their way to the Dipylon Hall in the Carlsberg Museum to discuss about the future of peer review with three experts: Lars Rasmussen, Editor-in-Chief of the Acta Anaesthesiologica Scandinavica, Iryna Kuchma, Open Access Program Manager at EIFL, and Victoria Babbit, publisher at Taylor & Francis.
Several points repeatedly came up during the discussion and seemed to concern researchers most: a merit system for reviewing, the lack of transparency in the peer review system, and the detection of misconduct during publication and reviewing. On the other hand, publishers and editors lamented the increasing challenge of finding qualified peer reviewers.
Being involved in peer review was described by researchers to be quite unsatisfactory, not least because of the lack of recognition in this task by their institution and during job recruitment. There is a need to quantify how often and how well a researcher reviews papers. One of the possible solutions for this, Iryna Kuchma said, are the five stars of transparent pre-publication peer review. Journals should publish the names of their reviewers, give feedback to the reviewers, and the review reports should be published online alongside the final article (a policy so far followed by only few journals, such as BioMed Central).
Along with the lack of transparency, the ethics of the publication process are a big challenge for high-quality peer review. During the discussion, Lars Rasmussen admitted that peer review is so far inefficient in catching misconduct, with hundreds of paper having been retracted in the past years in the field of anesthesiology alone. Reviewers have also been known to misuse the peer review system, delaying research of their competitors, stealing ideas and even attempting to review their own papers. In dealing with these challenges, the committee on publication ethics (COPE) plays a big role. But the scientific community as a whole also does the job, often pointing out flawed or forged research post-publication. In fact, some journals are turning towards open post-publication peer review as their peer review model, an approach championed by F1000 Research.
Common to new and traditional peer review models is the increasing challenge faced by publishers and editors to find qualified peer reviewers. The answer for our three experts: training PhD students. By introducing scientists early in their career to the peer review process, they will be able to perform this task independently even earlier than now.
Sadly, the session did not dwell too long on alternatives for peer review. Iryna Kuchma, though, did mention a call for proposals from the European Commission: “Innovative approach to release and disseminate research results and measure their impact”, within the Horizon 2020 framework program. It remains to be seen if this initiative will provide useful answers to the many questions the crisis of the peer review system offers. In the meantime, one of the session participants had a piece of advice for everyone: “we still do not know how well the new models for peer review work. But we are scientists, so try it out! Engage in experiments!”
Further reading: “Peer review – the nuts and bolts” guide for early career researchers by Sense About Science.