A Case for Anonymous Open Review

I recently reviewed a manuscript for the pioneering journal PeerJ. This presented me with a quandary. PeerJ’s experiment in open reviewing is nicely outlined in their recent post, and includes two steps: reviewers can sign their reports, and authors can publish the review history alongside their accepted paper. My quandary was this: I love the second idea, and think it is an important step forward in opening up the peer review process; but I don’t like to sign my reviews. Not because I want to hide behind anonymity - clearly, writing this post shows that I’m not going to any great lengths to hide my identity from the authors of the PeerJ manuscript - but rather because I think remaining anonymous makes me, personally, a better reviewer. So, on this occasion - despite producing what I consider to be a ‘good’ review, in that it was both pretty thorough, and very positive - I declined to sign. To explain why, here’s some history.

It started with so-called ‘double blind’ review, whereby manuscripts are anonymised before being sent to review. Or rather, it started with an argument about double-blind review. A paper said it benefited female authors. We disputed the evidence, and, although I know I’m predisposed to come down on my side of the argument, I honestly cannot see how anyone else can fail to agree with us - just look at our figure!!! And anyway, at a practical level how can it help, when only reviewers are blinded but editors make all the key decisions?

But I digress…

Thinking about double-blind review in turn led me to think about what I’d prefer to see in peer review, and openness seemed the way forward. At that time, only the first of PeerJ’s options was available, and for a while I started to sign all my reviews.

Well, I say ‘all’, but I noticed a trend: I was reluctant to sign my most critical reviews. This seems like basic human nature - it’s evident still in PeerJ, where reviewers are far less likely to sign reviews recommending rejection (see fig 5 here) - but is perhaps worth exploring more closely.

My particular field is relatively small, and I often know the authors of the manuscripts I review, at least well enough to say ‘hello’ to at conferences, sometimes much better than that. I have never seen this as a conflict of interest - I provide honest reviews whoever the author, and I have absolutely panned the work of some senior authors of very high standing - as well as some quite good friends - whose work I usually respect. I am much more comfortable doing this anonymously, not because there is anything in my comments that I would not, if forced, say to the face of the lead author; but simply because I would rather not be placed in that situation.

Yes, it all comes down to avoiding socially awkward situations. I will do almost anything to avoid face-to-face awkwardness. I am not one of those people who delights in pointing out a fatal flaw in someone’s work in the Q&A after a talk. I will find a million euphemisms for ‘crap’ if asked to comment on a (hypothetical, of course!) colleague’s substandard work. Whether you see that as a good or a bad quality in me probably depends on your cultural upbringing, but the simple fact is that I find the option of anonymity very appealing.

And so, having come to the conclusion that I preferred to remain anonymous when writing critical reviews, I felt the only morally consistent position for me to take was not to sign any reviews. Sometimes this is difficult. If I write an especially insightful (read: long) review of a piece by someone I really admire, it’s definitely tempting to sign. But no. Joey doesn’t share food, and Tom doesn’t sign reviews. Frankly - and I’m not suggesting for a moment that this is true of everyone - I think this makes me a better reviewer.

The other reason given for signing reviews is that it enables you to gain appropriate credit for your reviewing activity. I don’t really buy this - what kind of credit are you expecting? And how much? Let’s face it, writing a review can be hard work, but it’s much less demanding than writing the damn paper in the first place. My worry is that chasing formal credit encourages early career researchers to spend too long on reviews. I reviewed for Science a while back, and treated it with due seriousness: my review was several pages long, and really thorough, I thought. The other review stated, essentially: “Nah, not a Science paper”. I’m not saying this second review is something to aspire to, but you do need to learn to apportion time appropriately, and if you think a manuscript has very little merit, you probably don’t need six pages to say so.

Also: from whom are you expecting this credit from reviewing? You can already easily summarise your reviewing activity on your CV; I’m simply not convinced that adding a doi for each review will drastically increase your employment prospects or standing in the community. Or at least, it’s not something I feel I need at this point. For those who want credit, and feel like a doi gives them that, then of course it’s great to have the option.

I wouldn’t want any of the above to suggest that I am in any way against openness in peer review, which has numerous benefits. I would be delighted to see my (anonymous) reviews appended to published papers. There is of course an editorial issue here - it’s probably more useful to publish an essay-style review, à la Peerage of Science, than a numbered list of typos; and my experience is that many reviews themselves are riddled with spelling and grammatical errors. Who will review the reviews?! But in principle, yes, let’s open up the process. Transfer of reviews between journals - another form of openness, adding memory to the review process - is becoming more common too, especially within publishing houses, which is great, and ought to help avoid the kind of situation I wrote about here.

My point is that open, civil, and constructive reviews can still be conducted under anonymity. For the sake us shrinking violets who value its protection, I hope the publishing pioneers at PeerJ and elsewhere retain it as an option.