A Case for Anonymous Open Review

I recently reviewed a manuscript for the pioneering journal PeerJ. This presented me with a quandary. PeerJ’s experiment in open reviewing is nicely outlined in their recent post, and includes two steps: reviewers can sign their reports, and authors can publish the review history alongside their accepted paper. My quandary was this: I love the second idea, and think it is an important step forward in opening up the peer review process; but I don’t like to sign my reviews. Not because I want to hide behind anonymity - clearly, writing this post shows that I’m not going to any great lengths to hide my identity from the authors of the PeerJ manuscript - but rather because I think remaining anonymous makes me, personally, a better reviewer. So, on this occasion - despite producing what I consider to be a ‘good’ review, in that it was both pretty thorough, and very positive - I declined to sign. To explain why, here’s some history.

It started with so-called ‘double blind’ review, whereby manuscripts are anonymised before being sent to review. Or rather, it started with an argument about double-blind review. A paper said it benefited female authors. We disputed the evidence, and, although I know I’m predisposed to come down on my side of the argument, I honestly cannot see how anyone else can fail to agree with us - just look at our figure!!! And anyway, at a practical level how can it help, when only reviewers are blinded but editors make all the key decisions?

But I digress…

Thinking about double-blind review in turn led me to think about what I’d prefer to see in peer review, and openness seemed the way forward. At that time, only the first of PeerJ’s options was available, and for a while I started to sign all my reviews.

Well, I say ‘all’, but I noticed a trend: I was reluctant to sign my most critical reviews. This seems like basic human nature - it’s evident still in PeerJ, where reviewers are far less likely to sign reviews recommending rejection (see fig 5 here) - but is perhaps worth exploring more closely.

My particular field is relatively small, and I often know the authors of the manuscripts I review, at least well enough to say ‘hello’ to at conferences, sometimes much better than that. I have never seen this as a conflict of interest - I provide honest reviews whoever the author, and I have absolutely panned the work of some senior authors of very high standing - as well as some quite good friends - whose work I usually respect. I am much more comfortable doing this anonymously, not because there is anything in my comments that I would not, if forced, say to the face of the lead author; but simply because I would rather not be placed in that situation.

Yes, it all comes down to avoiding socially awkward situations. I will do almost anything to avoid face-to-face awkwardness. I am not one of those people who delights in pointing out a fatal flaw in someone’s work in the Q&A after a talk. I will find a million euphemisms for ‘crap’ if asked to comment on a (hypothetical, of course!) colleague’s substandard work. Whether you see that as a good or a bad quality in me probably depends on your cultural upbringing, but the simple fact is that I find the option of anonymity very appealing.

And so, having come to the conclusion that I preferred to remain anonymous when writing critical reviews, I felt the only morally consistent position for me to take was not to sign any reviews. Sometimes this is difficult. If I write an especially insightful (read: long) review of a piece by someone I really admire, it’s definitely tempting to sign. But no. Joey doesn’t share food, and Tom doesn’t sign reviews. Frankly - and I’m not suggesting for a moment that this is true of everyone - I think this makes me a better reviewer.

The other reason given for signing reviews is that it enables you to gain appropriate credit for your reviewing activity. I don’t really buy this - what kind of credit are you expecting? And how much? Let’s face it, writing a review can be hard work, but it’s much less demanding than writing the damn paper in the first place. My worry is that chasing formal credit encourages early career researchers to spend too long on reviews. I reviewed for Science a while back, and treated it with due seriousness: my review was several pages long, and really thorough, I thought. The other review stated, essentially: “Nah, not a Science paper”. I’m not saying this second review is something to aspire to, but you do need to learn to apportion time appropriately, and if you think a manuscript has very little merit, you probably don’t need six pages to say so.

Also: from whom are you expecting this credit from reviewing? You can already easily summarise your reviewing activity on your CV; I’m simply not convinced that adding a doi for each review will drastically increase your employment prospects or standing in the community. Or at least, it’s not something I feel I need at this point. For those who want credit, and feel like a doi gives them that, then of course it’s great to have the option.

I wouldn’t want any of the above to suggest that I am in any way against openness in peer review, which has numerous benefits. I would be delighted to see my (anonymous) reviews appended to published papers. There is of course an editorial issue here - it’s probably more useful to publish an essay-style review, à la Peerage of Science, than a numbered list of typos; and my experience is that many reviews themselves are riddled with spelling and grammatical errors. Who will review the reviews?! But in principle, yes, let’s open up the process. Transfer of reviews between journals - another form of openness, adding memory to the review process - is becoming more common too, especially within publishing houses, which is great, and ought to help avoid the kind of situation I wrote about here.

My point is that open, civil, and constructive reviews can still be conducted under anonymity. For the sake us shrinking violets who value its protection, I hope the publishing pioneers at PeerJ and elsewhere retain it as an option.

Reading, writing, and aestheticism

Last week my daughter turned one, and - as well as celebrating all the fun of her first year - I found myself reflecting on the growing list of ‘things I used to do’, in those dimly remembered days before the arrival of Webbs 2.0 and 2.1. There are the obvious activities - eating (and drinking) out, sport, long walks and lie ins. There are things that have come alarmingly close to making the list. ‘Doing my job properly’ springs to mind (though I think I have that under control again now…). Any parent will bore you with a similar list. But a growing concern this last year has been the fact that ‘reading for pleasure’ has receded from everyday activity to rare treat. I still read, of course, struggling like all academics to ‘keep up’ with the literature (what a laughable idea!) and to acquire some basic understanding of various topics relevant to assorted projects (and side-projects). At the moment, for instance, I am at varying distances through books on palaeobiology, oceanography, moral philosophy, and statistical graphics (and therein, some might claim, lies my problem - focus, man! - but I digress…) However, here’s a stark fact: I didn’t read a novel in 2013 (unprecedented in my literate life) and the stack next to my bed continues to yellow and gather dust. And while the shamefully low frequency of my contributions here has almost relegated ‘writing a blog’ to the list, I probably still write more posts than I read.

All this means that when I do get the chance to read something just for fun, it’d better be good. And by ‘good’ I mean the writing should offer, before anything else, what William Giraldi, in a lovely NYT review of Wendy Lesser’s Why I Read, calls ‘an ecstasy of aestheticism’. There, I’ve said it: I absolutely prize style over content. Which is why, if I get time, I will pick up the Review section from Saturday's Guardian before any science or tech supplement, to read long pieces by the likes of Will Self, Hilary Mantel, Geoff Dyer or Lionel Shriver. Now that's a pretty diverse quartet, but I know I'll get a good read out of any of them, regardless of the actual subject of their piece.

What this means, I’ve come to realise, is that I read very little that could (even loosely) be described as ‘science writing’. This despite the copious output of the many producers of absolutely brilliant science writing. More people are describing more science more clearly than ever before; but the focus (quite rightly) of these pieces is almost always on content (What cool stuff has been found? What’s the fascinating human story behind the discovery?) and much less on that elusive ecstasy of aestheticism. There is a huge public appetite for informative, readable science writing, and it seems perfectly appropriate that most science writing serves to sate this. But - given time constraints - I’m usually content to rely on 140 character chunks for pure information; for longer reads, I’m looking for brilliant writing first, with scientific content a distant second. I’m in search of the stylists of science writing. Who out there shuns the homogenising algorithms of ‘readability scores’, breaks all the rules of SciComm 101, and dares to stretch the reader with an esoteric vocabulary and the kind of (intricicate, recursive (sometimes (seemingly) infinitely so)) sentence structure that made the lamented David Foster Wallace’s essays such a challenging, pleasurable read?

I touched on some of these issues in a very early Mola mola post on Sciencey Fiction  and although that was focused (as you may have guessed) on fiction - largely novels, in fact - it does identify some writers who I think have nicely seasoned their fine writing with science. More recently, I was given a copy of Richard Hamblyn’s The Art of Science and within a paragraph of the introduction I felt I was in good hands. But it’s a long book, and sits, still, in that yellowing, dusty pile…

So then, let’s cut to the chase. Which writers - in any format - ought I at least try to squeeze in to my few spare weekly minutes to lend my recreational reading more of a sciencey flavour?

An Appreciation of John Steele

When I received the sad news, yesterday, that John Steele had died of the cancer that had afflicted him this last year, my instinct was to share the passing of a scientific hero as widely as possible. I duly tweeted, but given the general lack of response I wondered if perhaps his legacy is not as widely appreciated as I believe it should be. Hence this personal appreciation. I never met John, although I had been corresponding with him over the last couple of months, and was due to speak to him the morning after he was hospitalised for what turned out to be the final time. As an aside - the fact that I was approached, in such a generous manner (His first email to me ended: “This email is a rather long-winded way of saying - welcome; and I look forward to useful and illuminating discussions”) to collaborate with someone whose work, as you’ll see, has been an inspiration to me, is one of those great egalitarian things that happens from time to time in science, and I was thrilled to have this opportunity. But, as a result of this limited personal interaction - just a handful of emails - my appreciation is limited to John’s work, both his publications and this new, unpublished material to which I was contributing, which was buzzing with intriguing and innovative ideas.

Actually, I can’t hope to do justice to John’s wider scholarship here, and cover only really that small part of his work which addressed the issue of differences in temporal and spatial dynamics between marine and terrestrial ecosystems, an issue that has been central to my own research. As I set out in an earlier post, my own journey to the position now where I (reasonably confidently) call myself a marine ecologist has been rambling and convoluted. Along the way, certain of John’s papers stood out like beacons, reassuring me that there was indeed a path to follow, no matter how overgrown.

Mainly, these beacons consisted of a clutch of papers published in the early 1990s, in particular a 1991 paper in the Journal of Theoretical Biology (Can ecological theory cross the land-sea boundary?) and a 1994 Phil Trans paper with Eric Henderson on Coupling between phyiscial and biological scales). Similar ideas were further developed in papers in Ecological Research (Marine Ecosystem Dynamics: Comparison of Scales) and Bioscience (Marine Functional Diversity). All of these, in turn, were building on John’s 1985 review in Nature, A comparison of terrestrial and marine ecological systems.

Key to all of these papers is the idea of scale, both spatial and temporal, and especially how the scale of variability is different in marine than in terrestrial systems. Because the seas act as an enormous thermal buffer, variability is fundamentally different there than on land. I’ve been playing with some data to try to show this (see below), but the concept is simple: if you stand in one place for 24 hours on land, depending where on Earth you are, you might easily experience a temperature range of 20˚C or more. In most places, the temperature of the sea - even at its surface - won’t vary nearly this much in a year. Spatial variation is similar - you will typically find much more variability (along all kinds of axes, not just temperature) in a square kilometer of terrestrial habitat than in a square kilometre of sea. This clearly has impacts on the organisms living there: if you’re a lizard and you’re too hot, you can maybe move a metre or two from full sun into the shade. A marine fish might have to move hundreds of kilometres (or tens of metres deeper) to achieve a similar drop in temperature. So these patterns of environmental variation are clearly important in order to understand species’ responses to climate change, and can explain some of the subtle differences already seen between marine and terrestrial species (see for example recent papers by Sunday et al., Burrows et al.).

One of John’s major insights was that physical and biological processes were typically more closely coupled in space and time in marine than in terrestrial systems. This stemmed from his strong background in physical oceanography. Indeed, in our recent correspondence he confessed “I have no systematic training in biology”; rather he epitomised the interdisciplinary nature of fisheries science, in which connections between the physical environment and biological resources have always been recognised in a way that terrestrial ecologists have only relatively recently accepted. Despite this lack of formal training, his ecological insight was astute, as apparent throughout his 1974 book The Structure of Marine Ecosystems, from which, incidentally, I took the opening quote for the Royal Society Research Fellowship application which currently supports me: “The first impression one forms of any community is usually of the diversity of species present, and of the differences in numbers, with some species abundant and others scarce”. That book has some interesting parallels with Alec MacCall’s later Dynamic Geography of Marine Fish Populations, in that it’s ecological content (MacCall’s book is, in my view, an excellent primer on macroecology, though the word is not used) was destined to be overlooked by most ecologists because the word ‘marine’ appears in the title.

This has inevitably only scratched the surface of John’s work. What drove him, I think, to return time and again to the marine-terrestrial comparative idea is summed up best in the abstract of the Journal of Theoretical Biology piece: “It is proposed that theories developed in one sector can be tested most critically in the other, with potential for greater generality.” This idea has guided my own research, and was constantly in mind while writing several papers, for example this one which begins with a Steele quote (“I argue that we should attempt to address the question of [ecological] generalizations capable of crossing the land-to-sea boundary”) and, in particular, this opinion piece I published last year. A piece which, I was delighted to discover, John had seen: “I had read your recent TREE paper with interest, and of course, appreciated your references to my cry in the wilderness two decades ago.” I hope that in continuing this search for generality, and performing critical tests of theory, I might send up one or two small flares of my own which - even if they don’t light the path all that brightly - might at least lead others to John’s more illuminating beacons.