Peering into Peer Review

Peer review of papers and grant proposals is an important aspect of professional service—perhaps the most important one. Yet it sometimes gets to be too much, in obvious (unpaid extra work!) and less obvious ways.

I really hate to tear apart people’s work, even though I try to do so kindly and politely. A vast majority of papers, even rather crappy ones, represent a huge amount of labor, typically conducted and written up by junior researchers. I wish there were a commonly accepted way to let the authors know that you, as the reviewer, see and recognize the effort even though you have to deliver bad news (that the work isn’t publishable, sometimes in a particular prestigious journal but often even in principle). I try to do this, but worry about coming across as patronizing or, worse, touchy-feely. In my field, reviewers will commonly tear the authors a new a$$hole, leaving comments that are a hair’s breadth from calling the authors ignorant or stupid. Few deliver negative critique with any softeners, so maybe I should be used to the callousness. Maybe it shouldn’t bother me as much as it does. I always edit my reviews for tone, trying for at least matter-of-fact neutrality even when the paper is really bad and should be scrapped. I do sometimes wonder if there’s any point to my hand-wringing over being tactful when so many reviews are not… And what it would take for everyone to become just a little less unkind.

A related reason for why I’m sick and tired of reviewing is that writing up negative reviews takes an emotional toll. Reviewing is not an emotionally neutral activity for me. Having to highlight and summarize bad things about one paper after another ruins my mood and dampens my passion for my own work. Being steeped in negativity—even this perfectly justified negativity necessary for quality peer review—can bring down a reviewer who does it too much. When I catch myself being grouchy and bracing myself for disappointment before I even start reading a paper, it’s time for a reviewing break.

Manuscripts written by people I know tend to be solid and well-written. Sometimes they’re not flashy enough for a prestigious journal, but they’re generally decent. However, I’m occasionally put in an uncomfortable position, where a paper by colleagues I know and like is just plain bad. A recent such paper was not incorrect, just kind of pointless: no real motivation, no clear conclusion, obvious results, untimely. The type of work I might have my students do to test their code on, not something I would send to print. I did my peer reviewer duty as objectively as I could; if the paper had been written by anyone else, the authors would have received the same negative review from me. But despite being factual and tactful, and despite knowing that I hadn’t done anything wrong, I felt crappy, like I’d been disloyal and betrayed my colleagues.

How have your experiences with peer review been, blogosphere?

 


2 responses to “Peering into Peer Review”

  1. In econ, the norm has shifted to reviews being constructive. I think the reason for this is twofold– first, Larry Katz edits arguably our top journal and has been spreading the word about his views on the secondary purpose of reviews (to help authors get feedback to make their work publishable) and the chapter on writing reviews in the guide for young economists (or whatever it’s called) which says to be polite, although not to spend too much time on obvious rejects. The overall push for civility and not being racist or sexist from ASSA and NBER has probably also helped.

    It is really irritating as an editor to get a jerky review. The norm now is to save the jerkiness for the letter to the editor and to be polite in the author letter, though some people deviate (a very small number of older men, I think– when I was younger graduate students were often jerks and they became more polite with more experience, but that seems to have stopped).

    We are still very direct, and that translates to mean in some fields, but there is a huge difference between being clear with neutral language about the paper and personal attacks on the authors.

  2. Sarah S. Avatar
    Sarah S.

    This is interesting to me because the reviews that I have received in my STEM discipline have been overwhelmingly constructive and useful, with ideas that consistently improve my papers. I’m a professor at a SLAC so it’s a bit of a bubble and I only publish on an approximately annual basis. I’m sorry your field is so unpleasant. It makes me grateful that mine is so professional because sending off each paper always makes me nervous that our work is complete rubbish. I hope my reviewers are also at least a bit gratified to see that I do appreciate and incorporate their suggestions.

    I’m mostly am asked to review for our disciplinary Journal of Education, where I also publish regularly. These papers usually need only minor revisions and it’s generally enjoyable to see the new ideas. I do occasionally get papers that are so far off the mark that I wonder if the authors have even read the journal requirements, and I get cranky at the editors for sending me papers that are clearly not publishable any time soon. I should probably more carefully screen those papers and reject reviews for that journal more often, now that I think about it.

Leave a comment