I am really disillusioned by peer review today. No, it’s not my paper that got rejected. And no, it’s not the first time an incident like this happens.
I just got a notice that a paper I had reviewed for Prestigious Society Letters got editorially accepted for publication after 1st revision.
I reviewed the original manuscript but never saw the revision, although I said I wanted to see it (there is a check box to that effect). The editor appended the full correspondence to today’s email, following the paper’s acceptance. I can also see online the full timeline of the requests to various referees, when they submitted their reports, etc. My gripe for today is nestled between what was written (the correspondence) and when it was written (the timeline).
I was Referee 1 and recommended transfer to another journal (Reputable Society Journal, same publisher as Prestigious Society Letters). I raised several points because of which I did not consider the paper to be suitable for Prestigious Society Letters. Referee 2, who reviewed at the same time as me, liked the paper and said it should be published in PSL.
The paper gets revised and resubmitted. I never saw the revision, and I am guessing (based on the timeline) that Referee 2 didn’t either.
The revised and resubmitted manuscript gets sent to a tie-breaker Referee 3, who agrees with me (i.e., Referee 1) that the paper is more suitable for a more specialized journal and should be transferred to RSJ.
Now, for mortals like myself, that would be the end of the review process in PSL.
But no, not here, I am guessing because the lead senior author has the ability to throw his weight around, or something.
Let’s recap. After two rounds of review and Referee 3’s tie-breaker recommendation, basically 2 people are saying transfer the paper to RSJ and 1 is saying it’s OK for PSL.
I have had several papers in the same situation and this was always, 100% of the time, the end of it. If it were my paper, I would have to transfer.
I have also had papers in PSL where 2 referees liked the paper but one didn’t, and it was still rejected more than 80% of the time, with words such as “the reviews have failed to converge after two rounds of review.” (Occasionally I appealed, but the appeal always found in favor of the original editorial decision. These days I don’t waste my time and go elsewhere.)
What happens next is infuriating. The editor apparently didn’t like Referee 3’s report. The next day, the paper gets sent out to Referee 4, who eventually says the paper is OK, that the authors had responded to my comments adequately, and the paper should be accepted in PSL. The editor happily obliges.
Let’s summarize what we have learned about the peer review process based on this anecdote:
- If you are deemed a big-enough name, your paper gets reviewed in prestigious venues, bringing in as many new referees as needed through as many rounds of review as needed until the number of those who like the paper is at least equal to the number of those who don’t, at which point the paper is accepted.
- If you are plebs, all the referees have to be convinced the paper is great in no more than two rounds of review, otherwise the paper is rejected.
Not cool, PSL, not cool. I review a ton for you, but it seems that I should stop wasting my time.
xykademiqz 
I have had a few bad experiences with PSL in last couple of years.
Btw, did you make a complaint to the editor? Not about the decision, but the fact that you were never shown the corrections and the response to the comments that you spent so much thinking about.
If this goes on for far too long, I will be inclined to give a very short review, basically either up or down.
This is not the first time my review has been overturned on account of a heavy-hitter author (never for a no-name author, mind you). It’s a waste of my time, but, you know, I am not a heavy hitter, so my time must not be very valuable anyway.
I recently got asked to review a paper from PSL from a hotshot lab that had requested some sort of “rush review”. The editor told me this is a very important paper (or something like that) and asked me, with a straight face, to return the review within 4 days! Considering it made very (in retrospect overly) bold claims in a hot, competitive field I knew I would never get a good read in by then so turned it down.
The most annoying thing about this is that apparently this was the third journal the m/s had been submitted to (after Nature and Nature Physics), so if they had the time to mess around with all these different journals why couldn’t they give me the standard 2 weeks to review!
I’ve come to expect this sort of editorialism from Nature/Nature Physics (where more than once my review invitation email contained language that suggested they wanted me to accept quickly). But I though PSL was safe. Oh well.
PSL? I would think that pumpkin spice latte is only for the most basic papers.
🙂
Did you write to the editor? Maybe he/she needs to feel some pressure, even if it’s not immediately effective. To make it harder for him/her to do such things next time.
These kinds of posts are always bemusing to me. Astronomy is largely immune to paper rejection. The Astrophysical Journal publishes almost everything and if it doesn’t you can go to AJ, MNRAS, A&A, or in a pinch PASP (which approved our last paper in 12 hours). There are publications in Nature and Science but they don’t seem to have as much cachet as in other fields (especially when we hear tales of Nature requiring that authors tout a 2 sigma result to punch up the impact of their work).
Obviously, there are down sides to a system that rejects few papers, but at least we don’t have gatekeepers who blatantly favor big names when it comes to publications.
Wow. I did not think that this was happening at PSL. I review for them often and they have been really stingy about accepting papers in the last few years. If I really like the paper, I have to write an overly inflated review (kind of like the student recommendations) in order to give it a chance. I am dying to know who the editor was!
I had a paper rejected not long ago with three referees recommending it and one saying to send it to RSJ. Clearly I am not a heavy hitter.
they have been really stingy about accepting papers in the last few years. If I really like the paper, I have to write an overly inflated review
Oh, totally. It happened several times that I submit a detailed and positive review, but the editor comes back to ask me to argue further why the paper must be published in PSL (and not RSJ, for instance); i.e., it’s not enough to review, I now have to actually champion the paper. But I guess this too holds only if the authors are not big names.
Econ is like this too, though even worse we don’t have double-blind review, only single blind. But also there’s a huge amount of gate-keeping at the desk-reject level.
Now that I’m editing, I definitely feel a lot of these pressures myself but thus far I’ve been pretty good about mainly just looking at the abstract and tables when deciding whether or not to send out for review (as in, do their results support their claims) and then checking for appropriate robustness checks if they’re not in the tables. This is kind of important because people who don’t have the pedigree-training often don’t write their articles in a pedigree-standard format (Cambridge-style or UChicago-style, for example), which sends a signal about the authors’ training but not necessarily about the quality of the work. I’ve noticed reviewers tend to pick up on this too.
I haven’t done an R&R yet as an editor so I can’t say anything about how I will decide to send things back to reviewers. I do know that when I review sometimes I get the revision (sometimes revisions ad infinitum when I think the editor should be able to just deal with minor changes at that point) and sometimes I don’t. The only time I was bummed about not seeing the revision was for a Top Journal for which it was the first time I’d recommended R&R for a top general interest journal and it had actually gotten published. Usually I just forget about the referee report as soon as I send it in.