This one was inspired by a recent conversation in my group meeting.

Generally, the outcome of a review of a manuscript in the physical sciences is one of the following options (I am sure it’s basically the same in the biological and social sciences, and maybe even in the humanities, but I have no direct experience):

a) Accept as is

b) Reconsider with minor revisions

c) Reconsider with major revisions

d) Reject

As a referee, you will be asked to submit a report (to be transmitted to the authors and thus to be written in a collegial manner) along with a recommendation to the editor on the course of action. The recommended course of action is one of the options a) to d). Some journals offer further recommendation subtypes, such as “Accept with optional revisions” vs “Reconsider with mandatory revisions (minor)” vs “Reconsider with mandatory revisions (major)” vs “Reject and recommend transfer to another journal”.  Some have additional options for the referee, such as “I don’t need to see the paper again” or “I need to see the paper again.” But these are all nuances.

Reviews (report plus recommendation) are advisory to the editor. Again, reviews are advisory to the editor. Whether the review is positive or negative, write it so that the editor can understand what has governed your recommendation.

Each referee’s recommendation after a round of review is somewhere between a) and d). Based on these recommendation, the editor makes a  single decision between a) and d) — to accept as is, to invite resubmission with minor or major revisions, or to reject. This single decision is communicated to the authors. The authors will also see the referee reports, but may or may not see the recommendations of individual referees (depending on the field culture and journal), but the authors can usually tell from the content of the reports what each referee recommended. We focus on referee reports and the associated recommendations here.

When do you recommend “Accept as is”?

Option a), “Accept as is,” is usually not recommended by any referee after the first review unless the referee doesn’t give a toss. (It’s okay to “Accept as is” after the first or second revision.) As an associate editor, when I get an adulatory but shallow one-liner after the first review, “This paper is great, publish as is,” I roll my eyes. Such a report is completely useless. It offers me no advice, other than the advice that you as a reviewer didn’t take your job very seriously. Don’t be that reviewer. If you like a paper, your one-liner will not help against a scathing three-page report of another referee who hated the manuscript. If you really like the paper, give the authors something they can use to fight for it.

When do you recommend “Reconsider/accept with minor revisions”?

When you generally like the paper and its conclusion. You think the study is correct, the figures are clear, the conclusions are supported by the data, and the paper is written well. You were able to follow what they did and how, and you have enough information to determine that the technique is appropriate and correctly applied. The minor revisions are usually: missing relevant references (a small number), minor instances of unfortunate wording, some minor tangents that would be interesting to address as they link the paper to the broader field in a way the authors didn’t consider, clarifications in the title or abstract or intro, other clarifications of specific pieces or wording or details in the technique (experimental conditions, theoretical parameters), minor corrections to the figures (e.g., recommendation to choose different colors for better contrast in a 3D plot). Basically, the paper would not be awful to be published “as is” but it could be improved to full awesomeness with edits that are not overly time consuming.

When do you choose “Reconsider with major revisions” vs “Reject”?

Is there plagiarism/duplication of work? If yes, reject, and provide references where the overlapping work has appeared.

Is the paper topically inappropriate for the journal? If yes, then reject, and explain briefly why it doesn’t fit (these are often caught by the editors, so the paper is desk-rejected).

Is the paper not hot enough for the highfalutin journal? If the answer is affirmative, then reject, but please please explain why you think so. A negative one-line review is just as useless as a positive one. The editor can’t do much with your “gut feeling” that the paper is not cool enough for the journal, especially if that’s your only reason to reject the paper. (Unfortunately, what the gut of famous Prof. Greybeard has to say seems to have more weight than the opinions of younger guts). Your gut feeling should in principle be translatable into human speech, such as: all the references are old and there are no new ones, so this work is not timely enough for this journal; most of the references, especially recent ones and/or the ones with similar work, have been published in this other journal instead; the results are straightforward extension of published work and thus of very limited novelty; the results require unrealistic parameters or only occur under a very narrow set of conditions and are thus likely not robust, etc. [see comments for differences among fields].

Are the methods without a doubt inappropriate to address the problem at hand? Then reject. But if the method is one of several and is just not what you would use, that’s not a good enough reason alone to reject the work. Different methods have different strengths and often reveal different facets of the same phenomenon.

Now we come to the tough region.

Is the paper correct? Are the methods appropriate? Is it timely? Is it interesting? Does it present something novel about the world that is not obvious?

If the answer to all these questions is yes, then ask yourself if you can envision this paper being edited so as to become publishable. What would the authors have to do, specifically, to make it suitable for publication?

Does the language need considerable attention? Is the discussion of the techniques/methods unclear? Are the conclusions unclear? Can you write down what specifically is unclear?

If the answer is that you just hate all of the paper, that it’s boring or just awfully written, or that the necessary chances are comprehensive, pervasive in every aspect, then please reject outright. Try to to explain that the paper is far from publishable and that you cannot imagine it becoming publishable within the span of 1-2 revisions; that it would essentially have to become a completely different paper instead because of simultaneous issues with presentation, conclusions, figures, etc. It is much better to reject outright than to 1) torture yourself to try to list all the things that are wrong, 2) make the authors spend a lot of time entering those edits, only to 3) find out that even after all these edits you still think the paper is awful. Rejecting a paper because of pervasive issues is a kindness. For instance, imagine if you were to submit a first draft of a paper written by a second-year graduate student. These drafts usually require extensive edits and the advisor has to make several (many?) layers of corrections in order for it to become suitable to unleash upon the world. Similarly, there is no point in wasting the time of multiple referees “editing by peer review” something that’s as far from publishable as an early draft of a newbie student.

So, when do you say, “Reconsider with major revisions”? When you can envision a finite number of specific things that the paper needs in order to become publishable. Imagine receiving the paper with those revisions perfectly incorporated; if you would then have no problem accepting the paper, then that’s major revisions. Major revisions usually include: significant gaps in cited literature; missing data/figure(s) in order to support a conclusion; missing critical information that prevents a reader from following the exposition or assessing the correctness of the approach; poorly written abstract or conclusion.

How do you write a useful referee report? 

Start with a 2–3-sentence-long paragraph (Hyphen happy! Technically, the first one is a dash.) in which you state, in your own words, what the paper is about, how the authors do what they do, and what the main findings are. This helps show the authors and the editor that you have understood the paper.

Then say clearly, in a single-sentence paragraph,  what your position on the fate of the paper is. Do you feel it’s generally great, but have minor suggestions for improvement or minor but required edits? Do you think it’s inappropriate for publication in the present form, but expect it to become publishable if the authors satisfactorily address the specific problems outlined below? Or do you think the paper is simply not appropriate for publication in This Journal for reasons that are deal-breakers, and concisely explained?

If your are disposed towards rejecting, make sure you state why in a few sentences or a couple of numbered items/paragraphs.

If there are major issues with the paper, give a numbered list of major issues that the authors should address. Be specific about what you want them to do. Remember, if you are a good referee, this should be like a contract: if they do what you ask, you will recommend acceptance. Don’t be that douche who keeps moving the target and asking for new and varied things in subsequent reviews. Follow with a list of minor concerns, like the typos you caught, unfortunate wording, missing units, etc.

If you have identified minor or optional revisions, list them also in a numbered list. If something is optional to consider and you do not require that the authors comply, but just to seriously consider it, then say so.

Happy reviewing!

(See the comments for some differences between fields. Students and postdocs: your own advisor is your best guide regarding established refereeing practices in your field.)