Author: xykademiqz

On Becoming a Good Peer Reviewer

In my role as associate editor of a specialty journal, in the recent week or so I’ve been given, over and over, the crappiest of gifts: that two-line nonreview review, accompanied by a recommendation to reject. Some of these came from people who really should know better. These reviews are worthless. I cannot use them, because  they contain no information on which I could base my decision. I cannot reject and forward these non-reviews, because they are callous toward the authors who put a lot of work into  their manuscripts, and because, if I were to use them and forward them, I would show myself to be callous, too.

The question is: How does one learn to write useful referee reports/reviews of the technical work of others? (Other than being an editor, I also received recognition as an outstanding referee from a professional society, so I’d like to think I know a little bit about the topic.)

My students get training on how to write referee reports. After they’ve published a paper or two, they will get solicited for reviews whether they know what they’re doing or not, so it’s my job to make sure they do a service (rather than a disservice) to the scientific enterprise. First, I share (through the group’s password-protected document repository) a few of my own reviews with group members, which they can use as a style and structure guide. We also do joint reviews — certain society journals have this option, in particular for a supervised co-review with a (named!) junior person, whereupon the junior person also gets added to the reviewer base. I might ask a student to supply an additional review on papers where I am associate editor and the topic is in their wheelhouse.  I have never, ever asked a student to do an uncredited review instead of me: either we co-review or I decline and recommend them as a reviewer, but they always get properly credited for their work.

Before I proceed, a couple of disclaimers. In my field, there is only single-blind review (the referee is not known to the authors, but the authors are known to the referee). In most journals I referee for, the referee is explicitly asked to recommend a course of action (e.g., reject, resubmit with minor/major revisions, accept as is, transfer to another journal, and sometimes other finer steps in between); these are advisory to the editor, who can ultimately do what they like. If your field or subfield is not like that, for example if you’re actually forbidden from making even a hint of recommendation, please do not assume that all fields are like yours or that I don’t know what I am talking about.

One part of the training is helping students understand what it is that they should be recommending based on the report. Are the edits minor, but mandatory? Are they minor, but optional? Then say so. Can you envision the authors making certain edits that would eventually make you happy to recommend publication? Then tell them clearly what they need to do, even if it’s major, and don’t recommend rejection, but major revisions. Is there something in the paper that’s a complete deal breaker, so that you cannot envision how it would become appropriate for the journal without being a completely different paper? That’s a rejection. For example, the paper is not novel (all they claim to have done new had already been done by others, whom they didn’t cite); the paper is wrong or fraudulent or grossly misleading; the paper is poorly aligned with what the journal publishes (e.g., outside scope; too specialized for a generalist journal).

This is how I recommend to structure a review:

a) Summarize the paper in one to two sentences. If the paper were a screenplay, this would be the logline. You’re letting the authors and the editor know what you think the paper is about.

b) A few sentences regarding your general impressions of the paper. Not just the bad; the good, too. For instance, if it’s was written well or if it’s an engaging read, say so. If it’s an interesting topic, say so. If it is timely, say so. If it is a paper on the topic that had been beaten to death 10 or 50 years ago, say so.

c) A clear recommendation and the general reason for the recommendation, which should flow out of point b).

Btw, points a-c are usually one, maybe two paragraphs total. Not very long if you know what you are doing.

d) The specific points (with detailed arguments and references!) that led to your recommendation. I usually number these, and many referees do, so they’re easier to refer to during revision. I think of this list as a contract between the referee and the authors. If the authors do a good job of addressing these specific requests, you should be willing to recommend publication. I am royally pissed with referees who move these  goalposts. We had one like that a few years ago. We’d address all the many, many minutiae, only to have the referee come back asking for more and new stuff. It was maddening.

Doing d) properly takes time, but this is the heart of the review, and needs to be done properly for both the authors’ and the referee’s sake. In recent months, I have had the misfortune of reviewing several manuscripts where the authors pretended whole subfields didn’t exist and they didn’t cite anyone, presumably out of ignorance, but there’s also a nonzero chance it was on purpose, in order to elevate the perceived novelty of their work. In one notable example, it took me weeks to write the review, first because I was too pissed to write it, second because I had to sit down and look up key references to show them what they were missing. I explicitly had to say it was not an exhaustive list, that it was their job to do a literature review, but I pointed them toward where they could find out more.

The style of the review should be polite and matter-of-fact; being blunt is fine, but taking jabs, especially ad hominem, is not. I occasionally catch myself being snarky and have to drop the review until I can write more dispassionately. Don’t be mean or snarky. There are always junior scientists who poured their heart and soul into the paper. Do not be cruel; it can crush a young person’s spirit and contribute to them leaving science. The editors who let mean-spirited reviews through (presumably from untouchable giants in the field) are also to blame.

Miscellaneous: Do not be a pronoun jerk. When reviewing single-author papers, it’s easy to use “the author” and never even use a pronoun. It’s inclusive, avoids misgendering people, and prevents you from appearing blatantly sexist. (Based on my single-author-paper days and even today, when I submit solo-PI proposals, there’s always someone who relishes a bit too much in using the female pronouns to tell a female author that she sucks or that her work is garbage.)

Also, when you respond to referee reports, use “Referee A/1” and “they” because you have no idea who reviewed. While I’m used to being referred to as a he, presumably because my reviews display such kickassery and competence that no one can fathom them having sprouted from a feeble lady brain,  I am definitely pleased when the male gender is not assumed.

Academic blogosphere, what are your thoughts on becoming a good referee and on the peer-review process in general? 

Reader Question: To Pseud or Not to Pseud?

Dear fellow readers,

I am a STEM professor who decided to pour my frustrations into an essay. It’s written in a personal style, shaking my fist at the heavens about students who have never developed good skills in high school math and yet refuse to switch from a highly quantitative field to something else. I have tenure and I am a member of the union, but I am also at a university that is loudly trumpeting messages about Student Success. Also, while I never relate any specific incidents with identifying details, I do note that a lot of colleagues dismiss the notion of steering students elsewhere, and often say “What about diversity?” when I say that some students just don’t have the mathematical skills. (Note that many of these floundering students are white males from suburbia. The students whom I wish to steer elsewhere are less diverse and more privileged than my colleagues need to claim in order to fashion their stance into some semblance of morality.)

Would it be insanity to publish this under my real name?

— Blog Reader

Ba-dink-a Link

Wait, you thought I was done with links? BWAHAHAHAHAHA! No.

Link Clink

Lots to write about, but even more to do at work, which means I’ve hit a point of overwhelm at which I don’t feel like doing anything. But the work must be done, so, without further ado, Twitter links!

The BLMoment

I don’t do much politics on this blog, usually because I’m insufficiently informed to write anything particularly illuminating. But the ongoing protests, which I support, must not be ignored. I wanted to avoid performative posting, which seems to permeate social media, but I suppose I’m failing, so at least I’ll keep it brief.

I wish the abuse and murder of Black people would stop. I am White; I am trying to listen/read and learn, and I help out with donations. Battling deeply ingrained systemic racism is an ongoing issue, one that takes empathy, constant vigilance, and continued education.

I don’t have much original to say, and this isn’t about me anyway, so here are some links that might be of interest/use.


And a couple slightly more uplifting links:

Why We Plan (or Not)

The other day, I visited a blog I only occasionally read. It’s written by a lovely person whose approach to organizing time is very different from mine, someone who relies on structure and detailed, elaborate plans. The piece I read was an essay in which the blogger reflected upon the reasons behind planning. The blogger started with the death of a loved one who had lived a life full of “social engagements, travel, art, entertaining, and time with family,” then revealed that their devotion to planning was fueled by a desire to not waste time, to ensure that each small precious moment is spent on things that matter and serve a purpose.

This blogger seems like a really nice person, so I want it to be understood that I absolutely do not wish to throw any shade here. However, this essay, like many of the blogger’s posts, gave me metaphorical hives, because the blogger and I are so, SO different. Everything  described above strikes me as an extrovert’s definition of a satisfying life. An existence this full of social engagements, travel, and family sounds downright exhausting. There are so many people, nonstop people and nonstop movement in this story. There is also the assumption that one is able to perform whatever one planned to when one planned to do it. There is no room for procrastination and no room for pushing back (referred to as “bad attitude”).

Look, I agree that we should be grateful for the time we have. But I feel that focusing on never having a wasted moment is (for the likes of me, at least) just crazy-making. It’s a bit like this: If you ever had a baby, you probably remember the enormous pressure to take advantage of the nap times, all the things you wanted to squeeze into these tiny slots,  which often resulted in accomplishing nothing and feeling like a failure because you wanted to do too much, because the baby slept too little, because the baby slept too long and you reeeeeally should’ve taken advantage of THAT even though you had no crystal ball, because you couldn’t fall asleep yourself, because you couldn’t focus on work… Because, because, because. Detailed planning with the purpose on avoiding time waste gives me anxiety similar to trying (and failing) to make the most of a baby’s nap.

Please, please don’t let anyone tell you that sitting still and being comfortable in your head means you are wasting your life.

To me, unnecessary structure is stifling. Sure, I know there are meetings I have to attend and classes, office hours, and activities with the kids; I know there are deadlines and timeframes for various goals, but, other than that, I feel that a great many things do not, in fact, have to be scheduled, or at least not too rigidly. When it comes to raising kids,  I feel there are no big manufactured moments, just moments. Sure, kids like vacations and adventures, but, to me, the most meaningful, heartwarming moments come from goofing around in the kitchen, chatting in the car on our way back from sports, impromptu silliness while shopping together. Just being around the kids, feeding them and talking to them and doing homework with them and organizing playdates, the stuff of daily life — I am positive that’s what gets built into the kids’ psyche and makes it strong. Elaborate vacations are nice, but not critical. Sure, they make for great pictures, but to me vacations are terribly stressful precisely because so much hinges on not wasting every (expensive) moment.

My biggest priorities are spontaneity and free time in which anything can happen, and I prefer to spend most of this time by myself, inside my head. I have a lot of interests, many of them creative ones, which require solitude. I feel that the focus on people-centric activities and planning misses much of the creative work and the people who engage in it. I can do relatively mindless, superficial work in 15-min chunks between chores; God knows there’s plenty of that type of work to go around. But anything deep, science or art, requires focus and doesn’t always work on demand regardless of how upset with myself and my own bad attitude I might get.

In order to have my time best aligned with the things that I find meaningful — which include spontaneous moments of deep connection with my kids and a small number of adults who kinda sorta get me, as well as both the consumption and creation of science and art and all the crazy connections between disparate concepts and images that stem from those — I absolutely must NOT plan anything that can be left unplanned. Instead, I need to leave as much time as possible free for solitude, for the intermixing of the subconscious and the conscious and all the goodness that arises from a lack of structure.

What say you, blogosphere? What drives you to plan or not plan? 

Blink, Miss a Link

Link to the Brink

Haven’t forgotten about the blog and hope to be back with more frequent posts, as I promised when the quarantine started. But, for now, I have tons of links, several days worth. So, without further ado:

Hugs and Trips

qwinne’s post made me think. I don’t believe I have ever hugged a student. If a colleague goes in for a hug, I won’t avoid it, but I definitely won’t initiate it. I can’t recall if I’ve ever hugged my now-80-y.o. former mentor. Maybe? Probably not. I am not a touchy/huggy  person, in general. I don’t like to hug friends or most members of my ancestral family. (Again, I don’t avoid hugs, but anyone who’s paying attention will notice me stiffen and I never initialize any form of social touch, except maybe a handshake. There’s on-the-cheeks kissing as a form of greeting in my ancestral culture, which is just disgusting, and I thankfully don’t have to suffer through that anymore.)

However, I am very affectionate as a parent. I love LOVE hugging my kids and husband (and, before I was married, my boyfriends), but other that a couple of select family members (no, my parents are not among them), I don’t want anyone breaching the perimeter of my personal space.

Blogosphere, are you as prickly as me? Pricklier or cuddlier? How do you feel about (non-creepy and contextually appropriate, such as greetings or congratulations) hugs within a professional context?

A biennial conference in my field was to be held in Asia in 2021 in a picturesque but very remote and expensive touristy location. Amid Covid concerns, we discussed moving it to a cheaper and more accessible location that provides the organizers with added flexibility in terms of moving dates or going virtual, if need be. Yet, some members of the advisory board brought up the alternative location’s lack of picturesqueness as an issue. 

Even in the best of times (and we won’t have the best of times back for a long while), I hate HATE complicated conference travel. I don’t have the time or the energy to spend two days traveling inland via numerous transportation mechanisms to someplace pretty, only to be jetlagged in the conference hotel the whole time. I know the older (and richer) members of the community fly out several days early, bring their spouses, get over jetleg and relax, all before the conference starts. I don’t have the time, money, or will to do that. I have never wanted to attach vacations to my work. I want my work travel to be to places that are easily accessible; I want the hotel close to the venue; I want the presentation rooms to have comfortable temperature and seating, as well as enough outlets and Wi-Fi bandwidth; I also want the technical program to be enticing. These are the things I want; I am not going on vacation, so I don’t need or want the surroundings to be pretty. I want my vacations to be decoupled from my professional life, but I may be in the minority.

Blogosphere, how much do you care about location when you travel for work? Do you append vacations to conferences? 

Crisis vs Crisis

During these unsettling times, I find the self-indulgence of a midlife crisis to be almost comforting in its relative normalcy. It feels weirdly soothing to be pissed with the world because I’m no longer 20, because there’s so much I want to learn and do and be, yet  there just isn’t enough time or freedom to do it all. I need several lifetimes for what I want but I don’t get to have them, and getting myself properly fuckin’ infuriated over it is as good of an antidote as any for the low-level dread emanating from the internet.

Anyway, here are some comments I left over at Clarissa’s in her recent midlife-crisis thread.

A number of people IRL change careers or add a second career in the middle age, so it’s not just about the personal life, although it can be. I have been in the throes of a raging midlife crisis for several years, with no end in sight. I think the hallmark of a midlife crisis is the question “Seriously, this is all there is?” or “I’ve done everything I’ve set out to do. Now what?”Basically, re-evaluating of priorities and crafting a new set of goals that no longer hinge heavily on education or reproduction, but tend to be better aligned with a mature individual’s core values and their true passions (many of which were neglected since childhood in the pursuit of a lucrative career and/or mating prospects).

…there is research showing that a vast majority of people in the developed or semi-developed world do go through some form of midlife crisis. There’s psychoanalysis work dating all the way back to Jung and Freud on why it happens and why it’s developmentally appropriate.

So you did actually have a midlife crisis, but, since you were unencumbered by familial relationships, you could just leave once you decided you wanted to. Most people in their forties or thereabout cannot. I am the primary breadwinner and will be 60 when my youngest finishes college. I would love to just leave everything behind and go try to make a living as a screenwriter or move to Australia to do whatever or half a dozen other options, but I cannot, because there are people who depend on me financially and emotionally. Most folks have similar restraints. The midlife crisis stems from the conflict between the responsibilities of adulthood and the unfulfilled desires of youth, which comes to a head once a person has reached the end of the blueprint (education, mate selection/family, professional ascent).