So, today I received N (N=125^(1/3)) detailed written reviews of my competitive grant renewal (mail-in review). N-1 are positive, from completely glowing to enthusiastic but with some specific requests for clarifications.
The Nth is negative, as in — there is nothing new here and the proposal is just awfully written and diffuse and should have been narrower in focus because the PI attempts to do too much (not sure how the scope can be too broad if there’s nothing new, but whatever). The lack of novelty is detailed as this one specific group somewhere in Europe doing something related, so apparently no one anywhere else in the world gets to do anything similar. Another aspect is that there are all these well-known techniques, so why doesn’t the author use them? Because those techniques give A and I am after B, for which there are currently no techniques, hence I propose to develop one.
I don’t know if men ever get reviews like this one, but I do receive them with some regularity, for both grants (more often) and papers (less often). Basically, the reviewer has decided that he (likely he, from the statistics (few women in the field) and the general know-it-all tone) doesn’t like what I’m selling without necessarily investigating what that is. The lack of novelty is decided based on some vague idea that, somewhere else, someone more respectable and/or reputable than me (i.e., someone the reviewer knows) says they have done something that sort of sounds like what I say I’d do, so the more respectable and/or reputable individual must have completely solved the problem for all eternity and there’s no need to look into any of it by anyone ever again (in reality, the person may have published a paper or two to scratch the surface of the problem).
If there is something unclear, especially in a proposal, I am not given the benefit of the doubt that I do know what I am doing. The assumption is never competent until proven otherwise. The default is that I can’t possibly know what I am doing, that whatever I proposed couldn’t possibly be new or fulfill a legitimate open need. You know, if the problem were really as interesting and important and unsolved, as I claim, someone more worthy somewhere else would have thought of solving it already.
There is no such thing as a perfect proposal. This benefit of the doubt some people are given and others are not (despite high productivity with earlier funding) is the difference between getting funded and not. This is why, with another agency, I was recommended for funding but just below the funding line at least three times (as bloggy friend Alex says, “I’m the best of the worst”): When the funding is scarce and there is only money to fund 1-2 grants, you give the funds to those you trust will do a great job, and I am just not the most trustworthy of the lot. “But her emails!” must have been the grant reviewers’ thoughts.
Such bullshit, seriously.
xykademiqz 
This is why I’ve given up on working in research: I would very clearly explain what I was doing and why I was using technique x. Reviewer: “But why is she doing technique x?” I got the impression they never even read them…just looked at the institution of origin and threw them out. Except, there was one reviewer that noticed my husband and I were both on a grant (last names) and made some obnoxious comments.
It’s totally true that women get dinged for exactly the reasons you point out. The same judgments are made when reviewers see you (female OR male) are not from an R1 school (like the previous comment… “see the institution of origin and throw it out”), or that you are only an assistant or associate professor, at which point you are assumed to not know what you are doing. Or they can tell from your CV that you’re not a purist in that discipline, you’re an interdisciplinary researcher and thus must be an interloper into their sacred field and cannot be relied to know what you’re doing as compared to someone who got all their degrees in (physics, chemistry, whatever). Despite top publications in that area. I’d seen things in proposal reviews early in my career that really signalled that. Then in the last few years sitting on some review panels others were blatant about that (especially, ‘the PI doesn’t have tenure so can’t be trusted to truly be a Respectable Scientist…”- WTF?)! Gaaaah!!!! I pushed back hard…
Passing judgment based on stereotypes, or hierarchies alone, sucks.
In an ideal world, the reviews should be double blind, with some weightage given to previous work in a transparent manner that is independent of the proposal.
Well, I don’t know whether women get dinged this way on reviews more than men, but I am a mentor to many of my younger colleagues in my department, Having seen lots of reviews of colleagues (and seeing my reviews), this is a pretty common experience for both men and women. I think this is pretty much par for the course.
It’s frustrating, I agree. While there are lots of problems that are gender-biased in science (and we’d like to fix them, I agree), the statement that “the reviewer didn’t read my grant well, provided only vague complaints, and didn’t give me credit for knowing what I am doing, even though I’ve published extensively on it” is a pretty common occurrence and doesn’t seem to be obviously gendered in its application.
My take is that this comes from too many grants, too few reviewers, and not enough time to do it right. (And maybe not enough incentive to do it right?)
Of course, YMMV.
Knowing that these experiences would await me if I succeed far beyond my wildest dreams is one big reason why I pursued an industry job after PhD, and am glad I did, despite industry science having its own drawbacks.
Do you think there is a systemic change that would at least partly address this problem of biased reviewers and what feels like a not very meritocratic system? Like, would this be less of a problem if there was just more funding?
I think if there were more money around, things would be somewhat better. More close-but-no-cigar grants would be funded. Crappy reviewers would likely still be crappy, but maybe some of the good ones wouldn’t spend all their time chasing money (cause it would be easier to get some) and would agree to review more often.
qaz: I think this stereotype of incompetence affects women and minorities, definitely more than men overall and likely somewhat more than even just nonsuperstar men. A woman is either a veritable superstar (think a household name) or presumed incompetent , while for men there’s more recognition of a continuum of abilities (superstars, serious nonflashy scientists, various tiers of stereotypically ‘lesser’ scientists based on workplace, pedigree, purity of field as GoG said above, etc. ). I am sure everyone gets crappy reviews, but I would bet good money that the fraction of off-hand, summarily dismissive ones is higher for women, and I bet that it’s considerably higher for a nonsuperstar woman with a given seniority, publication record, and school rank than for a nonsuperstar man of similar professional standing.
This brings us to the most frustrating aspect of STEMming while female: you are never really sure if the crap you’re dealing with is gendered, because a lot of it is.
I know that my students who start out wanting to go to academia no longer want to do so by the time they’re done, because they don’t want to chase grants.
I don’t think that just throwing more money at the problem would solve it, at least not for long. That is essentially what NIH did, which resulted in huge numbers of postdocs being hired, turning into a huge pool of highly trained researchers, who ended up spending all their time writing grants to maintain soft-money jobs. Unless the money increases exponentially, the growth rate of trained researchers quickly surpasses the pool of money.
The growth of the pool of researchers has to be limited also (perhaps by not allowing postdocs to be funded by Federal funds?). Some fields where there is substantial industrial demand (currently, computer science) have limited the growth of unhired researchers, but unless the supply and demand are perfectly tuned, the problem then is that faculty can’t be found to teach courses, because industry pays better for easier work.
The line “best of the worst” reminds me of a line from They Might Be Giants:
“We were once so close to heaven,
Peter came out and gave us medals,
Declaring us the nicest of the damned.”
In terms of the money solution, we should remember that in the days when one could survive on a single NIH grant (because you were essentially guaranteed to get renewed if you were doing basically good work – which is the level that we would need to be to accommodate those reviews), we were putting 12% of GDP into science. (We put less than 3% in today.) Importantly, this would need to go beyond just NIH, it would need to be in all fields, including NSF, NASA, DOE, DOD, USDA, as well as NIH and CDC.
Importantly, the infamous NIH “doubling” was actually small and was washed out inflation-wise within a few years.
One of the things I always like to point out is that the real “moon-shot” cost $115 billion in today’s dollars. So when we say we’re going to put $100 million into a neuroscience or cancer “moon-shot”, we’re off by a factor of 1000. The Manhattan project cost $28 billion in today’s dollars.
In my extensive experience mentoring both male and female junior colleagues on their grant applications, women have their expertise questioned by reviewers substantially more frequently and more intensely than men.
I always go out of my way to make sure I put in all the evidence that I can do what I say I can do because I’m a relatively junior female PI. I would hope that that crap would stop once I get to be more senior, though. Older male PIs frequently give me feedback that what I say “should go without saying” or “is just obvious”. They have obviously never received the “she maybe cannot do it on her own without her PhD/postdoc supervisors” feedback.