Thursday 14 February 2013

Judging scientific excellence by the shape of the head

The Rat, 2013.



Hi rats,
At this point, you may be wondering: what the hell...!!!?? 

I can explain everything: being a parasitic animal, I recently joined the ranks of fellowship applicants, and I want to share my experience.

In case the comic is too subtle, I’ll say it in plain English: the final decision about your grant or fellowship is going to be made by non-experts, i.e., individuals who are actually not qualified to evaluate your scientific skills. The success of your application will depend on trifles such as whether you roman-numbered or not your project goals in the Background section of your proposal (veridical). So rather than trying to impress the referees with a creative and coherent research project, you may better put on a white lab coat and type furiously over an illuminated coloured computer keyboard that sounds “teetooteetootee” while a flask containing a green liquid bubbles in the background. You’re doing serious stuff here!

First, about your CV: have you ever been asked to list your ten best publications? Well, don’t make the common mistake of citing your ten favourite papers, because nobody is going to read them anyway; the non-expert referees will go straight for the journal and the number of citations.

Wait! Does that mean that any candidate with a Nature paper on entanglement sudden death will have preference over me!?

Yes, he will.

Incidentally, this reminds me that in the last two months different people have argued that this blog is detrimental for QI, because our community is, oh, so peaceful. By speaking my mind about certain topics, I’m creating a climate of belligerence and unhealthy competitivity that blah, blah, blah.

The thing is, I agree with these people! QI is a paradise... if you hold a permanent position. Because if you don’t, you have to compete for it against candidates with a paper on quantum discord with five hundred citations. So let’s call rubbish “rubbish”.

Coming back to your CV, funding agencies will typically require you to evidence your “capacity to build collaborations”. I don’t understand how this can be relevant, because some of the most dreadful calamities in History started as collaborations between talented individuals, like Glauber and Oppenheimer, or Jackie Chan and Chris Tucker. Fortunately, the non-expert referees can’t understand /don’t care about the quality of such collaborations. So anything fits in this section, like the time I sustained ****** *******berger’s head while he was vomiting in the toilet because he had drunk himself out after being rejected by all the girls in the quantum information workshop. What? Yes, the three of them.

Then, there comes the project. The requirements are clear and contradictory: on one hand, your project must be novel and imaginative. On the other hand, it has to be an extension of your previous results. 

This last demand shows how much science has changed since Richard Hamming gave his inspiring talk You and your research, where he advises researchers to change fields every seven years. Nowadays, you're supposed to work pretty much on the same topic since your graduation day till your retirement. Do you want to change fields? Become a gardener!

As you can see, applying for a scientific grant is infuriating enough to turn Mohandas Ghandi into the incredible Hulk. But the worst is yet to come: read the next paragraph and roar.

In the eve of every referee report deadline, the ghost of Research Results Yet to Come haunts the non-expert referees. Hence they will demand you to detail exactly how you intend to reach your project goals. Now, this may make sense in experimental particle physics, where collaborations involve thousands of people and improvisation is not an option. In theoretical matters, it is patently absurd.

But, absurd or not, proposals like: “I want to prove A, so I will try to prove B, C and D and then connect the results” will be rejected.

“But you are not explaining how to prove B, C and D! How are you going to achieve this?”

“I don’t know! That’s why I need a four-year funding, damn it! To do research and find out!”

Well, that’s not the right attitude. You have to convince the referees that there isn’t the slightest chance that the project is unsuccessful. Two plausible scenarios come to the mind:

a) the project is already finished, and you’re just listing results that you will “come up with” during the next four years;

b) the project is crap.

If we were living in a rational world, in either case the project would not be funded and the candidate would be sent to forced labour in Siberia. Unfortunately, we live in Bureaucracyland, where opinions must be objective, and the only way to get funding for a project is to finish it before the application deadline.

Finally, there comes the part of “national interests”. This is where you argue that what you’re doing is essential for the survival of the human species. Of course it isn’t, but anyone else is claiming so: if you want to stand a chance, I suggest you to claim it too. I pity those poor researchers who waste their time investigating banalities like cancer or global warming! For the same price, they could be working in something really important, like topological quantum error correction... or at least that is the impression that I get when I read certain grant applications on topological quantum error correction.

In this section, I would suggest the funding agencies to demand responsibility: “you said that your research would improve the French economy! Two years later, and government bonds are cheaper than ever! To the guillotine!”.

CONCLUSION

My previous posts have been criticized for being non-constructive (ironically, because that is also a non-constructive criticism). Nevertheless, today I want to finish with a positive note, because I believe that there is a fairer method to decide grants.

It is quite simple: if a grant is to be evaluated by, say, four referees, two experts and two non-experts, the latter should not have access to the candidate’s name, the CV or the detailed project proposal.

They should not have access to the candidate’s CV, because they are not qualified to emit judgement about his/her research capacity in an area which is alien to them. And idem with the project. If you let them have this information, eventually they will come up with a collection of arbitrary criteria to assess the proposal, like, e.g., the number of prizes awarded during the completion of the PhD studies. Those prizes do not exist in many countries; using such a criterion to evaluate international candidates is xenophobic at the very least. Likewise, any other “trick” they may distil from their long experience rejecting promising grant applications could be inadvertently racist or sexist. If you are a non-expert referee and intend to decide the outcome of a fellowship on similar grounds, let me ask you to consider tossing a coin, asking a spirit board or reading the entrails of a sacrificed chicken. At least that way you won’t be introducing systematic errors every time you’re asked to evaluate a bloody grant proposal!

And yet, there is a legitimate role for non-experts in the refereeing process.

Sometimes, certain research communities lose contact with reality and engage themselves in senseless (and endless) scientific production. It is then in the hands of a field outsider to set things right: the job of a non-expert should be to decide if the proposed project’s goals would suppose a benefit for society.

The task is easy: if the candidate states in his project description that “the aim of this project is to create an army of giant singing oysters to conquer Honolulu”, don’t fund him. If, on the contrary, the project description reads: “we will develop a flying voodoo doll that can communicate with pineapples”, accept the damn application and let the experts decide whether it is feasible or not!

Yours truly,

The Rat

15 comments:

  1. Dear Rat,

    please do not be constructive, we really miss the old style! More offences and truths please.

    ReplyDelete
    Replies
    1. Aha! You like blood, eh?

      Maybe in the next post...

      Delete
    2. ok. When is the "road to QIP" coming? The first installment was so good that it deserves a follow up!

      Delete
    3. The thing is, I said pretty much what I wanted to say in that post. More would be repetition...

      Delete
  2. Miguel, you might find it instructive to go to your library and read some bound copies of Scientific American from the year 1957.

    In paging through these volumes, it's evident that the scientific explicandum of the 1950s was vibrant and the job market (both public and private) was vibrant too.

    As for quality of scientific peer review, it is scarcely likely to have been any better in 1957, than it is at present, eh?

    Conclusion  If there is a deficiency in the vitality of the STEM culture of the early 21st century, relative to the vitality of the STEM culture of 1950s, it is more plausibly a deficiency of explicanda than a deficiency of peer review.

    ReplyDelete
    Replies
    1. … and for a thoughtful depiction of a future in which the explicanda of quantum information science are not markedly augmented, see Larry Cebula's No, You Cannot be a Professor and its sequel No, You Cannot be a Professor — the Reactions.

      It is prudent to be wary of Professor Quantum Sparkle-Pony! :)

      Delete
    2. Hi John,

      Perhaps peer review was as crappy in 1957 as it is nowadays. However, at that time it wasn't that relevant, at least for theoretical researchers. Actually, I'm not sure that even the concept of postdoc existed in 1957: my impression is that, in those golden times, one was granted a faculty position at the end of the PhD studies. Then it was not necessary to get a grant every O(1) years in order to pay the rent, etc.; rather, one could research whatever one felt like it.

      Back to the present, I would like to follow Richard Hamming's advice, i.e., leave nonlocality for a while and work on something else, say, quantum optics.

      But, alas, I cannot! If I stop my research in nonlocality and dedicate all my energies to quantum optics, I may not publish in a year or more, and so I will have troubles finding a postdoc when my current contract expires.

      The natural option would be to find a 3-5 year grant. However, no matter how vibrant my grant proposal is (and believe me, I could make it very vibrant for certain project I have been thinking about lately), I will never get funded, because I have no publications in quantum optics.

      I wished that referees read "You and your research" and gave me a chance, because it is clear that what I learned about quantum mechanics from my work on nonlocality can be applied to quantum optics as well. This won't happen, though, because for non-expert referees these two topics are worlds apart.

      So what I have been doing in the last two years is to research those other topics in my spare time. This has brought me a lot of publications in nonlocality and just three or four papers on unrelated areas. Definitely not enough volume to apply for a grant on any of these other fields.

      Conclusion:

      I find Richard Hamming's advice very healthy for group leaders, and a career suicide for postdocs. And maybe peer review used to be as intolerant with career discontinuities as it is today, but I don't see the connection with the way scientific results are explained.

      P.S.: on a more positive note, I liked a lot "you and your research" and "no, you cannot be a Professor". Please, keep posting wonderful links!

      Delete
    3. Dear Miguel: I wish I had more specific advice than Andy Grove's "You never get well on your old products" (asserted in regard to Intel's transition from memory chips to processor chips).

      Similarly, perhaps it's implausible to foresee that the QIT academic community will "get well" — in regard to an abundant supply family-supporting jobs and/or viable academic careers for young researchers — on its "old products" of quantum computers, quantum cryptography, and entanglement measures.

      What are the "new products" on which the future of QIT depends? Heck, isn't that what people read Schroedinger's Rat to find out? You tell us!

      Delete
  3. Dear Rat,
    Are you telling us that it isn't all about including the words "harness the power of quantum" in your proposal? Is there another masonic sign of recognition?

    ReplyDelete
    Replies
    1. HA! I knew that, if I waited long enough, some initiate would reveal the secret words!

      My previous attempts at guessing the code: "please", "please pretty please", "please pretty please with sugar on top" and "give me the grant or I will kill your offspring" had mixed results.

      Thanks, Lidia. I hope that the Grant Masters are merciful and spare your life.

      Delete
  4. Miguel, estas enojando y alienando a todos los politicos de la ciencia. Rezar 3 Padre Nuestros y 2 Ave Marias de penitencia, y parar este blog lo mas pronto posible o Dios te va a castigar.

    ReplyDelete
  5. Hi man,

    I hope you don't get a job! You should't take offence to this though as I'm not insulting you, I'm simply speaking my mind.

    ReplyDelete
  6. This blog post is also about the future of university teaching careers
    My Prediction For 2013: “Professor, I’ll have some fries with that exam”

    ReplyDelete
  7. For some reason(s), I like most of the told. Pitty, as I'm willing to spit on something. :)

    ReplyDelete
  8. "The notion of a quantum state encapsulates what is known of the preparation of a system. An unknown quantum state is self-contradictory, an oxymoron, just as is a "research proposal." Enrico Fermi said that when there is no surprise, it’s not research".
    Asher Peres (2004) "What is actually teleported?"

    ReplyDelete