30 May 2012

Peer Review

I have a suspicion that the people in public debate who treat peer review as the gold standard of science have rarely seen the sausage being made.

Peer review is more than a bit like a jury trial. Considering the alternatives, I'd prefer juries settle questions of guilt. But that doesn't make them infallible. Claiming a scientific study must be right because it appeared in a peer reviewed publication is just like claiming someone is guilty because they were found guilty by a jury. Both juries and peer review are indications of truth but they are not Truth.

Juries make mistakes. Far more than we care to recognize. Similarly peer reviewers make many mistakes. Especially since their job isn't to check to see if a submission is right, merely reasonable and interesting. They aren't answering "is this study true?" they're answering "is this worth sharing with the readers of this publication?"

I will now return to addressing the teeth-grindingly obtuse criticisms of the reviewers on my most recent paper. (In the name of St. Turing, how the hell was I supposed to know to cite a paper published in 2012 if I submitted this article in December, 2011?!) To be fair, they pointed out some problems with my manuscript, particularly with the rhetoric. But be that as it is, I don't have a goddamned time machine.

PS This. This is why I don't put my name on my blog. I know at least one of my reviewers googled me extensively before writing his review. This is not what I want future reviewers to see.

No comments:

Post a Comment