30 August 2011

I don't defer to scientists because I am a scientist.

Coyote Blog | Warren Meyer | Krugman Unintended Irony: Anyone Who Does Not Unquestioningly Believe Authorities is Anti-Science


It’s a wonder how, when over “97 percent to 98 percent” of scientific authorities accepted the Ptolomeic view of the solar system that we ever got past that. Though I could certainly understand why in the current economy a die-hard Keynesian might be urging an appeal to authority rather than thinking for oneself.

When, by the way, did the children of the sixties not only lose, but reverse their anti-authoritarian streak?

Postscript: I have always really hated the nose-counting approach to measuring the accuracy of a scientific hypothesis. If we want to label something as anti-science, how about using straw polls of scientists as a substitute for fact-based arguments?
I think Krugman's position of automatically deferring to scientists and determining truth by head-count is extremely foolish. And I think that specifically because I am a scientist.

I was at a small conference last week and I had to stand up in front of the room and give a talk, and one of the points motivating that talk is that a giant in my field is wrong about something. His view in incomplete, and his techniques are flawed.  His works gets hundreds, sometimes thousands of citations. It is widely accepted.  Mine is hardly read.  But his work is flawed, and I think I can show how.  No one in the room said "how dare you question this eminent researcher, and the legion of people who agree with him?"  They heard me out, and I think I convinced at least some of them my point has merit.

I was not the only one doing this. There were several other talks whose focus was errors in the work of our colleagues. One man's whole thesis was that the consensus -- yes, consensus -- about n-back training is incorrect. He couldn't replicate any of it. He thinks the statistics are sloppy and the methods are erroneous. He spoke for an hour about this, forcefully. It undermined the work of a lot of people in that room. But we all heard him out rather than shouting "But you're going against the accepted consensus! You're anti-Science!" And we heard him out specifically because we are scientists, and we listen to dissenting opinions rather than crouching behind a shield of popularity.  That's fine for pundits and columnists and talking heads, but in the lab that kind of behavior is unacceptable.

~ ~ ~

PS Even if consensus was the end of the story, it matter a lot who the consensus is among.  "Scientists" isn't helpful, because most scientists don't know a damn thing about the climate.  If I signed my name to an open letter saying I believe the Higgs boson exists you shouldn't care, because I don't know anything about particle physics.  That's the situation with much of the AGW "consensus."  Most of the people throwing their name onto the scales don't have any expertise in climatology or computational modeling or related fields.  Sociologists and geographers and geneticists are fine people, but how they feel about computational predictions of complex climactic systems is beyond irrelevant.

I think a lot of these signatories to "The Consensus" are suffering from some sort of blind spot bias.  If I stumbled into some debate on protein folding the proteomicists would have no trouble recognizing me as unqualified to support either side, even if I was trying to lend my weight to the popular one.  Hell, if most microbiologists stumbled into that debate the proteomicists would recognize them as being unqualified.  But they're more than happy to weigh in on something outside their own wheelhouse, for reasons which are both dubious and numerous.

~ ~ ~

PPS From Krugman's column:
Pay no attention to “fancy theories” that conflict with “common sense,” the Journal tells us. Because why should anyone imagine that you need more than gut feelings to analyze things like financial crises and recessions?
He has absolutely GOT to be shitting me. Does he have no recollection of writing this?
Europe’s economic success should be obvious even without statistics. For those Americans who have visited Paris: did it look poor and backward? What about Frankfurt or London? You should always bear in mind that when the question is which to believe — official economic statistics or your own lying eyes — the eyes have it.
I tore that statement apart, if I do say so myself, back when he made it.  It's one of my favorite posts, because Krugman is always so much fun to plane.


  1. I agree with this post and where you're coming from generally, but there is something of a minor contradiction here that you almost get to.

    "Consensus" isn't science, that's true. And consensus is often tallied using people who are not experts in the field, which helps in undermining the notion of so-called "scientific consensus."

    But isn't that the problem? I mean, even if a chemist isn't an expert in the field of climatology, he at least has some of the training to read the papers, challenge the math and methodology. And if he doesn't, how the heck am I--an English undergraduate heading toward a graduate degree in International Relations, or for that matter, what if I was one of the 17% of Americans whose highest educational attainment was high school?

    The point is that when scientists like you are conversing about the effectiveness on working memory of n-back training, most people wonder what kind of back an n-back is. That's jolly for you; you know what the leading theories are and their faults. The rest of the people just know that "scientists disagree" and that's enough for them to either refuse to question further or to just choose from the scientific theory that advances their socio-political predilections (although what kind of politics would come from a belief in the effectiveness of n-back training, I have no clue).

    If "scientists disagree" acted as an incentive for people to learn how to read scientific papers, that would be great and we could rest there. But we live in a world where people work, raise families, and have hobbies that don't involve developing a deep understanding of cognition, perception, or neurofuntionality.

    I try to keep up with as many scientific discoveries as I can, to make myself aware of the debate, but since I'm not a part of it in any real day-to-day sense, I have to defer, eventually, to authorities. I can't spend all day studying IR and also be a climatologist. So which authorities do I defer to? Well, I could rest with the fringe guy spouting numbers and methods I don't really understand, or I could go with the guy spouting numbers and methods I don't understand who stands with most of the other guys. It isn't perfect, but it's a kind of perpetual shorthand in a world of extreme diversification and specialization of knowledge.

  2. "I don't have the time or ability or inclination to study all the details and form my own opinion, so I'll believe the theory which is most widely held," is a perfectly fine thing to do. What I object to is "this theory is the most popular, therefore it is 100%, ironclad, don't-even-bother-to-ask-questions correct, and anyone doubting it must oppose the entire edifice of science, modernity and knowledge." That I have a huge problem with.

    Believing the popular position is right because it is popular is one thing. Knowing the popular position is right because it is popular is another.

  3. Oh, one other thing.

    I wish more people, myself included, felt comfortable not having an opinion when we're uninformed. I don't have time to learn about Chinese capital controls, or Swedish political parties, or any of an infinite number of things. It makes sense to defer to experts, and adopt their opinions. But I shouldn't pretend that my opinions carry any weight when I've done that. I'm still ignorant, even when I trust the majority of experts on some issue. I think epistemological humility demands that we acknowledge that even when we're being deferential to experts.

  4. Jim: if I had to guess, I would say a belief in n-back's efficacy is a belief in the malleability of intelligence, which is a strongly liberal sort of position. And contrariwise, belief that IQ is more or less fixed and has a strong genetic component is associated with conservatives like Charles Murray.

    > One man's whole thesis was that the consensus -- yes, consensus -- about n-back training is incorrect. He couldn't replicate any of it. He thinks the statistics are sloppy and the methods are erroneous. He spoke for an hour about this, forcefully. It undermined the work of a lot of people in that room.

    Has he published on this? Or a preprint maybe? (If he couldn't replicate, he did experiments, I take it.

  5. The speaker was Randy Engle. I haven't read through his recent papers, but I would check through those. This one [pdf] looks like a good bet. There will also be a book produced from the conference presentations, and I assume he will be authoring a chapter. It will be a while before that sees print though.

    I meant to actually leave a comment about this on your n-back training (which I've found very useful), but I must have been distracted before I got around to it.

  6. Ah, I actually already have that PDF. It's been sitting in my desktop for the last couple months, glaring at me, ever since it came up on the Brainworkshop mailing list.

    I know I should incorporate it, but it criticizes so many studies I feel it would take me an evening at least to work it in...

  7. I've only skimmed through it, but if Engle's talk and my brief look was any indication, it certainly doesn't suffer from an overly limited scope.