02 March 2010

Numbers don't lie, but they do deceive.

ProfessorBainbridge | Why Legal Scholarship Should Reject Quants and their Methods

There's a recent paper I had to read on executive compensation, whose author should remain nameless. The author found a statistically significant result that he didn't like. So he threw it under the bus by claiming that his regressions were flawed. Accordingly, he turned to panel data analyses that gave him a result he liked. All the while, another paper on the same topic had found the same results as our author's regressions. Who was it that said statistics don't lie?
[Emph. mine.]

Okay, point taken: people mis-use quantitative analysis. There is a lot of "faith-based econometrics," with people using numbers they drummed up to bolster whatever they already believed, as if statistics were so many chicken entrails.

Post title aside, I won't assert that number don't lie. But it is harder to lie with numbers than with words, just as it is harder to lie with pictures than with words. Provided your audience is not ignorant of the language of math, it is more difficult to wrangle numbers into saying what you want them to say.  (Of course, it is easier to lie in any language when you're dealing with someone who doesn't speak it, just as it is easier in general to lie to the ignorant.  This is nothing specific to numbers and innumeracy.)
“If in other sciences we should arrive at certainty without doubt and truth without error, it behooves us to place the foundations of knowledge in mathematics.”
— Roger Bacon
In most subjects we will never arrive at "certainty without doubt" or "truth without error," but that still ought to be the goal. To be sure, we need to be humble about how close to the goal we are (answer: not very), but that's no reason to abandon numerical inquiry entirely.

No comments:

Post a Comment