Offering a "most human human" prize is a mistake; it skews the incentives away from the purpose of the exercise.This is quite right. (Emphasis added.)
The purpose is to encourage the development of computers that can interact indistinguishably from humans engaging in normal human activity. But by providing a prize to the most human human, they've turned the challenge into "develop a computer that is indistinguishable from humans who are attempting to signal their humanity." Not only is that harder, but I think it's not as interesting or useful as the original concept.
It's been a long time since I actually read Turing's paper proposing his test, but I think the original thinking was predicated on the assumption that if a computer can do free-form conversation about anything then it can do more useful, topical conversation as well. I think that's understandable but wrong, just like the assumption that a computer should have been able to drive a car before flying a plane.
I think my version of the test would have more goal-oriented conversations, so that the computers would be trying to mimic not just a generic human, but a concierge, or a salesman at a cellphone store, or a HR benefits coordinator. The thinking is that we don't just want computers for the purposes of banal chatting. I don't need a computer if I want to prattle about the weather, all I need for that is to step into an elevator. I do want a computer that can help me pick up the rental car I reserved quickly and painlessly.
Maybe that's not as good of a test of some abstract intelligence, but I think it's a better test of what we actually want.
PS I'd also like to propose another test, since computing systems are often criticized (I think unfairly) for not being capable of doing "creative" things. My test would be a photography challenge. Set a robot and a person loose in an environment with digital cameras, and let the judges try and tell which photos were taken by whom.