15 February 2011

"Most Human Human"

eddie left a good comment in my previous post about the Loebner version of the Turing Test:
Offering a "most human human" prize is a mistake; it skews the incentives away from the purpose of the exercise.

The purpose is to encourage the development of computers that can interact indistinguishably from humans engaging in normal human activity. But by providing a prize to the most human human, they've turned the challenge into "develop a computer that is indistinguishable from humans who are attempting to signal their humanity." Not only is that harder, but I think it's not as interesting or useful as the original concept.
This is quite right.  (Emphasis added.)

It's been a long time since I actually read Turing's paper proposing his test, but I think the original thinking was predicated on the assumption that if a computer can do free-form conversation about anything then it can do more useful, topical conversation as well. I think that's understandable but wrong, just like the assumption that a computer should have been able to drive a car before flying a plane.

I think my version of the test would have more goal-oriented conversations, so that the computers would be trying to mimic not just a generic human, but a concierge, or a salesman at a cellphone store, or a HR benefits coordinator. The thinking is that we don't just want computers for the purposes of banal chatting. I don't need a computer if I want to prattle about the weather, all I need for that is to step into an elevator. I do want a computer that can help me pick up the rental car I reserved quickly and painlessly.

Maybe that's not as good of a test of some abstract intelligence, but I think it's a better test of what we actually want.

PS I'd also like to propose another test, since computing systems are often criticized (I think unfairly) for not being capable of doing "creative" things.  My test would be a photography challenge.  Set a robot and a person loose in an environment with digital cameras, and let the judges try and tell which photos were taken by whom.

1 comment:

  1. Speaking as one who has tried to make a computer drive a car, I *wish* the problem were anywhere near as easy as making a computer fly a plane. All that empty air... sigh. ;-)