On The Turing Test and AI

The news about yet another chatbot "passing" the Turing Test has been all over my news feed. It seems that few of the people writing on Eugene Goostman, the chatbot posing as a 13 year old boy from eastern Europe, have actually read the paper in question or have any kind of nuanced understanding of what the Turing Test entails. PZ Myers has summed up everything wrong with the reporting quite nicely. I just wanted to talk a little bit about something near and dear to my heart: the Turing Test.

The Turing Test, for those of you unfamiliar with the original source material, was originally proposed by computing genius and visionary Alan Turing, whose homosexuality was more important to the authorities of Great Britain than the fact that he was an instrumental member in the think tank that ultimately outwitted the Germans' Enigma machine. Oop.

It appeared in a longer piece called "Computing machinery and intelligence," published in Mind magazine in 1950. There is a lot of interesting stuff in there, and if you're at all interested in computer science....well, honestly, you've probably read it already, but there it is in case you haven't.

The Turing Test, Defined More Clearly

As an introduction to his proposal, Turing initially describes an imitation game played with three people: a man (A), a woman (B), and a judge (whose sex is irrelevant) (C). The goal is for the judge to correctly identify which player is the man and which the woman. There is, of course, a series of questions and answers involved, but it isn't the kind of interviewing that has been accepted as the Turing Test standard:

C: Will X please tell me the length of his or her hair?

Now suppose X is actually A, then A must answer. It is A's object in the game to try and cause C to make the wrong identification. His answer might therefore be:

"My hair is shingled, and the longest strands are about nine inches long."

In order that tones of voice may not help the interrogator the answers should be written, or better still, typewritten. The ideal arrangement is to have a teleprinter communicating between the two rooms. Alternatively the question and answers can be repeated by an intermediary. The object of the game for the third player (B) is to help the interrogator. The best strategy for her is probably to give truthful answers. She can add such things as "I am the woman, don't listen to him!" to her answers, but it will avail nothing as the man can make similar remarks. 

We now ask the question, "What will happen when a machine takes the part of A in this game?" Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, "Can machines think?" 

Then he proposes that the same game be played, only this time with a human and machine trying to fool the judge, instead of two humans.There is a level of trickery and strategy involved with the background of the Turing Test (what he calls "the imitation game"; Turing wasn't vain enough to name the test after himself in his own paper). It's not merely a conversation with ELIZA.

And while it seems like a very limited situation—a judge just getting written or secondhand feedback from players confined to different rooms—that's really just Turing trying to make the game as fair as possible, so that the machine could be judged on its cognition alone.

The question and answer method seems to be suitable for introducing almost any one of the fields of human endeavour that we wish to include. We do not wish to penalise the machine for its inability to shine in beauty competitions, nor to penalise a man for losing in a race against an aeroplane. The conditions of our game make these disabilities irrelevant. The "witnesses" can brag, if they consider it advisable, as much as they please about their charms, strength or heroism, but the interrogator cannot demand practical demonstrations.

To limit the idea of the Turing Test to a reiteration of the imitation game (even the more complex, deceit-filled model of the imitation game as described above) is missing Turing's larger point: can machines demonstrate the same behaviors that we associate with thinking and intelligence?


Polishing the Turing Test

Unsurprisingly, chatbots have been the favored method of would-be Turing Test passers, since the medium of instant messaging is basically identical to Turing's hypothetical setup. But daily life is mediated by computers and technology in ways that were probably unthinkable even to a visionary such as Turing. If he were writing "Computing machinery and intelligence" today, he'd perhaps make Facebook, Twitter, or Tumblr (ugh) the means of his contest. Surely it's not impossible to conceive of an AI that would post to social media (maybe even on multiple accounts), sharing thoughts and articles and quiz results and interacting with its followers the same way that people do. Some of the people who post in your news feed might already seem like thoughtless automatons.

The other problem with chatbots as Turing Test experiments is that it misses the spirit of the Test. Programming an AI only to pass the Turing Test and nothing more doesn't a thinking machine make. However, that doesn't mean the Turing Test is bullshit, or isn't a useful tool for evaluating AI, either. If you take something like Siri, for example, or Samantha from Her, which are programmed to do quite a number of tasks, we'd say they were "thinking" if they passed the Turing Test incidentally. Turing never mentions this incidental nature of the test but  it seems an intuitively important part of the test. When you consider how much of human cognition and intelligence is incidental to our brains and the problems they evolved to solve, this seems only fair. Or, as Harnad suggests, the Turing Test isn't something that should last for five minutes or fifteen or an hour, but indefinitely.


Objections to the Turing Test

Humans tend to assume that everyone else we come into contact with are human too, or are at least human-like. Which is why you get vegans assuming their cats should also be on a vegan diet, for example: we are biased towards believing things more or less "like us" and generally it takes more proof to disprove rather than prove. The Turing Test may, in fact, be too simple to be a good criterion.

There's also Searle's Chinese Room objection. I've never much cared for it or thought it to be a particularly strong objection, but it's a famous one nonetheless.


TL;DR

The AI singularity is not yet upon us and we will not be bowing before our robot overlord Eugeene Goostman. Goostman, really, could hardly be said to have "passed" the test; whether or not this implementation of the Turing Test even adheres to Turing's original vision is another question altogether.