[UPDATE: Other computer scientists are saying that the computer actually failed the test, and badly.]
People who have interacted with Siri, the helpful guide on the iPhone, are usually impressed with her ability to carry on what seems like a normal conversation. But it is not hard to discover that you are talking to a computer. How good would ‘she’ have to be to completely fool you?
The Turing test was first proposed by computer science pioneer Alan Turing in 1950 to address this question and there are many variations of it. It provides a benchmark in the development of artificial intelligence. It essentially consists of whether someone engaging in natural language conversations with an unseen entity can distinguish whether that entity is a human being or a machine. To pass the test, more than 30% of human interrogators should be unable to tell the difference after a series of five-minute conversations.
Via Machines Like Us, I learned that scientists at the University of reading report that for the first time, a machine has passed one version of the test by convincing 33% of the judges that it was a 13-year old Ukrainian boy named Eugene Gootsman.
The test was passed on the 60th anniversary of the death of Turing who committed suicide as a result of the awful treatment of him by the British government and legal system because he was homosexual.
So what does this mean for AI, that fascinating field whose advocates have claimed, for the past half-century, that human-like intelligence was just 10 years away from being achieved? I don’t know. It seems to me that the test results will depend on the quality of the judges and their ability to conceive of conversational initiatives that will elicit revealing responses. Ideally one would like to have some kind of standardized judges in order to monitor quality but I don’t know how one does that.
The 30% cut-off is also just a starting point. Passing it consistently with good judges will likely result in raising the bar from 30% to higher and higher levels until it gets close to 100%.