Eugene Goostman chatbot claimed to have passed Turing Test

It might be time to start being nicer to your laptop, because researchers at the University of Reading are claiming that a supercomputer program has passed the Turing Test for the first time in history. On Saturday, at the Turing Test 2014 organized by the University of Reading’s School of Systems Engineering, the chatbot Eugene Goostman reportedly convinced the judges 33 percent of the time that it was a human being and not a computer.

Devised by the mathematician Alan Turing in 1950 in his paper "Computing Machinery and Intelligence,” the Turing Test is considered the gold standard for gauging how far we’ve come in the field of artificial intelligence. The test is named after Turing, but the roots of it go back to René Descartes in the 17th century. It strikes not only at questions of artificial intelligence, but also at the limits of automata in general, the question of how we know if other people possess consciousness, and even the philosophical basis of materialism.

The idea is that an interrogator communicates with two contestants, one human and the other a machine, solely by text. The interrogator asks questions of each for five minutes and then decides which one is human. If the machine wins more than 30 percent of the rounds, it’s regarded as having passed the test. So far, no computer program has ever achieved this, but on Saturday, Professor Kevin Warwick, a Visiting Professor at the University of Reading and Deputy Vice-Chancellor for Research at Coventry University, says that the chatbot Eugene Goostman scored 33 percent.

There are actually several versions of the test, each with its own rules and criteria of what constitutes success. "Some will claim that the Test has already been passed,” says Professor Warwick. "The words Turing Test have been applied to similar competitions around the world. However this event involved more simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing's Test was passed for the first time on Saturday.”

The test is controversial because of the tendency of interrogators to attribute human characteristics to what is often a very simple algorithm. This is unfortunate because chatbots are easy to trip up if the interrogator is even slightly suspicious. Chatbots have difficulty with follow up questions and are easily thrown by non-sequiturs that a human could either give a straight answer to or respond to by specifically asking what the heck you’re talking about, then replying in context to the answer.

One of five supercomputer programs participating in the event, Eugene Goostman was developed in 2001 by Russian-born Vladimir Veselov, Ukrainian-born Eugene Demchenko, and Russian-born Sergey Ulasen in Saint Petersburg, and is designed to simulate a 13-year old boy in Odessa with an abrasive adolescent personality to match.

"Eugene was 'born' in 2001,” says Veselov. “Our main idea was that he can claim that he knows anything, but his age also makes it perfectly reasonable that he doesn't know everything. We spent a lot of time developing a character with a believable personality. This year we improved the 'dialog controller' which makes the conversation far more human-like when compared to programs that just answer questions. Going forward we plan to make Eugene smarter and continue working on improving what we refer to as 'conversation logic'."

The Turing Test 2014 was held in partnership with RoboLaw, an organization that examines the regulation of robotic technologies, and the judges included Red Dwarf actor Robert Llewellyn and Lord Sharkey.

"Of course the Test has implications for society today," says Warwick. "Having a computer that can trick a human into thinking that someone, or even something, is a person we trust is a wake-up call to cybercrime. The Turing Test is a vital tool for combating that threat. It is important to understand more fully how online, real-time communication of this type can influence an individual human in such a way that they are fooled into believing something is true ... when in fact it is not."

Sources: University of Reading, Eugene Goostman

Update (June 10, 2014): There is some controversy surrounding the claim that Eugene passed the Turing Test. The language of the story has been modified to reflect this.