Beyond the Turing Test of Stanford University in Palo Alto, Califor- Nia

NEWS | IN DEPTH COMPUTER SCIENCE photograph. But AI researchers want machines to understand and reason with what they see, says computer scientist Fei-Fei Li Beyond the Turing Test of Stanford University in Palo Alto, Califor- nia. The challenge Li will propose would ask Concluding that there is no one test for machine intelligence, machines to tell stories from pictures—not only identifying an object such as a coffee AI researchers develop a battery of research challenges mug, for example, but also noting that it sits half-empty on a table because someone By Jia You would motivate researchers to develop ma- drank from it. Such machines might one day chines with a deeper understanding of the interpret what she calls the “dark matter of s the movie The Imitation Game cel- world, argues Marcus, who is co-organizing the digital age”: images and videos, which ebrates British mathematician Alan the workshop. today’s search engines and bots can hardly Turing’s contributions to the Allied One set of proposed challenges focuses make sense of. victory in World War II, the artificial on common-sense reasoning, which re- For machines to truly assist people in their intelligence (AI) community is re- mains a tall order for machines yet is cru- daily lives, physical movement smoothly in- thinking another of his legacies: the cial for comprehending language. Take the tegrated with language and perceptual skills ATuring Test. sentence, “The trophy would not fit in the has to be part of the mix, says computer In 1950, Turing laid out an ap- scientist Charles Ortiz of the Nuance pealingly simple test for whether a Natural Language and AI Laboratory machine possesses human-level intel- in Sunnyvale, California. His proposed ligence: Will a person conversing with challenge would ask both a machine it in text mistake it for another human and a human to manipulate a robotic being? But more than 60 years later, arm in order to, say, play with a toy. it’s time for new criteria, says com- At the same time, they would carry on puter scientist Leora Morgenstern of a conversation about their actions. As on January 9, 2015 Leidos Inc. in Reston, Virginia. “We in Turing’s original test, a judge would now know a lot about AI and what’s evaluate the “humanness” of the com- needed to make progress. It’s a big puter’s performance. leap from Turing’s time.” Intelligence has one more dimen- At a 25 January workshop at the sion, says computer scientist Bar- 29th Association for the Advancement bara Grosz of Harvard University: of Artificial Intelligence conference teamwork. To effectively collaborate in Austin, Morgenstern and other re- with humans, machines will need to searchers will discuss proposals for understand their teammates’ prefer- www.sciencemag.org a new Turing Championship. In con- ences, share information appropriately, trast with Turing’s single litmus test, and handle uncertain environments. the proposed challenges acknowl- Grosz’s challenge would pair comput- edge that intelligence has multiple ers with people in group activities, such dimensions—from language compre- as formulating health care plans, to hension to social awareness—that are “We can only see a short distance test whether people overlook that their best tackled piece by piece. partners aren’t human. Over the years, Turing’s origi- ahead, but we can see plenty there Many more research challenges will Downloaded from nal idea has grown into a small be debated at the workshop, aimed at industry while drawing increasing crit- that needs to be done.” capabilities from long-term learning icism. Competitions such as the long- Alan Turing, 1950 to creativity. The goal, Marcus says, running Loebner Prize ask human is to winnow the proposals down to judges to text chat with either a person or brown suitcase because it was too big.” De- three to five competitions. A balance of am- a computer program for less than 30 min- ducing that “it” refers to the trophy, not the bition and realism is key, says computer sci- utes and then determine the converser’s suitcase, requires general knowledge that is entist Stuart Shieber of Harvard. “You want identity. In June, a computer program second nature for a person but difficult to to design competitions that are qualitatively named Eugene Goostman, which adopts program into a machine. Next fall, in what beyond the current level of AI, but not so far the persona of a 13-year-old Ukrainian boy, could be the first of the new Turing chal- that … it would be like setting an X prize for was declared to have passed a Turing Test lenges, the industry-sponsored Winograd space flight in da Vinci’s era,” he says. organized by the University of Reading in Schema Challenge will test machines’ com- Although it’s unlikely that consensus will the United Kingdom after fooling a third of prehension of such grammatically ambigu- emerge in January, the discussion will con- the judges in 5-minute conversations. Yet ous sentences. tinue at another AI conference in July, says researchers such as cognitive scientist Gary A second set of proposed challenges cen- co-organizer Manuela Veloso of Carnegie Marcus of New York University in New York ters on machine vision. With new machine- Mellon University in Pittsburgh, Pennsyl- City argue that such competitions put a learning techniques that train computers to vania. By early 2016, the organizers hope to premium on stock answers and other ruses. discern objects, researchers at places such stage a set of trial competitions that will be “It’s a parlor trick,” Marcus says. “There’s no as Google and Facebook are developing al- revised and repeated regularly. “If we don’t sense in which that program is genuinely gorithms that can guide a self-driving car move fast, it won’t happen,” Veloso says. intelligent.” The new Turing Championship or automatically identify any face in any “People will lose momentum.” ■ IMAGES IMAGES/GETTY IMAGES/HERITAGE ART FINE PHOTO: 116 9 JANUARY 2015 • VOL 347 ISSUE 6218 sciencemag.org SCIENCE Published by AAAS.

Load more