What Google's Winning Go Algorithm Will Do Next

Home , AlphaGo, DeepMind, Demis Hassabis, Lee Sedol

NEWS IN FOCUS

stimulation ultimately improved the athletes’ Federal University of Rio Grande do Norte brain stimulation. Some people do not respond jumping force by 70% and their coordination in Brazil, found similar increases in cyclists’ at all; others might respond only when stimu- by 80%, compared with the sham group, Halo performance when he stimulated the brain’s lated in a certain way. And even an individual’s announced in February. temporal cortex, which is involved in body response can differ from day to day. Edwards Troy Taylor, high-performance director for awareness and in automatic functions such as says that it is important to map these differences the USSA, is encouraged by the results — but breathing2. This suggests that the temporal and if tDCS is to be used therapeutically or for other concedes that they are preliminary. motor cortices are connected in ways that are purposes. “We’re moving toward customized not understood, or that tDCS does not target prescription of brain stimulation,” he says. PUSHING THE LIMITS OF ENDURANCE locations in the brain precisely, Okano says. Nonetheless, the use of tDCS in sports is Another study, presented on 7 March at the These results support the notion that the only likely to increase. Stimulating the motor Biomedical Basis of Elite Performance meeting brain manages exertion by collating feedback cortex, for instance, seems to increase dexter- in Nottingham, UK, suggests that tDCS may from the body and then slowing muscles to ity, so videogamers have been quick to take up reduce the perception of fatigue. Sports sci- prevent fatigue, says Dylan Edwards, a neuro- the technique. And it is increasingly easy to entist Lex Mauger of the University of Kent in physiologist at Burke Medical Research Institute acquire stimulation devices; Halo has begun to Canterbury, UK, and his colleagues found that in White Plains, New York3. “Even when you market its equipment for the express purpose of stimulating the motor-cortex region that con- think you’re exercising as hard as you can, there increasing athletic performance. trols leg function allows cyclists to pedal longer is always some reserve of ability,” he says. Taylor compares the use of brain stimulation without feeling tired. by athletes to eating carbohydrates ahead of an The researchers stimulated the brains of TRICKY TESTS athletic event, in the hopes of boosting endur- 12 untrained volunteers before directing the But Horvath cautions that little is known about ance. “It piggybacks on the ability to learn,” he athletes to pedal stationary bicycles until they the long-term effects of stimulating the brain. says. “It’s not introducing something artificial were exhausted. Every minute, they asked the And others are sceptical of the technique’s into the body.” cyclists to rate their level of effort. potential to increase performance. Vincent But Edwards worries that the availability of Volunteers who received tDCS were able to Walsh, a neuroscientist at University College tDCS devices will tempt athletes to try “brain pedal two minutes longer, on average, than were London, notes that the methods used in tDCS doping”, in part because there is no way to detect those who were given a sham treatment. They studies often differ between research groups — its use. “If this is real,” he says, “then absolutely also rated themselves as less tired. But there was and might not always be optimized. the Olympics should be concerned about it.” ■ no difference in heart rate or the lactate level in For instance, the fairly intense amount of 1. Cogiamanian, F. et al. Eur. J. Neurosci. 26, 242–249 the muscles between the treatment and control electricity that Mauger’s team used has been (2007). groups. This suggests that changes in brain per- shown to sometimes have complex and unin- 2. Okano, A. H. et al. Br. J. Sports Med. 49, 1213–1218 4 (2015). ception, rather than muscle pain or other body tended effects on the brain’s activity . 3. Noakes, T. D. Sports Med. 37, 374–377 (2007). feedback, drove the improved performance. Replicating such experiments is difficult 4. Batsikadze, G., Moliadze, V., Paulus, W., Kuo, M.-F. & Alexandre Okano, a biological engineer at because of variations in how people respond to Nitsche, M. A. J. Physiol. 591, 1987–2000 (2013).

ARTIFICIAL INTELLIGENCE What Google’s winning Go algorithm will do next AlphaGo’s techniques could have broad uses, but moving beyond games is a challenge.

BY ELIZABETH GIBNEY which was mainly learned, with a few ele- mastered draughts in 2007: “I expected them ments crafted specifically for the game — to use more computational resources and do a ollowing the defeat of one of its finest could be applied to problems that involve lot more learning, but I still didn’t expect to see human players, the ancient game of Go pattern recognition, decision-making and this amazing level of performance.” has joined the growing list of tasks at planning. But the approach is also limited. The improvement was largely down to the Fwhich computers perform better than humans. “It’s really impressive, but at the same time, fact that the more AlphaGo plays, the better it In a 6-day tournament in Seoul, watched by a there are still a lot of challenges,” says Yoshua gets, says Miles Brundage, a social scientist at reported 100 million people around the world, Bengio, a computer scientist at the University Arizona State University in Tempe, who stud- the computer algorithm AlphaGo, created by of Montreal in Canada. ies trends in AI. AlphaGo uses a brain-inspired the Google-owned company DeepMind, beat Lee, who had predicted that he would win architecture known as a neural network, in Go professional Lee Sedol by 4 games to 1. The the Google tournament in a landslide, was which connections between layers of simulated complexity and intuitive nature of the ancient shocked by his loss. In October, AlphaGo beat neurons strengthen on the basis of experience. board game had established Go as one the European champion Fan Hui. But the version It learned by first studying 30 million Go posi- greatest challenges in artificial intelligence of the program that won in Seoul is signifi- tions from human games and then improv- (AI). Now the big question is what the Deep- cantly stronger, says Jonathan Schaeffer, a com- ing by playing itself over and over again, a Mind team will turn to next. puter scientist at the University of Alberta in technique known as reinforcement learning. AlphaGo’s general-purpose approach — Edmonton, Canada, whose Chinook software Then, DeepMind combined AlphaGo’s ability

Professional Go player Lee Sedol (centre) after his 4–1 defeat by Google’s AlphaGo algorithm. to recognize successful board configurations might never beat the best human. But it’s an interaction involve a lot more uncertainty. with a ‘look-ahead search’, in which it explores important step, he says, because humans learn DeepMind is fuelled by a “very powerful the consequences of playing promising moves with such little guidance. cocktail” of the freedoms usually reserved for and uses that to decide which one to pick. DeepMind, based in London, also plans to academic researchers, and by the vast staff and Next, DeepMind could tackle more games. venture beyond games. In February the com- computing resources that come with being a Most board games, in which players tend to pany founded DeepMind Health and launched Google-backed firm, says Joelle Pineau, a com- have access to all information about play, are a collaboration with the UK National Health puter scientist at McGill University in Mon- now solved. But machines still cannot beat Service: its algorithms could eventually be treal. Its achievement with Go has prompted humans at multiplayer poker, say, in which applied to clinical data to improve diagnoses or speculation about when an AI will have a ver- each player sees only their own cards. The treatment plans. Such applications pose differ- satile, general intelligence. “People’s minds race DeepMind team has expressed an interest in ent challenges from games, says Oren Etzioni, forward and say, if it can beat a world cham- tackling Starcraft, a science-fiction strategy chief executive of the non-profit Allen Institute pion it can do anything,” says Etzioni. But deep game, and Schaeffer suggests that DeepMind for Artificial Intelligence in Seattle, Washing- reinforcement learning remains applicable only devise a program that can learn to play differ- ton. “The universal thing about games is that in certain domains, he says: “We are a long, long ent types of game from scratch. Such programs you can collect an arbitrary amount of data,” way from general artificial intelligence.” already compete annually at the International he says — and that the program is constantly DeepMind’s approach is not the only way General Game Playing Competition, which is getting feedback on what’s a good or bad move to push the boundaries of AI. Gary Marcus, a geared towards creating a more general type of by playing many games. But, in the messy real neuroscientist at New York University in New AI. Schaeffer suspects that DeepMind would world, data — on rare diseases, say — might be York City, has co-founded a start-up company, excel at the contest. “It’s so obvious, that I’m scarcer, and even with common diseases, label- Geometric Intelligence, to explore learn- positive they must be looking at it,” he says. ling the consequences of a decision as ‘good’ or ing techniques that extrapolate from a small DeepMind’s founder and chief executive ‘bad’ may not be straightforward. number of examples, inspired by how chil- Demis Hassabis mentioned the possibility of Hassabis has said that DeepMind’s algo- dren learn. In its short life, AlphaGo probably training a version of AlphaGo using self-play rithms could give smartphone personal assis- played hundreds of millions of games — many alone, omitting the knowledge from human- tants a deeper understanding of users’ requests. more than Lee, who still won one of the five expert games, at a conference last month. The And AI researchers see parallels between games against AlphaGo. “It’s impressive that firm created a program that learned to play less human dialogue and games: “Each person is a human can use a much smaller quantity of complex arcade games in this manner in 2015. making a play, and we have a sequence of turns, data to pick up a pattern,” says Marcus. “Prob- Without a head start, AlphaGo would probably and each of us has an objective,” says Bengio. ably, humans are much faster learners than take much longer to learn, says Bengio — and But they also caution that language and human computers.” ■

THE GO FILES MORE NEWS NATURE PODCAST

Updates from ● Horse-sized dinosaur sheds light on Retrieving lost a battle of T. rex’s origins go.nature.com/ihlkx2 memories; nailing MORE man versus ● Peculiar pattern found in ‘random’ China’s emissions; ONLINE machine over prime numbers go.nature.com/hz11qo and is Alzheimer’s ● GOOGLE DEEPMIND the Go board Genetic study links ‘good’ transmissible? go.nature.com/ cholesterol mutation to heart disease nature.com/nature/ j26o3w go.nature.com/mjils9 podcast