COMMENT HISTORY Two centuries of EXPLORATION On the trail of FUNDING African grant-giving OBITUARY Vladimir Voevodsky, distorting publication an ill-fated expedition to bodies need more data to pioneer in algebraic geometry metrics p.163 map the Arctic p.166 guide investments p.168 and computer proofs p.169 BSIP/UIG/GETTY A man with a spinal-cord injury (right) prepares for a virtual cycle race in which competitors steer avatars using brain signals. Four ethical priorities for neurotechnologies and AI Artificial intelligence and brain–computer interfaces must respect and preserve people’s privacy, identity, agency and equality, say Rafael Yuste, Sara Goering and colleagues. onsider the following scenario. feels frustrated with the experimental team. illustrates some of the challenges that society A paralysed man participates in a clin- Later, his robotic hand crushes a cup after might be heading towards. ical trial of a brain–computer interface taking it from one of the research assistants, Current BCI technology is mainly focused C(BCI). A computer connected to a chip in his and hurts the assistant. Apologizing for what on therapeutic outcomes, such as helping brain is trained to interpret the neural activ- he says must have been a malfunction of the people with spinal-cord injuries. It already ity resulting from his mental rehearsals of an device, he wonders whether his frustration enables users to perform relatively simple action. The computer generates commands with the team played a part. motor tasks — moving a computer cursor or that move a robotic arm. One day, the man This scenario is hypothetical. But it controlling a motorized wheelchair, for ©2017 Mac millan Publishers Li mited, part of Spri nger Nature. All 9ri gNOVEMBERhts reserved. 2017 | VOL 551 | NATURE | 159 COMMENT example. Moreover, researchers can already nations and people of varying religions, applications and send e-mails3. Meanwhile, interpret a person’s neural activity from func- ethnicities and socio-economic backgrounds researchers at Duke University in Durham, tional magnetic resonance imaging scans at will have differing needs and outlooks. As North Carolina, have shown that three a rudimentary level1 — that the individual is such, governments must create their own monkeys with electrode implants can oper- thinking of a person, say, rather than a car. deliberative bodies to mediate open debate ate as a ‘brain net’ to move an avatar arm It might take years or even decades until involving representatives from all sectors collaboratively4. These devices can work BCI and other neurotechnologies are part of society, and to determine how to trans- across thousands of kilometres if the signal of our daily lives. But technological devel- late these guidelines into policy, including is transmitted wirelessly by the Internet. opments mean that we are on a path to a specific laws and regulations. Soon such coarse devices, which can world in which it will be possible to decode stimulate and read the activity of a few people’s mental processes and directly manip- INTELLIGENT INVESTMENTS dozen neurons at most, will be surpassed. ulate the brain mechanisms underlying their Some of the world’s wealthiest investors are Earlier this year, the US Defense Advanced intentions, emotions and decisions; where betting on the interplay between neuro- Research Projects Agency (DARPA) individuals could communicate with others science and AI. More than a dozen companies launched a project called Neural Engineer- simply by thinking; and where powerful worldwide, including Kernel and Elon Musk’s ing System Design. It aims to win approval computational systems linked directly to start-up firm Neuralink, which launched this from the US Food and Drug Administration people’s brains aid their interactions with the year, are investing in the creation of devices within 4 years for a wireless human brain world such that their mental and physical that can both ‘read’ human brain activity and device that can monitor brain activity using abilities are greatly enhanced. ‘write’ neural information into the brain. We 1 million electrodes simultaneously and Such advances could revolutionize the estimate that current spending on neuro- selectively stimulate up to 100,000 neurons. treatment of many conditions, from brain technology by for-profit industry is already Meanwhile, Google, IBM, Microsoft, injury and paralysis to epilepsy and schizo- US$100 million per year, and growing fast. Facebook, Apple and numerous start-ups are phrenia, and transform human experience Investment from other sectors is also building ever-more-sophisticated artificial for the better. But the technology could also considerable. Since 2013, more than neural networks that can already outperform exacerbate social inequalities and offer cor- $500 million in federal funds has gone humans on tasks with well-defined inputs porations, hackers, governments or anyone towards the development of neurotechnol- and outputs. else new ways to exploit and manipulate ogy under the US BRAIN initiative alone. Last year, for example, researchers at the people. And it could profoundly alter some Current capabilities are already impressive. University of Washington in Seattle demon- core human characteristics: private mental life, A neuroscientist paralysed by amyotrophic strated that Google’s FaceNet system could individual agency and an understanding of lateral sclerosis (ALS; also known as Lou recognize one face from a million others. individuals as entities bound by their bodies. Gehrig’s or motor neuron disease) has used Another Google system with similar neural- It is crucial to consider the possible a BCI to run his laboratory, write grant network architecture far outperforms well- ramifications now. travelled humans at guessing where in the The Morningside Group comprises neuro- world a street scene has been photographed, scientists, neurotechnologists, clinicians, PROTECTING PRIVACY demonstrating the generality of the tech- ethicists and machine-intelligence engineers. nique. In August, Microsoft announced that, It includes representatives from Google Federated learning in certain metrics, its neural network for rec- and Kernel (a neurotechnology start-up ognizing conversational speech has matched in Los Angeles, California); from inter- When technology companies use the abilities of even trained professionals, national brain projects; and from academic machine learning to improve their who have the option of repeatedly rewind- and research institutions in the United States, software, they typically gather user ing and listening to words used in context. Canada, Europe, Israel, China, Japan and information on their servers to analyse And using electroencephalogram (EEG) Australia. We gathered at a workshop spon- how a particular service is being used data, researchers at the University of Freiburg sored by the US National Science Foundation and then train new algorithms on in Germany showed in July how neural at Columbia University, New York, in May the aggregated data. Researchers networks can be used to decode planning- 2017 to discuss the ethics of neurotechnolo- at Google are experimenting with related brain activity and so control robots5. gies and machine intelligence. an alternative method of artificial- Future neural networks derived from a We believe that existing ethics guidelines intelligence training called federated better understanding of how real ones work are insufficient for this realm2. These include learning. Here, the teaching process will almost certainly be much more powerful the Declaration of Helsinki, a statement of happens locally on each user’s device even than these examples. The artificial net- ethical principles first established in 1964 without the data being centralized: works in current use have been inspired by for medical research involving human sub- the lessons aggregated from the models of brain circuits that are more than jects (go.nature.com/2z262ag); the Belmont data (for instance, the knowledge 50 years old, which are based on recording the Report, a 1979 statement crafted by the US that the word ‘weekly’ can be used activity of individual neurons in anaesthetized National Commission for the Protection of as an adjective and an adverb) are animals6. In today’s neuroscience labs, Human Subjects of Biomedical and Behav- sent back to Google’s servers, but researchers can monitor and manipulate the ioural Research (go.nature.com/2hrezmb); the actual e-mails, texts and so on activity of thousands of neurons in awake, and the Asilomar artificial intelligence (AI) remain on the user’s own phone. behaving animals, owing to advances in statement of cautionary principles, pub- Other groups are exploring similar optical methods, computing, molecular lished early this year and signed by business ideas. Thus, information systems with engineering and microelectronics. leaders and AI researchers, among others improved designs could be used to We are already intimately connected to (go.nature.com/2ihnqac). enhance users’ ownership and privacy our machines. Researchers at Google calcu- To begin to address this deficit, here we lay over their personal data, while still lated this year that the average user touches out recommendations relating to four areas enabling valuable computations to be their phone nearly one million times annu- of concern: privacy and consent; agency and performed on those data. ally (unpublished data). The human brain identity; augmentation; and bias. Different controls auditory