DOCSLIB.ORG

A Survey of Research Questions for Robust and Beneficial AI

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Survey of Research Questions for Robust and Beneficial AI A survey of research questions for robust and beneficial AI 1 Introduction Artificial intelligence (AI) research has explored a variety of problems and approaches since its inception, but for the last 20 years or so has been focused on the problems surrounding the construction of intelligent agents|systems that perceive and act in some environment. In this context, the criterion for intelligence is related to statistical and economic notions of rationality|colloquially, the ability to make good decisions, plans, or inferences. The adoption of probabilistic representations and statistical learning methods has led to a large degree of integration and cross-fertilization between AI, machine learning, statistics, control theory, neuroscience, and other fields. The establishment of shared theoretical frameworks, combined with the availability of data and processing power, has yielded remarkable successes in various component tasks such as speech recognition, image classification, autonomous vehicles, machine translation, legged locomotion, and question-answering systems. As capabilities in these areas and others cross the threshold from laboratory research to economically valuable technologies, a virtuous cycle takes hold whereby even small improvements in performance are worth large sums of money, prompting greater investments in research. There is now a broad consensus that AI research is progressing steadily, and that its impact on society is likely to increase. The potential benefits are huge, since everything that civilization has to offer is a product of human intelligence; we cannot predict what we might achieve when this intelligence is magnified by the tools AI may provide, but the eradication of disease and poverty are not unfathomable. Because of the great potential of AI, it is valuable to investigate how to reap its benefits while avoiding potential pitfalls. The progress in AI research makes it timely to focus research not only on making AI more capable, but also on maximizing the societal benefit of AI. Such considerations motivated the AAAI 2008{09 Presidential Panel on Long-Term AI Futures [61] and other projects and community efforts on AI impacts. These constitute a significant expansion of the field of AI itself, which up to now has focused largely on techniques that are neutral with respect to purpose. The present document can be viewed as a natural continuation of these efforts, focusing on identifying research directions that can help maximize the societal benefit of AI. This research is by necessity interdisciplinary, because it involves both society and AI. It ranges from economics, law, and philosophy to computer security, formal methods and, of course, various branches of AI itself. The focus is on delivering AI that is beneficial to society and robust in the sense that the benefits are guaranteed: our AI systems must do what we want them to do. This document is an attempt to lay out some of the research topics that we think will be most useful to do now in order to shape the future impact of AI. We will surely find that some questions are less useful or timely than others, and some important ones are missing. We hope this guide will be a helpful source of suggestions, but also that potential grantees won't be discouraged from approaching us with similarly relevant topics we didn't think of. We will try to publish future versions that are up to date with progress in the field. We are very grateful to the many people who have contributed to this document, in particular Daniel Dewey, Stuart Russell, and Max Tegmark for their invaluable work on the research priorities document, Luke Muehlhauser for his list of potential strategic research projects, Nate Soares and MIRI for their technical agenda, and the MIRIxOxford research workshop analyzing and expanding on the MIRI technical agenda. Many people at FLI have contributed lists of additional research projects and directions, including Jim Babcock, Steve Greidinger, J´anosKram´ar,Richard Mallah and Max Tegmark, and many more have provided 1 helpful feedback and suggestions, including Anthony Aguirre, Erik Brynjolfsson, Meia Chita-Tegmark, Daniel Dewey, Owain Evans, Eric Gastfriend, Ales Flidr, Katja Grace, Evan Hefner, Viktoriya Krakovna, Colleen Messing, Howard Messing, Chase Moores, Luke Muehlhauser, Se´an O´ hEigeartaigh,´ Tristan Plumb, Stuart Russell, Murray Shanahan, Nate Soares, Marin Soljacic, Michele Reilly, Anders Sandberg, John Sturm, Jaan Tallinn, Jacob Trefethen, Nick Ryder, Michael Vassar, and Alexander Wissner-Gross, Eliezer Yudkowsky. Thanks also to Nick Bostrom for writing the excellent and seminal book \Superintelligence". Like the rest of the document, this list is at an early stage - we're looking forward to receiving your constructive feedback and comments! 2 Contents 1 Introduction 1 2 Short-term research priorities 4 2.1 Optimizing AI's Economic Impact . .4 2.1.1 Measuring and Forecasting Economic Impact of Automation and AI . .4 2.1.2 Policy research . .5 2.1.3 Managing potential adverse effects of automation and AI . .5 2.2 Law and Ethics Research . .6 2.3 Computer Science Research for Robust AI . .7 2.3.1 Verification (\Did I build the system right?") .......................7 2.3.2 Validity (\Did I build the right system?") .........................8 2.3.3 Security (\How can I prevent unauthorized access?") ...................9 2.3.4 Control (\OK, I built the system wrong, can I fix it?") ..................9 3 Long-term research priorities 11 3.1 Some perspectives on the long term . 11 3.2 Verification . 12 3.3 Validity . 12 3.3.1 Ethics . 13 3.3.2 Ensuring goal stability . 15 3.4 Security . 16 3.4.1 Software containment . 17 3.4.2 Psychological containment . 18 3.4.3 Hardware containment . 18 3.4.4 Tripwires: Detection & Response . 19 3.4.5 Detecting intent to deceive . 19 3.5 Control . 19 3.5.1 Corrigibility and Domesticity . 20 3.5.2 Safe and Unsafe Agent Architectures . 21 4 Forecasting 23 4.1 Motivation . 23 4.2 Methodology . 23 4.3 Forecasting AI progress . 24 4.4 Forecasting AI takeoff . 26 4.5 Brain emulations (uploads) . 26 5 Policy and Collaboration 27 3 2 Short-term research priorities 2.1 Optimizing AI's Economic Impact The successes of industrial applications of AI, from manufacturing to information services, demonstrate a growing impact on the economy, although there is disagreement about the exact nature of this impact and on how to distinguish between the effects of AI and those of other information technologies. Many economists and computer scientists agree that there is valuable research to be done on how to maximize the economic benefits of AI while mitigating adverse effects, which could include increased inequality and unemployment [72, 21, 39, 40, 107, 84, 69]. Such considerations motivate a range of research directions, spanning areas from economics to psychology. Below are a few examples that should by no means be interpreted as an exhaustive list. 2.1.1 Measuring and Forecasting Economic Impact of Automation and AI When and in what order should we expect various jobs to become automated [39]? How will this affect the employment and wages of various professions, including less skilled workers, creatives, and different kinds of information workers? Some have argued that AI is likely to greatly increase the overall wealth of humanity as a whole [21]. However, increased automation may push income distribution further towards a power law [22], and the resulting disparity may fall disproportionately along lines of race, class, and gender; research anticipating the economic and societal impact of such disparity could be useful. 1. It is possible that economic measures such as real GDP per capita do not accurately capture the benefits and detriments of heavily AI-and-automation-based economies, making these metrics unsuitable for policy purposes [72]. Research on improved metrics could be useful for decision-making. 2. What has been the historical record on jobs being displaced by automation? What have the average rate and distribution of displacement been - has it been clustered in time, industry, and geography? How long before displaced workers found new jobs? Did displacement contribute to inequality? 3. Is there anything different about the advancement of artificial intelligence happening now that would lead us to expect a change from our centuries-long historical record on jobs being displaced by automa- tion? 4. What factors make an industry more amenable or less amenable to automation? Machines histori- cally performed rote mass-production, but we've been expanding their capabilities with advances in information processing and artificial intelligence. 5. Which markets are most susceptible to disruption as automation advances? Significant parts of the economy { including finance, insurance, actuarial, and many consumer markets { could experience upheaval through use of AI techniques to learn, model, and predict agent actions. These markets might be identified by a combination of high complexity and high rewards for navigating that complexity [69]. Are there other features? 6. Some jobs could be done by machines in principle but require a particular advance in artificial intelli- gence before it becomes feasible. What are some of those prerequisite advances? 7. Based on the above factors, how far in advance can we predict that an industry is likely to be largely automated? Is this something we can predict and prepare for 20 years ahead? 10 years? 6 months? Not at all? 4 2.1.2 Policy research What is the space of policies that could/should be considered for helping AI-assisted societies flourish? For example, Brynjolfsson and McAfee [21] explore various policies for incentivizing development of labor- intensive sectors and for using AI-generated wealth to support underemployed populations. What outcomes are these policies likely to lead to? What are the key uncertainties in these outcome predictions, and what research would help reduce these uncertainties? 1. What factors contribute to a `winner-take-all' dynamic of software-based industries? The low cost of reproduction and increasingly global nature of the information economy, for example, make it easier to concentrate wealth.
Load more

Recommended publications
  • Safety of Artificial Superintelligence
    Environment. Technology. Resources. Rezekne, Latvia Proceedings of the 12th International Scientific and Practical Conference. Volume II, 180-183 Safety of Artificial Superintelligence Aleksejs Zorins Peter Grabusts Faculty of Engineering Faculty of Engineering Rezekne Academy of Technologies Rezekne Academy of Technologies Rezekne, Latvia Rezekne, Latvia [email protected] [email protected] Abstract—The paper analyses an important problem of to make a man happy it will do it with the fastest and cyber security from human safety perspective which is usu- cheapest (in terms of computational resources) without ally described as data and/or computer safety itself with- using a common sense (for example killing of all people out mentioning the human. There are numerous scientific will lead to the situation that no one is unhappy or the predictions of creation of artificial superintelligence, which decision to treat human with drugs will also make him could arise in the near future. That is why the strong neces- sity for protection of such a system from causing any farm happy etc.). Another issue is that we want computer to arises. This paper reviews approaches and methods already that we want, but due to bugs in the code computer will presented for solving this problem in a single article, analy- do what the code says, and in the case of superintelligent ses its results and provides future research directions. system, this may be a disaster. Keywords—Artificial Superintelligence, Cyber Security, The next sections of the paper will show the possible Singularity, Safety of Artificial Intelligence. solutions of making superintelligence safe to humanity.
    [Show full text]
  • Liability Law for Present and Future Robotics Technology Trevor N
    Liability Law for Present and Future Robotics Technology Trevor N. White and Seth D. Baum Global Catastrophic Risk Institute http://gcrinstitute.org * http://sethbaum.com Published in Robot Ethics 2.0, edited by Patrick Lin, George Bekey, Keith Abney, and Ryan Jenkins, Oxford University Press, 2017, pages 66-79. This version 10 October 2017 Abstract Advances in robotics technology are causing major changes in manufacturing, transportation, medicine, and a number of other sectors. While many of these changes are beneficial, there will inevitably be some harms. Who or what is liable when a robot causes harm? This paper addresses how liability law can and should account for robots, including robots that exist today and robots that potentially could be built at some point in the near or distant future. Already, robots have been implicated in a variety of harms. However, current and near-future robots pose no significant challenge for liability law: they can be readily handled with existing liability law or minor variations thereof. We show this through examples from medical technology, drones, and consumer robotics. A greater challenge will arise if it becomes possible to build robots that merit legal personhood and thus can be held liable. Liability law for robot persons could draw on certain precedents, such as animal liability. However, legal innovations will be needed, in particular for determining which robots merit legal personhood. Finally, a major challenge comes from the possibility of future robots that could cause major global catastrophe. As with other global catastrophic risks, liability law could not apply, because there would be no post- catastrophe legal system to impose liability.
    [Show full text]
  • Classification Schemas for Artificial Intelligence Failures
    Journal XX (XXXX) XXXXXX https://doi.org/XXXX/XXXX Classification Schemas for Artificial Intelligence Failures Peter J. Scott1 and Roman V. Yampolskiy2 1 Next Wave Institute, USA 2 University of Louisville, Kentucky, USA [email protected], [email protected] Abstract In this paper we examine historical failures of artificial intelligence (AI) and propose a classification scheme for categorizing future failures. By doing so we hope that (a) the responses to future failures can be improved through applying a systematic classification that can be used to simplify the choice of response and (b) future failures can be reduced through augmenting development lifecycles with targeted risk assessments. Keywords: artificial intelligence, failure, AI safety, classification 1. Introduction Artificial intelligence (AI) is estimated to have a $4-6 trillion market value [1] and employ 22,000 PhD researchers [2]. It is estimated to create 133 million new roles by 2022 but to displace 75 million jobs in the same period [6]. Projections for the eventual impact of AI on humanity range from utopia (Kurzweil, 2005) (p.487) to extinction (Bostrom, 2005). In many respects AI development outpaces the efforts of prognosticators to predict its progress and is inherently unpredictable (Yampolskiy, 2019). Yet all AI development is (so far) undertaken by humans, and the field of software development is noteworthy for unreliability of delivering on promises: over two-thirds of companies are more likely than not to fail in their IT projects [4]. As much effort as has been put into the discipline of software safety, it still has far to go. Against this background of rampant failures we must evaluate the future of a technology that could evolve to human-like capabilities, usually known as artificial general intelligence (AGI).
    [Show full text]
  • Letters to the Editor
    Articles Letters to the Editor Research Priorities for is a product of human intelligence; we puter scientists, innovators, entrepre - cannot predict what we might achieve neurs, statisti cians, journalists, engi - Robust and Beneficial when this intelligence is magnified by neers, authors, professors, teachers, stu - Artificial Intelligence: the tools AI may provide, but the eradi - dents, CEOs, economists, developers, An Open Letter cation of disease and poverty are not philosophers, artists, futurists, physi - unfathomable. Because of the great cists, filmmakers, health-care profes - rtificial intelligence (AI) research potential of AI, it is important to sionals, research analysts, and members Ahas explored a variety of problems research how to reap its benefits while of many other fields. The earliest signa - and approaches since its inception, but avoiding potential pitfalls. tories follow, reproduced in order and as for the last 20 years or so has been The progress in AI research makes it they signed. For the complete list, see focused on the problems surrounding timely to focus research not only on tinyurl.com/ailetter. - ed. the construction of intelligent agents making AI more capable, but also on Stuart Russell, Berkeley, Professor of Com - — systems that perceive and act in maximizing the societal benefit of AI. puter Science, director of the Center for some environment. In this context, Such considerations motivated the “intelligence” is related to statistical Intelligent Systems, and coauthor of the AAAI 2008–09 Presidential Panel on standard textbook Artificial Intelligence: a and economic notions of rationality — Long-Term AI Futures and other proj - Modern Approach colloquially, the ability to make good ects on AI impacts, and constitute a sig - Tom Dietterich, Oregon State, President of decisions, plans, or inferences.
    [Show full text]
  • Exploratory Engineering in AI
    MIRI MACHINE INTELLIGENCE RESEARCH INSTITUTE Exploratory Engineering in AI Luke Muehlhauser Machine Intelligence Research Institute Bill Hibbard University of Wisconsin Madison Space Science and Engineering Center Muehlhauser, Luke and Bill Hibbard. 2013. “Exploratory Engineering in AI” Communications of the ACM, Vol. 57 No. 9, Pages 32–34. doi:10.1145/2644257 This version contains minor changes. Luke Muehlhauser, Bill Hibbard We regularly see examples of new artificial intelligence (AI) capabilities. Google’s self- driving car has safely traversed thousands of miles. Watson beat the Jeopardy! cham- pions, and Deep Blue beat the chess champion. Boston Dynamics’ Big Dog can walk over uneven terrain and right itself when it falls over. From many angles, software can recognize faces as well as people can. As their capabilities improve, AI systems will become increasingly independent of humans. We will be no more able to monitor their decisions than we are now able to check all the math done by today’s computers. No doubt such automation will produce tremendous economic value, but will we be able to trust these advanced autonomous systems with so much capability? For example, consider the autonomous trading programs which lost Knight Capital $440 million (pre-tax) on August 1st, 2012, requiring the firm to quickly raise $400 mil- lion to avoid bankruptcy (Valetkevitch and Mikolajczak 2012). This event undermines a common view that AI systems cannot cause much harm because they will only ever be tools of human masters. Autonomous trading programs make millions of trading decisions per day, and they were given sufficient capability to nearly bankrupt one of the largest traders in U.S.
    [Show full text]
  • The Future of AI: Opportunities and Challenges
    The Future of AI: Opportunities and Challenges Puerto Rico, January 2-5, 2015 ! Ajay Agrawal is the Peter Munk Professor of Entrepreneurship at the University of Toronto's Rotman School of Management, Research Associate at the National Bureau of Economic Research in Cambridge, MA, Founder of the Creative Destruction Lab, and Co-founder of The Next 36. His research is focused on the economics of science and innovation. He serves on the editorial boards of Management Science, the Journal of Urban Economics, and The Strategic Management Journal. & Anthony Aguirre has worked on a wide variety of topics in theoretical cosmology, ranging from intergalactic dust to galaxy formation to gravity physics to the large-scale structure of inflationary universes and the arrow of time. He also has strong interest in science outreach, and has appeared in numerous science documentaries. He is a co-founder of the Foundational Questions Institute and the Future of Life Institute. & Geoff Anders is the founder of Leverage Research, a research institute that studies psychology, cognitive enhancement, scientific methodology, and the impact of technology on society. He is also a member of the Effective Altruism movement, a movement dedicated to improving the world in the most effective ways. Like many of the members of the Effective Altruism movement, Geoff is deeply interested in the potential impact of new technologies, especially artificial intelligence. & Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces. He was the co-creator of Photosynth, software that assembles photos into 3D environments.
    [Show full text]
  • Public Response to RFI on AI
    White House Office of Science and Technology Policy Request for Information on the Future of Artificial Intelligence Public Responses September 1, 2016 Respondent 1 Chris Nicholson, Skymind Inc. This submission will address topics 1, 2, 4 and 10 in the OSTP’s RFI: • the legal and governance implications of AI • the use of AI for public good • the social and economic implications of AI • the role of “market-shaping” approaches Governance, anomaly detection and urban systems The fundamental task in the governance of urban systems is to keep them running; that is, to maintain the fluid movement of people, goods, vehicles and information throughout the system, without which it ceases to function. Breakdowns in the functioning of these systems and their constituent parts are therefore of great interest, whether it be their energy, transport, security or information infrastructures. Those breakdowns may result from deteriorations in the physical plant, sudden and unanticipated overloads, natural disasters or adversarial behavior. In many cases, municipal governments possess historical data about those breakdowns and the events that precede them, in the form of activity and sensor logs, video, and internal or public communications. Where they don’t possess such data already, it can be gathered. Such datasets are a tremendous help when applying learning algorithms to predict breakdowns and system failures. With enough lead time, those predictions make pre- emptive action possible, action that would cost cities much less than recovery efforts in the wake of a disaster. Our choice is between an ounce of prevention or a pound of cure. Even in cases where we don’t have data covering past breakdowns, algorithms exist to identify anomalies in the data we begin gathering now.
    [Show full text]
  • Leakproofing the Singularity Artificial Intelligence Confinement Problem
    Roman V.Yampolskiy Leakproofing the Singularity Artificial Intelligence Confinement Problem Abstract: This paper attempts to formalize and to address the ‘leakproofing’ of the Singularity problem presented by David Chalmers. The paper begins with the definition of the Artificial Intelli- gence Confinement Problem. After analysis of existing solutions and their shortcomings, a protocol is proposed aimed at making a more secure confinement environment which might delay potential negative effect from the technological singularity while allowing humanity to benefit from the superintelligence. Keywords: AI-Box, AI Confinement Problem, Hazardous Intelligent Software, Leakproof Singularity, Oracle AI. ‘I am the slave of the lamp’ Genie from Aladdin 1. Introduction With the likely development of superintelligent programs in the near future, many scientists have raised the issue of safety as it relates to such technology (Yudkowsky, 2008; Bostrom, 2006; Hibbard, 2005; Chalmers, 2010; Hall, 2000). A common theme in Artificial Intelli- gence (AI)1 safety research is the possibility of keeping a super- intelligent agent in a sealed hardware so as to prevent it from doing any harm to humankind. Such ideas originate with scientific Correspondence: Roman V. Yampolskiy, Department of Computer Engineering and Computer Sci- ence University of Louisville. Email: [email protected] [1] In this paper the term AI is used to represent superintelligence. Journal of Consciousness Studies, 19, No. 1–2, 2012, pp. 194–214 Copyright (c) Imprint Academic 2011 For personal use only -- not for reproduction LEAKPROOFING THE SINGULARITY 195 visionaries such as Eric Drexler, who has suggested confining transhuman machines so that their outputs could be studied and used safely (Drexler, 1986).
    [Show full text]
  • Intelligence Explosion FAQ
    MIRI MACHINE INTELLIGENCE RESEARCH INSTITUTE Intelligence Explosion FAQ Luke Muehlhauser Machine Intelligence Research Institute Abstract The Machine Intelligence Research Institute is one of the leading research institutes on intelligence explosion. Below are short answers to common questions we receive. Muehlhauser, Luke. 2013. “Intelligence Explosion FAQ.” First published 2011 as “Singularity FAQ.” Machine Intelligence Research Institute, Berkeley, CA Contents 1 Basics 1 1.1 What is an intelligence explosion? . 1 2 How Likely Is an Intelligence Explosion? 2 2.1 How is “intelligence” defined? . 2 2.2 What is greater-than-human intelligence? . 2 2.3 What is whole-brain emulation? . 3 2.4 What is biological cognitive enhancement? . 3 2.5 What are brain-computer interfaces? . 4 2.6 How could general intelligence be programmed into a machine? . 4 2.7 What is superintelligence? . 4 2.8 When will the intelligence explosion happen? . 5 2.9 Might an intelligence explosion never occur? . 6 3 Consequences of an Intelligence Explosion 7 3.1 Why would great intelligence produce great power? . 7 3.2 How could an intelligence explosion be useful? . 7 3.3 How might an intelligence explosion be dangerous? . 8 4 Friendly AI 9 4.1 What is Friendly AI? . 9 4.2 What can we expect the motivations of a superintelligent machine to be? 10 4.3 Can’t we just keep the superintelligence in a box, with no access to the Internet? . 11 4.4 Can’t we just program the superintelligence not to harm us? . 11 4.5 Can we program the superintelligence to maximize human pleasure or desire satisfaction? .
    [Show full text]
  • Hello, World: Artificial Intelligence and Its Use in the Public Sector
    Hello, World: Artificial intelligence and its use in the public sector Jamie Berryhill Kévin Kok Heang Rob Clogher Keegan McBride November 2019 | http://oe.cd/helloworld OECD Working Papers on Public Governance No. 36 Cover images are the European Parliament and the dome of Germany’s Reichstag building processed through Deep Learning algorithms to match the style of Van Gogh paintings. tw Hello, World: Artificial Intelligence and its Use in the Public Sector Authors: Jamie Berryhill, Kévin Kok Heang, Rob Clogher, Keegan McBride PUBE 2 This document and any map included herein are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. 1. Note by Tukey: The information in this document with reference to ‘Cyprus’ relates to the southern part of the island. There is no single authority representing both Turkish and Greek Cypriot people on the island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the ‘Cyprus issue’. 2. Note by all the European Union Member States of the OECD and the European Commission: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. HELLO, WORLD: ARTIFICIAL INTELLIGENCE AND ITS USE IN THE PUBLIC SECTOR © OECD 2019 3 Foreword Artificial Intelligence (AI) is an area of research and technology application that can have a significant impact on public policies and services in many ways.
    [Show full text]
  • Ethical Artificial Intelligence
    Ethical Artificial Intelligence Bill Hibbard Space Science and Engineering Center University of Wisconsin - Madison and Machine Intelligence Research Institute, Berkeley, CA 17 November 2015 First Edition Please send typo and error reports, and any other comments, to [email protected]. Copyright © Bill Hibbard 2014, 2015 Preface Recent research is giving us ways to define the behaviors of future artificial intelligence (AI) systems, before they are built, by mathematical equations. We can use these equations to describe various broad types of unintended and harmful AI behaviors, and to propose AI design techniques that avoid those behaviors. That is the primary subject of this book. Because AI will affect everyone's future, the book is written to be accessible to readers at different levels. Mathematical explanations are provided for those who want details, but it is also possible to skip over the math and follow the general arguments via text and illustrations. The introductory and final sections of the mathematical chapters (2 −4 and 6 −9) avoid mathematical notation. While this book discusses dangers posed by AI and proposes solutions, it is only a snapshot of ongoing research from my particular point of view and is not offered as the final answer. The problem of defining and creating ethical AI continues, and interest in it will grow enormously. While the effort to solve this problem is currently much smaller than is devoted to Internet security, ultimately there is no limit to the effort that ethical AI merits. Hollywood movies such as the Terminator and Matrix series depict violent war between AI and humans.
    [Show full text]
  • Artificial Intelligence Safety and Cybersecurity: a Timeline of AI Failures
    Artificial Intelligence Safety and Cybersecurity: a Timeline of AI Failures Roman V. Yampolskiy M. S. Spellchecker Computer Engineering and Computer Science Microsoft Corporation University of Louisville One Microsoft Way, Redmond, WA [email protected] [email protected] Abstract In this work, we present and analyze reported failures of artificially intelligent systems and extrapolate our analysis to future AIs. We suggest that both the frequency and the seriousness of future AI failures will steadily increase. AI Safety can be improved based on ideas developed by cybersecurity experts. For narrow AIs safety failures are at the same, moderate, level of criticality as in cybersecurity, however for general AI, failures have a fundamentally different impact. A single failure of a superintelligent system may cause a catastrophic event without a chance for recovery. The goal of cybersecurity is to reduce the number of successful attacks on the system; the goal of AI Safety is to make sure zero attacks succeed in bypassing the safety mechanisms. Unfortunately, such a level of performance is unachievable. Every security system will eventually fail; there is no such thing as a 100% secure system. Keywords: AI Safety, Cybersecurity, Failures, Superintelligence. 1. Introduction A day does not go by without a news article reporting some amazing breakthrough in artificial intelligence1. In fact progress in AI has been so steady that some futurologists, such as Ray Kurzweil, project current trends into the future and anticipate what the headlines of tomorrow will bring us. Consider some developments from the world of technology: 2004 DARPA sponsors a driverless car grand challenge. Technology developed by the participants eventually allows Google to develop a driverless automobile and modify existing transportation laws.
    [Show full text]