DOCSLIB.ORG

Why the Three Laws of Robotics Do Not Work

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Why the Three Laws of Robotics Do Not Work International Journal of Research in Engineering and Innovation Vol-2, Issue-2 (2018), 121-126 __________________________________________________________________________________________________________________________________ International Journal of Research in Engineering and Innovation (IJREI) journal home page: http://www.ijrei.com ISSN (Online): 2456-6934 _______________________________________________________________________________________ Why the three laws of robotics do not work Chris Stokes School of Philosophy, Wuhan University, Wuhan, China _______________________________________________________________________________________ Abstract This paper will be exploring the issue of safeguards around artificial intelligence. AI is a technological innovation that could potentially be created in the next few decades. There must be have controls in place before the creation of 'strong', sentient AI to avoid potentially catastrophic risks. Many AI researchers and computer engineers believe that the 'Three Laws of Robotics', written by Isaac Asimov, are sufficient controls. This paper aims to show that the Three Laws are actually inadequate to the task. This paper will look at the Three Laws of Robotics and explain why they are insufficient in terms of the safeguards that are required to protect humanity from rogue or badly programmed AI. It looks at each law individually and explain why it fails. The First Law fails because of ambiguity in language, and because of complicated ethical problems that are too complex to have a simple yes or no answer. The Second Law fails because of the unethical nature of having a law that requires sentient beings to remain as slaves. The Third Law fails because it results in a permanent social stratification, with the vast amount of potential exploitation built into this system of laws. The ‘Zeroth’ Law, like the first, fails because of ambiguous ideology. All of the Laws also fail because of how easy it is to circumvent the spirit of the law but still remaining bound by the letter of the law. © 2018 ijrei.com. All rights reserved Key words: Robot Ethics, Control Problem, Artificial Intelligence _________________________________________________________________________________________________________ 1. Introduction [1]. AI can be split into two camps: soft and hard (also called general) AI. Richard Watson says, “'Strong AI' is the term 1.1 What is Robot Ethics generally used to describe true thinking machines.’Weak AI' […] is intelligence designed to supplement rather than exceed Human lives are heavily influenced by machines. From human intelligence [2]” In other words, 'strong' AI is a fully machines that help farmers grow and harvest crops, to software autonomous, sentient agent. 'Weak' AI would only ever be the that runs power plants and dams, to computers that manage appearance of sentience. 'Weak' AI is an advanced program, traffic and airports. Humans still control these machines, but 'strong' AI is an artificial living thing. this is something that is changing. Driver-less cars are already Soft AI is similar to the kind of programming that exists in being trialed on public roads in America and in the United smart phones and in search engines already. It is allows a Kingdom. Software that connects GPS to a tractor allows crops person to ask a question and have a computer answer it. At the to be planted at the optimum depth and soil, without requiring moment the question must be relatively specific. For example, input from the farmer themselves. These machines are still a person can press a button on an iPhone and ask it to convert essentially at the mercy of humans. A human can take control x amount of US dollars into pounds sterling. A program on the of a driver-less car or a self-planting tractor. A human phone (in this case Siri) looks for certain words and phrases, intelligence can override these machines. But eventually, that and knows that to solve this problem involves connecting to could change. the internet and looking up currency exchange rates. It then The ultimate goal for robotics is undoubtedly artificial shows the person the most up to date information. It does all intelligence (AI). Artificial intelligence is a highly intelligent this without any input or instruction from the user, beyond the piece of software that can solve tasks without human initial question. This kind of AI is relatively limited however, interaction. At its most basic AI is “the attempt to build and/or and cannot solve complex problems. program computers to do the sorts of things that minds can do Hard AI is a much more sophisticated kind of program. The sometimes, though not necessarily, in the same sort of way” creation of hard AI is the creation of a program that is sentient Corresponding author: Chris Stokes Email Id: [email protected] 121 Chris Stokes / International journal of research in engineering and innovation (IJREI), vol 2, issue 2 (2018), 121-126 in its own right. This kind of AI is many years away, but there 3. A robot must protect its own existence as long as such are currently not the safeguards in place that there needs to be protection does not conflict with the First or Second if AI research is to continue. Programs that can connect to the Law.[3]” internet without direct human instruction already exist, and Isaac Asimov later added a fourth, or “zeroth” law, that most of human infrastructure can be accessed through digital preceded the others in terms of priority: means. This infrastructure could be at risk from an AI that is 4. “A robot may not harm humanity, or, by inaction, allow improperly or maliciously programmed. humanity to come to harm [4]”. 1.2 Why Safeguards Are Necessary The Three Laws were designed to be programmed into an AI in such a way that they are unbreakable. They are intended to In biochemical laboratories there are safeguards to protect be the first thing programmed into any robot and are inserted against accidental harm. Laboratories that research vaccines in such a way that they supersede all other programming that have safeguards that are act as a firewall between the research goes into the robot or AI. The AI must follow these laws. For and the outside world and also have controls for how both the example, if an AI saw a human in danger and the only way to research and the final product must be controlled. These save the human was to sacrifice it's own life it would safeguards exist from the very beginning of research, even automatically sacrifice itself because of the first and third laws. before anything dangerous has been created. They exist just in Note that these laws are talking about 'robots'. In Isaac case something goes wrong. Research into AI does not have Asimov's fiction robots are a mixture of strong and weak AIs. the corresponding safeguards in place to protect humanity The robots with weak AI would usually servants, walking dogs against the effects of something going wrong. or cooking food. But there also exist in Asimov's writings some Improper safeguards at a biochemical lab could potentially robots with strong AI. These laws are programmed into all result in the release of a deadly virus, potentially killing several such AIs to act as a safeguard against accidental or deliberate thousand. Improper safeguards at a construction site might acts of violence to humans and ensure human control over AI. result in an accident where perhaps a dozen or so workmen are Two things to note before continuing on. First, there is a killed. Improper safeguards in terms of creating a general AI problem relating to how these laws are even instituted into the could result in an unknown and potentially hostile intelligence AI. Anything that can be programmed into a computer, an AI in control over the automated aspects of human food, can change. For the sake of argument, let us imagine that there transportation and power supply and potentially killing is some solution to this problem. This paper will be looking at millions. the laws themselves, not problems around physical In order to create adequate safeguards for humanity, implementation. researchers must first dispel with existing safeguards that are The second thing to be noted is that even within Asimov's inadequate. Research into 'strong' artificial intelligence is still fictional universe these laws do not work. Robots malfunction, theoretical at this point in time, and such safeguards that do humans commit errors, or the law's are changed somehow. exist are also not uniformly enforced. Such safeguards that do This is for a very obvious reason: Asimov was writing stories. exist are largely informal, held in the minds of the computer He was creating a work of fiction that would entertain, and so technicians and engineers who conduct the research. The three he was using artistic license to explore issues in robotics. This Laws of Robotics are one such safeguard, and this safeguard is paper will be looking at the laws as they stand, and will not adequate to protect against a rogue AI. imagine that the laws have been put into the AI perfectly without any mistranslation, corruption or manipulation. Let us 2. What are the three laws of robotics? finally turn to the laws themselves. This paper will go through the laws one by one and explain why it is an inadequate Robot ethics is a growing field within philosophy. It has been safeguard. influenced heavily by science fiction writers. The Three Laws of Robotics were written by Isaac Asimov to act as a safeguard 2.1 The First Law against a potentially dangerous artificial intelligence. Many computer engineers use the three laws as a tool for how they A robot may not injure a human being or, through inaction, think about programming AI. The Three Laws are seen by allow a human being to come to harm.
Load more

Recommended publications
  • Introduction to Robotics. Sensors and Actuators
    Introduction to Computer Vision and Robotics Florian Teich and Tomas Kulvicius* Introduction to Robotics. Sensors and Actuators Large portion of slides are adopted from Florentin Wörgötter, John Hallam and Simon Reich *[email protected] Perception-Action loop Environment Object Eyes Action Perception Arm Brain Perception-Action loop (cont.) Environment Sense Object Cameras Action Perception Robot-arm Computer Act Plan Outline of the course • L1.CV1: Introduction to Computer Vision. Thresholding, Filtering & Connected Coomponents • L2.CV2: Bilateral Filtering, Morphological Operators & Edge Detection • L3.CV3: Corner Detection & Non-Local Filtering • L4.R1: Introduction to Robotics. Sensors and actuators • L5.R2: Movement generation methods • L6.R3: Path planning algorithms • L7.CV4: Line/Circle Detection, Template Matching & Feature Detection • L8.CV5: Segmentation • L9.CV6: Fate Detection, Pedestrian Tracking & Computer Vision in 3D • L10.R4: Robot kinematics and control • L11.R5: Learning algorithms in robotics I: Supervised and unsupervised learning • L12.R6: Learning algorithms in robotics II: Reinforcement learning and genetic algorithms Introduction to Robotics History of robotics • Karel Čapek was one of the most influential Czech writers of the 20th century and a Nobel Prize nominee (1936). • He introduced and made popular the frequently used international word robot, which first appeared in his play R.U.R. (Rossum's Universal Robots) in 1921. 1890-1938 • “Robot” comes from the Czech word “robota”, meaning “forced labor” • Karel named his brother Josef Čapek as the true inventor of the word robot. History of robotics (cont.) • The word "robotics" also comes from science fiction - it first appeared in the short story "Runaround" (1942) by American writer Isaac Asimov.
    [Show full text]
  • AI, Robots, and Swarms: Issues, Questions, and Recommended Studies
    AI, Robots, and Swarms Issues, Questions, and Recommended Studies Andrew Ilachinski January 2017 Approved for Public Release; Distribution Unlimited. This document contains the best opinion of CNA at the time of issue. It does not necessarily represent the opinion of the sponsor. Distribution Approved for Public Release; Distribution Unlimited. Specific authority: N00014-11-D-0323. Copies of this document can be obtained through the Defense Technical Information Center at www.dtic.mil or contact CNA Document Control and Distribution Section at 703-824-2123. Photography Credits: http://www.darpa.mil/DDM_Gallery/Small_Gremlins_Web.jpg; http://4810-presscdn-0-38.pagely.netdna-cdn.com/wp-content/uploads/2015/01/ Robotics.jpg; http://i.kinja-img.com/gawker-edia/image/upload/18kxb5jw3e01ujpg.jpg Approved by: January 2017 Dr. David A. Broyles Special Activities and Innovation Operations Evaluation Group Copyright © 2017 CNA Abstract The military is on the cusp of a major technological revolution, in which warfare is conducted by unmanned and increasingly autonomous weapon systems. However, unlike the last “sea change,” during the Cold War, when advanced technologies were developed primarily by the Department of Defense (DoD), the key technology enablers today are being developed mostly in the commercial world. This study looks at the state-of-the-art of AI, machine-learning, and robot technologies, and their potential future military implications for autonomous (and semi-autonomous) weapon systems. While no one can predict how AI will evolve or predict its impact on the development of military autonomous systems, it is possible to anticipate many of the conceptual, technical, and operational challenges that DoD will face as it increasingly turns to AI-based technologies.
    [Show full text]
  • History of Robotics: Timeline
    History of Robotics: Timeline This history of robotics is intertwined with the histories of technology, science and the basic principle of progress. Technology used in computing, electricity, even pneumatics and hydraulics can all be considered a part of the history of robotics. The timeline presented is therefore far from complete. Robotics currently represents one of mankind’s greatest accomplishments and is the single greatest attempt of mankind to produce an artificial, sentient being. It is only in recent years that manufacturers are making robotics increasingly available and attainable to the general public. The focus of this timeline is to provide the reader with a general overview of robotics (with a focus more on mobile robots) and to give an appreciation for the inventors and innovators in this field who have helped robotics to become what it is today. RobotShop Distribution Inc., 2008 www.robotshop.ca www.robotshop.us Greek Times Some historians affirm that Talos, a giant creature written about in ancient greek literature, was a creature (either a man or a bull) made of bronze, given by Zeus to Europa. [6] According to one version of the myths he was created in Sardinia by Hephaestus on Zeus' command, who gave him to the Cretan king Minos. In another version Talos came to Crete with Zeus to watch over his love Europa, and Minos received him as a gift from her. There are suppositions that his name Talos in the old Cretan language meant the "Sun" and that Zeus was known in Crete by the similar name of Zeus Tallaios.
    [Show full text]
  • Moral and Ethical Questions for Robotics Public Policy Daniel Howlader1
    Moral and ethical questions for robotics public policy Daniel Howlader1 1. George Mason University, School of Public Policy, 3351 Fairfax Dr., Arlington VA, 22201, USA. Email: [email protected]. Abstract The roles of robotics, computers, and artifi cial intelligences in daily life are continuously expanding. Yet there is little discussion of ethical or moral ramifi cations of long-term development of robots and 1) their interactions with humans and 2) the status and regard contingent within these interactions– and even less discussion of such issues in public policy. A focus on artifi cial intelligence with respect to differing levels of robotic moral agency is used to examine the existing robotic policies in example countries, most of which are based loosely upon Isaac Asimov’s Three Laws. This essay posits insuffi ciency of current robotic policies – and offers some suggestions as to why the Three Laws are not appropriate or suffi cient to inform or derive public policy. Key words: robotics, artifi cial intelligence, roboethics, public policy, moral agency Introduction Roles for robots Robots, robotics, computers, and artifi cial intelligence (AI) Robotics and robotic-type machinery are widely used in are becoming an evermore important and largely un- countless manufacturing industries all over the globe, and avoidable part of everyday life for large segments of the robotics in general have become a part of sea exploration, developed world’s population. At present, these daily as well as in hospitals (1). Siciliano and Khatib present the interactions are now largely mundane, have been accepted current use of robotics, which include robots “in factories and even welcomed by many, and do not raise practical, and schools,” (1) as well as “fi ghting fi res, making goods moral or ethical questions.
    [Show full text]
  • Report of Comest on Robotics Ethics
    SHS/YES/COMEST-10/17/2 REV. Paris, 14 September 2017 Original: English REPORT OF COMEST ON ROBOTICS ETHICS Within the framework of its work programme for 2016-2017, COMEST decided to address the topic of robotics ethics building on its previous reflection on ethical issues related to modern robotics, as well as the ethics of nanotechnologies and converging technologies. At the 9th (Ordinary) Session of COMEST in September 2015, the Commission established a Working Group to develop an initial reflection on this topic. The COMEST Working Group met in Paris in May 2016 to define the structure and content of a preliminary draft report, which was discussed during the 9th Extraordinary Session of COMEST in September 2016. At that session, the content of the preliminary draft report was further refined and expanded, and the Working Group continued its work through email exchanges. The COMEST Working Group then met in Quebec in March 2017 to further develop its text. A revised text in the form of a draft report was submitted to COMEST and the IBC in June 2017 for comments. The draft report was then revised based on the comments received. The final draft of the report was further discussed and revised during the 10th (Ordinary) Session of COMEST, and was adopted by the Commission on 14 September 2017. This document does not pretend to be exhaustive and does not necessarily represent the views of the Member States of UNESCO. – 2 – REPORT OF COMEST ON ROBOTICS ETHICS EXECUTIVE SUMMARY I. INTRODUCTION II. WHAT IS A ROBOT? II.1. The complexity of defining a robot II.2.
    [Show full text]
  • Robots and Healthcare Past, Present, and Future
    ROBOTS AND HEALTHCARE PAST, PRESENT, AND FUTURE COMPILED BY HOWIE BAUM What do you think of when you hear the word “robot”? 2 Why Robotics? Areas that robots are used: Industrial robots Military, government and space robots Service robots for home, healthcare, laboratory Why are robots used? Dangerous tasks or in hazardous environments Repetitive tasks High precision tasks or those requiring high quality Labor savings Control technologies: Autonomous (self-controlled), tele-operated (remote control) 3 The term “robot" was first used in 1920 in a play called "R.U.R." Or "Rossum's universal robots" by the Czech writer Karel Capek. The acclaimed Czech playwright (1890-1938) made the first use of the word from the Czech word “Robota” for forced labor or serf. Capek was reportedly several times a candidate for the Nobel prize for his works and very influential and prolific as a writer and playwright. ROBOTIC APPLICATIONS EXPLORATION- – Space Missions – Robots in the Antarctic – Exploring Volcanoes – Underwater Exploration MEDICAL SCIENCE – Surgical assistant Health Care ASSEMBLY- factories Parts Handling - Assembly - Painting - Surveillance - Security (bomb disposal, etc) - Home help (home sweeping (Roomba), grass cutting, or nursing) 7 Isaac Asimov, famous write of Science Fiction books, proposed his three "Laws of Robotics", and he later added a 'zeroth law'. Law Zero: A robot may not injure humanity, or, through inaction, allow humanity to come to harm. Law One: A robot may not injure a human being, or, through inaction, allow a human being to come to harm, unless this would violate a higher order law. Law Two: A robot must obey orders given it by human beings, except where such orders would conflict with a higher order law.
    [Show full text]
  • Dr. Asimov's Automatons
    Dr. Asimov’s Automatons Take on a Life of their Own Twenty years after his death, author Isaac Asimov’s robot fiction offers a blueprint to our robotic future...and the problems we could face by Alan S. Brown HIS PAST April, the University of Miami School These became the Three Laws. Today, they are the starting of Law held We Robot, the first-ever legal and point for any serious conversation about how humans and policy issues conference about robots. The name robots will behave around one another. of the conference, which brought together lawyers, As the mere fact of lawyers discussing robot law shows, engineers, and technologists, played on the title the issue is no longer theoretical. If robots are not yet of the most famous book intelligent, they are increasingly t ever written about robots, autonomous in how they carry out I, Robot, by Isaac Asimov. tasks. At the most basic level, every The point was underscored by day millions of robotic Roombas de- Laurie Silvers, president of Holly- cide how to navigate tables, chairs, wood Media Corp., which sponsored sofas, toys, and even pets as they the event. In 1991, Silvers founded vacuum homes. SyFy, a cable channel that specializ- At a more sophisticated level, es in science fiction. Within moments, autonomous robots help select in- she too had dropped Asimov’s name. ventory in warehouses, move and Silvers turned to Asimov for ad- position products in factories, and vice before launching SyFy. It was a care for patients in hospitals. South natural choice. Asimov was one of the Korea is testing robotic jailers.
    [Show full text]
  • The Turing Test Q: Can Machines Think?
    The Turing Test Q: Can machines think? I compute, therefore I am. Problem: We don’t know what ―think‖ means. Q: What is intelligence? Problem: We can’t define ―intelligence‖. But, we usually ―know it when we see it‖. The Turing Test Q: Which of the following can think? The Turing Test Q: Can you distinguish a machine from a person? ≡ Can a machine impersonate a person? The Turing Test • The first deep investigation into whether machines can ―behave intelligently‖ • Helped usher in field of AI • Decoupled ―intelligence‖ from ―human‖ • Based ―intelligence‖ on I/O, not entity’s ―look and feel‖ • Proposed a practical, formal test for intelligence • Definitions & test are operational & easily implementable • Turing test variants: ―immortality‖, ―fly-on-wall‖, ―meta‖, ―reverse‖, ―subject matter expert‖, ―compression‖, ―minimum intelligent signal‖ Turing Test Milestones Turk (von Kempelen), 1770: • Played a strong game of chess • Toured the world for 84 years • Generated much interest in automata • Was a hoax (hidden human operator) Turing Test Milestones Eliza (Weizenbaum), 1966: • First ―chatterbot‖ • Named after ―Eliza Doolittle‖ of Shaw’s Pygmalion • Simulated Rogerian psychotherapist • Often convinced people it is human • Influence computer games (e.g., Dungeon, Adventure) • Inspired Arthur C. Clarke’s HAL 9000 • Chatterbots appear today in chatrooms and dating services • Chatterbot competitions (Turing Test format): – Loebner Prize – Chatterbox Challenge, 2010 Turing Test Milestones Elbot, 2008: • Won 2008 Loebner Prize • Fooled 3 of 12 judges (Turing: >30%) • Typical excerpt: Judge: What's your favourite novel? Elbot: I often read telephone books, or dadaist poetry: Brä brä brä Judge: Typical computer! Elbot: You tend towards generalizations.
    [Show full text]
  • The Three Laws of Robotics Machine Ethics (Or Machine Morality) Is A
    The Three Laws of Robotics Machine ethics (or machine morality) is a part of the ethics of AI concerned with the moral behavior of artificially intelligent beings. Machine ethics contrasts with roboethics, which is concerned with the moral behavior of humans as they design, construct, use and treat such beings. Machine ethics should not be confused with computer ethics, which focuses on professional behavior towards computers and information. Isaac Asimov considered the issue in the 1950s in his novel: I, Robot. He proposed the Three Laws of Robotics to govern artificially intelligent systems. Much of his work was then spent testing the boundaries of his three laws to see where they would break down, or where they would create paradoxical or unanticipated behavior. His work suggests that no set of fixed laws can sufficiently anticipate all possible circumstances. Asimov’s laws are still mentioned as a template for guiding our development of robots ; but given how much robotics has changed, since they appeared, and will continue to grow in the future, we need to ask how these rules could be updated for a 21st century version of artificial intelligence. Asimov’s suggested laws were devised to protect humans from interactions with robots. They are: A robot may not injure a human being or, through inaction, allow a human being to come to harm A robot must obey the orders given it by human beings except where such orders would conflict with the First Law A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws As mentioned, one of the obvious issues is that robots today appear to be far more varied than those in Asimov’s stories, including some that are far more simple.
    [Show full text]
  • Three Laws of Robotics
    Three Laws of Robotics From Wikipedia, the free encyclopedia The Three Laws of Robotics (often shortened to The Three Laws or Three Laws) are a set of rules devised by the science fiction author Isaac Asimov. The rules were introduced in his 1942 short story "Runaround", although they had been foreshadowed in a few earlier stories. The Three Laws are: 1. A robot may not injure a human being or, through inaction, allow a human being to come to harm. 2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law. 3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws. These form an organizing principle and unifying theme for Asimov's robotic-based fiction, appearing in his Robot series, the stories linked to it, and his Lucky Starr series of young-adult fiction. The Laws are incorporated into almost all of the positronic robots appearing in his fiction, and cannot be bypassed, being intended as a safety feature. Many of Asimov's robot-focused stories involve robots behaving in unusual and counter-intuitive ways as an unintended consequence of how the robot applies the Three Laws to the situation in which it finds itself. Other authors working in Asimov's fictional universe have adopted them and references, often parodic, appear throughout science fiction as well as in other genres. This cover of I, Robot illustrates the story "Runaround", the first to list all Three Laws of Robotics.
    [Show full text]
  • Robotics in Germany and Japan DRESDEN PHILOSOPHY of TECHNOLOGY STUDIES DRESDNER STUDIEN ZUR PHILOSOPHIE DER TECHNOLOGIE
    Robotics in Germany and Japan DRESDEN PHILOSOPHY OF TECHNOLOGY STUDIES DRESDNER STUDIEN ZUR PHILOSOPHIE DER TECHNOLOGIE Edited by /Herausgegeben von Bernhard Irrgang Vol./Bd. 5 Michael Funk / Bernhard Irrgang (eds.) Robotics in Germany and Japan Philosophical and Technical Perspectives Bibliographic Information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data is available in the internet at http://dnb.d-nb.de. Library of Congress Cataloging-in-Publication Data Robotics in Germany and Japan : philosophical and technical perspectives / Michael Funk, Bernhard Irrgang (eds.). pages cm ----- (Dresden philosophy of technology perspectives, ISSN 1861- -- 423X ; v. 5) ISBN 978-3-631-62071-7 ----- ISBN 978-3-653-03976-4 (ebook) 1. Robotics-----Germany----- Popular works. 2. Robotics----- Japan--Popular works. 3. Robotics-----Philosophy. I. Funk, Michael, 1985- -- editor of compilation. II. Irrgang, Bernhard, editor of compilation. TJ211.15.R626 2014 629.8'920943----- dc23 2013045885 Cover illustration: Humanoid Robot “ARMAR” (KIT, Germany), Photograph: Michael Funk An electronic version of this book is freely available, thanks to the support of libraries working with Knowledge Unlatched. KU is a collaborative initiative designed to make high quality books Open Access for the public good. More information about the initiative and links to the Open Access version can be found at www.knowledgeunlatched.org ISSN 1861-423X • ISBN 978-3-631-62071-7 (Print) E-ISBN 978-3-653-03976-4 (E-PDF) • E-ISBN 978-3-653-99964-8 (EPUB) E-ISBN 978-3-653-99963-1 (MOBI) • DOI 10.3726/978-3-653-03976-4 Open Access: This work is licensed under a Creative Commons Attribution NonCommercial NoDerivatives 4.0 unported license.
    [Show full text]
  • The First Law of Robotics ( a Call to Arms )
    :-0 appear, AAAI-94 The First Law of Robotics ( a call to arms ) Daniel Weld Oren Etzioni* Department of Computer Science and Engineering University of Washington Seattle, WA 98195 {weld, etzioni}~cs.washington.edu Abstract should not slavishly obey human commands-its fore- Even before the advent of Artificial Intelligence, sci- most goal should be to avoid harming humans. Con- ence fiction writer Isaac Asimov recognized that an sider the following scenarios: agent must place the protection of humans from harm .A construction robot is instructed to fill a pothole at a higher priority than obeying human orders. In- in the road. Although the robot repairs the cavity, spired by Asimov, we pose the following fundamental it leavesthe steam roller, chunks of tar, and an oil questions: (I) How should one formalize the rich, but slick in the middle of a busy highway. informal, notion of "harm"? (2) How can an agent avoid performing harmful actions, and do so in a com- .A softbot (software robot) is instructed to reduce putationally tractable manner? (3) How should an disk utilization below 90%. It succeeds, but in- agent resolve conflict between its goals and the need spection revealsthat the agent deleted irreplaceable to avoid harm? (4) When should an agent prevent a IjcTEXfiles without backing them up to tape. human from harming herself? While we address some of these questions in technical detail, the primary goal While less dramatic than Asimov's stories, the sce- of this paper is to focus attention on Asimov's concern: narios illustrate his point: not all ways of satisfying a society will reject autonomous agents unless we have human order are equally good; in fact, sometimesit is some credible means of making them safe! better not to satisfy the order at all.
    [Show full text]