ROBOTS AND MORAL CONCEPTS Are Asimov’s three laws of robotics sufficient to guide the actions of contemporary robots or do they need moral concepts to guide their actions?

Date: September 22nd, 2016

Subject: Master thesis

Universty: Tilburg University

Program: MA Philosophy

Track: Ethiek van Bedrijf en Organisatie

Supervisor: dr. Alfred Archer

Second reader: dr. Bart Engelen

Student & anr: Matthijs Smakman, 759122

ACKNOWLEDGEMENTS For making this journey possible I would like to thank the Institute of ICT at the HU University of Applied Sciences Utrecht. For the many papers they reviewed and the discussions that followed I am especially thankful to my colleague Alex Bongers and my dear friend Frank Wassenaar. Also, I am grateful to Willemijn for her love and patience, since it is not always easy living with a student of philosophy.

For his guidance and enthusiasm during this process and his encouragement to present some of my work at the Designing Moral Technologies conference in Ascona and the Research Conference Robophilosophy in Aarhus, I am thankful to my thesis supervisor Alfred Archer.

Finally, I would like to thank al my colleagues, friends and family who supported me during the writing process of this thesis: without your backing, this work would not have been done.

INTRODUCTION To answer the research question “Are Asimov’s three laws of robotics sufficient to guide the actions of contemporary robots or do they need moral concepts to guide their actions?”, this thesis will use four chapters. In the first chapter, I will outline the field of contemporary robotics to familiarise the reader with the subject at hand. This will meanly be done by describing and analysing existing work in the field social robotics and machine ethics. In the second chapter, I will examine Asimov’s three laws of robotics and provide the reader with the philosophical and practical issues concerning these laws. In the third chapter, I will use arguments for well-being as the ultimate source of moral reasoning, to argue that there are no ultimate, non-derivative reasons to program robots with moral concepts such as moral obligation, morally wrong or morally right. In chapter four, I will argue that implementing well-being as a guiding principle might be problematic in the field of robotics. Secondly, I will examine arguments given by Ross (1961) that the complexity of moral judgement requires more than just a concept of well-being, and then argue that there is no need for these extra concepts. Finally, I will argue that certain roles bring certain obligations, role obligations. Therefore, robots need different “moral” programming for the different roles robots are going to perform, but that even in such a theory there is no need for moral concepts. Roles bring certain reasons to act in specific ways, but robots do not need concepts such as moral obligation to apply this.

2 CHAPTER 1 - ROBOTS In this first chapter, I will firstly give a working definition of what a robot is and give some examples of contemporary robots. Secondly, I will analyse why people, especially in the last 50 years, have written extensively about their fears of robots, although the particular circumstances they feared were, and according to some still are, decades away. Thirdly, I will state that this thesis is about moral action, and not moral agency. The main purpose of this chapter is to show the relevance of this thesis and to familiarise the reader with the subject at hand.

WHAT IS A ROBOT?

A robot is an “engineered machine that senses, thinks and acts” (Lin, Abney, & Bekey, 2011). A machine senses using sensors. These sensors are used by the machine to obtain data about the external world, for example by using cameras, GPS-receivers or other ways to acquire data. This data needs to be processed if the robot is to react. This process is called thinking. It can be argued that this classification of 'thinking' is false, but the classification will be adequate for the question of this thesis because it will not effect the arguments. The software process that I call thinking uses rules to process the data from its sensors and to make decisions on how to react. At the basis of these rules is a human, but on the basis of the human programmed code, the robot can learn how to react to new unknown situations.

Machine learning software enables machines to learn from past experiences. The definition of machine learning is: “A computer program is said to learn from experience ‘E’ with respect to some class of task ‘T’ and performance measure ‘P’, if its performance at tasks in ‘T’, as measured by ‘P’, improves with experience ‘E’ (Mitchell, 1997). In other words, if a robot can improve how it performs a certain task based on past experience, then it has learned. Robots that are programmed to use machine learning improve their actions, therefore situations can arise which the engineer or programmer of that robot could possibly be unaware of.

Next to the abilities of a robot to sense and think, it must be able to react autonomously. This reaction follows from its ability to sense and to think. The reaction of the robot happens in the real world, the world humans are living in. Since robots can act autonomously without being under the direct control of a human and make decisions based on sense data, a number of questions arise. For example, how do we ensure that robots don’t harm humans? Lin et. al. (2011) describe an interesting list of ethical questions such as: “whose ethics and law ought to be the standard in robotics”, “if we could program a code of ethics to regulate robotic behaviour, which ethical theory should we use” “should robots merely be considered tools, such as guns and computers, and

3 regulated accordingly” and “will robotic companionship (that could replace human or animal companionship) for other purposes, such as drinking buddies, pets, other forms of entertainment, or sex, be morally problematic?”.

If robots are not programmed with the capacity to make moral decisions disastrous situations can arise, for example, robots forcing people against their will. Because of this Wallach & Allen (2009) argue that robots should be provided with the ability for ethical reasoning and ethical decision- making. There have been many that have taken on the challenge of analysing and implementing moral decision making into robots, such as: Anderson (2008), Coeckelbergh, (2010), Crnkovic, & Çürüklü (2012), Malle (2015), Murphy and Woods (2009), and Wallach (2010).

One possible strategy for the implementation of morality and for answering some of the ethical questions formulated by Lin et. al. (2011) is to program robots in such a way that they are guided by (the software equivalents of) moral concepts such as moral obligation, morally wrong, morally right, fairness, virtues and kindness. Another scenario to implement morality is to create strict, deontic like laws that a robot is to uphold. In the 1940’s Isaac Asimov (1942), in his novel Runaround, proposed three laws of robots that were to guide the action of robots in such a way that their action would by adequate and safe. Both strategies will be discussed in this thesis as ways to implement morality into machines.

EXAMPLES OF CONTEMPORARY ROBOTS

As robots will become part of our daily lives in the next decades it is necessary to ensure that their behaviour is morally adequate (Crnkovic & Çürüklü, 2012). A recent case shows how the behaviour of a robot with artificial intelligence (AI) can go wrong. In 2016 Microsoft launched an AI chat robot on Twitter called Tay. Tay learned by the conversations it had with real humans. After only 24 hours it was taken offline because of tweets like “Bush did 9/11” and “Hitler would have done a better job than the monkey we have got now” (Horton, 2016). Some suggest that hackers deliberately provided Tay with information so that it would learn immoral statements, but this is not confirmed. What this example shows is that a robot, without the right moral guidelines, can start to act immorally. Up until this day Microsoft has not yet re-launched Tay.

While in the 1950s robots were mainly used as industrial tools, today robots have become more social. Already robots are being developed in areas such as entertainment, research and education, personal care and companions, military and security (Lin et. al. 2011). In this paragraph, I will introduce contemporary robots that are being applied in the field of education and healthcare.

4 In the field of education, experiments are being done in which robots, partially, or completely replace teachers. One example is an experiment at a secondary school in Uithoorn, the Netherlands. Although the scientific results of this experiment are not jet published the researchers, Elly Konijn and Johan Hoorn, gave an interesting interview to Martin Kuiper from the NRC (Kuiper, 2016). In the experiment, a small robot called Zora, made by the French company Aldebaran and programmed by the Belgian company QBMT, helps children practice with mathematics. According to Konijn and Hoorn, Zora can help with routine tasks done previously by the teacher, for example explaining a mathematical formula over and over again (Kuiper, 2016). Although more research needs to be done in the area of human-robot interaction studies suggest that college students are more likely to follow behavioural suggestions offered by an autonomous social robot in comparison to a robot directly controlled by a human (Edwards, Edwards, Spence, & Harris, 2016). Experiments using artificial and actual lecturers in insets in video lectures show that: “participants who saw the inset video of the actual lecturer replaced by an animated human lecturer recalled less information than those who saw the recording of the human lecturer” (Li, Rene,́ Jeremy, & Wendy, 2015). Zora may not look like an animated human lecturer, but according to Konijn and Hoorn (Kuiper, 2016), this is not a bad thing since children are very sensitive to facial expressions and are quickly distracted. Another field where Zora, and other robots, are being applied is in healthcare and companionship for elderly people with dementia. This field might be less dynamic or complex than teaching children mathematical formulae but no less interesting from the moral point of view.

Another contemporary robot is Paro, an interactive therapeutic robot that looks like a baby seal. An area where robots such as Zora and Paro are implemented is healthcare, especially dementia care. Since the number of people suffering from dementia is expected to strongly increase due to the increased life expectancy, the medical cost will also rise (RIVM, 2016). Dementia is already the costliest illness in the Netherlands, at five billion euro’s a year. Even with the most optimistic prediction, this will steeply increase to 7.5 billion euro’s a year in the coming years (Alzheimer Nederland, 2013). Besides the cost for society, dementia comes with burdens on the social environment of the patient. According to Bharucha, et al. (2009) intelligent assistive technologies can compensate for the specific physical and cognitive deficits of older adults with dementia, and thereby reduce caregiver burden. Technology can therefore not only improve the standard of living for the patient but in doing so also improves the life of the caregiver by reducing burden. To accomplish this four categories of robots are being implemented into this field, namely: rehabilitation robots, service robots, telepresence robots and companion robots (Ienca, Jotterand, Vica, & Elger, 2016). There are multiple ethical questions that arise with the introduction of these

5 robots, such as the question regarding obtaining informed consent, privacy, and safety. As there is a long tradition of fearing robots, as will be argued in the next paragraph, “Why are we so afraid”, for now, I will focus on safety. According to Ienca, et. al. (2016): “good system safety norms require that a robot used in health care or as a commercial application is safe and that its use does not cause any increased risk of harm for users”. One of the ways safety should be obtained is through ethical-social strategies (Ienca, et. al. 2016). This thesis will assess arguments concerning how moral concepts can be applied in the field of robotics in chapter three and four.

According to Ienca, et. al. (2016) “safety largely translates into the concept of non-maleficence, i.e. the principle of avoiding (preventing and not-inflicting) harm”. They link this with the principle of beneficence, which in the field of robocare:

“require a careful assessment of the balance between therapeutic, assistive or psychosocial benefit, on the one hand, and potential risks or distress, on the other hand. The promotion of the best interest of the user would also require a careful and continuative evaluation of their positive and negative experiences, with the knowledge that the user’s preferences and experiences may change over the progression of the disease and that their ability to communicate those preferences and experiences may decrease over time” (Ienca, et. al. 2016).

The way a robot is to judge what is in the best interest of a patient, therefore, needs to be flexible. A static concept of what would be in the best interest of the patient may not take into account the changing preferences and experiences of a patient.

In this subsection, I have shown two examples of contemporary applied robotics in the field of education and healthcare & companionship. With these two examples, I have argued that there are practical and theoretical ethical issues with contemporary robotics, ranging from: “how does a teacher-robot react to inappropriate language”, to “how can a robot promote the best interest when this changes over time, like with patients suffering from dementia”. Next to these ethical issues I have also argued that there are expected advantages using robots in these fields, reaching from teachers spending more time on personal attention as the robots do the routine tasks, great financial benefits reducing the cost of healthcare, to the improvement of the lives of patients and caretakers. Although these can be great benefits to the uses of robots in these fields there seems to be an underlying fear of robots, especially when combined with Artificial Intelligence (AI). In the next subsection, I will examine this fear for intelligent robots.

WHY ARE WE SO AFRAID?

6 There is a long-standing tradition in fearing that man would breach into the realm of god by creating life (McCauley, 2007), for example, the legend of Gollum (Bilefsky, 2009) and the myth of Prometheus how stole the fire from the Greek gods and gave it to men. Isaac Asimov called this fear the Frankenstein Complex (Asimov, 1978). Lee McCauley argues that this “Frankenstein Complex” is still clearly visible in Hollywood movies like: “Terminator (all three); I, Robot; A.I.: Artificial Intelligence; 2010: “a Space Odyssey; Cherry 2000; D.A.R.Y.L; Blade Runner; Short Circuit; Electric Dreams; the Battlestar Galactica series; Robocop; Metropolis; Runaway; Screamers; The Stepford Wives; and Westworld” (McCauley, 2007). More recent examples are the movie Ex Machina and the TV series Humans. In all of these movies, robots try to harm people in some way. But, as (McCauley, 2007) points out, not only Hollywood has given us reason to fear but also academics such as Ray Kurzweil (2006), Kevin Warwick (2004), Bill Joy (2000), and Hans Moravec (1998). More recent accounts include Stephen Hawking who said AI-robots “could spell the end of the human race” (Cellan-Jones, 2014); Elon Musk, owner of the Tesla car company, called AI “our biggest existential threat” (Gibbs, 2014); and Steve Wozniak, co-founder of Apple, said: “computers are going to take over from humans, no question… if we build these devices to take care of everything for us, eventually they'll think faster than us and they'll get rid of the slow humans to run companies more efficiently” (Smith, 2015). According to McCauley (2007) the strongest point in the arguments to fear robots to take over: “hinges on the assumption that the machines will become too complicated for humans to build using standard means and will, therefore, relinquish the design and manufacture of future robots to intelligent machines themselves”.

The fear of building something we cannot control goes to the core of the Frankenstein Complex. Even though to some this may still sound unrealistic, companies like Google are already implementing interruption policies, more popular known as “big red kill switches” (Burgess, 2016). These policies are being built into algorithms to safely stop a machine to prevent artificial intelligence from spinning out of control. In a joint paper by Google Deepmind and The Future of Humanity Institute at the University of Oxford, a framework is being proposed “to allow a human operator to repeatedly safely interrupt a reinforcement learning agent while making sure the agent will not learn to prevent or induce these interruptions” (Orseau & Armstrong, 2016). The question how to build a piece of software that can be safely switched off at all time is clearly a technical one. But the reason for building this switch, to make sure that when a robot is doing harm it can be stopped, is relevant for this thesis and shows it relevance.

7 Artificial intelligent robots are “unlikely to behave optimally all the time” (Orseau & Armstrong, 2016). Although the biggest fear is the total extermination of the human race, as feared by some prominent experts and exploited by Hollywood, it seems hard to see robots like Zola or Paro as constituting a threat to the human race. But, as Orseau & Armstrong (2016) point out, robots are unlikely to behave optimally all the time, especially in the current experimental stage.

THIS THESIS IS ALL ABOUT ACTION

The American author and professor of biochemistry at Boston University Isaac Asimov described three laws of robotics that guide robots towards “good” behaviour (Asimov, 1942). The question this thesis tries to answer is: should contemporary robots be programmed with moral concepts as a guiding mechanism, or do Asimov’s three laws of robotics suffice? This will be done by first exploring Asimov’s laws of robotics. And secondly, examine what role moral concepts should play in a human life, and based on the answer to this question the thesis will examine how moral concepts can play a role in robotic decision making. This thesis is not going to try to argue that robots will be full Autonomous Moral Agents, but will examine how moral concepts as moral obligation, moral wrongness, kindness, virtuous and fairness, should guide the actions of robots.

CHAPTER 2 - ASIMOV’S LAWS OF ROBOTICS, DO THEY SUFFICE? In 1942 Isaac Asimov published his Three laws of robotics (Asimov, 1942). These rules should govern the behaviour of robots and make sure their behaviour is adequate. Although Asimov was a scientist - he was an instructor in biochemistry at Boston University School of Medicine - he is perhaps most known for his science fiction novels. His three laws of Robotics first appeared in his novel Runaround (1942). The three laws of robotics included “the essential guiding principles of a good many of the world's ethical systems” (Asimov, 1946). Although the three laws originally were not published in a scientific journal, this does not make the laws less prestigious. They have become so common “among practising roboticists and software developers that they influenced, if only subconsciously, the course of robotics” (Clarke, 1993). Since the three laws of robotics have had an influence on practising roboticists and software developers it seems fit to start with an analyses of these three laws.

The three laws of robotics are:

• first law: A robot may not injure a human being, or, through inaction, allow a human being to come to harm;

8 • second law: A robot must obey orders given it by human beings, except where such orders would conflict with the First Law; • third law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

The three laws, at first glance, look relatively simple and attractive; three laws that build on each other. The fist law secures humans from any harm being done by robots, and at the same time obligates robots to prevent humans from harm. The second law makes sure a robot will not act in ways humans don’t want to, because a machine must obey human orders, as long as they are not in conflict with the first law. When these two laws are not in conflict, the third law creates a kind of self-sustaining mechanism, making the robot: “seek to avoid its own destruction through natural causes or accident; defend itself against attack by another robot or robots; and defend itself against attack by any human or humans” (Clarke, 1993). Although these laws seem appealing there are blind spots that, when applied, cause difficulties. These difficulties are a main theme in Asimov’s novels. To analyse the difficulties of the three laws this chapter will look at Murphy & Woods (2009), Anderson (2008), Sloman (2006) and Clarke (1993).

PROBLEMS WITH THE FIRST LAW

The first law states that a “robot may not harm a human being, or through inaction allow a human being to come to harm” (Asimov, 1942). According to the Robin Murphy and David Woods, the biggest problem with the first law is: “that it views safety only in terms of the robot – that is, the robot is the responsible safety agent in all matters of human-robot interaction” (Murphy & Woods, 2009). This brings problems in, for example, a legal perspective since the robot is a product and not a responsible agent. Although a robot could be causally responsible, it would seem strange to hold a robot accountable. The robot is programmed to behave in a certain way, or it is programmed to learn how to behave in a certain way, in both cases it is not accountable for its own behaviour. Imagine a robot vacuum cleaner that sucks up your cat which was lying on the floor, thereby killing it. In this case, the robot is certainly causally responsible for the cat dying, but not accountable. If your brother were to suck up your cat with a vacuum cleaner it would be a totally different case, assuming that the brother is an autonomous moral agent. Murphy and Wood’s (2009) point out that the first rule suggests that the robot is the responsible agent, but that is not the case, at least not legally. That is why they propose an alternative first law: “A human may not deploy a robot without the human robot work system meeting the highest legal and professional standards of safety and ethics” (Murphy & Woods, 2009). This law focuses more on the engineer and the organisation that is responsible for the designs and thereby also for the behaviour of the robot. By

9 changing the first law in a guideline for engineers Murphy and Woods (2009) change the law from a direct guideline for robots to an indirect one. Although the first law doesn’t say that robots are responsible agents, it suggests that the robot is, thereby diverting the responsibility of engineers.

Another problem recognised by Murphy and Woods (2009) is that a robot cannot recognise humans without mistakes. This shortcoming is also mentioned by Roger Clarke (1993). The problem emerges because it is difficult to programme that is to be recognised as a human. A human can, for example, be recognised by its skin colour, temperature or form, but this will inevitably exclude some humans. The exclusion of some humans could have disastrous consequences. Therefore, because the first law cannot be implemented flawlessly, the law is non-absolute. But the same argument could be made for pets or other animals. If a robot were to see animals as equivalent to a paving stone and treat it in the same way, this would lead to undesirable behaviour. This brings up another problem focused on by Aaron Sloman (2006), speciesism. Asimov’s laws seem only to take into account the prevention of human harm, and no other species, except maybe the robot itself in the third law.

An “anachronism” is what Murphy and Woods (2009) call the Asimov’s first law because it is already widely broken. Murphy and Woods (2009) claim that some armies already allow weaponized robots to kill people. Although it may be true that this law is not being implemented in some military robots, better known as “killer robots”, I argue that it does not follow that it should not be implemented in robots operating in other areas such as healthcare and education. It is clear that Zora killing a school child is not the desired behaviour. But even the uses of killer robots in warfare is still highly controversial and widely debated by for example Sparrow (2007), Krishnan (2009) and Leveringhaus (2015). Therefore Murphy and Woods (2009) argument may show that the first law is already being broken by killer robots, but it is not to say that this is ethically permissible, even for military purposes. Even if it would be permissible for military purposes it may not be for teaching robots or robots caring for elderly people suffering from dementia. This suggests that that Asimov’s laws are not as universal as they are meant to be and that different laws are needed for various situations.

The concept of harm

Regardless of the possibly already existing “killer robots”, robots may not be able to perceive the intent human have or reliably interpret contextualised scenes enough to obey the first law (Murphy & Woods, 2009). This is in line with what Sloman (2006) calls one of the main problems, vagueness. The concept of harm is not specified in the three laws, and what counts as harm may

10 differ in contextualised scenes that robots can’t interpret as posted by Murphy and Woods (2009). The concept of harm and how exactly to define harm is contested within philosophy until this day; “there are significant disagreements about what counts as a harm” (Bradley, 2012). Ben Bradley (2012) argues in his paper Doing Away With Harm that the concept of harm should be replaced by other concepts. Bradley argues that existing theories of harm are inadequate and he proposes seven desirable features for theories of harm: Extensional adequacy, Axiological neutrality, Ontological neutrality, Amorality, Unity, Prudential importance and Normative importance. According to Bradley (2012), the comparative account of harm is the most widely discussed but faces strong counterarguments that seem to make it unsustainable. I will give a couple of examples talked about by Bradley to show that the concept of harm is a complex one, perhaps even a concept we should get rid of.

A comparative account of harm can be explained as: “a harmful event is an event that makes things go worse for someone, on the whole, than they would have gone if the event had not happened” (Bradley, 2012). Bradley argues that there are three problems with this view, involving: pre- emption, omission and the non-identity problem. The pre-emption problem is that there are cases in which the comparative account fails to count a harmful event as harmful. The omission issue involves situations in which a person doesn't get a benefit. And “failing to benefit someone moves that person down on the well-being scale, and therefore counts as a harm” (Bradley, 2012). Bradley uses a Batman example to clarify his omission argument.

“Suppose Batman purchases a set of golf clubs with the intention of giving them to Robin, which would have made Robin happy. Batman tells the Joker about his intentions. The Joker says to Batman, “why not keep them for yourself?” Batman is persuaded. He keeps the golf clubs. The comparative account entails that Batman has harmed Robin, because Robin would have been better off if Batman had not kept the clubs” (Bradley, 2012).

If the event “the keeping of the clubs” had not taken place, Robin would have been better off. But, “merely failing to benefit someone does not constitute harming that person” (Bradley, 2012). In this case an event that is not harmful is branded harmful.

Matthew Hanser (2008) provides an event-based account of harm. According to Hanser “Someone suffers a harm if and only if he suffers a harm of some level with respect to some basic good” (Hanser, 2008). This is what he calls a non-derivative harm. Peter is, for example, harmed if he is deprived of a share of basic good. Hanser says about these goods: “Goods are not states or conditions that it is good to be in. Rather, they are things that it is good to have”. These basic

11 goods are the building blocks for a happy life. Next to this form of non-derivative harm there is a derivative harm. A derivative harm is being prevented from gaining a basic good. Peter is, for example, harmed when he is hindered in obtaining a share of a basic good. These derivative harms can come in different levels. The second is that Peter is prevented from getting a basic good, the third is Peter being prevented from being prevented from not getting a basis good.

I would like to argue against robots using such a layered concept of harm, because it would make them ineffective. A robot depriving a person from a basic good would be doing non-derivative harm. When a robot steals money from Peter, Peter is harmed. Also when a robot is blocking Peter’s way to the ATM Peter would be harmed as the robot deprives Peter from getting a good. If this is the case, a robot standing in a queue in a supermarket would be depriving someone on some level of a basic good, thus the robot is harming someone. This would lead to a situation in which a robot would constantly be harming someone.

With the example above I have tried to show that the concept of harm is hard to define. According to Bradley (2012) the reason for this could have to do with moralizing. Bradley points out that “we are more likely to call an act harmful if we think it is wrong” (Bradley, 2012). Bradley (2012) beliefs that the best action is to avoid to notion of harm in moral theorizing. According to Bradley (2012), consequentialists do not need the concept as they can use the intrinsic values of consequences of acts, and deontologists don’t need the concept as they can use concept such as the violation of rights or the inflicting of pain or injury. Perhaps, in retrospect, Asimov should have never used the concept of harm. Because he did not elaborate on the concepts of harm, the main problem concerning Asimov’s first law is the problem of vagueness.

Other issues concerning the first law

The first law does not recognise the possibility of an action where one human threatens a whole group of humans. The first law would not allow a robot to harm the human who threatens the group. As a response to this objection Asimov made an adapted version of his first law, namely: “The Machine cannot harm a human being more than minimally and that only to save a greater number” (Asimov, 1950). Here, like in the example of defining humans, the first law is non- absolute and the value of humans is based on numbers. Next to the law not being absolute, it seems that harming an innocent person for benefits of a group is not obviously the right action.

As argued, Asimov’s first law has practical and philosophical issues. Some issues are stronger than others. The practical issues regarding the difficulty to unmistakably recognise humans and harming humans as minimally to save a greater number seem less pressing then the problem of vagueness.

12 Because the concept of harm is so crucial in the first law, it should be clear what it entails. But, as argued, there is significant disagreement about what counts as harm. Therefore, I argue, since the law does not give any guidance to what it means by harm, that the first law needs further development before being applied in the field of robotics. Although this is a serious issue it would be premature to dismiss all Asimov’s laws on this basis. The next paragraph will assess the second and third law of robots and give a philosophical, Kantian critique to Asimov’s laws. Thereafter, this chapter will end with the conclusion that Asimov’s three laws combined are not easily implemented and have both practical and philosophical issues differing in weight.

PROBLEMS WITH THE SECOND LAW

The second law, “a robot must obey orders given it by human beings, except where such orders would conflict with the First Law”, also has difficulties when being applied. According to Murphy & Woods (2009), one of the challenges concerns the understanding of language: “robust natural language understanding still continues to lie just beyond the frontiers of today’s AI” (Murphy & Woods, 2009). I argue that this argument is only a contemporary problem that will be solved in the future, just like the great challenges computer science faced in the past. I will provide three examples of previous challenges by which I argue that the problem of understanding language will be solved.

In 1995, IBM's supercomputer Deep Blue defeated the former world chess champion Garry Kasparov (McPhee, Baker, & Siemaszko, 1997). In 2011, IBM's supercomputer Watson defeated two of the best players in the game show Jeopardy, a game in which candidates receive the answer to a question and then need to formulate the correct question (Markoff, 2011). Now, in 2016 Google's supercomputer AlphaGo defeated one of world's best Go players, Lee Sedol (Borowiec, 2016). This 2500-year-old strategy game has more board positions than there are atoms in the universe, according to Deep Mind's CEO Demis Hassabis (Borowiec, 2016). This vast number of possibilities “make it difficult to follow a particular strategy, and mastering the game means using intuition to react to any number of possible twists or turns” (Borowiec, 2016). The intuition master Go-players rely on, was thought to be not possible for computers to reproduce, but it was. It seems that IBM Watson already can adequately understand language, but if it does not, language understanding will be solved eventually, just like chess, Jeopardy and Go.

Murphy & Woods (2009) add to the language understanding problem one more dimension, that of nonverbal communication. A big part of human’s communication is nonverbal and the combination of verbal and nonverbal communication will make it even more complex for robots to understand their orders.

13 Next to the complexity in the way the orders are given and interpreted, there is another interesting point, that of conflicting orders. Clarke (1993) brings up a case where conflicting orders are being given by two humans. In this case, the robot should prioritise the conflicting orders. Which brings up questions like, do all humans have equal standing? In the case, of two contradicting orders with equal strength, a robot would not be able to make a decision. According to Clarke, the robot should: “either choose arbitrarily among the alternative strategies or arbitrarily modify an arbitrarily chosen strategy variable (say, move a short distance in any direction) and re-evaluate the situation” (Clarke, 1993).

There seems to be a problem with the way a robot universally has to choose between conflicting orders. I argue that this problem is mostly caused by the universality of Asimov’s laws. If the problem is framed in a specific situation it changes. Imagine for example that a healthcare robot in a hospital is ordered by a nurse to clean a patient, but the patient doesn’t want cleaning and orders the robot not to. Now the robot is faced with conflicting orders. A solution could be a chain of command system. In a chain of command system, not every user has the same ordering rights. For example, an order given by the nurse cannot be undone by the patient, but when a patient absolutely refuses to be touched by the robot, then the robot can not force the patient. Like in almost any software application users can have different rights, as a way to overcome the conflicting order problem. A chain of command system may allow robot to prioritise orders, but this causes other ethical issues. For example, what if the robot is ordered to administer the patient with a life-saving drug and the patient refuses, is it to headlock a patient and force the pills down the throat? A simple chain of command system with rights to order is clearly not enough to ensure the adequate behaviour in these situations, but a complex system of moral concepts may be. What this chain of command should be is a separate ethical discussion for which I recommend reading Pontier and Hoorn (2012).

PROBLEMS WITH THE THIRD LAW

The third law, “a robot must protect its own existence as long as such protection does not conflict with the First or Second Law”, gives the robot a kind of self-defence mechanism. The problem with the third law according to Murphy and Woods (2009) is that it will not be implemented, because no one wants to pay for it. It will require extra sensors and consumers will not be willing to pay for it, because they are “wildly overconfident that trouble and complexities outside the bounds of expected behaviour rarely arise” (Murphy & Woods, 2009). This confidence makes it not less proper to program robots with a third law, but more difficult to implement. Also, why can

14 a robot not be programmed with the third law without extra sensors? In this cases the robot would just protect itself using the sensors it has available.

A KANTIAN PERSPECTIVE ON ASIMOV’S THREE LAWS

Susan Leigh Anderson (2008) gives a Kantian argument why Asimov’s three laws of robotics do not suffice. According to Anderson “in following any ethical theory the agent must consider at least him/her/itself, if he/she/it has moral standing, and typically others as well, in deciding how to act” (Anderson S. L., 2008). Therefore, in analysing if Asimov’s laws should be used as guiding theory for the behaviour of robots, the question of whether robots have moral standing must be discussed. Anderson provides five different strategies that all fail to provide argument for robots to have moral standing:

1. In Jeremy Bentham’s theory “sentience” is the leading criterion for moral standing. For a robot to have moral standing it must have feelings. According to Anderson, “it would be very difficult to determine whether a robot has feelings… even though it is [also] difficult to determine whether even another human being has feelings like oneself” (Anderson S. L., 2008). 2. Immanuel Kant uses the concept of “self-conscious” as a leading criterion. For a robot to have moral standing it must be conscious of itself. According to Anderson, a robot could sound like it is self-conscious, “but to prove that it really understands what it is saying and that it has not just been “programmed” to say these things is another matter” (Anderson S. L., 2008). 3. For an entity to have moral standing, according to Michael Tooley (Tooley, 1972), it must desire it, which involves conscious experience. As in the previous examples, this would be problematic when applied to the field of robotics. Again a robot can act if it desires something, but “there would be those who would claim that the robot had simply been “programmed” to do and say certain things” (Anderson S. L., 2008). Even if a robot where to desire some good, it seems problematic for a robot to have a conscious experience. 4. According to Tibor Machan for an entity to have moral rights it needs to be a moral agent, an entity that is exacted to behave morally. So if a robot would respect the rights of others, it would have rights itself. Even if this is right: “just because a machine’s behaviour is guided by moral principles does not mean that we would ascribe moral responsibility to the machine” (Anderson S. L., 2008). 5. Mary Anne Warren argues that an entity needs to be a member of a moral community. This “membership” is not restricted to human-beings. One of Warren’s characteristics that

15 define personhood is emotionality. But, since emotionality can also cloud ethical judgement and delude the morally right action Anderson (2008) is not convinced that emotionality should be a serious criterion.

I agree with Anderson (2008) that it is problematic to determine which criteria should be used to give a robot moral standing. Therefore, as stated in the chapter one, this thesis will focus on the concepts that should guide the behaviour of robots and for now, assume that robots do not have moral standing. From this premise, Anderson gives a Kantian perspective on Asimov’s three laws of robots which I will examine below.

By looking at Asimov’s The Bicentennial Man (1984) Anderson (2008) states that the three laws of robotics allow humans to mistreat robots. Not in a way that robots have moral standing, but the three laws allow humans to, for example, vandalise the robot. Anderson uses a similar situation discusses by Kant (1963) in his Lectures on Ethics to reflect on Asimov’s laws that allow humans to mistreat robots. According to Kant, although animals do not have moral standing, humans should still not mistreat them because “[t]ender feelings towards dumb animals develop humane feelings towards mankind” (Anderson S. L., 2008). Projecting this at robots we do not have a direct duty to be kind to robots, but we have an obligation towards then, as “indirect duties towards humanity” (Kant, 1963). Using Kant’s argument, Anderson (2008) argues that any ethical law humans make must be in favour of the respectful treatment of entities that lack moral standing themselves. Not because of the entities, but because it may affect the ethical behaviour of humans in a negative way. Anderson even dares to state that “the three laws make it likely that having such laws will lead to humans harming other humans as well” (Anderson S. L., 2008). Anderson tries to argue that Asimov’s laws, by letting human mistreat robots, will have a negative effect on how humans treat each other. And since the ethical guidelines of robots should prevent humans from harm, the three laws of robots are no good ethical guidelines for robots. Anderson (2008) finished her argument by stating that: “since Asimov’s Three Laws permit humans to mistreat robots/intelligent machines, they are not, according to Kant, satisfactory as moral principles that these machines should be required to follow” (Anderson S. L., 2008).

After closely looking at the Kantian argument I do not find it convincing. If Anderson’s argument would be that the three laws are harming humans indirectly by letting people mistreat robots, and therefore they are not good principles, I would probably agree. Just as I agree on her Kantian argument that we should not mistreat animals or robots, because it would have a bad effect on our character. But it does not follow that ethical guidelines that do not enable a robot it to defend itself against mistreatment are wrong. This would be to say that defenceless animals are morally harmful

16 because they cannot defend themselves against malicious humans, which is obviously false. Kant’s original argument gives reasons for humans not to harm animals. The same argument can be used to argue that humans should not mistreat robots, but not that Asimov’s laws are unsatisfactory. Anderson (2008) does not seem to consider laws that regulate people, like laws that prohibit people from mistreating animals or ways a robots may discourage people from mistreating them.

PROVISIONAL CONCLUSIONS

In this chapter I have argued that all three of Asimov’s laws have problems that differ in weight and complexity, varying from philosophical to practical issues. Some of the technological issues I argue will be solved in time, like the other challenges faced in the past, whereas the more philosophical issues like the concept of harm will need ongoing reflection. The first and most important law, as it is recurring in the second and third law, has problems concerning vagueness, the difficulty to interpret contextualised scenes, the recognition of humans, existing first-law- breaking robots and issues concerning the liability of a robot. Next to the list of problems concerning the first law, the second and third law face issues regarding conflicting orders, the understanding of language and the probability to be implemented.

I have argued that although robots do not have moral standing there are indirect reasons to not mistreat them, namely because it would have a negative effect on human behaviour. But, it does not follow that Asimov’s laws are unsatisfactory because they enable robots to be mistreated. The strongest argument that makes Asimov’s laws unsatisfactory is that the concept of harm is vague.

CHAPTER 3 – ROBOTS AND MORAL CONCEPTS In Asimov’s The Caves of Steel (2011) a person called Baley sits in a car with a robot named R. Daneel. Baley suggests that a code of behaviour, like that of the bible, is higher than any law, including the three laws of robotics. Baley then quotes a famous peace of the bible in which a woman is found guilty of adultery, but according to Jesus, she should not be stoned as the law prescribes. “He that is without sin among you, let him first cast a stone at her”, Jesus said. When Baley refers to concepts as mercy, forgiveness and justice R. Daneel replies: “I am not acquainted with those words” (Asimov, 2011). The idea that something goes beyond the three laws is for R. Daneel a statement that is necessarily false: “Higher than law? Is that not a contradiction in terms?” (Asimov, 2011). In this chapter, I will examine if robots should be programmed with moral

17 concepts as mentioned by Baley in The Caves of Steel and if robots should be guided by these concepts.

By programming robots in such a way that they are guided by (the software equivalents of) moral concepts, such as mercy, forgiveness and justice some of the ethical questions formulated by Lin et. al. (2011) could possibly be answered. This chapter will assess whether the behaviour of robots should or should not be programmed to be guided by moral concepts by examining arguments given by Roger Crisp (2006) and Brian McElwee (2010).

According to Crisp (2006), we should talk about morality without moral concepts because morality provides only non-ultimate reasons. Well-being is the ultimate source of all moral reasons (Crisp, 2006). And because morality does not provide ultimate reasons to act, we should not start with the conception that people have any moral obligation when analysing morality. In this section, I will show the arguments Crisp (2006) gives for demoralising ethics and how this can help understand why robots should or should not be programmed with moral concepts. Crisp (2006) claims that: “The kind of reasons philosophical ethics should be most concerned with are ultimate or non-derivative in nature”, actions which will advance well-being.

According to Crisp (2006), nearly any human society has: “a set of cognitive and conative states, including beliefs, desires, and feelings, which leads its possessors among other things to (a) view certain actions as wrong (that is, morally forbidden) and hence to be avoided, (b) feel guilt and/or shame as a result of performing such actions, and (c) blame others who perform such actions.” He calls this positive morality to correlate with the term positive law. He uses these terms to strengthen his claim that there is a strong analogy between morality and law.

POSITIVE LAW AND POSITIVE MORALITY

The first analogy is that both law and morality are: “mechanisms for guiding human action towards similar kinds of goals” (Crisp, 2006). The focus here is firstly on criminal law, not civil law. He assumes that the two concepts have only one function, to tell what we have most reason to do, which will be argued against later in this chapter. The second analogy is that both law and morality: “involve the forbidding of certain actions, and the infliction of sanctions on those who perform these actions” (Crisp, 2006). The third analogy is that both law and morality have a similar origin. Our ancestors developed morality so that the group would function better, and because of that survive. At first these would be just basic emotions such as anger and fear, but after humans developed language, more complex emotions became possible. With the analogies of positive law and positive morality Crisp (2006) argues that positive morality does not provide non-ultimate or

18 derivative reasons. So the moral reason to be kind does not come from the moral obligation to be kind, but because it increases well-being. But for morality to function effectively, people should take it to be ultimately reason giving (Joyce, 2001). According to Crisp (2006), this is clear because emotions of guilt and shame involve the thought of wrongness.

THE FUNCTION OF MORAL CONCEPTS

The goal of normative ethics, says Crisp (2006), is to tell what we have most reason to do. Moral concepts such as moral obligation, moral wrongness, kindness, virtue and fairness are not the ultimate source of our reasons. These only provide non-ultimate or derivative reasons. The ultimate source for all moral reasoning is to promote well-being, according to Crisp (2006). Because of this, according to Crisp (2006), there is no need to consider other philosophical views. Another reason to avoid moral concepts is that they come with strong normative emotions like shame, guilt and blame. These emotions can cloud our judgement and for that reason, they need to be avoided. To answer the question “what makes people's lives better” there is no need for concepts such as moral wrongness, obligation, cruel, good or bad. All that we need to know, is what acts are going to best promote people's well-being.

So integrating moral concepts in robots is not only difficult, it is best to be avoided. Although robots can't feel emotions like humans do, at least not yet and not for the foreseeable future according to Lin et. al. (2011), we should still try to avoid moral concepts as a cowardly robot or a loving robot. The robot should be aimed at increasing well-being and this is what the focus of the debate should be. Moral concepts, as shown, do not independently provide reasons for actions, and this is why they should be eliminated or at least be avoided.

OTHER FUNCTIONS OF MORALITY

Brian McElwee (2010) responds to Crisp’s argument that moral concepts and ethics need to be reason-giving. If the goal of morality is to maximise well-being, then it seems strange to think that moral obligation would give additional reasons to act morally, reasons that are not exhausted by the reason to promote well-being. According to McElwee (2010), the concept of moral obligation involves the idea of reason implying. Hence, moral concepts are not reason providing, but reason implying. Therefore, moral wrongness does not give reasons not to act in a certain way, but it implies that there are reasons not to do it. These reasons do not come from the moral wrongness of it, they come from the facts that make it wrong. The facts that make an act wrong are also the facts that give reasons not to perform the act. The wrongness in itself does not provide any reasons, according to McElwee (2010). This view does not conflict with Crisp’s view that moral

19 concepts do not provide reasons for action. Therefore, I argue that since ethics should point people to reasons to act, and moral obligations do not provide reasons to act, they still should play no role, and should be abandoned.

Crisp (2006) assumes a situation where all the reasons to act in a certain way are known. In this situation, there would be no role for moral obligation. Moral obligation would then not be telling us anything extra, and because of this, it would not be useful. But even with Crisp’s argument, there are still reasons not to eliminate moral obligation. Agreeing that the function of ethics is to guide behaviour does not mean this is the only function. According to McElwee (2010), there are other functions. The function of, for example, moral obligation is not only to give reason for actions, but its function can also be found in other areas.

The function of moral obligation, according to McElwee (2010), is to tell us how to react to agents that behave in particular anti-social ways. He argues that: “even if morality does not itself provide reasons for action, the moral categories nevertheless have a central role to play in ethical theory: they allow us to make crucial judgements about how to feel about and react to, agents who behave in antisocial ways, and they help motivate us to act altruistically” (McElwee, 2010). So knowing something is morally required can help motivate the performance of actions. This means the concept of moral obligation helps an agent to make a choice of which actions he or she should perform in a certain situation. In McElwee’s view, moral obligations help agents to react to people in certain ways. There are three functions of moral obligation according to McElwee (2010). The first is that moral obligation offers an assessment of the agent. To say something is morally wrong is to say it is an unacceptable form of behaviour. The second is that we need the concept of moral obligation to take a stand. We need the concept to make a stand to which actions can be tolerated and which actions cannot be tolerated. The final and third function is to play a motivational and reinforcing role. To believe something is morally required can reinforce our motivation to perform an action. These three functions can’t be reduced to their reason-giving function.

TOTAL RATIONAL AGENTS

McElwee (2010) has shown that the function of moral obligation can be wider than just reason giving. One response of Crisp (2006) is to have a thought experiment. In a society with only complete rational agents, that know what actions will promote the most well-being, there is no need for morality to guide actions. This group of rational agents would not be missing anything if they had no concepts such as moral obligation. The absence of for example moral wrongness, according to Crisp (2006), doesn't seem to be problematic in this society.

20 This thought experiment shows that in a certain ideal society, there would be no need for moral obligation or other derivative moral concepts. In this society there is no need for the functions McElwee (2010) presents: an assessment of the agent, to take a stand or to motivate and reinforce behaviour. Inhabitants of such a society would use well-being to determine what they have most reason to do. But in reality, there is no such ideal society. Humans are not completely rational beings and are often guided by evolutionary and sociological processes in their moral decision making (Haidt & Kesebir, 2010). One example is conformation bias.

Conformation bias is the “tendency to search for confirmatory, but not disconfirmatory, evidence for the hypotheses [people] believe” (Mandelbaum, 2014). An example of this bias could be an American orientating on how to vote in the election for the next president of America. Imagine this American having a feeling that the system is rigged. When she hears Donald Trump saying the system is rigged she will have a stronger tendency to think he is right then Hillary Clinton saying the system is fine, although neither of them will provide solid evidence. Social media can also be seen as a way the biased beliefs are being strengthened. Users who uses sites that offer personalised content, like Google, Facebook, Twitter and Instagram are likely to view more content that confirm their beliefs opposed to content that challenge their beliefs. This strengthens the already existing belief.

Another example of irrationality is that apparently, when it comes to protecting lives, one plus one is less than two. This is what Paul Slovic calls “psychic numbing” (Slovic, 2007). Slovic’s research shows that the value of a human lives decreases when the numbers of lives at risk increase. One of the fundamental mechanisms that play a role here is the capacity to experience affect, “the positive and negative feelings that combine with reasoned analysis to guide our judgments, decisions, and actions” (Slovic, 2007). The statistics of, for example, the 13.5 million people in need of humanitarian assistance inside Syria (European Commission, 2016) fail to spark emotions or feelings that will lead to action. According to Slovic (2007) we should look at moral arguments and international law to guide us to action, because moral feelings cannot be relied upon.

Next to the biases and failure of moral feelings to properly motivate us to guide our action, people are selfish. Haidt and Kesebir (2010) describe a functionalist approach of morality, a morality that: “binds individuals together, suppresses selfishness and directs people's strongest moral passions” (Haidt & Kesebir, 2010). This morality, partially shaped by evolutionary selection and living in groups, guides behaviour towards good actions and punishes bad behaviour. What Haidt and Kesebir (2010) abandon is the search for ethical truth, without saying that anything goes. Morality

21 has a function, to guide our action towards good interactions for living in groups, and to suppress selfishness.

As argued the function of moral concepts is to guide or start actions. For humans this is important since we suffer from biases, psychic numbing and selfishness that can be overcome by reasoning with the use of moral concepts. As McElwee (2010) has argued, we use these concepts to take a stand to which actions can be tolerated and which actions cannot be tolerated and reinforce our motivation to perform an action. If these are the only functional roles moral concepts have, do robots need these guiding concepts? It seems that robots do not suffer from these irrational processes and thereby can be programmed to be fully rational, like in the thought experiment of Roger Crisp. A robot can be programmed to value each human live as one, no matter how great the numbers at risk will be. If robots at all can be programmed with feelings, it can be programmed to review arguments not based on earlier “feeling” but on data. Thereby avoiding the conformation bias. And finality it seems that a robot does not have selfishness, or can be programmed not to have it. Therefore, it does not need moral concepts to suppress selfishness. This means that robots do not need the reinforcing role of moral concepts such as moral obligation. Therefore, I argue that there is no reason to program a robot to have moral concepts as guiding principles.

If there is no ultimate or derivative reason to program robots with moral concepts such as fairness or kindness, the concepts that guide us to the right moral attitude, there must be another way to integrate morality into these machines. This is important since we do not have complete control over the learning process of a robot and its relationship to its environment. A robot can maybe learn using machine learning but its core purpose will be to maximise a score x. That x should be well-being since this is the only non-derivative reason to act and therefore the only “ethical” concept a robot should be guided by and thus needs.

CHAPTER 4 – WHY WELL-BEING IS NOT ENOUGH In the previous chapter I have argued that robots do not need moral concepts as guiding principles and that a concept of what promotes well-being is sufficient. There are no ultimate or derivative reasons to program robots with moral concepts such as mercy, forgiveness and justice, which Baley, in Asimov’s novel The Caves of Steel (2011), suggested to be higher than the three laws. In this chapter I will firstly show that implementing the concept of well-being that Crisp supposes might be problematic in the field of robotics. Secondly, I will examine arguments given by Ross (1961) that the complexity of moral judgement requires more than just a concept of well-being, and then argue that there is no need for these extra concepts. Finally, I will argue that certain roles

22 bring certain obligations, role obligations. Therefore, robots need different “moral” programming for the different roles robots are going to perform, but that even in such a theory there is no need for moral concepts. Roles bring certain reasons to act in specific ways, but robots do not need concepts such as moral obligation to apply this.

ISSUES REGARDING CRISP’S ACCOUNT OF WELL-BEING

Crisp suggests well-being as a source of an ultimate reason: “the property some action has of furthering the agent’s own well-being is a reason for that agent to perform that action, a reason which varies in strength in proportion to the degree of promotion” (Crisp, 2006). This property of an action, “that counts, for the agent, in favour of its performance by that agent” is a normative reason according to Crisp (2006). Crisp (2006) divides these reasons in pro tanto and overall reasons. According to Crisp these two types of reasons “are fundamental and along with the ultimate/non- ultimate distinction provide a comprehensive basis for a theory of normative reasons and hence of ethics” (Crisp, 2006). It is these concepts that provide reasons, not moral concepts. If, for example, Jane thinks she will enjoy reading a magazine on a terrace, she has a normative pro tanto reason to do so. If reading a magazine on a terrace would provide Jane with a score 5 on enjoyment, and visiting the zoo a score 4, then Jane has normative pro tanto reasons for both actions. She will enjoy both actions, but overall she has a reason for reading a magazine on a terrace. Crisp suggests that any reason must be grounded on the promotion of well-being, “either the agent’s or that of others” (Crisp, 2006). Well-being is the value, and the pro tanto and overall reasons are the normative reasons. But Crisp leaves room for more values, “this view… does not imply that well-being is the only value” (Crisp, 2006).

To come up with a normative principle Crisp uses a form of intuitionism which leads him to defend a self-interest (SI) principle. He argues that this principle is graspable by intuition and thereby it is one reason for an agent. The self-interest principle is:

“Any agent at time t who has (a) a life that can go better or worse for her and (b) a range of alternative action available to her at t which will affect whether that life does go better or worse overall for her has a reason to act at t in any way that makes her life go better overall, the strength of such a reason varies directly in proportion to the degree of promotion of her well-being” (Crisp, 2006).

This principle may be graspable by intuition and therefore have some justification as a principle for human moral agents, but this does not automatically provide an argument for being a principle for robots. Even Crisp admits that he cannot be entirely certain about this principle as it is justified

23 by an appeal “to the initial strong credibility of SI” (Crisp, 2006). As the principle cannot be certain Crisp uses Sidgwick’s (1981) four conditions; clarity, reflection, consistency and consensus to strengthen his argument. These conditions are to establish a proposition that is: “in the highest degree of certainty attainable” (Sidgwick, 1981). Next to the issue that principles for human moral agents are not necessarily the principles for robots, there might be difficulty programming Crisp’s view of well-being. Crisp (2006) does not defend, for example, objective list theories, but he defends hedonism as a theory of well-being: “of what is ultimately good for any individual” (Crisp, 2006). The hedonist, as he wants to understand her, will say that: “what makes accomplishment, enjoyable experiences, or whatever good for people is their being enjoyable, and that this is the only ‘good-for-making’ property there is” (Crisp, 2006). This concept of being enjoyable might be problematic for robots, since robots don’t feel enjoyment or suffering. Feeling enjoyment is a capacity only humans and to some degree animals have, not machines.

MORALITY IS TOO COMPLEX FOR JUST WELL-BEING

The self-interest principle combined with hedonism might not be applicable as a principle for the guidance of robots, since robots don’t feel enjoyment or suffering. Nevertheless, a part of the argument Crisp uses to defend his SI principle might be helpful. His argument seems to be built on intuition and this approach might be helpful in deciding what a robot has most reason to do. An other philosopher who uses intuitionism is W. D. Ross (1961). Ross (1961) argues that single rule ethical theories, like Utilitarianism or Hedonism, are not sufficient because they fail to grasp the complexity of an ethical situation. These single rule theories neglect the diverse relation that one might have in different circumstances. Ross (1961) argues that there are prima facie duties which we should try to follow and that relations such as: “promisee to promiser, of creditor to debtor, of wife to husband, of child to parent, of friend to friend, of fellow countryman to fellow countryman and the like” (Ross, 1961) are all foundations of these prima facie duties. There are, according to Ross, seven prima facie duties: fidelity, reparation, gratitude, justice, beneficence, self- improvement and nonmaleficence. Ross (1961) points out that relations influence what is intuitively regarded as “the good action”. One argument made by Ross illustrates that the right outcome cannot intuitively account for an action being right.

“that the fulfilment of a promise to A would produce 1,000 units of good for him, but that by doing some other act I could produce 1,001 units of good for B, to whom I have made no promise, the other consequences of the two acts being of equal value should we really think it self-evident that it was our duty to do the second act and not the first? I think not.” (Ross, 1961)

24 Ross’ concept of prima facie duties suggest that moral situations cannot just be settled by looking at what will promote the most well-being in the way Crisp describes it, as enjoyment. Anderson et al. (2004) also argues against the use of single rule theories and argues that Ross’ prima facie duties combined with the reflective equilibrium approach of Rawls (1951) works better than single rule theories in the field of machine ethics. According to Anderson et. al (2004), Ross’ theory of prima facie duties seems to give a better account of ethical obligations that most of us recognize than some single theories, such as act utilitarianism. But Anderson et. al. (2004) also point out a flaw in Ross theory: “Ross gives us no decision procedure for determining which duty becomes the strongest one, when, as often happens, several duties pull in different directions in an ethical dilemma” (Anderson et. al, 2004). To counteract this Anderson et. al. (2004) suggests Rawls’ reflective equilibrium approach which weighs the duties in particular circumstances.

Next to Ross’ and Anderson’s objections against single rule theories there are other arguments why implementing robots with just a concept of well-being might not be sufficient. One of which is that a robot needs to be almost omnipotent to be able to calculate all the effects of an action. And what is considered the right outcome can differ from culture to culture, and can change over time (Haidt & Kesebir, 2010). What the right outcome of an action performed by a teacher in China is, therefore does not have to be considered the right outcome in The Netherlands. But this argument only adds up to Ross’ and Anderson’s argument that moral situations are too complex for single rule theories.

Ross (1961) and Andersons et. al. (2004) both point out that programming robots just to promote well-being will ultimately fail because the notion of well-being is too complex. Do robots therefore need concepts such as duty, gratitude and justice, concepts Ross uses? Not according to Crisp. Crisp shows, in a mentioning of Normative Rossianism, that Ross’ prima facie duties can be described without the uses of moral concepts, like so:

“Any agent has the following ultimate reasons: to abide by her contracts; to provide certain goods to those whom she has treated in particular ways detrimental to them; to express thanks for benefits; to distribute well-being according to certain rules concerning the status of possible beneficiaries; to increase the degree to which she possesses intelligence and certain traits of character; not to cause decreases in the overall well-being of others” (Crisp, 2006).

25 Like described, the prima facie duties of Ross can be stated without the uses of moral concepts. Although Ross did not have robots in mind when he argued for his prima facie duties, they show that the right action of a robot can depend on the kind of relationship it has with its environment. Therefore, the right action of a teaching robot such as Zora, can differ from the right action of a cleaning robot, because they have a different relation to the humans they interact with. But since these situations are completely new, as there has never been a teaching robot, or companion robot or health care robot, the ultimate reasons a specific robot should follow are still unclear. What, for example, is the relation between a teacher robot and a child, or a cleaning robot and the house owner, or a companion robot and an elderly person? These relations are not straightforward and simple, as I would like to illustrate by an example.

Alice is a care robot for lonely elderly people and elderly people suffering from dementia. In one of the experiments Alice registers that an elderly woman had not gone outside that day. When Alice asked the elderly woman if she would like to go outside with her the woman replies: “Yes, if you want to” (Burger, 2015). This reply illustrates the complexity of the relation between a robot and a human. The relations between social robots and humans is not comparable to the relations between humans and simple tools, such as a washing machine. What the relation between robots and humans is, will – presumably – only become clear when robots are becoming more common in these social areas.

DIFFERENT ROLES MEAN DIFFERENT RIGHT ACTIONS

Since prima facie duties are based on relationships and the relationships of humans and robots is new and unclear it cannot be a satisfactory base for moral reasons. Therefore, I propose role obligation as a means to generate reasons robots should use as a guideline in ethical situations. Role obligation is “a moral requirement, which attaches to an institutional role, whose content is fixed by the function of the role, and whose normative force flows from the role” (Hardimon, 1994). A role obligation is attached to an institutional role, for example a teacher or a nurse. These role obligations give teachers ultimate reasons to perform actions. Humans can have different roles in different situations, like being a father and at the same time a brother and a bus driver, which refer to different role obligations. In these cases, deciding which obligation to follow can face people with a dilemma. Robots, however, can be created for one individual role, like Zora is created to be a teacher. This means that different sets of reasons apply to robots in varies fields. A military robot will have a different set of moral reasons than a teaching robot. According to Hardimon (1994) the structure of an institution of which a role is a part determines what is required of the role. Therefore, what is required of a teacher robot is determined by the structure of schools and

26 universities. As long as the relationships between humans and robots are still unclear, these role obligations derived from institutions can give a robot guidance. This also keeps in mind the cultural differences of what is considered a morally right action.

Traditionally role obligation is discussed using moral concepts, such as moral obligation, but this is not indispensable. Role obligation can be programmed without moral concepts, in the same way the prima facie duties of Ross don’t necessarily need moral concepts. That is to say, the reasons a robot has to perform an action follow from the rules linked to a particular role and do not require moral concepts. This approach allows for different sets of reasons based on the roles robots will occupy, but does not require a robot to be programmed with moral concepts. It gives attention to the complexity of an ethical situation, more than just a universal concept of well-being. However, the role-related reasons can be linked to a concept of well-being as a means to determine which overall reason a robot has to perform an action.

27 CONCLUSION I have argued that Asimov’s laws of robotics are not easily implemented and have both practical and philosophical issues differing in weight. The strongest argument against the uses of Asimov’s laws as they stand to guide the behaviour of contemporary robots is that the concept of harm is not defined. The concept harm is vague and there are significant disagreements about what counts as a harm which Asimov does not consider.

One possible strategy to guide robots is by some (software equivalent) of moral concepts, such as moral obligation, ought, duty, supererogation, morally praiseworthy, moral must, morally good, morally bad, moral value, fairness, virtue, vice, justice and generosity. Using Roger Crisp’s arguments for well-being as the ultimate source of moral reasoning, this thesis has argued that there are no ultimate, non-derivative reasons to program robots with moral concepts. Although these moral concepts should not be used to program robots, they are not to be abandoned by humans since there are still reasons to keep using them, namely: as an assessment of the agent, to take a stand or to motivate and reinforce behaviour. This is important since humans suffer from biases, psychic numbing and selfishness that can be overcome by reasoning with the use of moral concepts. Because robots do not suffer from these sometimes irrational processes and can be programmed as completely rational agents, they don’t need these additional motivations, they can suffice with a concept of what promotes well-being.

To program robots to be guided only by the outcome of their action might be problematic, because the right outcomes do not always entail the best moral action. In Hedonism or Utilitarianism, relations and the complexity of ethical situations is misrecognised, and therefore Ross proposes prima facie duties. But even in applying these duties there is no need for moral concepts. According to Ross relationships influence what is considered the right action, they are the foundations of the prima facie duties. But, because the relations between robots and humans is unclear, these duties cannot be a satisfactory base for ultimate reasons.

I propose role obligation as a useful way to think about actions robots should perform. Role obligation gives attention to the complexity of an ethical situation, and can be stated and understand without the uses of moral concepts. The structure of an institution of which a role is a part determines what is required of the role. Because these institutions are not new they can be a satisfactory base for guiding principles. Therefore, role obligations, without using moral concepts, seems to be the best approach to define reasons robots have to perform certain actions.

28 BIBLIOGRAPHY Alzheimer Nederland. (2013, 07 18). Kans op dementie daalt, aantal patienten stijgt. Retrieved 07 20, 2016, from www.alzheimer-nederland.nl: http://www.alzheimer- nederland.nl/nieuws/onderzoek/2013/juli/aantal-pati%C3%ABnten-blijft-stijgen.aspx

Anderson, M., Anderson, S. L., & Chris, A. (2004). Towards machine ethics. AAAI-04 workshop on agent organizations: theory and practice. San Jose, CA.

Anderson, S. L. (2008). Asimov's "Three Laws of Robotics" and Machine Metaethics. AI & Society - Special Issue: Ethics and artificial agents, 22(4), 477-493.

Ashok J. Bharucha, V. A. (2009). Intelligent Assistive Technology Applications to Dementia Care: Current Capabilities, Limitations, and Future Challenges. The American Journal of Geriatric Psychiatry, 17(2), 88–104.

Asimov, I. (1942). Runaround. Astounding Science Fiction , March.

Asimov, I. (1946). Evidence. Astounding Science Fiction.

Asimov, I. (1950). The Evitable Conflict. Astounding Science Fiction.

Asimov, I. (1978). The Machine and the Robot. Science Fiction: Contemporary Mythology.

Asimov, I. (1984). The Bicentennial Man. Philosophy and Science Fiction, 183-216.

Asimov, I. (2011). Caves of steel (Vol. 2). Spectra.

Bilefsky, D. (2009, 5 10). Hard Times Give New Life to Prague’s Golem. Retrieved 07 21, 2016, from The New York Times: http://www.nytimes.com/2009/05/11/world/europe/11golem.html?_r=0

Borowiec, S. (2016, 03 09). Google's AlphaGo AI defeats human in first game of Go contest. Retrieved from The Guardian: https://www.theguardian.com/technology/2016/mar/09/google- deepmind-alphago-ai-defeats-human-lee-sedol-first-game-go-contest

Bradley, B. (2012). Doing Away with Harm. Philosophy and Phenomenological Research, 2, 390-412.

Burger, S. (Director). (2015). ik ben Alice [Motion Picture].

Burgess, M. (2016, 06 06). Google's 'big red' killswitch could prevent an AI uprising. Retrieved 07 22, 2016, from Wired: http://www.wired.co.uk/article/google-red-button-killswitch-artificial- intelligence

29 Cellan-Jones, R. (2014, 12 02). Stephen Hawking warns artificial intelligence could end mankind. Retrieved 07 21, 2016, from BBC: http://www.bbc.com/news/technology-30290540

Clarke, R. (1993). Asimov's Laws of Robotics Implications for Information Technology part 1. Computer, 26(12), 53-61.

Coeckelbergh, M. (2010). Moral appearances: emotions, robots, and human morality. Ethics and Information Technology, 12.3, 235-241.

Crisp, R. (2006). Reasons and The Good. Oxford: Clarendon Press.

Crnkovic, G. D., & Çürüklü, B. (2012). Robots: ethical by design. Ethics and Information Technology, 14.1 , 61-71.

Edwards, A., Edwards, C., Spence, P. R., & Harris, C. (2016). Robots in the classroom: Differences in students’ perceptions of credibility and learning between “teacher as robot” and “robot as teacher”. Computers in Human Behavior.

European Commission. (2016, 5). ECHO Factsheet Syria crisis. Retrieved from ec.europa.eu: https://ec.europa.eu/echo/files/aid/countries/factsheets/syria_en.pdf

Gibbs, S. (2014, 10 27). Elon Musk: artificial intelligence is our biggest existential threat. Retrieved 07 21, 2016, from theguardian: https://www.theguardian.com/technology/2014/oct/27/elon- musk-artificial-intelligence-ai-biggest-existential-threat

Haidt, J., & Kesebir, S. (2010). Morality. In D. G. S. Fiske, Handbook of social Psychology 5th Edition (pp. 797-832). Hobeken, NJ: Wiley.

Hanser, M. (2008). The Metaphysics of Harm. Philosophy and Phenomenological Research, 77, 421-50.

Hardimon, M. (1994). Role Obligations. The Journal of Philosophy, 91(7), 333-363.

Horton, H. (2016, 03 24). Microsoft deletes 'teen girl' AI after it became a Hitler-loving sex robot within 24 hours. (Telegraph Media Group) Retrieved 08 08, 2016, from The Telegraph: http://www.telegraph.co.uk/technology/2016/03/24/microsofts-teen-girl-ai-turns-into- a-hitler-loving-sex-robot-wit/

Ienca, M., Jotterand, F., Vica, C., & Elger, B. (2016). Social and Assistive Robotics in Dementia Care: Ethical Recommendations for Research and Practice. International Journal of Social Robotics, 1-9.

30 Joy, B. (2000). Why the future doesn't need us. Wired.

Joyce, R. (2001). The Myth of Morality. Cambridge University Press.

Kant, I. (1963). Our Duties to Animals. Lectures on Ethics. New Yrok, NY: Harper & Row.

Krishnan, A. (2009). Killer robots: legality and ethicality of autonomous weapons. (L. Ashgate Publishing, Ed.)

Kuiper, M. (2016, 01 14). Wow, zegt de lesgevende robot, Een robot helpt met wiskunde op een school in Uithoorn. Hij kan tot in het oneindige uitleggen. Retrieved from www.NRC.nl: http://www.nrc.nl/nieuws/2016/01/14/wow-zegt-de-lesgevende-robot-1577125- a263541

Kurzweil, R. (2006). The Singularity Is Near: When Humans Transcend Biology. Penguin Books.

Leveringhaus, A. (2015). Ethics and Autonomous Weapons. Springer.

Li, J., Rene,́ K., Jeremy, B., & Wendy, J. (2015). Social robots and virtual agents as lecturers for video instruction. Computers in Human Behavior.

Lin, P., Abney, K., & Bekey, G. (2011). Robot ethics: Mapping the issues for a mechanized world. Artificial Intelligence, 175.5, 942-949.

Malle, B. (2015). Integrating robot ethics and machine morality: the study and design of moral competence in robots. Ethics and Information Technology, 1-14.

Mandelbaum, E. (2014). Thinking is Believing. Inquiry: An Interdisciplinary Journal of Philosophy, 57(1), 44-96.

Markoff, J. (2011, 02 16). Computer Wins on ‘Jeopardy!’: Trivial, It’s Not. Retrieved from New York Times : http://www.nytimes.com/2011/02/17/science/17jeopardy- watson.html?pagewanted=all.

McCauley, L. (2007). AI Armageddon and the Three Laws of Robotics . Ethics and Information Technology, 9(2), 153-164.

McElwee, B. (2010). Should We De-moralize Ethical Theory? Ratio, 23(3), 308-321.

McPhee, M., Baker, K. C., & Siemaszko, C. (1997, 05 12). Kasparov has Deep Blues after losing. The New York Daily News.

31 Mitchell, T. (1997). Machine Learning, International Edition. Singapore,: McGraw-Hill Book Co.

Moravec, H. P. (1998). Robot: Mere Machine to Transcendent Mind. Oxford: Oxford University Press.

Murphy, R. R., & Woods, D. D. (2009, 07 24). Beyond Asimov: The Three Laws of Responsible Robotics. IEEE intelligent systems, 24(4), 14-20.

Orseau, L., & Armstrong, S. (2016). Safely Interruptible Agents. 32nd Conference on Uncertainty in Artificial Intelligence (UAI 2016). Association for Uncertainty in Artificial Intelligence.

Pontier, M. A., & Hoorn, J. F. (2012). Toward machines that behave ethically better than humans do. 34th international annual conference of the cognitive science society. 12, pp. 2198-2203. Austin, TX: Cognitive Science Society.

RIVM. (2016). Trend in aantal mensen met dementie. Retrieved 07 20, 2016, from Volksgezondheidenzorg.info: https://www.volksgezondheidenzorg.info/onderwerp/dementie/cijfers- context/trends#definities

Ross, W. D. (1961). The right and the good. Oxford: Clarendon Press.

Sidgwick, H. (1981). The methods of ethics (7th Edition ed.). Macmillan and Company, Limited.

Sloman, A. (2006, 07 06). Why Asimov's three laws of robotics are unethical. Retrieved from University of Birmingham, School of Computer Science: http://www.cs.bham.ac.uk/research/projects/cogaff/misc/asimov-three-laws.html

Slovic, P. (2007). "If I look at the mass I will never act" Psychic numbing and genocide. Judgement and Decision Making , 2(2), 79-95.

Smith, P. (2015, 03 23). Apple co-founder Steve Wozniak on the Apple Watch, electric cars and the surpassing of humanity. Retrieved from The Australian Financial Review: http://www.afr.com/technology/apple-cofounder-steve-wozniak-on-the-apple-watch- electric-cars-and-the-surpassing-of-humanity-20150320-1m3xxk

Sparrow, R. (2007, 02). Killer robots. Journal of applied philosophy, 24(1), 62-77.

Tooley, M. (1972). Abortion and infanticide. Philosophy & Public Affairs, 37-65.

Wallach, W. (2010). Robot minds and human ethics: the need for a comprehensive model of moral decision making. Ethics and Information Technology, 12.3, 243-250.

32 Warwick, K. (2004). I, Cyborg (Reprint edition (July 19, 2004) ed.). University of Illinois Press.

33