Linköping University | Department ITN Master’s thesis, 120 credits | Design Spring 2020 | LiU-ITN-TEK-A--20/016--SE

A critical review of the intersection between design, ethics and technology: the social importance of designers and how ethics can truly be promoted through design. The development of autonomous transportation as a use case to demonstrate designers’ social responsibility and facilitate a discussion on the role of design for questions of social significance.

Jana Voykova

Supervisor: Prof. Stefan Holmlid Examiner: Prof. Jonas Löwgren

Linköping University SE-581 83 Linköping, Sweden +46 013 28 10 00, www.liu.se

Table of Contents

ABSTRACT

1. BACKGROUND AND MOTIVATION 1

2. RESEARCH QUESTIONS 9 2.1 Research question #1: the social responsibility of designers 10 2.2 Research question #2: social and ethical implications of autonomous transportation 15 2.3 Connection between the research questions 16 2.4 Delimitations 17

3. METHOD 19 3.1 Research through design (RtD) 19 3.2 Limitations of RtD 22 3.3 Speculative and critical design 22 3.4 Limitations of speculative and critical design 26 3.5 Literature and desk research 28 3.6 Interviews 29 3.7 Surveys (Questionnaires) 30 3.8 Applying SCD in a prototype exhibition format 31

4. THEORETICAL FRAMEWORK 33 4.1 Design ethics 33 4.2 Technology ethics 48 4.3 Autonomous vehicles (machine) ethics 56 4.4 Business ethics 73

5. RESEARCH PROCESS 93 5.1 Planning and concept development 93 5.2 Interviewing and surveying 94 5.3 Reiteration of second research question 97 5.4 Prototyping an exhibition 97

6. RESEARCH OUTCOMES AND DISCUSSION 110 6.1 Research question #1: the social responsibility of designers 110 6.2 Research question #2: social and ethical implications of autonomous transportation 133 6.3 Virtual exhibition feedback 146 6.4 Limitations of research outcomes 150 6.5 Discussion 151

7. CONCLUSION AND FUTURE WORK 156

REFERENCES 160

ACKNOWLEDGMENTS 173

APPENDICES 174

MSc Design Thesis: Jana Voykova

Abstract

In his speech during the 2016 Speculative Design Symposium, held at the University of California, San Diego, Benjamin Bratton1 rightly argued that the job of 21st century design is to undo (much of) the design of the 20th. A number of recent controversial designs and practices in the business and public sphere have suddenly made ethical design (design ethics2) a hot topic in the design community. This master thesis is a highly critical and fairly philosophical examination of the design profession in the context of the current socio-technical landscape. It analyses the convergence between the fields of design, ethics and disruptive technology. Autonomous transportation is taken as an example to illustrate what circumstances (should) drive designers’ social engagement. Hopefully, it also accommodates for a productive reflection on the place of ethics in a broader social context. By utilising speculative and critical design approaches, the thesis aims to stimulate, provoke and ideally maintain a public discourse on the direction of development of technology and modern societies, and inspire designers to be more critical to the vocational portrayal of their profession.

Keywords: ethical design, design ethics, critical design, speculative design, ethics, morality, technology, disruptive technology, innovation, autonomous vehicles, smart cities.

1 Benjamin Bratton is an American sociologist and Professor of Visual Arts of the Center for Design and Geopolitics at the University of California, San Diego.

2 I will be using the terms ethical design and design ethics interchangeably throughout this thesis.

MSc Design Thesis: Jana Voykova

1. Background and Motivation

Changing our nature to fit the world, rather than the other way around, is actually the deepest form of disempowerment. It distracts us from reflecting critically on the world and deadens the impulse to social and political improvement.

Michael Sandel3

During the international conference of the World Economic Forum (WEF) in Davos, Switzerland in January 2017, Johnson Control’s4 CEO Alex Molinaroli made an enticing observation about why technology is so challenging today. His statement was interesting not because he was the only person to express this view, or because it was a breakthrough discovery, but because it is worth reflecting on where it comes from, what it means and why we hear it so often today. Technology, in Molinaroli’s words (which I am paraphrasing), is moving faster than people and so our perception of what is possible constantly changes which puts us in a state of reluctance to move along with it. In other words, the pace of evolution of technology is so fast today, that people simply cannot comprehend it or catch up it and eventually, they lose interest in trying to do so.

In his book The case against perfection: Ethics in the age of genetic engineering (2007), Michael Sandel provides what, in my view, constitutes a great explanation of our current technological puzzlement. He writes that ‘when science moves faster than our moral understanding, as it does today, men and women struggle to articulate their unease’ (Sandel, 2007, p. 9). In this context, as naïve as this question might sound, I think it is critical to ask: who steers technology and why is it so difficult for us to keep up with its pace of advancement? Is it not we, humans who decide what technology does and how fast it moves?

I find statements like Molinaroli’s problematic because, while being true, they imply a sense of inevitability and inertia about our technological evolution, which humans cannot slow down, fathom, or influence in any way. This, in its turn creates the illusion that the future only has one possible and viable direction to take: the one which technological, business, political or scientific communities determine, and we cannot, or worse — shouldn’t challenge it.

3 Michael Sandel is an American political philosopher and professor in political theory at Harvard university Law School. 4 Johnson Controls is an international company, headquartered in Ireland which produces fire, security and ventilation equipment for buildings, software technologies and services.

1 MSc Design Thesis: Jana Voykova

Many people say that 21st century is the most fascinating time to live in. While I agree with them in a certain measure — compared to earlier decades, the present age is much more favourable for a good portion of the world’s population, science has made staggering progress, and there is, at least in theory, more balanced global economic and political cooperation, 21st century is also a time of massive social disruption and great bewilderment about what the future holds. We seem to be assuming that all challenges that we face in our modern societies should and will be resolved with innovation and technology. We also seem to be assuming that both innovation and technology are inevitable and if someone dares to take a step back and suggest that we should be more careful with how we design technology and what we demand from science, they are instinctively labelled as narrow-minded enemies of progress and evolution.

It is not disputable that technology was the catalyst for many processes that lead to tremendous improvements in various facets of life — revolutionary ways to learn, work, entertain, travel, faster means to communicate and share information, immediate access to innovative consumer products, at least for the privileged classes. However, contrary to the idea which too often gets emphasised in the public domain, technology is neither a neutral, nor a net-positive phenomenon. Often times, disruptive technology comes with a list of side effects and trade-offs, which are conveniently omitted from the advertising campaigns and dressed-up as agents of the greater good. Eventually, these trade-offs become the quid pro quo which explains the intrinsically broken design of most of the off-springs of the third modernity5 and our instinctive acceptance of it.

5 In his book Towards the Third Modernity (2008), the French sociologist Alain de Vulpian revises the cultural, social and political changes that we have witnessed in the last two centuries. He identifies the period after the Second World War and the mass consumption that came with it as the First Modernity, the years after the 1960s when people sought to set free from the authority of local institutions and corporations as the Second Modernity. The Third Modernity is the digital era which started taking shape after the 2000s. Vulpian discusses the change in attitudes which mark each of these periods, for example the move from a highly religious to an atheistic society, the sexual revolution, the different outlook on marriage, moral norms and political rule, individualism, humanism, environmental awareness.

2 MSc Design Thesis: Jana Voykova

Figure 1. One of the major reasons why technology is so puzzling today, in my opinion, is that, the way we develop it puts to the test our perceptions of what critical aspects of civic life, such as power, knowledge, privacy, trust, freedom, behaviour, really mean. We literally find ourselves in a state of constant confusion about what is going on.

Unsurprisingly, there is an extreme polarisation of opinions when it comes to technology — some people fanatically believe in its ability to save humanity of, if not all, at least a great majority of the problems we struggle with today, others are more skeptical of technology’s capacity to serve as a panacea and instead, apply, what I call “a common-sense-logic” to it. This, by no means suggests that this second group, to which I belong, does not appreciate scientific and technological evolution and instead proposes that we give up all progress made and go back to 18th century lifestyles. But when our striving for constant innovation leads us to a threshold that continually stretches our moral understanding, undermines what being human means and makes us wonder if it is a good or a bad thing, it becomes imperative that we stay cognizant to these processes.

Sadly, I think we have already crossed a number of thresholds with the developments in genetic sequencing, with the design of algorithms that tirelessly monitor our habits, thus extracting valuable information about our cognitive abilities and personalities, with devices that demand our attention at all times, and with robotic systems which humanity and the environment will supposedly benefit from immensely, but which are nonetheless accompanied by dubious ethical repercussions. Is there even a limit? In a reality where literally anything is possible and there are no boundaries, it’s difficult to say whether there is anything at all which we shouldn’t do. Where is our scientific knowledge and boundless

3 MSc Design Thesis: Jana Voykova curiosity going to lead us? What line do we yet have to cross before we realise that we have gone too far? Sometimes advancements of science and technology which, at first glance, seem harmless and virtuous, like artificial insemination and surrogacy for example, can have serious ethical and economic implications, which can lead to the establishment of dangerous power dynamics. These are questions which are connected with the role of markets in civic life and which I will revisit later on in this thesis.

Similar ethical dilemmas apply in even stronger measure when it comes to more recent developments based on artificial intelligence and machine learning6. At the 2017 Summit conference of Singularity University7 (I think the word university here is used in an extremely superficial sense), Maarten Steinbuch8 announced that by 2045, we will overcome death and ageing will be regarded as a disease. His proclamation produced alarmingly ecstatic applauses from the audience. Steinbuch’s prediction might be wildly optimistic but nonetheless too serious to be dismissed lightly. Before we get too excited about the unlimited opportunities of science and technology, I think we should ask ourselves: who will be able to reap the fruits of human curiosity (or stupidity, depending on how we look at things)?

Certainly not everyone who lives on planet Earth and is going to be born in the following decades will be entitled to immortality, especially in a reality of scarce natural resources and an all the more likely climate catastrophe. As Michael Sandel maintains throughout his book The case against perfection: Ethics in the age of genetic engineering (2007), ‘the fundamental question is not how to assure equal access to enhancement but whether we should aspire to it’ (Sandel, 2007, p. 16). Sandel argues, accurately I believe, that the trouble with artificial enhancements lies not so much in the denying of those methods to classes who cannot afford them, but in the dehumanisation of those who can and will. Chance in life becomes choice and we distort our appreciation for effort, achievement, freedom and responsibility. The bigger problem with this however, according to Sandel, is ‘the Promethean aspiration to remake nature to serve our purposes and satisfy our desires’ (Sandel, 2007, p. 27).

6 Although these terms are widely known, for the sake of clarity, I will provide a definition later on in this thesis. 7 Singularity University, located in Silicon Valley, is a private venture institution founded in 2008 by Ray Kurzweil and Peter Diamandis, and sponsored by big technology conglomerates like Google, SAP and Deloitte, with the mission to create “a global learning and innovation community using exponential technologies to tackle the world’s biggest challenges and build a better future for all”. It offers programs and courses to organisations and individuals in artificial intelligence, robotics and biotechnology. 8 Maarten Steinbuch is a professor in Systems- and Control and head of the Control Systems Technology group of the Mechanical Engineering Department of Eindhoven University of Technology.

4 MSc Design Thesis: Jana Voykova

Regrettably, the sound observations of intellectuals like Michael Sandel have not echoed clearly enough in our scientific, political and social communities. In 2005 Ray Kurzweil, one of the founders of the aforementioned Singularity University and a highly esteemed public figure, published a book called The Singularity is Near: When humans transcend biology.

With the not so subtle confidence of a prodigy of his generation, Kurzweil explains his futuristic visions and fascination with machines and technology from a very early age. His determination to, by means of science and technology, overcome the ‘profound limitations’ of human biology and cognition, as he repeatedly declares in his book, and the affluent position he has assumed, are horrifying at least on two accounts: 1) he seems to be neglecting the complexity of the human mental and physical nature by treating it as something transient which we are obliged to surpass, and 2) this antihuman and remarkably egocentric position, as Douglas Rushkoff9 points out, is ‘driving the philosophy of the most capitalized companies on the planet’ (Rushkoff, 2019, p. 135). Instead of raising public outrage, as it should have, Kurzweil’s inclinations were celebrated and admired by critics, publishers and contemporaries. He was called “one of the most important thinkers of our time”, “a brilliant scientist and futurist” and having “adamantine intellectual integrity”!

As much as I find bioengineering to be a key element in the discussion about scientific evolution and the (technological) future of humanity, my thesis will examine the development of a disruptive technology which will allegedly have a more immediate and humbling impact on our societies and cities, namely autonomous10 transportation. I will do that in order to contextualise my research on the social importance of design and find the place of ethics in our public affairs.

The proponents of autonomous vehicles, smart homes and connected cities, seem to be viewing innovative technologies as the silver bullet that will eliminate present-day challenges in highly urbanised areas. They promise grand improvements in terms of optimising space, minimising road accidents, addressing climate change, increasing human efficiency, and single out technology as the oracle which holds the answers to all our problems, even the purely political ones. Still, they miss to steer the public conversation towards the more

9 Douglas Rushkoff is Professor of Media Theory and Digital Economics at City University of New York, author of books and comics, columnist, host of a podcast, commentator for CNN. He lectures on media, technology, economics, culture and ethics. 10 I will be using the terms autonomous and self-driving throughout this thesis because these are the most common terms used to describe the technology in media and industry. However, the technology is better described by the term automated because the current level of automation achieved does not eliminate completely the need for human intervention (which is the meaning of autonomous). The terminology and levels of automation will be explained further on in this thesis.

5 MSc Design Thesis: Jana Voykova important risks which new technologies bear and the way they reengineer essential human faculties.

At this point in my introduction, one might ask: what does autonomous transportation have to do with design and ethics? The short answer is: more than meets the eye. It might not be a popular view, but designers have always been major contributors to how societies are shaped and matters like autonomous vehicles should occupy their attention namely because they won’t just revolutionise transportation but redesign social life in its entirety. As Ezio Manzini explains in his book Design when everybody designs (2015), ‘in a world in rapid and profound transformation, we are all designers’ (Manzini, 2015, p. 1) What Manzini means is, that the ability to design is an intrinsic human ability but one that needs cultivating.

In a time of overwhelming social and environmental turbulence, designers are the social agents equipped with the tools to gather political, economic, and business experts along with citizens to resolve social challenges together. Furthermore, designers are not simply artistic creatures who care about colour theory and alignment. They are professionals whose skills can and should be used to truly address social problems. ‘Information and communication technologies’, Manzini writes, ‘rapidly penetrating into society, they have been quickly “normalized.” (Manzini, 2015, p. 16).

In this predicament, which I call “a technological mess”, design is simply a handy instrument and designers — a convenient cog in the wheel. It seems to me that designers have forgotten their social role as gatekeepers which Victor Papanek11 brilliantly identified in his book Design for the Real World (1971). We — designers, citizens, technologists, policy makers, have failed to ask the hard questions that are salient to understanding the real-life impact of the designs we have helped create.

Although this is a question with an obvious answer, I do think we have to enquire: should designers also be ethicists and more importantly, how? We might get confused about the answer because the image of designers as mere followers of the trends which powerful players in society dictate, is constantly being reinforced in industry and media. As Matthew Beard and Simon Longstaff12 point out in their paper Ethical by design: Principles for good

11 Victor Papanek is a designer born in Austria and emigrated to the United States during the Second World War. He dedicated his practice to critique of contemporary design culture and used design as a political tool to create a more democratised and socially responsible design, especially for marginalised societies. 12 Dr Matthew Beard is an Australian moral philosopher with background in military and applied ethics. He is an ethics university lecturer and a fellow at The Ethics Centre, a non-profit organisation in Australia. Dr Simon Longstaff is an Australian philosopher who holds a PhD in philosophy from the University of Cambridge. He is the Executive Director at The Ethics Centre, fellow at the World Economic Forum and a respected advisor on questions related to political philosophy, applied ethics and strategic business development.

6 MSc Design Thesis: Jana Voykova technology, ‘the fact that we can do something does not mean that we should’ (Beard and Longstaff, 2018, p. 4) but this view seems to be dangerously unpopular in scientific and business circles today.

The intersection between the fields of design, ethics and disruptive technology is namely the backbone of my thesis research. As far as design is concerned, my focus is twofold: on the one hand, I will examine the social importance of the design profession and the ethical obligations designers have; on the other hand, I will explore how design can draw the public attention to an unpopular outlook on current social phenomena whose future impact may not necessarily be of a positive character.

From a technological perspective, I will analyse the development of autonomous transportation in terms of its social importance and ethical implications. It will serve as a use case to illustrate under what circumstances the social responsibility of designers emerges and how it can be articulated. In this discussion, I will try to find the place of ethics in our professional and public life and hopefully accommodate for a productive reflection on what we can do to truly promote ethical behaviour not only in design, but in a broader social milieu. Therefore, my argumentation will touch upon a fairly wide range of aspects which I regard critical to yielding an objective assessment of these topics.

Ethics is a term which denotes what is the right, good thing to do and what one should do. ‘Ethics allows us to judge the world — what should happen? Of all the ways you might act, which is the best? Which of all the possibilities should you bring into reality? What ought one to do?’ (Beard and Longstaff, 2019, p.16). There is often a disagreement both in scientific communities and society at large regarding the difference between ethics and morality. I think Andrew Crane and Dirk Matten13 offer a great definition, which explains the distinction between the two:

‘Morality is concerned with the norms, values and beliefs embedded in social processes which define right and wrong for an individual or a community. Ethics is concerned with the study of morality and the application of reason to elucidate specific rules and principles that determine right and wrong for a given situation. These rules and principles are called ethical theories’ (Crane and Matten, 2010, p. 8). In this sense, morality precedes ethics. Therefore, when I discuss ethical behaviour in design and technology, I refer to the moral principles we should follow as human beings.

13 Andrew Crane is a former Professor in Business Ethics in the Schulich School of Business at York University in Canada and current Director, Center for Business, Organisations and Society at the University of Bath School of Management in Bath, UK. Dirk Matten is a Professor at the Schulich School of Business where he holds the Hewlett-Packard Chair in Corporate Social Responsibility. He is a visiting professor at the University of London, the University of Nottingham and at Sabancı University in Istanbul.

7 MSc Design Thesis: Jana Voykova

Additionally, this thesis draws a parallel between ethical behaviour and social responsibility, both in the context of design and the technology/business/public spheres, thus, I will be using these two terms as synonyms.

By applying a strongly critical lens both to the design profession and the direction which society’s evolution is taking (by way of discussing autonomous transportation as a socio- technical case study), I will try to steer the public attention to a more pragmatic perspective on current global events and offer a highly critical revision of the vocational portrayal of the design profession.

8 MSc Design Thesis: Jana Voykova

2. Research Questions

People often want to know what design is. The question is what design isn't. One of my pet peeves is that people, when you say the word design, they think oh, you mean the beautiful part, the sexy part, the handsome part. It's not about that. It isn't about how it looks. It's about how it works, how it feels, how it changes the person's life, that’s what design is about. It's not about the aesthetics. It's about the outcome. So, design in its simplest form is the humanization of a technology that has become commoditized. The problem is that businesses must understand design to improve the processes and their outcome.

John Maeda14

The fields of design, ethics and technology are so vast that each of them separately can be researched infinitely. Even when we narrow down our focus to the social purpose of design or the ethical considerations and social impact of autonomous vehicles, these topics can still take years to examine and a myriad of factors to consider. For this reason, my thesis aims to bring academic knowledge contributions to the area where these three fields converge, with the ultimate and ambitious goal to challenge the public conversation and the rushed fascination with the potential of new technology as a limitless resource and, by means of speculative and critical design, secure a room for a more sensible outlook on present societal challenges.

The backdrop of this discussion is the position which designers should take within society and how they, having the power to reach a wide audience with their work, should use this power to educate, question, inspire, provoke and bring forward questions that are fairly unpopular in the design community but nonetheless vital for the welfare of society.

14 John Maeda is a former president of the Rhode Island School of Design. He holds degrees in electrical engineering and computer science from MIT, MBA from Arizona State University and a PhD from the University of Tsukuba in Japan. He is the author of several books among which The Laws of Simplicity, Creative Code and Redesigning Leadership. Maeda is currently the head of computational design and inclusion at the company Automattic which stands behind open-source projects like WordPress, Akismet, Tumblr.

9 MSc Design Thesis: Jana Voykova

Figure 2. The purpose of this thesis is to intersect the fields of design, ethics and technology in order to argue for a future that is right by design.

2.1 Research question #1: the social responsibility of designers

With regard to the convergence of design, ethics and technology, this thesis will aim to answer two research questions which might be perceived as rather dissimilar. Although the connection between them might not be immediately apparent, I would argue that they can and should be seen as akin (explanation of this will be provided further below). The first of these two research questions is:

1. How do design practitioners perceive their social responsibility and the social importance of their profession, and how can they demonstrate this responsibility in their practice15?

15 Here “practice” is meant to denote ‘the work situation of design professionals, in which they typically work for a client, to a brief which may include the point that the clients’ offerings are part of the designed solutions, with commercial constraints of time, means, and budget’ (Stappers & Giaccardi in The Encyclopedia of Human- Computer Interaction).

10 MSc Design Thesis: Jana Voykova

Social responsibility is a loose term which pertains to a wide array of aspects. This makes it quite challenging to describe. In more recent times, with the growing presence of the private sector in social life, academic research has focused on defining what corporate social responsibility (CSR) means. Although my research is not going to specifically deal with CSR, I will be analysing business ethics further on and therefore, the term CSR has a relative bearing on my perception of social responsibility.

Singh and Singh define corporate social responsibility as ‘the obligation of decision makers to take actions which protect and improve the welfare of society as a whole along with their own interests. Its suggests two active aspects of social responsibility — protecting and improving. To protect the welfare of society implies the avoidance of negative impacts on society. To improve the welfare of society implies the creation of positive benefits for society’ (Singh and Singh, 2013, p.17).

Aleksander Kobylarek offers an interesting perspective on social responsibility, analysing the term in the context of scientific research and academia. I think his interpretation of scientific social responsibility is very useful in a thesis which analyses how we perceive scientific and technological progress. Kobylarek argues, rightly I think, that scientific responsibility can be understood in implicit and explicit sense. In implicit terms, socially responsible science is concerned with the responsibility to produce truthful and unbiased knowledge which enriches academic research. In explicit terms, socially responsible science ‘should be looking at how particular areas (knowledge, academics, educational institutions) can or should fit in with the needs of the non-scientific community’ (Kobylarek, 2019, p.6).

In other words, socially responsible science is ‘a thorough and expert verification of knowledge which can be the starting point for practical action – designing and altering reality, correcting errors, avoiding mistakes, predicting, constructing comfort zones, and finally making further development of the world possible (ibid.). The opposite of socially responsible science, as defined by Kobylarek is ‘creating fake science and constructing the gullible theories detached from reality and an inadequate vision of the world around us…in order to justify some ideology or some practical enterprise designed purely for profit, such as alternative medicine (Kobylarek, 2019, p.7). In that sense, science shouldn’t be seen as an absolute value because its dependencies on financial support for example, can sometimes harm its credibility and distort its objectives.

11 MSc Design Thesis: Jana Voykova

When I talk about social responsibility and ethical behaviour, I refer to the more general delineation of the term. In order to make it more concrete, though, I will be using the definition, provided by the Pachamama Alliance organisation16:

“Social responsibility is an ethical theory, in which individuals are accountable for fulfilling their civic duty; the actions of an individual must benefit the whole of society. In this way, there must be a balance between economic growth and the welfare of society and the environment. If this equilibrium is maintained, then social responsibility is accomplished. Every individual has a responsibility to act in a manner that is beneficial to society and not solely to the individual”.

An important addition to this definition, is that the welfare of society should not necessarily be understood in utilitarian17 terms because the preferences and interests of the majority are not always in line with upholding our moral values (for example, if a great majority endorses taking drugs, this doesn’t automatically make the act morally permissible).

The assertion which I will try to verify through my research as far as my first research question goes, is that designers do realise how big a social responsibility they bear, however, they are often not in a position to defend their social responsibility because the business and economic models which contextualise their work, either have little actual regard to ethical principles or neglect ethics as a background endeavour, confined to the realms of philosophy (I should however clarify that I do think there are some designers who, for one reason or another, have a more cynical understanding of the social status of their profession as an elitist occupation and they don’t necessarily perceive themselves as having to be socially responsible. That view is rather rare, though). In other words, I will argue that designers are not given enough freedom to demonstrate their social responsibility because of significant deficiencies in our current social fabric but personal circumstances and dependencies should not be used as a justification for unethical behaviour.

16 The Pachamama Alliance is a non-profit organisation established in 1996 in support of the Achuar indigenous tribe in the Amazon rainforest whose culture was threatened with the exploitation of their land by the oil industry. 17 Utilitarianism is an ethical theory which commands that ‘the morally right act or policy is that which produces the greatest happiness for the members of society’ (Kymlicka, 2002, p.10). Utilitarianism was popularised by the 18th century English philosopher Jeremy Bentham who believed that people have two masters: pain and pleasure, and the morality of people’s actions is determined by minimising pain and maximising pleasure for the greatest number. Although the idea to always seek to maximise happiness is sometimes hard to contest, utilitarianism disregards individual rights and liberty, and endorses illegitimate preferences and principles, things which egalitarian and democratic societies, at least in theory, hold dear.

12 MSc Design Thesis: Jana Voykova

To a great degree, this is a result of the popular worldview that design has no place at the decision-making table where the fate of the society is settled, and design can only aid social affairs insofar as visual appeal to consumers is sought after. Consequently, I will argue that this worldview is wrong. This is one of the reasons, according to me, as to why there is a harsh contrast between social design and mainstream design, the former of which is seen as ethical design, while the latter is understood in a strictly business sense.

Until those two kinds of design are reconciled and ideally merged together, any attempts at designing ethically will be just an irregularity, a trend which will not change much about the status quo.

2.1.1 Can we measure social responsibility?

Since social responsibility and ethical behaviour, to which it is closely related (as explained earlier, I am using the two terms interchangeably), are quite broad and sometimes vague terms, it is a challenge to specify reliable criteria to assess and measure them. Furthermore, agreeing on a common definition of behaviour also seems to be difficult for the research community. From a biological standpoint, behaviour is studied through observation, while in psychological and social science contexts (which study mainly human behaviour), questionnaires and interviews are a more popular method. Jana Uher, of The London School of Economics and Political Science presents five different definitions of “behaviour”. I find the following one to be most precise:

“The internally coordinated responses (actions or inactions) of whole living organisms (individuals or groups) to internal and/or external stimuli, excluding responses more easily understood as developmental changes” (p.108) (Uher, 2016, p. 5).

The British researcher Jake Lomax describes behaviour as synonymous with action. He borrows the definition given by Davis et al. to define it:

“Anything a person does in response to internal or external events. Actions may be overt (motor or verbal) and directly measurable or, covert (activities not viewable but involving voluntary muscles) and indirectly measurable; behaviours are physical events that occur in the body and are controlled by the brain” (Davis et al. 2015) (Lomax, 2020, p. 3).

If we unite those two definitions, we can conclude that behaviour is a deliberate or subconscious mental process (thinking, attitude), translated into actions or inactions.

13 MSc Design Thesis: Jana Voykova

Lomax provides a simple yet useful framework (the Mechanisms of Social Change) to track behaviour change. It is based on Aristotle’s Nicomachean Ethics, which Aristotle used to describe important characteristics of human action:

• Who — the actors of an action/inaction;

• What — the action/inaction performed;

• Where — the contextual setting of the action/ inaction;

• When — the timing of the action/inaction;

• With what — pointing to the resources used;

• Why — the underlying reason for the action/inaction;

• How — the way the action/inaction is performed (how much, how often).

I think this framework can be used as a set of measurement criteria when assessing an individual’s or an organisation’s social responsibility, although this measurement will not be definitive.

In their paper Are “ethical” or “socially responsible” investments socially responsible? (2006), Hellsten and Mallin express their concern that it has been unclear what the ratio between individual and organisational ethical and moral commitments should be when socially important questions are addressed and how much this is a contradiction in terms.

They raise some important considerations like what particular actions or inactions of corporations are to be regarded as ethical or unethical; whether markets can also be seen as public servants as they supply commodities and productivity-stimulating products, not purely look to expand their own profits. Hellsten and Mallin bring up the question about the motivations of companies to invest in ethical behaviour, if they do it out of altruistic emotion or to polish their image, and if certain ethical practices in one country can be conflicting with those in another. ‘The social responsibility of business can be defined in a number of ways philosophically, morally and practically, but measuring social responsibility remains a major challenge’ (Hellsten and Mallin, 2006, p. 398).

I acknowledge that ethics is not a straightforward discipline and it is not easy to put a numeric value next to social responsibility but on the other hand, I don’t think there is room for much dispute on what a morally permissible or morally abhorrent action is in many circumstances (for example employing people in exploitative conditions, doing genetic modifications, allowing child labour, disrespecting natural resources, disregarding people’s dignity (human trafficking), bribing, can hardly be seen as grey-area actions). In this sense, I believe we should describe, understand and agree on what we mean by social responsibility rather than measure it. All people bear a social responsibility, not only designers, but also other professionals, ordinary citizens, governments and corporations,

14 MSc Design Thesis: Jana Voykova and social responsibility is not a pie out of which every person should take 10%, for example, in order to fulfil their civic duty. It has more to do with the fact that we are human beings, rather than designers or technologists or politicians, etc. We may indeed be able to measure social responsibility, but this is not what this thesis will do. Rather, it will provide my perspective on what social responsibility means and what factors around it, impact how much it can be upheld.

2.2 Research question #2: social and ethical implications of autonomous transportation

The second research question which I will address in this thesis is:

2. What are the ethical and social implications of autonomous transportation and how can design be used to aid the public understanding of important social phenomena?

As mentioned earlier, the development of autonomous vehicles will be considered as a use case to show a specific technological context where designers’ social responsibility is brought forward to stimulate a critical perspective on the way society is built. In view of this, my discussion on autonomous transportation will not concern the purely technical (engineering) side of the technology but rather its ethical and social worth.

Further to this, I will not give much attention to the conditional factors that will enable the success of the technology — like the necessity to redesign urban infrastructure, the change of perspective when it comes to human-machine interaction or the changes needed to legislation and safety standards. These are obvious steps for the technology to function as expected and I don’t think it’s very productive to discuss them when my critique and consideration of the social and ethical limitations of the technology contests the very concept behind its development.

Questioning the very existence of the technology and the motivations behind it is not a popular approach in the research field and I also see it as a great example to verbalise my comprehension of the social role of designers within a wider social environment.

With my research on this second research question I will try to verify the assumption that disruptive technologies, (and autonomous transportation, more specifically) come with obvious drawbacks that shouldn’t be taken light-heartedly. I will make the claim that these drawbacks should be brought to the public discourse more definitively than they currently are in order to allow a more objective representation of the technology and consequently

15 MSc Design Thesis: Jana Voykova engage citizens more effectively in decisions which will affect them personally. My understanding is that at present, the technology is not portrayed objectively by the companies that develop it, and people in general either have a skewed perception of the social consequences and benefits of the technology, or are not interested enough to proactively look for the information which is absent from the social space, thus disengaging themselves from important social conversations.

Relevant to this question are phenomena like corporate social responsibility and business ethics, as well as social justice and the power dynamics which private and public institutions administer on society. Those aspects will be deemed crucial to my argumentation; however, this thesis will not claim any capacity to resolve such complex social, political and economic problems. They will be brought to the discussion in order to show that design can and should draw the public attention to questions which often fall outside of the public discourse but have great relevance for the future of society.

2.3 Connection between the research questions

While I have made clear that this thesis will examine the convergence between the fields of design, ethics and disruptive technology, the connection between the research questions may not be evident at first glance. Although they are very distinct in a sense and I will discuss them somewhat separately throughout the thesis, my argumentation will aim to explain how they are related and why they both have a place in my research. If we think about design beyond its purely aesthetic functions, we can say that everything is design: design and create are synonyms, and everything around us is designed in a certain way.

When addressing the social importance of designers and their responsibility to work ethically, questions with enormous social impact, like autonomous transportation are extremely relevant to this discussion for at least two reasons: 1) social responsibility presupposes a relationship between people and groups of people; thus, when we talk about designers’ social responsibility, we don’t mean responsibility towards abstract ideologies but towards other human beings; in that sense, questions like autonomous transportation are revealing of what drives designers’ social responsibility as these are creations which have tangible consequences for people; 2) such questions directly affect the way we conceive of ethics and morality, and determine the social context in which designers create. The examination of the two questions and the yielded outcomes will hopefully supply a meaningful academic material and seek to validate the position I have taken with regards to the social role of designers, namely, to assess critically their surroundings rather than simply resolve business problems.

16 MSc Design Thesis: Jana Voykova

2.4 Delimitations

Due to time and resource constraints, as well as the nature of the topics I chose to analyse, this thesis is not going to present an exhaustive research either on the social role of designers and the social importance of design or the ethical and social limitations of technology and autonomous transportation. However, it will aim to bring up prospects which are not traditionally discussed when addressing design and technology ethics. That said, this thesis should be rather seen as a design philosophy than a strictly scientific piece: while the undertaken research does not constitute a skewed vision of the yielded outcomes or the previous research done in the field, it is analysed rather critically which presupposes a certain degree of bias in my argumentation. Furthermore, the zoomed-out perspective which I will demonstrate throughout the thesis might be interpreted as an unnecessary stray from the central topics discussed. I believe this view is mistaken because the backdrop argumentation I will outline, while not being complete, is vital for understanding the point this thesis is trying to make.

Given these limitations, as long as autonomous vehicles are concerned, insights will not be gathered from important stakeholders, such as governments, technology companies, automotive manufacturers, ethicists, sociologists, economists etc., although an attempt to engage some of them has been made. Instead, the perspective of some of these stakeholders will be discussed in the context of the literature research and analysis as part of the project.

As far as the social importance of designers is analysed, the results presented on this research question should be seen as a qualitative sample. It is unrealistic to expect that the constrained research done in the course of a short period of time (4 months) is a thorough representation of a subject matter which hinges on quite complex factors.

In essence, by applying a critical approach to the intersection of the areas of design, ethics and technology, this thesis will rather raise questions than provide straightforward answers and offer a holistic outlook on the present social context of design, ethics and technology. The analysis and discussion, or parts of them, can then be taken further and developed in the course of a doctorate thesis, for example. A summary of the delimitations is provided below.

17 MSc Design Thesis: Jana Voykova

This thesis will NOT:

• Supply a roadmap for how design will be given a more important role in society although possible steps towards this will be explored in the research process;

• Provide a solution to tackling current societal challenges which led to the development of disruptive technologies like autonomous vehicles (urbanisation, congestion, pollution, road accidents, sustainable production). However, these aspects will be outlined in the thesis to a certain extent and possible alternatives will be suggested;

• Resolve ethical and moral dilemmas in society as this is a conversation which design alone cannot address; not to mention that these questions have been part of the philosophical debate for centuries;

• Outline an ethics code which will save the world from corporate greed and make designers martyrs of the greater good. This thesis claims no such thing. The principles of ethical behaviour, although being controversial and sometimes hazy, they already exist, and designers are seen, by no means, as the single upholders of our moral laws.

What this thesis will do is:

• Examine how designers perceive their profession and social responsibility, and explore how designers’ social engagement can be displayed in practice;

• Use critical and speculative design to demonstrate how design can raise the public awareness to challenges of social importance;

• Urge designers and other professionals to be more critical in their work (and life);

• Discuss the social and ethical limitations of autonomous vehicles in a broader socio-technical context and use the technology as an outlet of designers’ social responsibility;

• Ideally inspire reflection as to what a desirable future society may look like and encourage the general population to seek more accountability from public and private power structures by deliberate decisions and actions in their daily lives.

18 MSc Design Thesis: Jana Voykova

3. Method

Designing that does not already Future, Fiction, Speculate, Criticise, Provoke, Discourse, Interrogate, Probe, Play, is inadequate designing.

Cameron Tonkinwise18

In order to answer my research questions and achieve the outcomes I have set out to achieve with this thesis, I am going to apply a combination of methods in my research. I will use desk research as a starting point to gain background knowledge about the previous research of the main topics of my thesis and select theories which will aid my own argumentation. I will conduct interviews and surveys so as to find an answer to my specific research questions.

As a culmination of my research, I will apply speculative and critical design in a virtual exhibition format as a way to verify my predictions around the research questions and share my critique with an audience outside of my academic cohort. In general terms, my master thesis can be described as a Research through design-inspired project. This means that its purpose is to produce academic knowledge contribution which can serve as a basis for subsequent exploration in the respective research field.

3.1 Research through design (RtD)

Research through design (RtD) denotes a ‘practice-based inquiry that generates transferrable knowledge’ (Durrant et al., 2017). As Sir Christopher Frayling19 argues, there has been a misconception and doubt among designers and craftspeople as to how research fits into their respective design practices. Traditionally, research is referred to as something concerned with generating abstracted and unpractical knowledge, with words rather than deeds, while art and design are seen as pragmatic endeavours which produce tangible results. Frayling rightly points out that divorcing craft and design from social, technical and cultural context, disregards the intrinsic connection between art and design on the one hand, and cognitive science and research, on the other. The image of the artist as ‘an expressive lunatic’ whose craft is rooted in feeling and instinct rather than thinking and research, is just

18 Cameron Tonkinwise is the Director of Design Studies and Doctoral Studies at the School of Design, Carnegie Mellon University. He teaches and researches design philosophy and design for sustainability. 19 Christopher Frayling is former rector of the Royal College of Art in London and professor in history of art and design.

19 MSc Design Thesis: Jana Voykova as misleading, as is the assumption that a scientist is ‘a critical rationalist’ who does textbook research in laboratories and universities. ‘Doing science is much more like doing design’ (Frayling, 1993, p. 4). The point which Frayling essentially communicates, is that it is ‘conceptually strange and artecidal20 to separate art and design from all other practices’ since what we perceive as “practice”, namely action which follows reflection and reflection which follows action, can be found both in art and design, and in research and science. In view of this, research through design is an appropriate method to apply in my research to the extent that my discussion on the social role of designers aims to show that artistic professions like design should not be seen as detached from analytical occupations which rely on scientific research. In my thesis, this goes to say that the research done as part of a design project can make valuable contributions to the scientific research community by proposing an unconventional perspective on how design should be seen in the wider social landscape.

Frayling summarises the research that is done in art and design in three categories, borrowing the idea from the English art historian and critic Sir Herbert Read’s “teaching through art”:

• Research into art and design — historical research but also research into a variety of theoretical perspectives on art and design: social, economic, political, cultural, technical, ethical;

• Research through art and design — material explorations but also development and action research: for example, customising a piece of technology to do something nobody has considered before and communicating the results, here the focus is on what is being communicated through the activities of art, craft and design, the end product is knowledge which is both embedded in the artefact but also stands outside of it;

• Research for art and design — the gathering of reference materials rather than proper research, where the end product is the artefact and the visual and imagistic communication is key.

In more recent years, RtD was recognised as a methodology to ‘integrate interaction design with technology and behavioural science in support of HCI21 education and research’ (Zimmerman et al., 2007, p.1) because ‘in the past decades, interaction design and other forms of design were growing their academic basis’ (Stappers and Giaccardi, 2017). The primary concern was the lack of understanding about how design research can contribute to

20 Frayling uses a reference to the Scottish politician Stuart McDonald, which I assume is a wordplay derived from the word “suicidal”. 21 HCI or Human-Computer Interaction is a field that emerged in the 1980s to denote and examine the problems that people had with using computer systems, deciphering instructions and recovering from errors. In this sense, it is closely related to humans’ psychological and cognitive abilities (Bannon, L. J. and Ehn, P., 2012).

20 MSc Design Thesis: Jana Voykova

HCI research by adding the perspective of design thinking into this research field. According to Zimmerman et al., design thinking in RtD is ‘the application of a design process that involves grounding — investigation to gain multiple perspectives on a problem; ideation — generation of many possible different solutions; iteration — cyclical process of refining concept with increasing fidelity; and reflection’ (Zimmerman et al., 2007, p. 2). RtD equips designers with the ability to address so called “wicked problems” which are by definition not approachable using scientific or engineering modes of enquiry, by integrating true knowledge (performing the upfront research for a design project) with how knowledge (the technical opportunities demonstrated by engineers)’ (Zimmerman et al., 2007).

The creation of design artefacts as an outcome of an RtD project enables us to envision a future preferred state as opposed to the current one. The design research which is done as part of RtD projects is different from the design research done in commercial settings to the extent that it produces ‘knowledge that others can use in other areas than the producer of the knowledge is working on’, rather than create a specific solution to be used in the real world (Stappers and Giaccardi, 2017). In other words, as Zimmerman et al. describe it, ‘design researchers focus on making the right things, while design practitioners focus on making the commercially successful things’ (Zimmerman et al., 2007, p. 7). In an RtD context, the artefact ‘creates the possibility for people and products to engage in interactions that were not possible before, and these can come into existence — indeed, become observable — through the design’ (Stappers and Giaccardi, 2017).

Stappers and Giaccardi, like Frayling, draw a distinction between research for design and research through design. In the former, the gathering of scientific and technological information is crucial for the solution for which the design is made (doing research is part of doing design), in the latter, doing design is part of the research and the design activities facilitate the interaction between an artefact (prototype) and people. Zimmerman et al. propose a definition of RtD which is well-suited to explain the purpose of this master thesis: ‘a designerly enquiry focused on making of an artefact with the intended goal of societal change’ (which is referred to as ‘social design’ or ‘critical design’ elsewhere) (Zimmerman et al., 2010, p. 311). They define RtD as an opportunity for the research community ‘to focus on research of the future…allowing us to consider the ethics of what we design’ (Zimmerman et al., 2010, p. 310).

21 MSc Design Thesis: Jana Voykova

3.2 Limitations of RtD

As the academic literature on RtD shows, there is still a level of uncertainty around various aspects of the methodology such as what structure should be followed in an RtD project, what type of knowledge is to be conceived, how it should be communicated and to whom, and how the knowledge contribution is to be applied to other fields of research. Furthermore, RtD ‘is a term that is used primarily in academic work in the design communities’ (Stappers & Giaccardi, 2017) but often its purpose is to drive social change and start a discussion about an issue outside of the academic realm. The distinction between research done in RtD and research as part of a “real-life design practice”, where the artefacts created as part of each of those processes serve very distinct purposes, only make it more difficult for RtD to actually be applied in real-life, in my opinion. I also don’t necessarily agree with the widespread notion in RtD that the created artefact (prototype) should be in an unfinished form and sometimes even purposely unusable in real life even though I do understand why there is a need to differentiate a commercial product from an RtD artefact.

3.3 Speculative and critical design

As mentioned above, RtD is sometimes also referred to as speculative and critical design. The purpose of my master thesis and the topics of my research make the use of speculative and critical design an obvious method22 to achieving the goals of the project. As Ivica Mitrovic of the University of Split, Croatia writes in her paper Speculative — Post-Design Practice or New Utopia (2016), speculative and critical design practice points to the 1960s and 1970s and the radical architecture and design movement whose purpose was to re-think the profession through a political and social prism. As much then as today, speculative and critical design revolts against the modernist technological progress and consumerist ideology. Mitrovic rightly points out that it remains yet to be seen whether this speculative practice will become ‘the new, post-design practice or yet another utopia and historical reference’ (Mitrovic, 2016, p. 12). My application of the speculative and critical design approach accompanies both my theoretical argumentation and the design artefacts that I created as an expression of my critique. It also enables me to share my untraditional view of the convergence between design, ethics and technology with a wider audience thus verifying my assumptions.

22 Dunne and Raby prefer to describe speculative and critical design as a position that designers take on, rather than a methodology they use.

22 MSc Design Thesis: Jana Voykova

Speculative and critical design are popular methods when formulating critique towards social phenomena like the role of technology in everyday life. These terms, both within and outside of the academic space, are often used interchangeably, or unified in the abbreviation SCD, and sometimes referred to as design fiction. In actuality, the differences between them are very subtle and they serve one and the same purpose. As James Auger23 notes, ‘they all remove the constraints from the commercial sector that define normative design processes; use models and prototypes at the heart of enquiry; and utilise fiction to present alternative products, systems or worlds’ (Auger, 2013, p. 1). Gert Pasman of the Delft University of Technology in the Netherlands describes design fiction as ‘a practice which applies design as an instrument to generate awareness, raise concerns or challenge values about (the use of) new, emerging and future technologies, products and services’ (Pasman, 2016, p. 512). Nevertheless, it seems that design fiction belongs more to the world of art, cinema and literature, and rather celebrates technology than questions it. A common tool in design fiction is film and video which depict ‘an idealized and utopian world, in which people interact fluently and effortless with large amounts of data through the use of interactive applications’ without reference to ‘any social meanings or implications that might distract’ (Pasman, 2016, p. 513).

Design fiction is often mentioned in line with the work of the science-fiction writer Bruce Sterling. His definition of the term is ‘the deliberate use of diegetic prototypes to suspend disbelief about change’ (Dunne and Raby, 2013, p. 100). But the focus of design fictions is narrower than what speculative and critical design (SCD) concentrates on. Design fiction deals with single objects rather than entire worlds, social and political trends. It is more of a technological storytelling with design objects and a reference to the already known than a construction of ‘glitchy, strange, disruptive’ (Dunne and Raby, 2013, p.100) socially oriented critiques of the present.

The term critical design was coined by the British designers and researchers at the Royal College of Art in London Anthony Dunne and Fiona Raby in the 1990s as an opposition to affirmative design (mainstream design) — design that reinforces the status quo. In their book Speculative everything (2013), Dunne and Raby describe critical design as ‘an attitude that uses speculative design proposals to challenge narrow assumptions, preconceptions and givens about the role products play in everyday life’ (Dunne and Raby, 2013, p. 45).

Critical design is sometimes tarnished as mere complaining and negativism towards already established and accepted lifestyles or attitudes within the society. This is a mistaken perception because critical design aims to make us think and question whether the already

23 James Auger is an Associate Professor at the Madeira Institute of Interactive Technologies, Portugal. His uses speculative and critical design to question the role of technology in everyday life.

23 MSc Design Thesis: Jana Voykova established is indeed the right/best way to go about something. It allows us to envision and demand better futures from industry, society and political spectrums. Critical design provides constructive alternatives to how the present could be instead of raising critique for the sake of being critical. As Dunne and Raby maintain, critical design reveals critical thinking about design itself, but it is also ‘directed at the technology industry and its market- driven limitations, and beyond that, general theory, politics and ideology’ (Dunne and Raby, 2013, p. 46). The designer is the facilitator and catalyst of this type of conversations rather than a self-righteous moralist. He/she constructs a space where this conversation is possible and desired. Instead of preaching from a higher position and providing the answers, by means of critical design, designers encourage the audience to ask the questions, make the effort and think themselves. Critical design is idealistic, it shows that change is in the power of people; to quote Dunne and Raby, ‘sometimes we have more effect as citizens than as designers’ (Dunne and Raby, 2013, p. 37). It can be expressed in many ways, it borrows from other artistic disciplines like cinema, literature and music, but critical design is not art. Usually, the result is unsettling, sometimes bewildering and disturbing but unlike art, critical design needs to be closer to the everyday, ‘if it is too weird, it will be dismissed as art, if too normal, it will be effortlessly assimilated’ (Dunne and Raby, 2013, p. 54). As Dunne and Raby suggest, critical design doesn’t aim to replace mainstream design or provide a map, a solution. It is not a prediction but rather a compass which shows direction with many possible solutions.

Speculation is a key ingredient of critical design; this is why the terms critical and speculative design are usually used conversely (or merged together, as I will do in the rest of this thesis). By creating prototypes (or provotypes as they are also referred to), designers provoke and encourage the audience to re-think the reality we live in and project either a future that is more desirable or one that we fear might become a reality (depict a dystopian future, as I will do in my application of SCD in this thesis). Dunne and Raby see speculative design as the bridge between the real and the imagined reality. We need this bridge because we normally think of speculation and fiction as something that doesn’t belong to the everyday. By placing the actual and the fictional (speculative) reality in one room, we can envision them together which is essentially the first step towards conceiving a different and attainable social and political order. In this sense, speculative and critical design has the potential to exit the academic and the artistic fields and enter the mundane and ordinary which is where actual change can happen.

‘Further, SCD strives to encourage designer-user dialogue and public debate about preferable societal development, among others with help of creative, narrative and aesthetic methods’….A main communicative feature in SD and CD can be connoted by the term ‘discourse’, which e.g. implies the goal to raise questions and encourage debate, not to provide answers or create solutions. The term ‘discourse’ refers to postmodernist Michel

24 MSc Design Thesis: Jana Voykova

Foucault. Foucault discusses the relation between discourse, power and knowledge. A discourse displays ways of organising and producing knowledge, while language is a crucial mean of exerting power, for example by influencing what is true, acceptable and practical. Discourse power is executed by controlling communication, for example defining who is ill or healthy, what is legal or prohibited, what is a useful or inadequate product, etc. (Johannessen et al., 2019, pp. 1624-1625). ‘Proposing a critical attitude of the designer has roots in design history. The ideas of William Morris, and the mid-19th century Arts and Crafts movement in Britain, were based on values challenging those of the contemporary industry (Raizman and King, 2003). Morris’s ethos refused capitalist and consumerist ideas, an ideal that later became an inspiration for the Weimar schools of craft and after that the Bauhaus. Tharp and Tharp describe anti-design and radical design, two avant-garde postmodernist movements of the 1960s and 1970s, as predecessors of SCD (Johannessen et al., 2019, p. 1626).

Table 1. In their book Speculative everything, Dunne and Raby describe speculative design as the A/B manifesto. In the A column we find the principles behind mainstream design while the B column describes the principles that drive speculative design.

A B

Affirmative Critical

Problem solving Problem finding

Provides answers Asks questions

Design for production Design for debate

Design as solution Design as medium

In the service of industry In the service of society

Fictional functions Functional fictions

For how the world is For how the world could be

Change the world to suit us Change us to suit the world

Science fiction Social fiction

Futures Parallel worlds

The “real” real The “unreal” real

Narratives of production Narratives of consumption

Applications Implications

Fun Humour

Innovation Provocation

25 MSc Design Thesis: Jana Voykova

A B

Concept design Conceptual design

Consumer Citizen

Makes us buy Makes us think

Ergonomics Rhetoric

User-friendliness Ethics

Process Authorship

3.4 Limitations of speculative and critical design

Undoubtedly, speculative and critical design is a constructive philosophy not only for designers but for other professionals, too. However, there are certain pitfalls to SCD which are worth pointing out. Although it does allow for the merging of the everyday with the imaginary and provides a space where public discourse on important topics can happen, there is the risk that what has been reflected on and considered within those spaces, remains there after the event (even though the purpose of the critiques/speculations is the exact opposite). I agree with Dunne and Raby that change starts with the individual, however, change on societal and political level doesn’t happen either individually or within a few hours/days. Already accepted behaviours are hard to alter.

Unfortunately, speculative and critical design, just like RtD, have not (yet) grown substantially outside of academia. The rigid distinction between speculative and critical design on the one hand, and mainstream design — whose purpose is to ‘make money for the industry and ‘solve problems for industry, address client’s needs, and conform existing cultural, economic and technical expectations of society’ (Johannessen et al, 2019, p. 1628) on the other hand, restricts the real impact SCD can successfully achieve.

26 MSc Design Thesis: Jana Voykova

Table 2. Comparison between mainstream design and speculative and critical design (SCD), Tharp & Tharp, a contracted version of Dunne and Raby’s A/B manifesto.

MAINSTREAM DESIGN SPECULATIVE AND CRITICAL DESIGN

Attitude Normative Critical

Foundation Information Speculation

Mindset Pragmatic; Productive Idealistic; Dreaming

Commercial: Satisfy industry’s need Discursive Purpose to make money Spur debate on the development of society Develop solutions: Provide answers by Explore ideas: Find problems by asking Goal solving problems questions Serve a user: In seriousness provide Provoke an audience: Use ambiguity to make Intent clarity satire

While I agree with Estelle Hary and Bastien Kerspern that ‘reaching for clients with a design fiction or speculative design posture sounds a bit paradoxical’ (Mitrovic, 2016, p. 77), we need to move this conversation from academic research to the business realm if we are to achieve real change. Critical attitude naturally begins within academia and it is education’s privilege to support and stimulate critical perspectives strongly enough in design students so that they continue to apply them in their practice after they graduate. Still, bringing up what is plausible, probable, desirable or dystopian shouldn’t be a single, occasional event but a position/mindset that is interwoven in the whole practice of designers. We need to find the balance between making speculative design too mainstream to the point when it has become just a label for something else (the so-called innovation trap24), and the unfolding of its power to truly change the status quo. Speculative design has also sometimes been chastised for its ‘Eurocentrism’, in other words criticising from a privileged Western position.

‘Critical designers at the Royal College of Art for instance, imagine what they believe to be dystopian scenarios in a distant future, when in fact people in other parts of the world are already living versions of those lifestyles’ (Cameron Tonkinwise in Mitrovic, 2016, p. 24). ‘SCD seemingly has many norms of its own, and most topics revolve around the ‘domestication of technology’ and can be classified as ‘white man’s’ problems’ (Pierce in Johannessen et al, 2019, p. 1630).

24 Matt Ward, Head of the Design Department at Goldsmiths, University of London talks about the hype around technology and how it disrupts markets. In this process, “visionaries”, people who can imagine the unimagined are crucial for businesses and investors which might make speculative design attractive in this space. However, as Ward points out, ‘the big worry is that critical and speculative design become the advertising arm of venture capital’ (Mitrovic, 2016, p. 20).

27 MSc Design Thesis: Jana Voykova

Figure 3. A high-level overview of my application of SCD in this thesis research: on the one hand, its critical approach serves as the basis of my argumentation, desk and primary research; on the other hand, I use it to translate my critique into a more practical and approachable format — a prototype exhibition with the design artefacts I created in the course of my research in order to criticise, unsettle, provoke, speculate and question the way ethics, design and technology are perceived today. The concepts behind the artefacts will be explained in more detail in the Research Process chapter.

3.5 Literature and desk research

Performing desk research, or ‘reviewing previous research findings to gain a broad understanding of the research question’ (Travis and Hodgson, 2019, p. 51), is a helpful and natural first step towards conducting primary research25. As David Travis and Philip Hodgson note in their book Think like a UX researcher (2019), it is unlikely that one research project is 100% another one alike. Despite this though, it is likely that another research project has tried to answer similar questions. Reviewing previous research makes the research more objective and credible and eliminates doing unnecessary work. The data collected previously can also steer the actual research into more productive directions than initially anticipated.

25 Research where you go out and discover stuff yourself (Travis and Hodgson, 2019, p. 51).

28 MSc Design Thesis: Jana Voykova

I focused my desk research on previous examinations in the fields of ethical design/social responsibility of designers, social and ethical considerations of autonomous vehicles as well as research and theoretical framings from the field of business ethics. These resources will be discussed and analysed in finer detail in the Theoretical Framework chapter of this thesis. As might be expected, my argumentation and review of them will be highly critical.

3.6 Interviews

In order to answer my research questions and verify the validity of my assumptions, I conducted a series of interviews. Steve Portigal describes interviewing as ‘conducting contextual research and analysing it to reveal a deep understanding of people that informs design and business problems’ (Portigal, 2013, p.41). As he writes in his book Interviewing People: How to uncover compelling insights (2013), interviewing is among the most popular, reliable and quick ways to gather information about needs and pain points, confirm hypotheses, study behaviours, analyse opinions, to name a few. Although Portigal discusses doing interviews from a business perspective (as part of a UX research process), interviews play a crucial role in academic research and ethnographic studies, as well.

I did two rounds of interviews each targeted at answering the two research questions. In the first one, I talked to practicing designers about their views on the design profession and what designers’ social responsibility encompasses. I chose to connect with practicing designers instead of researchers or students because I wanted to see how social responsibility is perceived in “real-life”. This allowed me to gauge opinions from a more practical and realistic pool as opposed to examine how the term social responsibility is understood through an idealistic lens (in academic research). I knew some of the designers I interviewed personally, others I encountered in online design communities. I contacted designers with various design backgrounds and profiles so as to enrich my research with different points of view. The majority of the designers are Bulgarian, however, there are representatives of other countries, as well.

In the other round of interviews, I discussed the development of autonomous transportation and its social impact with “ordinary” people who, in one way or another, have a relation to the topic (displayed either by interest, profession or other personal investment). Since I didn’t manage to engage other relevant stakeholders in this discussion: automotive companies, technology companies who are developing autonomous vehicles software, lobby organisations, I decided to talk to citizens and collect their opinions on the technology as they were more approachable. I interviewed friends but also people I didn’t know personally.

29 MSc Design Thesis: Jana Voykova

My target group was people who worked either in automotive or autonomous vehicle divisions but also people whose work was not directly connected with the technology. More details about my interviews are to be found in the Research Process chapter of this thesis. Because of the time limitations, I only talked to 20 people in total (10 for each category), which is why the insights I gathered have a rather qualitative than a quantitative character (detailed description and analysis of the interviews is provided in the Research outcomes and Discussion chapter of this thesis).

3.7 Surveys (Questionnaires)

Combining several methods of research is a good way to achieve objective results for analysis. To expand on the information and insights I collected during my interview sessions, I introduced two short online surveys for each research question and shared them with the same groups of people which I targeted in my interviews:

• Practicing designers of different design profiles (disciplines);

• People from my personal/professional network.

Both of the groups included people of various nationalities, ages, professions and sexes. Alan Aldridge and Ken Levine define surveys as an effective method for gathering and analysing data in sociological and psychological research. Surveys allow researchers to ‘collect the same information (variables or types of data) about all the cases in the sample’ (Aldridge and Levine, 2001, p. 5). They provide the opportunity to obtain both qualitative (by means of open-ended questions) and quantitative (by means of closed questions) data quickly. Some scholars criticise surveys for being atomistic: reducing a whole population to the answers provided by a sample of individuals. However, as Aldridge and Levine point out, surveys might be atomistic, but they are not concerned with individuals at all. Instead, they produce aggregate data, and this is exactly the type of data collected in my research — it doesn’t pretend to reflect the opinion of whole countries or continents, but rather — of a selected sample of the population, and this is clearly stated as a limitation in the analysis of my results.

Aldridge and Levine argue, and I agree, that any type of research conducted poorly will yield poor results, so the factor that matters most, is the quality of the execution and analysis, not the method itself. According to them, surveying international groups of people can be problematic because this may result in a fragmented data. I decided to proceed with surveying international respondents anyway because the purpose of my research is to explore opinions of people from different backgrounds and circumstances, professional,

30 MSc Design Thesis: Jana Voykova national and personal, instead of confining the research just to my home developing country whose political and economic reality is quite distinct from the countries where the design profession and the development of autonomous transportation are more mature and the economic situation is much more stable. Further details of the process are to be found in the Research Process chapter, the collected insights are analysed in the Research outcomes and Discussion chapter.

3.8 Applying SCD in a prototype exhibition format

In order to validate my research, put my critique into a more practical dimension, and share my position with other people than my professors, I created several design artefacts which I was planning to display in an exhibition. This way, I was hopefully going to reach a broader audience in a more approaching way and offer a critical inspection of the kind of future we might be currently designing as a society. The artefacts communicate my concerns about the importance we attribute to technology in our daily lives and question the viability of the idea behind the so-called smart cities which autonomous vehicles are going to infuse. The exhibition was to spark reflection on how our behaviour and modern lifestyles disempower us to reflect critically on our social landscape and challenge normalised habits and visions.

The purpose of the exhibition was also to validate my arguments by gauging the reactions towards this critical outlook and hopefully inspire conversations on topics that normally do not make it to the dinner table. I chose not to focus specifically on autonomous transportation but depict a more general dystopian present-future intentionally.

My critique of the technology is not directed at its shortcomings as such because I do believe that it has the potential to impact societies positively if the risks around it are taken seriously. My critique of the technology is rather connected with the redesign of important human values and practices as a result of the development of disruptive technologies like autonomous vehicles, and the subsequent widening of social division within and between countries. I also contest the idea that we should use innovative technology to amend any gaps in our human nature.

In that sense, my critique goes beyond the narrow context of this technology and looks at the consequences of technology in general in a much wider and a fairly philosophical sense.

By combining speculative and critical design with design fiction I hoped I would inspire people to reflect on how the already established and normalised can have detrimental effects for marginalised social groups and how technology is often used to attack human weaknesses

31 MSc Design Thesis: Jana Voykova

(consciously or otherwise) than to enhance and amplify human mentality. More details on the design process and the concepts around the exhibition are outlined in the Research Process chapter.

Due to unforeseen circumstances, I had to make slight changes to the exhibition and set it up as a virtual experience instead (hence it became a prototype of an exhibition). Details about this will be explained in the Research Process chapter.

32 MSc Design Thesis: Jana Voykova

4. Theoretical Framework

This chapter of the thesis represents a reasonably detailed discussion of some of the previous research done in the fields of design and machine ethics (ethics of technology/autonomous vehicles). It highlights resources which I believe back up my motivation and seek to explain in further detail the connection between the research questions. Throughout my theoretical argumentation and more focused in the Business ethics chapter, I will analyse past and current economic, political and technological phenomena and theories relevant to the topics of my research, which have a direct or indirect impact on design, ethics and the evolution of technology.

While my outline of these events and perspectives is not exhaustive, and it might be perceived as a deviation from my research topics, I consider a more systemic outlook on these questions crucial to achieving a fruitful debate on the intersection of design, ethics and technology and absolutely necessary so as to avoid my motivations and argumentation to be completely misunderstood. Furthermore, these questions, being so sensitive and political, seem to be traditionally omitted or just briefly mentioned in the design ethics papers and research, and therefore, might spark a productive deliberation in the design community as well as academia.

4.1 Design ethics

Designers make the world’s most beautiful trash.

Scott Ewen

The above is a quote from the book Do good design (2009) by the Canadian graphic designer David Berman. Like many other designers, Berman was completely oblivious to the gravity of his profession until a female friend of his called him out on the fact that ‘graphic designers were responsible for destroying forests in support of the systematic objectification of women by using pictures of their bodies to help sell products…’ (Berman, 2009, p. 26). Although his book is rather a critical statement against the advertising age and the wicked power of consumerism which, has unapologetically sucked graphic designers in its profit-generating machine for decades, it also pays homage to the ethical aspect of the design practice.

33 MSc Design Thesis: Jana Voykova

Berman identifies design as ‘the most efficient (and most destructive) tool of deception in human history’ (Berman, 2009, p. 2). He talks about corporate branding and the habits of overconsumption that corporations foster. His book urges designers to use their persuasive skills to distribute ideas which have more to do with creating a socially just future than with endorsing unsustainable standards of living. Inspired by the Canadian journalist Naomi Klein’s book No logo, Berman draws attention to the immoral practices of companies from various industries — retail, fashion, technology, food, automotive, which do not only create a false image of themselves (as agents of the public good) but also manipulate the general population to identify with brands and base their moral values on the products they buy.

Ironically, designers are no innocent bystanders in this process. They have submitted themselves, either subconsciously or forced by certain circumstances, to serve a system which has little regard to serving people’s real needs. In this scenario, designers’ professional success depends on their not asking questions or thinking critically. As David Berman notes, when leveraging graphic designer’s power to cleverly craft messages and images in order to deceive people, these messages become a lie. Naturally, this doesn’t mean that all advertising is, by nature bad. But I agree with Berman that ‘good design should be about what is good about the product, not what is vulnerable in the buyer’ (Berman, 2009, p. 92).

The ideas presented by Berman, are not notorious. He is not the first one to bring awareness to the social responsibility of designers and suggest that designers should follow an ethics code, similar to the Hippocratic Oath in medicine. In 1964 the British graphic designer Ken Garland wrote the First things first manifesto in London, which was signed by 22 other designers. In essence, it was an advocacy of designers’ ability to put their creative skills to socially important causes instead of devoting their time to advertising and commerce. In 2000, David Berman, together with other designers, wrote the Code of Ethics for designers in Canada, which, in its turn, inspired the American designer association AIGA (in 2005) and the Norwegian graphic designers and illustrators’ association (in 2008) to do the same.

I fully support the belief that designers should swear to an ethics code, but they should also be held accountable to abide by it. While Berman lays out some timely and adequate remarks, I disagree with him that it is enough for designers to dedicate at least 10% of their time to virtuous work. As Mike Monteiro ardently affirms in his book Ruined by Design (2019), doing unethical things 90% of the time and using the 10% left to make up for them, is like putting a lipstick on a pig. Designers don’t work for their employers, they work for ‘the people who aren’t in the room’, as Monteiro puts it (2019, p. 217).

Whenever I bring up the general absence of ethics in business, I often hear that “this is how the world works” as if this is a good enough justification that something isn’t quite right. The world isn’t broken, though. ‘It’s working exactly as it was designed to work. And we’re the

34 MSc Design Thesis: Jana Voykova ones who designed it’ (Monteiro, 2019, p. 56). We just didn’t ask why and said no often enough, as Monteiro holds in his book. He wrote a designers’ code of ethics, the first principle of which, states: ‘before you are a designer, you are a human being’ (Monteiro, 2019, p. 35). This means that being a designer who has brilliant typographic, layout or UX skills, doesn’t override caring about how these skills affect people. In fact, paying close attention to how our designs impact people should transcend our artistic flair.

Monteiro was greatly influenced by Victor Papanek’s book Design for the Real World (1971). In the late 1960s, Papanek identified (industrial) design as one of the most dangerous occupations. He wrote Design for the Real World, because there were no books at the time about designers’ social responsibility. This was Papanek’s attempt to change designers’ own perception of their role in society and educate the public on what design should really be about. Unsurprisingly, Papanek’s book was met rather with outrage than awe when it came out. Stating that ‘social and moral judgment must be brought into play long before the design process begins’ (Papanek, 1971, p. 55), might have been perceived as giving an ultimatum to either choose virtue or make a profit, as if the two are mutually exclusive (sadly, it does seem that in practice, they are mutually exclusive as I will argue further on in this thesis).

Surprisingly large number of people find it slightly odd to talk about the social responsibility of designers, because traditionally, the profession has rather belonged to the cosmetics department. Perhaps this is one reason why the cognitive scientist Donald Norman wrote a book (The design of everyday things, 2002) about why designers should design with human psychology in mind and coined the term “human-centred design”, also known as the less benign user-centred design26. For better or worse, in the past few years, we have been recklessly using this term to describe every possible activity we undertake as designers, regardless of whether we actually do or not. Probably that is because it seems that today, it is more important how things look like than how they actually are.

In the 1960s, Papanek wrote that ‘many glamorous jobs, the majority of industrial design students in the United States are educated to deal with and delighted to have, just happen to be with firms whose policies and practices are far from enlightened in terms of the public interest or people’s desire for well-made, ecologically responsible, and aesthetically pleasing products (Papanek, 1971, p. 335). Disappointingly, things haven’t changed much on this front since then. Despite Papanek’s appeal to ‘design for people’s needs rather than their wants, or artificially created wants’ (Papanek, 1971, p.234), it appears we have done just that — designed artificial needs. Part of the success of this process, is the cleverly marketed

26 The American designer Adam Lefton writes, ‘labelling people as users strips them of complexity. It reduces humans to a single behaviour, effectively supporting a view of people as more like robots whose sole function is to use a product or feature’. This quote is taken from Lefton’s article “As a designer, I refuse to call people users” published on the online publishing platform Medium.

35 MSc Design Thesis: Jana Voykova

“exclusivity” of the design profession. Calling designers “creatives” and “pixel-pushers”, however, is not glamorous, it’s derogatory and implies that designers are some lunatic creatures who can only make pretty things and follow orders. Such categorisations appeal to a lot of (young) designers because they make them feel special and give them a sense of rarity. If companies and by extension, designers were seriously invested in designing for humans, though, we would not be living in societies where human’s addictive nature is constantly being exploited, the idea that what we have rather than who we are is continually bolstered and our so-called democratic principles and sustainability awareness lead us in the exact opposite direction of a sustainable and just world order.

In Design when everybody designs (2015), Ezio Manzini talks about the move from the conventional mode of ‘doing things as they have always been done’ to the design mode of combining three key ingredients of design: ‘critical sense (the ability to look at the state of things and recognize what cannot, or should not be acceptable), creativity (the ability to imagine something that does not yet exist), and practical sense (the ability to recognize feasible ways of getting things to happen)’ (Manzini, 2015, p. 31). Manzini argues, that as a result of the second industrial revolution, and especially with the increase of connectivity and the digital age, societies have started thinking and functioning in completely unconventional ways and ‘design mode has become dominant in all fields, at all levels of human activity, and for every kind of subject, whether individual or collective’ (Manzini, 2015, p. 32). While I see where this statement comes from, I am not entirely convinced that ‘all organizations (whether public or private) are genuinely becoming design-driven’ as Manzini suggests, simply because the political and economic imperatives which govern those organisations, seem to be unable to accommodate for social innovation27 and ethical behaviour, and this makes them pretend to be design-driven (I’ll expand on this later).

According to Manzini, not all design is design for social innovation. Social innovation is concerned with ‘sociotechnical transformation driven by and oriented towards social change’ (Manzini, 2015, p. 63) and is different from social design. Social design, according to Manzini, is concerned with exceptionally problematic situations such as extreme poverty while social innovation can be directed at problems that concern both poor, and middle and upper classes. I disagree with Manzini on this point, although he is not alone in asserting that social design and design for social innovation are two different things. I think social design and design for social innovation should be seen as synonymous. Ideally, they should be merged with mainstream design, submitting it to their principles, not being dissolved in it. Social design is normally regarded as a solely charitable activity, stimulated by ethical principles, while mainstream design is usually seen as detached from doing social good and

27 Manzini defines social innovation as ‘everything that expert design (professional design) can do to activate, sustain, and orient process of social change toward sustainability’.

36 MSc Design Thesis: Jana Voykova concerned mainly with making money. Still, putting mainstream design and design for social innovation (or social design, for me they are the same thing) in two different categories, in which the former is promoted by economic goals while the latter — by ethical motivations, restrains what design can actually do in order to improve how society is built. This binary conception mistakenly suggests that ethical design is always doomed to be done for free as it is not economically viable. Although Manzini acknowledges the so-called “dark side” of the Internet and the digital age it habituated, he seems to consider the ubiquitous connectivity of the 21st century as the only condition to achieve global cooperation and social innovation in local and global communities. While ubiquitous connectivity eases communication and collaboration, there are other, just as effective analogue ways to organise, facilitate and promote social innovation. They simply require more engagement and effort — things we seem to be increasingly reluctant to give in modern societies.

In his thesis paper, Daniel Scott discusses the responsibility which graphic designers bear to improve their surroundings through their work, instead of serving the consumption-driven market economy. He questions the common perception of designers as people who make things look nice in order to help sell more products and defines designers as educators of their audience about right and wrong. According to Scott, designing for social change is a way for the designer to put their work to use in projects that address social issues and improve the community a designer creates for. The definition which he uses to denote social design is ‘design or a process of design that contributes to improving human well-being or society’ (Scott, 2012, p. 10). He refers to Andrew Shea’ book Designing for social change (2012), where Shea developed a framework for designers to utilise when designing for social causes.

This framework consists of ten strategies: 1. Immerse yourself, 2. Build trust, 3. Promise only what you can deliver, 4. Prioritise process, 5. Confront controversy, 6. Identify the community’s strengths, 7. Utilise local resources, 8. Design with the community’s voice, 9. Give communities ownership and 10. Sustained engagement. I find the fifth one — confront controversy, particularly interesting. It commands that designers should not shy away from discussing controversial topics through their work, topics that the community they live in, does not necessarily want to hear or think about. This, in my opinion, is at the heart of what it means for a designer to be critical and socially responsible.

If I can make a generalisation in this regard, I believe people do try to avoid thinking or talking about the negative aspects of life, probably because they have enough negativity in their personal lives and also because they rely on creative professions like design to not bring forward painful conversations, to give them a break and show them the beautiful side of life. Design may be an artistic occupation, but it is nonetheless an intellectual one too. Just as in theatre, there is comedy and tragedy, so too in design there should be a place for catharsis,

37 MSc Design Thesis: Jana Voykova for meaningful and not necessarily pleasant deliberation. Also, burying our heads in the sand isn’t going to resolve our problems, it’s going to exacerbate them. Scott quotes Paul Nini’s article In Search of Ethics in Graphic Design (2004), published on AIGA’s webpage where Nini identifies that designers’ ‘most significant contribution to society would be to make sure that the communications we create are actually useful to those for whom they're intended’.

I agree with Nini that ‘the client’s desire for profits, and our desire for visual sophistication (and peer recognition) should come after the needs of our audience have been met’. However, I think we should use the word “needs” with great care and conceive critically of those needs that we, as designers are addressing because “needs” can be rather a distortion than a reflection of reality (as Victor Papanek observed in his book Design for the Real World, and as I would argue, creating fictitious needs seems to be a widespread and lucrative practice in business today).

An interesting aspect of Scott’s thesis is his discussion on the role of propaganda in (graphic) design. As he points out, ‘the connection between propaganda and advertizing is so great that it could almost be seen as a topic that should be focused on more within visual communication or graphic design education’ (Scott, 2012, p. 15). Although we tend to associate the word “propaganda” with negative bias and brainwash, I agree with Scott that ‘propaganda is just a form of communication’, ‘a message delivered to the masses through various media forms with the purpose to ‘influence the behaviour, opinions and decisions of the people it is directed at’ (Scott, 2012, p. 14, citing Jowett & O’Donnell). All communication is propaganda in a way. The difference between a good (ethical) or a bad (misleading) propaganda, lies in the sources it comes from, the effects it wants to instil in its audience and the audience’s capability to assess the communicated message critically.

While I agree with Scott’s overall line of reasoning, I certainly cannot endorse his statement that designers’ work does not always reflect the kind of individuals they are as sometimes the solutions they create are not necessarily in support of causes they defend. I realise he claimed this because of the contingent factors which often restrict designers’ idealist expression. On the other hand, though, when we sign our names under a design, we testify our approval of it. Whether we did what we did because our boss told us or because we had families to feed, can only ask for sympathy and understanding but by no means excuses our behaviour. This doesn’t mean that we should completely disregard the contingencies that obstruct designers to demonstrate their social engagement, but it will be just as wrong to use those contingencies to wash our hands. If we do, nothing will ever change because contingencies are never going to disappear.

38 MSc Design Thesis: Jana Voykova

Megan Prusynski of the University of Idaho used editorial design in her bachelor thesis to bring up awareness to the social responsibility of designers. She designed a magazine called unplug where she expressed her position as a citizen and a designer against the use of design to promote unethical practices like consumerism, labour exploitation, the ‘deteriorating connection between humans and the natural world’ (Prusynski, 2005, p.18).

Much like Scott, Prusynski stands up against designers utilising their creative skills to drive forward advertising and consumerism. In her view, activism — ‘an action taken in opposition to, or in support of a cause’ (Prusynski, 2005, p. 5), can use design as a tool for social change. She mentions cultural movements like Futurism, Constructivism and Dada in the early 20th century as a reflection of designers’ protest and political critique. As she points out, although many designers used their ability to reach a broad audience to communicate socially important messages, design’s social role was still sporadic, and this intensified with the emergence of brand identity design.

Although I find Prusynski’s observations on the design profession and its social significance to be logical, I would argue that designers are not entirely ‘servants to corporations (the largest polluters and most powerful criminals)’ (Prusynski, 2005, p. 8), as she calls them. Employees do depend on their employers in many respects, but they are not always victims of private companies and private companies are not the only ‘criminals’ out there. We all contribute to the flourishing of consumerism: after all, if one designer refuses to support unethical behaviour through their work and another 20 have no problem with it, the latter can hardly be seen as victims, they are just as blameworthy as the corporations assigning the projects, whatever their motivations. Like Scott, Prusynski seems to be viewing corporations’ economic greed and designers’ dependency on having to have a means of subsistence as a good enough justification for unethical actions. Personal life contingencies do exist but there should be a place where we draw the line.

Prusynski pays attention to the charitable nature of social design and, like Ezio Manzini, categorises it as primarily pro-bono. Interestingly, a lot of large and powerful organisations whose work addresses good causes (like UN and WWF), also see social design as a charity endeavour. They themselves seem to be assuming that if a designer (or anybody else) wants to contribute to a noble cause, they should not expect remuneration28. Why is that? Is it because these same good cause organisations are looking out for other interests as well? Or because we simply don’t see doing good and doing business as compatible activities? Maybe

28 A lot of large non-profit organisations offer unpaid internships and organise campaigns for voluntary work and use their prominent image to sell the engagement to designers and other professionals who need experience or just want to contribute to a good cause. The same goes for some well-known design agencies. Something must indeed be wrong with an economy where young people ‘chase unpaid internships with more energy than those in previous generations sought paying work’ (Rushkoff, 2016, pp. 108-109).

39 MSc Design Thesis: Jana Voykova both. In any case, when they are trying to recruit external help by selling the engagement as exposure and a portfolio jewel, I am seriously doubting their good intentions. But this is simply how the world works, isn’t it? We shrug our shoulders, we roll our eyes and move on with life. Governments, private players, non-profit organisations, and individuals, should stop viewing social engagement and doing good as a random voluntary experience, paid for with a status-boost before the public. We cannot possibly expect much to actually change until we actually start fixing the system at its core. Everything we say or do until then, is just rhetoric and clearing our guilty conscience.

Richard Devon and Ibo Van De Poel made an attempt to look at these questions holistically in their paper Design Ethics: The Social Ethics Paradigm. They argue for a social ethics approach which treats ethics as a collective endeavour that stems from social relations, norms and decision-making policies as opposed to the mistaken mainstream individualistic regard to ethical design and technology. They claim, rightly I believe, that ethical considerations and repercussions are embedded in the whole design process and the sum of decisions taken as part of it.

The point they are trying to make is that who makes the decisions, rather than what decisions are made, is more crucial. I would argue that, from an ethical standpoint, both of these components of the decision-making process are equally important. As the authors note, ‘different arrangements in the social arrangements for making a decision about technology can have very different ethical implications and hence should be subject for conscious reflection and empirical enquiry in ethics’ (Devon and Poel, 2004, p. 463).

One of the examples they provide in support of this argument, is the following hypothetical situation: an engineer, David, is confronted with an ethical dilemma of exposing the unethical behaviour of his employer. Whatever decision David might make (to expose his employer, confront them or create a different technical solution to make up for the questionable outcome), this is not a decision which ‘entails permanently changing any social arrangements’ as ‘decisions are almost always social, with many stakeholders, and it may be ethically suspect to reduce such problems only to individual dilemmas’ (ibid.). In this sense, abiding by ethics on the individual level is desirable and necessary but an individual ethical act does not automatically translate to the collective social behaviour. If a person X abides by their individual ethics code throughout their life, this does not make their ethical behaviour a default behaviour for all other people in his or her community (although I would say it gives a good example).

Another example Devon and Poel provide, is taken from a real-life situation. In 1972 an engineer, Dan Applegate, drew the attention to a faulty design in the McDonnell Douglas DC-10 jet airliner which could lead to a plane crash under certain circumstances. The

40 MSc Design Thesis: Jana Voykova contract between McDonnell Douglas and a subcontractor, Convair, did not allow Convair to report this to the Federal Aviation Administration (FAA). As the authors maintain, a social ethics approach would not be addressing the individual ethical dilemmas of the engineers with the subcontractor or those of McDonnell Douglas who dismissed the concern. Instead, it would examine the contractual agreements between the two parties and the authority of FAA who learned about the technical issue through test results. A social ethics method would focus on the flow of information in such situations, rather than point fingers at individual “whistle-blowers”.

Devon and Poel identify social ethics as empirical and claim that the social ethics paradigm presupposes that we explore many different alternatives when making decisions about social constructs and weighing the trade-offs of all of them (the so-called reflective equilibrium29). A key idea here is that the structure of the design process in an organisation is not always deliberate but can be an inherited custom. The social ethics paradigm treats such processes as prone to change and improvement, though, which is an important principle to keep in mind when arguing for social change. Devon and Poel talk about two types of ethical responsibility: active and passive. Active responsibility is a conscious proactive ethical behaviour where an action considers consequences and obligations before the action is taken. It is dependent not only on personal character but on the institutional setting. Passive responsibility is a reactive ethical behaviour, it relates to accountability or liability after something undesirable has occurred.

While I agree with Devon and Poel that ‘the fact that people can be held accountable afterwards will give them an additional motive to act in a responsible way beforehand’ (Devon and Poel, 2004, p. 465), I disagree that a person’s accountability depends on whether they had a determining voice in the decision-making process. Ethical restraint shouldn’t be dependent on authority. I strongly believe that a person is accountable for their actions irrespective of whether they had the final say or bore formal responsibility (a CEO should be just as accountable as a receptionist for their ethical or unethical conduct). Otherwise, we will be doing the right thing for the wrong reason30: behave ethically or give the impression

29 reflective equilibrium’ is a method, coined by the American moral philosopher John Rawls in his book A theory of Justice (1971). It draws a comparison between existing principles and their practical application in order to find a solution which works both in theory and in practice. Essentially, it is a reverse-engineering of existing and commonly accepted judgments and specific experiential circumstances, adjusting and altering either of them in order to find an ‘initial situation that both expresses reasonable conditions and yields principles which match our considered judgments duly pruned and adjusted’ (Rawls, 1971, p.18). 30 According to the 17th century German philosopher Immanuel Kant, what is important when judging if an action is morally right, is the motive: we do the right thing not because we have an incentive (like getting praise or money in exchange) but because we are autonomous beings, capable of reason who respect others as ends in themselves, not mere means to our ends (Sandel, 2009, p.61). This reflects the deontological account of morality: we do the right thing because it is our duty to do the right thing, not because we will get something in return.

41 MSc Design Thesis: Jana Voykova of being ethical because this portrays us in a good light, not because our principles dictate it. The authors quote Ulrich Beck on ‘the organised irresponsibility’ in our technological development, where nobody in particular is responsible for negative side-effects, especially if they are ‘unforeseen’. This alludes to the point that Matt Beard and Simon Longstaff make, namely, that a good technology is such that is mindful of the potentially harmful side-effects it may have. That is why it is morally unacceptable for companies to justify a morally dubious design which resulted in harm by saying that they did not know or did not foresee the harmful effects.

Devon and Poel suggest that, during the design process, sometimes choices are made explicitly, other times implicitly. They claim that a lot of decisions are taken following the principle of “this is how things are done” which can incidentally lead to an undesirable result. ‘Many moral problems in design seem to stem not so much from a deliberate immoral decision, but from a range of decisions that in themselves are morally dubious’ (Devon and Poel, 2004, p. 466).

They borrow Vaughan’s term “organisational deviance” to define norms that are seen as unethical outside of an organisation but are regarded as normal and legitimate within an organisation. This is not a result of deliberate rogue practice of an organisation but rather a pattern of implicit decisions. In order to avoid such pitfalls, the authors recommend that ethical criteria like explicitness, inclusiveness and the possibility to revise a decision are taken into account. I agree with Devon and Poel, that the design process doesn’t stop with the release of a certain product on the market.

The way products impact the people who use them should be a leading principle in their redesign and improvement. However, as the authors argue, the impact of design on the society is generally unknown beforehand. Therefore, the people for whom the product is intended should participate actively in the design process and more importantly, should be made aware of its ramifications and potential risks before they get to use it. To put this into the perspective of my research on autonomous vehicles, the potential risks of this technology should be addressed and defined in the public space before the product is released on the mass market, regardless of whether such questions are controversial and may harm the competitive advantage of the companies that develop the technology.

At the centre of the social ethics paradigm is the notion that individual and social ethics don’t exclude each other but that ‘individual without social ethics is powerless’ (Devon and Poel, 2004, p. 468). This means that reforms in malleable social arrangements constrain and shape the morality of individuals, as argued previously and as I will demonstrate in the outcomes of this thesis research. In order to achieve those reforms in social arrangements,

42 MSc Design Thesis: Jana Voykova we need design processes that involve ‘studying many alternatives, that engage many stakeholders and weigh empirical evidence very strongly’ (ibid.).

4.1.1 A critical examination of design ethics

Hypocrisy is the hill on which trust goes to die.

Matt Beard

What some designers call design ethics or ethical design, occasionally misinterprets the ethics part of the expression. Design ethics is sometimes fighting against the very essence of ethics, especially when we try to reconcile ethical design with contemporary business. I think this is an important distinction to make so as to avoid misunderstanding what the role of ethics in design should really be. A parallel that comes to mind is the way we conceive of activities like corporate social responsibility, which quite often boils down to pure greenwashing31 — something that we say we are doing, either because it does well on our image or because it is politically correct, while in reality we are doing the exact opposite. As argued earlier, it is difficult to measure the social responsibility of individuals and corporations but on the other hand, as Laufer posits, corporations have effective tools to exempt themselves from liability, such as lobbying, pushing accountability and blame down the hierarchy and away from the firm (reverse whistle-blowing), or hiding deviance by issuing sustainability reports.

‘As with greenwashing, the defensive strategies employed by firms to protect against entity liability are aimed both inside and outside the organization. Internally, Confusion flows naturally from the complex nature of the corporate form, reliance on decentralized decision making, and the practices of managerial winking. Fronting is accomplished through the representations of retained counsel, compliance officers, ethics officers, and ethics committees. Posturing seeks to convince internal customers, as much as external stakeholders, of the organization’s collective commitment to ethics. Finally, both sets of strategies, interestingly, rely heavily on the advice of a large cottage industry of public relations and reputation management firms… If there is one striking similarity, it is the

31 Greenwashing or bluewashing (washing through the reputation of the United Nations), is a term which denotes the ‘disconnect between a corporation’s public statements of compliance or social responsibility, and its genuine efforts, particularly without external, third party verification and monitoring’ (Laufer, 2003, p. 257). This is understood in the context of both environmental and social practices that a corporation engages in, for example addressing air pollution and climate change, fair labour conditions, money laundering and bribery etc.

43 MSc Design Thesis: Jana Voykova potentially perverse nature of these strategies. Both internal and external strategies have the potential to give an organization the appearance of ethicality and leadership, when no such commitment exists. (Laufer, 2003, pp. 256-257). What this means is the fact that a company issues a sustainability report or appoints an ethics consultant, doesn’t mean that it actually implements changes to their operations in order to behave more responsibly and ethically.

Should someone suspect that Laufer’s observations are rather academic fiction, than hard- evidenced truth, a quick trip to the findings of WEF’s Net-Zero Challenge: Fast-Forward to Decisive Climate Action Report (2020), will prove them wrong. In fact, we don’t need a report to see that we have made insufficient progress towards meeting UN’s Sustainable Development Goals since the Paris Agreement from 2015. Worse, we act as if we just found out about climate change, while in reality, people sounded the alarm already 60-70 years ago. To add insult to injury, we fool ourselves that we will magically stop global warming in the next 10 years, when clearly, our reactive nature, combined with social and political negligence, has prevented us to do that for nearly a century!

‘Despite commitments from individual governments and companies over the past decade, emissions have risen by 1.5% per year. Should this pattern continue, the world is projected to warm by 3°C to 5°C by 2100, with catastrophic effects on human civilization’ (WEF, Net- Zero Challenge: Fast-Forward to Decisive Climate Action Report (2020). Out of the 125 countries who have committed to be carbon-neutral by 2050, none are among the top five emitters and some, like the US are ‘openly denying climate science and backtracking on previous regulations and international commitments…Individuals need to drive climate action in their roles as consumers, voters, leaders and activists’ (ibid.). This is indeed paramount, but can we seriously expect that individuals will do that when consumerism is still at the top of the human values’ pyramid?

Demand for energy and plastics continues to increase, freight demand is expected to triple by 2050 and even though some countries (Nordic countries and the Netherlands, in particular) have truly taken decisive steps towards sustainable production and lifestyle, their efforts are not enough to offset the irresponsibility of the rest of the world. Quite frankly, it’s hypocritical and tragic to think that some companies/countries/ individuals are excused from responsibility when it comes to matters of global importance. WEF projects that these trends are not going to change for the better in the next 10 to 20 years unless the international community (public and private sector and citizens) starts treating the problems at hand as a collective threat, implementing measures in reality instead of on paper.

I am certainly not claiming that what some designers call ethical design is quite the same as greenwashing, although sometimes it does resemble it. Ethical design can be challenging to

44 MSc Design Thesis: Jana Voykova achieve in a business environment, and this might make us less ambitious about the discussion on the topic and the actual outcome we will accomplish.

Just a few weeks ago (March 2020), Smashing Magazine, a German resource for practical advice and information for web designers and developers, published a book called Ethical Design Handbook (2020). The book aims to explain to designers and practitioners working in the digital industry what ethical design is and how it can be applied in business. Without disregarding or disrespecting the efforts of the people who contributed to the realisation of this project, I found some of the statements in the book quite vexing. I agree with the authors that professionals need resources to guide them on how to create more ethical products without sacrificing their business. I also realise that ethical design in tech is an overwhelming field and so they had to narrow down the scope of their book.

However, their saying ‘we had to exclude some areas, like artificial intelligence and the dark web because these fall outside of our area of expertise, and we quite simply don’t have answers to give on questions like: “How do we make ethically designed AI?”, makes me wonder whether they actually understand what ethics means in the context of design ethics. Ethics does not provide straightforward answers, it raises questions which lead to the right solutions. Ethicists and philosophers still cannot find the answers to age-old ethical dilemmas, let alone the technological and business bafflements of the 21st century, designers are in no capacity to do that alone, either.

Having said that, we cannot simply dismiss such questions when addressing ethics in digital design just because they are difficult or because we don’t have the answers to them. Furthermore, ‘ethical design strategy’, I read in the introduction of the book, ‘will not make you rich quick’ (Falbe et al., 2020, xiv). As much as I have taken this out of context, I wonder why it appears to be a problem that ethical design will not make you rich quickly? Is it because of the assertion that, in a free market society, the goals of businesses other than to ‘make as much money for their stockholders as possible’ is a ‘subversive doctrine’ (Friedman, 2002, p. 133)?

I think we have embraced too blindly the short-sighted way respected economists! like Milton Friedman have established the principles that (should) govern business. I am not arguing that businesses should function as charity organisations or that money has nothing to do with economics and entrepreneurship. Quite the contrary. Even so, excusing businesses and governments, or anyone else for that matter, from any form of accountability towards the society or calling corporate social responsibility (if it exists at all) ‘a suicidal impulse’ (Friedman, 1970, p. 178) and proposing that corporations be excused from taxes in exchange for them to demonstrate social responsibility, as Mr. Friedman proposed in his book Capitalism and Freedom (2002), is absurd! This is exactly why we need to talk about

45 MSc Design Thesis: Jana Voykova bringing back ethics into the fields of economics, design, technology and politics and we don’t get to have the luxury to omit the inconvenient, complicated, challenging or painful aspects of the conversation. If we truly want to make a difference, that is.

Also, ethical design is not a business strategy which CEOs and leaders in the digital industry can define and educate their employees on. Codes of business conduct are supposed to do this. Ethics and ethical design are a public endeavour. They start at the collective level and encompass all spheres of public life: political, economic, educational, cultural, entrepreneurial. When we say ‘There will always be a business for those who want to make money fast. Designers from the dark side read other books, not this one, and let us leave it at that’ (Falbe et al., 2020, xvi), are we not saying that it is fine for some businesses to be unethical and it is okay for some designers to succumb to the dark side, but this is not what we are discussing in a handbook on design ethics?!

I can imagine that it is a tough job to write a book on ethical design that takes the business side of the question into account and make the book practically viable at the same time. Nonetheless, ‘if the business people in the organisation succeed with ethical design, then the designers and developers would have a much easier time succeeding with ethical design too’ (Falbe et al., 2020, xix) is a morally ambiguous statement. What happens if the businesspeople fail to succeed with ethical design? Do designers and developers follow suit without any repercussions? As stated earlier, washing our hands with contingencies and corporate hierarchy is not an acceptable solution.

Another thing that worries me about the Ethical Design Handbook is that the authors seem to assume that ethics in design concerns primarily people’s privacy. They describe the principles for good technology which Matt Beard and Simon Longstaff explain in their paper Ethical by Design: Principles for good technology as ‘not very specific and actionable’ (Falbe et al., 2020, p.62) and instead turn to The Privacy by Design framework, developed by Ann Cavoukian whose four principles only shed light on ethical design around privacy. Privacy is an important aspect of it, but ethics is not just about privacy and I would argue that The Ethics Centre’s principles for good technology32 are indeed very actionable and specific, and address ethics in technology comprehensively, something designers should be doing, as well.

Even though the authors of the book lay out some important ideas about how we can design our digital spaces more ethically and fuse ethics into the business models of companies, it

32 The principles for good technology which Matt Beard and Simon Longstaff outline in their paper can be found under Appendix A. The Ethics Centre: Principles for good technology at the end of this thesis. I decided to provide them as an appendix because I wouldn’t do them justice if I just mentioned them as bullet points here without further explanation of what they stand for.

46 MSc Design Thesis: Jana Voykova still gives me the impression that design ethics is seen as a social trend which was stimulated by recent tech scandals, and there is a step-by-step guide on how to make our companies and products more ethical. As if we just discovered that human beings should have morality. For all it’s worth, we are where we are because we have forgotten that ethics shouldn’t be divorced from design, business or any other aspect of life. This allusion of design ethics as a tendency or a business metric is something that I have seen elsewhere in the design space, and it is doing a disservice to the whole point of debating ethics in design.

If we just repeat what is already written in the codes of business conduct and bundle up ethics to fit business, while trimming the underlying problems, what are we actually doing? Ethical design is not a performance indicator which secures a monetary prize if we succeed with it, it’s something we should do as a matter of principle, because it’s the right thing to do — all design should be ethical design.

Designers appeal to taking a digital detox and things like that which are not innately bad recommendations, but this is not the solution we should be looking for. Instead of treating the symptoms, we should be eradicating the disease. Ethical design should override economic outcomes, not try to work around them. As Cennydd Bowles33 points out, ‘relying solely on a business case for ethics is a sure route to failure… A moral argument that hinges entirely on financial consequences is unwittingly agreeing that ethics is subservient to profit; it will leave you powerless to oppose future moral harms that are profitable nevertheless’ (Bowles, 2018, p. 198).

Erik Spiekermann34 argues that after adopting or signing a designers’ ethics code, we will not magically become “do-gooders” the next morning, sending commercial clients away’ and waiting for more worthy projects to find their way to us (Berman, 2008, p. 7). This does sound a bit cynical but I believe what Spiekermann means, is not that people go to work, intentionally wanting to hurt others (although sometimes we do close our eyes before unethical actions), but that we live in a business and social environment that demands from us to be very selective in defending our moral reasoning and that is where we should focus our energy more vigorously. Simply put, certain moral principles35 shouldn’t be overridden by any circumstances whatsoever.

33 Cennydd Bowles is a London-based designer and consultant, whose work focuses on the ethics of emerging technology. 34 Erik Spiekermann is a German graphic designer and typographer, head of Edenspiekermann, a creative agency with offices in Berlin, Amsterdam, Chicago, Los Angeles, San Francisco and Singapore. This quote is taken from the foreword to David Berman’s Do good design, where Spiekermann refers to the 2000 reiteration of Ken Garland’s First things First manifesto from 1964. 35 As already clarified, morality is defined differently by different ethical theories but there are actions which, I think, cannot and shouldn’t be categorised as grey-area, no matter what ethical theory we believe in.

47 MSc Design Thesis: Jana Voykova

4.2 Technology ethics

Meritocracy has long historical roots, but it also has a new face. It proclaims greater equality of opportunity for more people than ever before. We have been encouraged to believe that if we try hard enough, we can make it: that race or class or gender are not, on a fundamental level, significant barriers to success. To release our inner talent, we need to work hard and market ourselves in the right way to achieve success. It is characterised by the sheer extent of its attempts to atomise people as individuals who must compete with each other to succeed, by extending entrepreneurial behaviour into the nooks and crannies of everyday life… It promises opportunity whilst producing social division.

Jo Littler36

As stated in the introduction of this thesis, our deterministic conception of technology today makes for an excellent case to project what kind of social events should prompt designers to demonstrate their social responsibility. In this section, I will expand on my motivations to focus on disruptive technology in my critical review of the design profession and introduce the relevance of autonomous transportation for this thesis. Further on, I will analyse the previous research done with regard to the ethics of autonomous vehicles and discuss this use case in more detail. I will conclude this chapter with a fairly detailed discussion on some theories and ideas from the business ethics field.

In 2016, the founder and executive chairman of the World Economic Forum Klaus Schwab, identified 21st century as The Fourth Industrial Revolution in his book, bearing the same name. Schwab argues, rightly I think, that the fourth industrial revolution is fundamentally different in scope, complexity and scale from its predecessors, even though it was enabled by the digital revolution of the 1960s-2000s. It is ‘the fusion of technologies like quantum computing, gene sequencing and nanotechnology across physical, digital and biological domains’ (Schwab, 2016, p.10), that make the fourth industrial revolution unique. However, unlike the preceding three revolutions in human history, Schwab maintains, the present one is characterised by two primary challenges which limit its potential to be effectively and cohesively realised: 1) ‘the required level of leadership and understanding of the changes underway are low when contrasted with the need to rethink our economic, social and political systems’ and 2) ‘the world lacks a consistent, positive and common narrative that outlines the opportunities and challenges of the fourth industrial revolution’ (Schwab,

36 Jo Littler is Professor of Social Analysis and Cultural Politics at the School of Arts and Social Sciences at City University of London. Her research focuses on society, culture, meritocracy, inequality, consumerism, cultural politics.

48 MSc Design Thesis: Jana Voykova

2016, p. 13). These challenges are not technological and scientific per se, their roots can be traced back to the way we started viewing politics and by extension — economics and civic life, in the post-Cold War period (I will come back to this argument in the following sections of this chapter).

Technology has become so dexterous that this naturally raises the question of whether there will, sooner or later, be a time when it will render humans redundant and make us superfluous. Many people have legitimate concerns about the future of their jobs, but it is much more that is at stake than our job market. Technology is not confined to just one space anymore, digital or physical. In fact, it has become so precariously embedded not only in our daily routines but our psyche, that even a temporary downtime of the technologies we use, makes us extremely anxious. At least in the more urbanised, developed places, there are very few tasks we can conveniently perform without relying on technology. Or so we have come to believe.

Technology-optimists are too quick to reassure people who are more agnostic about the positive effect of technology on the future of work (a phenomenon known as technological unemployment) that, like in previous ages of economic and industrial transition, new jobs will emerge to compensate for the losses in present workplaces. Those new jobs, however, will be primarily in sectors, demanding programming, analytical and engineering skills, which mainly serve the needs of large corporations because progress is quite unevenly distributed between the private and the public sphere.

Technology-optimists forget that not everyone can easily transition and educate themselves on those particular subjects. Moreover, not everyone is willing to do so, and this should not be a choice that is ever taken away from people. ‘Most of technological unemployment studies have been conducted using aggregated data that detailed the extent of job loss by industry and profession. These studies are informative but fail to identify individual-level perceptions about job prospects and the sociodemographic characteristics, job skills and level of human capital of individuals who have experienced job loss or wage loss due to automation’ (Dodel and Mesch, 2020, p. 2).

According to the self-interest principle, unsurprisingly, ‘those in stronger labor market positions report positive views on the advantages of technology for their career development and are less likely to experience job loss and wage loss due to automation’ (ibid.). As Dodel and Mesch argue, there are several reasons for this: 1) if people in stronger labour market positions become unemployed, they are more likely to find a new job relatively easily; and 2) their labour contracts are likely more favourable to their rights as employees. In support of the claim that technological unemployment worries are rather anxieties than evidence-based predictions, researchers like Mokyr and Arntz argue that automation and the resulting

49 MSc Design Thesis: Jana Voykova displacing of jobs is a long process and can sometimes meet resistance from unions and political institutions. Job fluctuations may indeed span over years, but we see less and less resistance towards automation even from public sector institutions because technology is so prevalent that refusing or failing to harness technological change creates a glass ceiling which both individuals and organisations can hardly overcome.

Technology-optimists claim that automation complements certain professions like computer engineering and new occupations are emerging, which compels them to conclude that ‘computerization may not predict the end of human labor and may even provide advantages for employees who are more skilled, adaptable and creative (Morikawa, 2016) (Dodel and Mesch, 2020, p. 4). But what happens to the people who are less skilled and adaptable?

The delusional logic of technology-optimists dismisses these vulnerable classes because 1) they are invisible in the cosy cubicles of tech companies for one group of people and for another group, they are even stronger an incentive to ‘cling to their jobs all the more desperately, leaving them less likely to question the deeper processes at play’ (Rushkoff, 2016, p. 13); and 2) employment opportunities today seem to rest on the fragile meritocratic views37 incubated in Silicon Valley and automatically adopted as a rationale for moral desert in virtually all industries and domains of life.

Schwab rightly argues that ‘talent, more than capital, will represent the critical production factor’ and therefore, ‘scarcity of a skilled workforce rather than the availability of capital is more likely to be the crippling limit to innovation, competitiveness and growth’ (Schwab, 2015, p. 47). Schwab foresaw that this will result in a segregated job market, divided between low-skill/low-pay and high-skill/high-pay segments. This means that the successful transition between jobs will directly depend on the ability of workers to adapt continuously and learn new skills throughout their lifetime. Schwab’s dystopian predictions did not only come true but worsened in recent years38.

37 Meritocracy dictates that success is solely dependent on one’s merits and if somebody fails at something (usually in professional sense), this is because they lack ingenuity and intelligence, not because of luck or circumstance or other external factors. This theory simulates a philosophy of “to each what they deserve” while in reality, it seems to be a theory of “survival of the fittest”. As Jo Littler argues, the term has changed its meaning in the course of history and from denoting ‘an elite group of people who govern and who have been able to arrive at such a position ‘on the basis of individual ability or achievement’ (Littler, 2018, p. 24), it now refers to ‘the ‘open’ system of access to that elite, a gap that is bridged by the image of travelling up the ladder’ (ibid.). 38 According to WEF’s report Jobs of Tomorrow 2020, by 2022, there will be 133 million new job opportunities: 37% of them will be in the care economy, 17% in sales and marketing, 16% in data and AI, 12% in engineering and cloud computing, and 8% in people and culture. The jobs with highest demand will be Artificial Intelligence specialists, medical transcriptionists, data scientists, customer success specialists and full stack engineers while the ones with lower skills will be landfill biogas generation system technicians, social media assistants, wind turbine service technicians, green marketers and growth hackers. As optimistic as this data seems, the threat of

50 MSc Design Thesis: Jana Voykova

While making our lives easier, which in many respects is subjective, technology (or rather the meritocratic beliefs behind it) has simultaneously created a world of winners and losers, the former being the more adaptable, eager to change and re-skill, the latter being unable to harness change and embrace life-long learning. What happens if I want to remain a driver or a teacher and not become a computer scientist or a data analyst? A technology-optimist would probably object to this line of reasoning by saying that, as long as individuals are not willing to keep up with progress, businesses have no obligations (moral or legal) to endorse their behaviour.

But this isn’t about denying innovation. It’s about having the right to choose. Technology should augment humans, not urge us to helplessly compete with it. Besides, why should technology be the ultimate measure of excellence? A geography teacher is just as valuable to the society as a programmer, I would argue even more because a teacher doesn’t simply teach their students facts about the world but helps them shape themselves as individuals while a programmer’s job is an artificially inflated occupation by an industry whose dividends it helps grow. Who knows, maybe 10 years from now, the growth of financial assets will be more dependent on other professions and being a developer might be just as unglamorous as being a teacher is today in some countries.

As Dodel and Mesch point out, job polarisation — increase in the demand of high-skill jobs, decrease in the need for mid-level jobs, and an increase in the share of low-skill jobs, is a consequence of computerisation. As a result, people who lose their mid-level jobs will orient themselves to low-skill jobs and while computerisation won’t destroy a large number of jobs, ‘less qualified workers would be the ones bearing the brunt of the adjustment’ (Dodel and Mesch, 2020, p. 5). While these predictions vary across industries and geographies, job polarisation is a painful outcome of technological progress, which shouldn’t be taken leniently. For this reason, my speculative future, which I will present in my prototype exhibition, will depict namely this labour and class division in an intentionally exaggerated form, seeking to inspire deeper thought on this subject.

The relationship between private corporations which provide the tools to automate jobs, and the public sector which have the regulatory instruments to restrict and control it, is relevant to the analysis of automation and job loss. The former German Minister of Labour and Social Affairs Andrea Nahles asserts39 that multibillion tech companies like Google, Airbnb and Facebook simply don’t need the state and see it rather as an obstruction to their business.

unequal opportunity, job displacement and widening income inequality seem ever more present and the most important question is whether public and private sector will use the tools the digital age is giving them ‘wisely and in the service of workers’, as written in the report. 39 Quote is taken from Deutsche Welle’s documentary Will robots steal our jobs? (2017).

51 MSc Design Thesis: Jana Voykova

More alarmingly, they have the financial power to back up this attitude and conduct their business however they please. In theory, as Yuval Noah Harari40 maintains in his book 21 lessons for the 21st century (2018), ‘government regulation can successfully block new technologies even if they are commercially viable and economically lucrative’ (Harari, 2018, p. 46). Ironically, humans’ imperfect nature, inability to work as fast as machines, emotional vulnerability, among other things, make them a liability for their employers. One potential solution to the job loss issue, discussed both by Schwab and Harari, is the debate in a number of countries to introduce a universal basic income as a precautionary measure to handle the problem.

While such initiatives are not illogical, they are somewhat problematic. This is not so much because they raise an array of new tough questions, but because such mitigation plans only prove our collective perplexity regarding the current technological rise.

Roland Paulsen41 offers an interesting perspective on the economic, political and technological reality we live in. He explains this reality by analysing the way labour management looked around 40 years ago and the way it looks now. Before, Paulsen rightly argues, we focused on providing more favourable work conditions for workers, like giving them more vacation, lowering the pension age, shortening the workday, introducing healthcare and unemployment insurance. Today on the other hand, even though productivity has doubled (what could previously be done by two people can now be done by one), jobs are ever more demanding, and we are continually stretching the work week. This is not sustainable either for people’s quality of life or for the stability of the economy and the environment.

Paulsen sees this as a by-product of the risk politics which many governments have embraced today. According to him, this approach is irrational because, whilst politicians focus on mitigating potential future harms, they miss to address real threats that are solvable by political effort here and now. As he points out, many politicians win over campaigns by promising to take measures against sexual harassment by minority groups, but no political party has come to power because they laid out a plan on how to address climate change, whose devastating impact on society and environment is already a reality.

When it comes to automation and robotisation, Paulsen sees universal basic income as an alternative solution to the current capitalist economic models of many countries. He questions the widely accepted notion that one should always have a job (he calls this arbetstvång or compulsion to work) in order to be able to sustain themselves and envisions

40 Yuval Noah Harari is a professor in history at the Hebrew University in Jerusalem. 41 Roland Paulsen is a Swedish sociologist and Associate professor at Lund university in Sweden.

52 MSc Design Thesis: Jana Voykova universal basic income as an alternative instrument for distributing wealth in society. In Paulsen’s view, this compulsion to work requires from us to always seek to open new jobs while the bigger portion of productivity gains goes to owners of capital, not holders of jobs. Douglas Rushkoff shares Paulsen’s vision in this regard and sees guaranteed minimum income in combination with other changes in the economic system as a possible solution to the fragility of today’s (digital) capitalism. I will expand on this argument in the Business ethics section of this chapter.

Along with the job loss unease, automation and digitisation come with serious data privacy trade-offs which seem to have been widely accepted. Harari rightly argues that data is the most important asset today, whoever owns the data, has the power to control the world in the future. As dreadful as this sounds, it looks like it is indeed true. Recent scandals, like the 2013 revelations by the former CIA agent Edward Snowden on the National Security Agency’s (NSA) mass surveillance of the American and international population, as well as the 2016 presidential elections which led Donald Trump to the White House, using Facebook as a brainwashing tool, unsurprisingly caused the public’s scepticism towards both governments’ and private players’ capacity to defend their interests. It’s like choosing between two evils, trying to pick the smaller one. In this context, it is worth reflecting on the increasing power of large tech companies to shape people’s perception of democracy and liberal values.

As the British journalist Carole Cadwalladr writes regarding the Facebook scandal, people are still suffering vertigo and confusion about the story because of the ability of those involved in it to spin the facts however it suits them. Their power to determine citizens’ political views and actions, and authorities’ failure to restrict it, is a clear sign of democracy’s fallout.

The lack of trust in governments is not a new phenomenon, though. It has existed more or less throughout history, and for good reasons. Although a good majority of the population in less democratic societies would rather outsource their data protection to private corporations than give its safe keeping to their governments, I would say that using laws to regulate and restrict what private companies have the freedom to do, is currently our best bet if we want to avoid the apocalyptic future we already, to an extent, are living in.

Still, we should ask ourselves whether regulation is the right answer. To quote Rachel Botsman42, what are we actually regulating? Are we regulating human behaviour or

42 Rachel Botsman is a Trust Fellow at Oxford University’s Saïd Business School. She holds a BA in Fine Arts from the University of Oxford and is the author of two highly acclaimed books — What’s mine is yours (2010) and Who can you trust (2016). This is a quote from the Ethics & Tech: Panel discussion, part of the Purpose Conference.

53 MSc Design Thesis: Jana Voykova corporate power, or the limits of morality? The story which tech giants tell us about them making our lives easier and serving our needs by giving us platforms to stream our private lives at all times, and submit to our most primitive instincts, might in fact be, as Douglas Rushkoff puts it ‘a camouflage for the real activity occurring on these networks: the hoarding of data about all of us’ (Rushkoff, 2019, p. 60).

Persuasive technology is a real strategy, which, by means of subtle manipulation, convinces people that they need more applications, new gadgets, smarter homes, smarter cities, and new technology is the best way to tackle current social and political challenges. In this fairytale, designers and technologists are merely the weapons that fire the bullets. As Rushkoff writes, today we design so that ‘people change their attitudes to match their behaviours’, not the other way around (Rushkoff, 2019, p.63). Sadly, we eagerly give our consent in fear that we might be left out of progress. The end result is: we obediently let ourselves to be treated as lab mice. Our rarely asking “why” and questioning the motivations behind modern innovations, makes it far easier for disturbing and abnormal behaviour to become normalised.

While I find it outrageous to declare that there is no difference between humans and robots, as the Osaka university professor Hiroshi Ishiguro43 audaciously believes, I agree with Harari that the information on artificial intelligence and machine learning that is fed to the general public doesn’t come from scientific knowledge on genetic engineering but rather from science fiction pop culture. As a consequence, as Harari argues, we fear a battle between humans and robots whereas the real peril might lie in a battle between ‘a small superhuman elite empowered by algorithms, and a vast underclass of disempowered Homo sapiens’ (Harari, 2019, p. 286). This scenario is not a precedent in history, of course. But while in the 1940s, societies resembled Orwell’s 1984, today’s look more like Huxley’s Brave New World, which is even worse.

The polished marketing campaigns which promote innovations are too often deprived of adequate plans regarding how to address the social turmoil they result in. This is not what marketing does, one might object. Obviously, nobody lives in a happy jar (well, some people do but not in the sense I mean it). Most likely, the majority of people do know that economic gains are a key component of the motivation that drives technological and scientific progress today. However, the assumed innovation inevitability and our total disengagement from reflecting critically on what happens around us, might just result in the average citizen being caught in the crosshairs if we take a wrong turn in our quest for perfection (sadly, this is already a reality in a lot of economically and politically vulnerable places).

43 In the BBC documentary Hyper Evolution: Rise Of The Robots (2018), Ishiguro stated that he doesn’t see any difference between humans and robots.

54 MSc Design Thesis: Jana Voykova

There is a lot of discussion around fake news and truth when technology is concerned. I agree with Harari, however, that history doesn’t seem to point to any era of truth. It is just that, today, the means of distorting and manipulating the truth have become so sophisticated that we can barely tell the difference between a truth and a lie.

In addition to all of the above, the ingenious nature of technology in 2020, puts in jeopardy all efforts of the international community to bridge the economic and social divide within and between societies. Klaus Schwab writes that ‘the convergence of the physical, digital and biological worlds that is at the heart of the fourth industrial revolution offers significant opportunities for the world to achieve huge gains in resource use and efficiency’ (Schwab, 2015, p.63). He identifies the Internet of Things44 (IoT), circular economy and the democratisation of information which comes from digitised assets as drivers of this positive change. But as he himself mentions, there are still parts of the world which haven’t seen the benefits of the second industrial revolution, let alone the fourth.

According to Credit Suisse Global Wealth Report (2019), there are enormous differences in average wealth across countries and regions. The richest 10% of wealth holders own 82% of the global wealth and the top 1% alone own 45%. Nations with wealth per adult above USD 100,000 reside in North America, Western Europe and the richer Asia-Pacific and Middle Eastern countries. North America and Europe together account for 57% of total household wealth but only 17% of the world adult population lives there. Wealth inequality decreased in China and other emerging markets like Brazil, Argentina, Czech Republic, Indonesia after the financial crisis in 2008 but rose in all the wealth-rich countries (where technological innovations are accelerated with utmost determination).

Among the key reasons for this, specified in the report, is that financial assets (equity shares) have outpaced non-financial assets and this trend is projected to continue in the next five years. In their Global Risks Report (2020), the World Economic Forum listed digital fragmentation — unequal access to next generation technologies, lack of a unified technology governance framework and cyber insecurity, as one of the major threats before the international community in 2020. ‘Powerful economic, demographic and technological forces are shaping a new balance of power. The result is an unsettled geopolitical landscape — one in which states are increasingly viewing opportunities and challenges through a unilateral lens’ (World Economic Forum, 2020, Global Risk Report). Notably, asymmetries

44 Internet of Things (IoT) is ‘a network of connected devices with unique identifiers in the form of an IP address which have embedded technologies or are equipped with technologies that enable them to sense, gather data and communicate about the environment in which they reside and/or themselves’ (Basnayaka et al., 2020, p. 2). The key components of IoT are data, connectivity, intelligence, communication, action, ecosystem and things. It is estimated that by 2050, up to 100 billion things will be connected electronically by internet to provide services like smart mobility, healthcare, digital government.

55 MSc Design Thesis: Jana Voykova in the distribution of power and resources are among the main drivers for the lack of social cohesion and the vulnerable geopolitical arrangement globally.

All these problems and the moral quandaries that stem from them cannot be addressed just in the corporate technology labs or the chambers of parliament. As Whitworth and Adnan explain, ‘to try and build a community as an engineer builds a house is a levels error, choosing the wrong level for the job’. Engineers alone cannot build communities; many people do that together’ (Whitworth and Adnan, 2013, p. 58). In the preface to The social design of technical systems: Building technologies for communities (2013), Whitworth rightly points out that ‘people in a community are ethical not because they foolishly ignore their self-interest but because they intuitively see community gains beyond themselves’…. ‘Ethics teaches humans what is right and we need to apply social requirements to technology design as well as work design, because technologies now mediate social interactions’ and ‘the core socio-technical principle is still “just because you can, doesn’t mean you should” (Whitworth, 2013, p. 53).

4.3 Autonomous vehicles (machine) ethics

Machines that try to infer the motives of people, that try to second-guess their actions, are apt to be unsettling at best, and in the worst case, dangerous.

Donald Norman

Following the imminent logic of technology, we have come to a widespread prophecy that autonomous transportation is the future. The enormous social, economic and political difficulties that came with industrialisation and intensified with the expansion of consumerism and urbanisation, posed a number of serious challenges before the so-called “developed” world, which naturally impact developing nations, as well. The World Economic Forum has estimated that by 2030, 60% of the world’s population will be living in cities and the limiting factor for urban mobility in the future will be land, not affordability. They have also assessed that between 2019 and 2030, the number of delivery vehicles (as a result of expanding e-commerce), will increase by 36% and these vehicles will emit an additional 6 million tonnes of CO2. As a consequence, the average commute time for each passenger will increase by up to 21% and air pollution will aggravate the effects on the climate45.

45 Data is taken from WEF’s report The Future of the Last-Mile Ecosystem from January 2020.

56 MSc Design Thesis: Jana Voykova

The shrinking space in big cities, the climate fluctuations and the alleged concerns about the increasing number of traffic fatalities led to the ambition among influential tech and automotive companies to design a vehicle which would drive completely on its own thus eliminating the so-called human error factor, minimising the number of cars in large cities and boosting human productivity.

The concept of a car that is driving autonomously, as might be expected, is not novel at all. Humans have had aspirations to design a vehicle that drives itself long before the scientific breakthroughs of the digital age. Developments in radio technology and aviation in the early 1900s laid the foundation for automated46 vehicles. In this sense, the history of self-driving cars originated in the military rather than the automotive industry.

One of the first attempts at designing a self-driving vehicle (which was rather a driverless47 one) was done in the 1920s in Manhattan, US when the inventor Francis Houdina, who worked as an electrical engineer in the US Army, demonstrated how a car “drove itself”. Houdina was controlling the car remotely with a radio-transmitter from another car at a close distance and even though this technique seems rather rudimentary today, for that time it was revolutionary. In the 1930s and late 1950s, concepts of self-driving cars appeared in magazines, books and on screen which might be the reason why the general population thought of the idea as utopian and fictional rather than something achievable in reality.

In the mid-1930s, the American oil and auto industries, together with urban planners, policy makers and architects started developing the concept of the highways of the future. This was the first idea of automated traffic as opposed to remotely controlled one. As innovative as the idea was, it was mainly seen as propaganda to restore trust in capitalism and technological progress after the Great Depression (1929-1939). On 14 February 1958, the first “automatically guided automobile” completed a test route of one mile at General Motors’ Technical Center in Michigan. The engineers had fitted the front area of a 1958 Chevrolet with two electronic sensors that followed a wire laid in the road which adjusted the steering wheel accordingly.

In the late 1950s, the futuristic idea of an automated vehicle was abandoned and replaced by a more feasible product — the autopilot, developed by Chrysler and marketed as a “supergadget”. The self-conscious vehicle was left to the fantasy writers. Isaac Asimov’s short

46 Automated is a more precise term than autonomous because the word autonomous implies “self- determination within the scope of a moral law” (Maurer et al., 2015, p. 10) and conscience which machines (still) don’t have. 47 Although self-driving and driverless are used synonymously, there is a difference between the two: self-driving is synonymous with autonomous, driving on its own while driverless is just without a driver, it can be controlled remotely but this doesn’t mean the machine is autonomous.

57 MSc Design Thesis: Jana Voykova story Sally from 1953 portrayed vehicles in an anthropomorphic way, having feelings and behaving as humans. In the years after, the idea was picked up by other writers and used in dystopian scenarios with the hostile machine that wanted to kill people.

In 1977, a research team from the Mechanical Engineering Laboratory in Tsukuba, Japan presented a vehicle that could record and process pictures of guide rails on the road via two cameras. Around the same time, the first driver-assistance systems emerged as a result of the increased use of electronics in vehicle technology. The first on-board computer was designed for the BMW 7-series and in 1978 the ABS was introduced. In the 1980s and 1990s the research in autonomous vehicles was taken up more seriously by academia and industry. In 1984 a team from the University of the Federal Armed Forces in Munich (Germany) conceptualised the first vehicle that used dynamical models for visual autonomous guidance: a 5-ton Mercedes 508D which managed to drive autonomously with the help of cameras at a speed of up to 96km/h.

In 1994 the same team, together with Mercedes Benz developed two twin S-Class robot vehicles which drove autonomously more than 1000km on three-lane highways around Paris at a speed of up to 130km/h. This marked a paradigm shift in the autonomous vehicle project: computer vision was ensured by cameras and steering, throttle and brakes were controlled through a computer code. It wasn’t until the early 2000s though, when breakthroughs in artificial intelligence (AI), machine learning (ML), deep learning (DL) and automation48 enabled the design of a more sophisticated autonomous vehicle.

In 2004 the Defense Advanced Research Projects Agency (DARPA) in the US ran a competition for autonomous vehicles, called The Grand Challenge. The purpose was to encourage American ‘ingenuity to accelerate the development of autonomous vehicle technologies that could be applied to military requirements’49. They offered a $1 million

48 There is no universally accepted definition of AI, ML, and DL. AI is a general term that describes a machine’s ability to mimic cognitive functions that humans possess, such as problem-solving, learning and thinking. It is essentially a program code which guides the machine to behave in certain ways in certain situations. There is narrow AI, general AI and super AI. Narrow AI means that the machine is very good at performing just a single task, say playing chess. It pulls data from a pre-programmed pool of data and whenever one attempts to steer its “attention” to another task, it fails to perform. General AI is a machine which exhibits human intelligence. It is conscious, driven by self-awareness. General AI does not exist yet. It is expected to function like a human, able to reason, solve problems, be creative, make judgements, completely on its own. Super AI supersedes human intelligence in all aspects of the human brain. Techno-optimists envision a mergence between humans and super AI in the aforementioned singularity. ML and DL are subcategories of AI. ML is algorithms that are fed large amounts of data sets which they learn from and make informed decisions based on the learned insights. ML leads to the automation of various tasks. DL is a subfield of ML. DL uses a multi-layered neural network and like ML, requires vast amounts of data. DL is what enabled the development of self-driving cars, personal assistants, chatbots and what will potentially enable to creation of general and super AI. 49 The quote is taken from DARPA’s website.

58 MSc Design Thesis: Jana Voykova prize but during the first round, on March 13, 2004, no one of the 15 competitors managed to complete the route from Barstow, CA, to Primm, NV. A year later, the autonomous vehicle Stanley of the Stanford Racing Team, led by Sebastian Thrun, professor in computer science and electrical engineering at Stanford University, managed to conquer the 132 miles on a desert terrain and claimed the prize of $2 million. Today, tech giants, small start-ups and automotive companies are zealously competing to become the first to successfully design and launch a fully automated car50 to the average consumer.

While the evolution of the concept behind autonomous vehicles is fascinating even for sceptics like me, the potential impact of the technology on various aspects of society is much more important to bring up and analyse.

As often emphasised in industry and academia, the car is not just a means of transportation. ‘Steering a car is the only area where the love of power and imagination still has free rein, observed the semiologist Roland Barthes in 1963 [3, p. 241]. The sociologist Henri Lefebvre also argued that the automobile was the last refuge of chance and risk in an increasingly controlled and managed society’ (Maurer et al., 2015, p. 41). This is one of the reasons why autonomous vehicles are sometimes met with criticism and indignation rather than excitement and awe, especially in the car enthusiast communities.

As Christian Gerdes points out, ‘the very act of driving conjures a range of strong and very human emotions. Whether it is the feeling of freedom that the mobility of the car provides, the frustration of being stuck in traffic, the panic when realising a potential collision looms or the joy of an open road with a favourite song on the radio, driving is a human experience. With automated vehicles, however, that experience changes — both for passengers in the automated car and other road users who must walk or drive alongside it as part of the social experience of traffic. The car ceases to be simply an extension of its human driver and becomes an agent in its own right, navigating through the highways and rules of human society’ (Maurer et al., 2015, p. 37).

Naturally, autonomous transportation is going to transform much more than just the human-vehicle relationship. The development of the technology and its appearance on the mass market is connected with monumental changes in terms of labour force, infrastructure, policy making, insurance, security, and rethinking of morality and ethics. There is an extensive academic and industry research done on the social impact of the technology in all

50 A fully automated (autonomous) vehicle, or Level 5 of automation, means that the car can drive itself in all conditions and under all circumstances without the need for human intervention. This definition is beyond the BASt (German Federal Highway Research Institute) autonomy scale as it defines the vehicle and not the degree of automation’ (Maurer et al., 2015, p. 35)

59 MSc Design Thesis: Jana Voykova those areas. However, little attention is given to a more critical outlook on the motivations behind the technology and the vast improvements we can do to road infrastructure, better training, stricter regulation, more stringent penalties for disobeying traffic rules, and to using technology to enhance driver-assistance systems instead of replacing drivers. Cars don’t need to be autonomous and conscious so that they can help us become better drivers or use physical spaces more rationally.

Most of these alternative measures I point to, might be perceived as rather naïve since we have failed to make substantial progress in a lot of the above-mentioned areas so far. But assuming and reinforcing the assertion that automation in all aspects of life is going to save humanity of its imperfect human nature, is a dangerous and morally questionable proclamation. Just as equally dangerous tactic would be to contest the technology by arguing that machines are not 100% error-free (yet) and glitches in the system will inevitably occur. Picking up on machine vulnerability, however, is hardly an objective interpretation of the question at hand. We cannot expect that a machine will perform flawlessly 100% of the time, especially when it is thrown in an unpredictable, non-conformist human environment. Neither will humans and this debate shouldn’t be about humans against machines.

It is hardly deniable that accidents and fatalities will be minimised substantially (provided the technology has reached the required level of sophistication and the necessary changes to road infrastructure, legislation and security standards are in place). The benefits of autonomous vehicles are indeed compelling: optimisation of space and traffic, millions of saved lives, access to more convenient transportation for marginalised groups of the society (e.g. disabled people, elderly members).

Just as compelling are the risks, like the threat of compromising the system for the fulfilment of nefarious intentions and the unresolvable moral and ethical implications of the technology. As stated in WEF’s Global Risk Report, ‘using “security-by-design” principles to integrate cybersecurity features into new products is still secondary to getting products quickly out into the market’ (WEF, 2020, Global Risk Report 2020). Security vulnerabilities can have devastating consequences for the economy and society. ‘It is estimated that a total shutdown of the internet would result in a daily GDP loss of 1.9% in a high-connectivity country and 0.4% in a low-connectivity country’ (ibid.). Not to mention that hacking connected technologies can be used as a military weapon with disastrous ramifications.

We can turn to regulation and policy making in this regard, too. Unfortunately, the fact that laws are ratified and adopted, doesn’t mean that they cannot be disobeyed and twisted, and this is one of the areas where we should channel our financial and intellectual efforts more eagerly. According to the Global status report of road safety for 2018, issued by the World Health Organisation (WHO), 123 countries have road traffic laws in line with best practice

60 MSc Design Thesis: Jana Voykova for one or more key factors for road safety. As of 2016, the number of road traffic accidents resulting in death, was 1.35 million and continued to climb. Unsurprisingly, the mortality rate due to road accidents is three times higher in low-and middle-income countries51 than it is in high-income countries, both across and within continents.

Improvements in road infrastructure and stricter penalties led to significant reduction of road fatalities in countries like Colombia, Brazil and South Korea. The enforcing of legislation, which remains a major challenge in most countries, on key risk factors including speed, drink-driving, helmet and seat-belt use, are cited as critical components of an integrated strategy to prevent road traffic deaths. Driver distraction (mobile phone use), driving under the influence of substances, speeding, unsafe roads are among the primary causes of road accidents identified in the report. Vehicle safety is crucial to the prevention of crashes, however, not all new and used cars are required to be equipped with internationally recognised safety standards.

As of 2018, 40 high-income countries apply seven to eight of the eight priority UN vehicle safety standards, 11 countries apply two to six, while 124 countries apply one or none. Clearly, there is still a lot more to be done on a political level in tackling the increasing traffic death rate and resolving urbanisation threats.

Let us suppose, however, that we have exhausted our options on the above points and there are no alternatives for coping with the issue than introducing autonomous transportation. Unless autonomous vehicles are given for free or are cheap enough for the average consumer to afford, which, at least in the beginning I highly doubt will be the case, only a small elite in high-income countries will be able to buy them. This is a weak argument, of course. Although traditional auto manufacturers are marketing the technology to the masses, other companies (like Embark in the US) focus primarily on replacing public transportation and large fleets like delivery trucks, taxis and car sharing services, not the vehicles of individual buyers. Additionally, with time, even high-end technologies like autonomous vehicles will become affordable for the masses.

Even so, why aren’t we trying to solve the root causes for truck driver shortages, driver distraction and urbanisation? We don’t need autonomous cars to address driver shortage, for example; the profession just needs to become more appealing to more people. This can be achieved by employing more drivers to cover shorter routes and switch more often, for instance. Cars don’t need to be autonomous in order to reduce pollution, they can just as well

51 Low-income countries are countries which have lower than $1,025 income per capita, they are also known as developing countries or emerging markets. The majority of low-income countries are located in Africa. Middle- income countries have a per capita gross national income (GNI) between $1,026 and $12,3755. The majority of middle-income countries are located in Africa and Asia.

61 MSc Design Thesis: Jana Voykova be electric but electric cars need to be affordable for the average driver to buy and more importantly, the state should regulate and oversee the replacement of combustion-engine cars with electric ones.

Pollution and the number of cars will be dramatically reduced if means of subsistence are equally distributed between big cities and smaller towns and the infrastructure allows people to bike more or walk (especially in developing countries). These are measures which are apparently working when there is a strong political will in place. Instead of using technology to enhance humans mentally and improve people’s working and living conditions, we simply use it as a replacement of humans or at the very least, as a tool to commend our weaknesses.

Instead of judging, reprimanding and penalising irresponsible behaviour more effectively, we encourage it and work around the problem by relieving drivers from the “stressful” experience of driving. Driving as such is not stressful for a lot of people who do love to drive. Quite the opposite. What is stressful about driving is our inability to educate people to be responsible and channel state budgets into truly productive solutions. Going around the problem, as argued earlier, is putting a lipstick on a pig.

Introducing autonomous vehicles and connected devices will not stop either urbanisation or the constant pile-up of cement in cities, because people won’t live in their cars, be they autonomous or not, nor will it eliminate the depletion of rural areas because they cannot provide subsistence.

62 MSc Design Thesis: Jana Voykova

Figure 4. In big cities, sights like these are the rule, not the exception. I don’t even have to go to the other end of the city to find them, I just need to look out my window. From what I see and hear, this is not the reality only in Bulgaria. Instead of trying to get people out of the cities, we are doing our best to cram more and more people in. Strong political will is what we should be demanding, not more housing and office buildings. We don’t need connected and smart cities; we need smart citizens and governments.

Autonomous vehicles are part of the innovative urban planning known as smart cities of the future but little or not at all is discussed about why we continue assuming that cities are a form of next level evolution and we treat cities as the default sustaining environment by focusing nearly all of our economic growth, production, enhanced social life, and political effort in them. Little is discussed about the widening gap within and between countries as a result of technological progress and innovation concentrated in urbanised territories.

63 MSc Design Thesis: Jana Voykova

As Francesco Gonella52 argues, ‘the meaning of the term “Smart City” is poorly defined… The element that seems to be shared by all of the various approaches to city smartness is the application of Information and Communication Technologies (ICT) to basic infrastructural services (Washburn et al., 2010)’ (Gonella, 2019, p. 1).

According to the European Union, which seems to be active in the development of the concept, smart cities are ‘a place where traditional networks and services are made more efficient with the use of digital and telecommunication technologies for the benefit of its inhabitants and business’ (ibid.). Gonella rightly points out that the current narratives of smart cities are deeply flawed because they are detached from the systemic nature of the urban metabolism, and ‘real problems that are the root of the “un-smartness” of a city, like poverty, inequality, unemployment, illiteracy, corruption, lack of sanitary and educational structures, are just not addressed whatsoever’ (Gonella, 2019, p.2).

Furthermore, these narratives assume that citizens’ wellbeing and happiness is solely dependent on the interconnectedness of their cities and access to smart mobility and innovation. One example which Gonella gives is London. London is reported to be the smartest city in the world by a number of agencies like the IESE Business School. Its smartness is attributed to its cultural vibrancy, the number of smart business makers (whatever that means as Gonella puts it) and ICT interconnectedness.

However, according to official government data from the London Datastore, 2.4 million people live in poverty and in 2018, London overtook New York City in the murder rate. Probably what is most troubling in the current smart city narratives is not so much the deranged descriptions of technology companies about their vision for the future urban landscape but the readiness with which governments and high-rank organisations such as UN and EU accept these visions and call them a “holistic approach” to tackling social and environmental crises.

‘The European Union Smart Cities Information System recently published a report entitled “The making of a smart city: best practices across Europe”. In its 256 pages, the words “mobility” and “business” appear 114 and 67 times, respectively. The words “children,” “poverty,” “violence,” “disability,” “inequality,” “welfare,” and “homeless” never appear. This clarifies not only the meaning of “smartness” in the EU policy, but also how policy-makers consider their commitment to solve real and serious problems in the cities as something that can be managed separately — as if the two realities of a fully ICT-interconnected and a socially devastated city refer to two different, parallel worlds’ (ibid.).

52 Francesco Gonella is Professor of Physics, Department of Molecular Sciences and Nanosystems at the Ca’ Foscari University of Venice.

64 MSc Design Thesis: Jana Voykova

‘The architect Kevin Rogan refers to the SC narrative as “making it look like they’re doing something they’re not, like making a neighborhood instead of building a huge behavioral data farm’ (Gonella, 2019, p.3). Governments, as Gonella points out, should see people as citizens, not consumers.

This deterministic philosophy on technology says very little about the end goal we are seeking to achieve and where we are headed. The future is disruptive technology. Why? Who decides that? For whom is it the future? Why aren’t we considering more rational measures to address the challenges of modern cities? Disappointingly, the public, political, and increasingly academic dialogue has been depleted of moral judgment and we focus so intensely on the future that we become completely detached from the present. We embrace change and innovation as a one-way street to prosperity and greatness but miss to see the thorns standing right in front of us.

Technology does have the potential to equip us with tools to combat biodiversity loss and global warming, autonomous vehicles do have the potential to reduce road accidents and save millions of lives, bio engineering does have the potential to advance medicine and healthcare, but the ultimate prize for the success of these projects seems to be competitive advantage and growing cashflow for single actors, not global cooperation against threats before humanity and nature. We talk about equality, human rights and social justice but these things look very differently in different parts of the world. Is the right to have access to healthcare and education equal to the right to be able to buy a sports car on the human rights scale?

Different moral philosophies address these questions differently. Equality, as Will Kymlicka53 argues, is ‘an abstract idea because it can be interpreted in various ways’ (Kymlicka, 2002, p. 4): it can be measured against opportunity, liberty, and wealth whereas the more idealistic account on equality would be to treat people as equals. Concentrating opportunity, knowledge and power in just one area thus intensifying social division and indoctrinating the ideology that good quality of life depends solely on access to new technology, is anything but treating people as equals. That is why I will question the concept of the smart city in my speculative future by referring to it as “smart” ironically, with the purpose to make people more suspicious of the validity of the concept.

53 Will Kymlicka is a Canadian political philosopher and Professor in philosophy at Queen’s University at Kingston.

65 MSc Design Thesis: Jana Voykova

As far as the ethical implications of autonomous transportation go: the so-called trolley problem54, this is one of the major reasons one should be against such technologies entering real life. I claim this not because I doubt that we will find a good enough practical solution to these ethical dilemmas but because we shouldn’t have to in the first place. We can and probably will figure out what is best from a liability and legal standpoint in the case of an accident where there is no human driver to take responsibility. However, none of these solutions will be morally acceptable because no ethical theory has been accepted as the ultimate criterion to what is morally right and just. Whether we will prefer to give priority to the passenger or to a group of pedestrians in a hypothetical collision, none of the choices we’ll make will be morally right. Theorists and researchers urge us to teach machines to be ethical in ethical dilemma situations with autonomous cars, but they forget that even humans shouldn’t have to make such choices deliberately. I will park this argument for now and look at how the social and ethical impact of autonomous vehicles has been approached in other research projects.

‘When designing any advanced technology that is directly interacting with humans’, Krzysztof Korbaczynski writes in his master thesis, ‘it is impossible to avoid the discussion about new ethical challenges posed by new technological solutions…A quote from Noam Chomsky stating that “as soon as questions of will or decision or reason or choice of action arise, human science is at a loss” (Chomsky 1978) could describe the possible scenario of a future in which technology has highly outpaced the ethics. In order to avoid the above described situation, and protect ourselves from such course of events, debate on ethics of intelligent technology is an absolute necessity’ (Korbaczynski, 2018, p. 34).

Without underestimating the potential benefits of the technology, Korbaczynski analyses rationally the development of automated55 vehicles and the information that is available in the public space regarding them. As he argues, ‘the forces that act upon the automotive industry are a part of a larger dynamic of change that unfolds both within technologies e.g., expectation related to digital lifestyle, and between technology and society’ (Korbaczynski, 2018, p. 11). The fact that in recent years, the automotive industry has shifted from a more industrial production to a more service-oriented one, Korbaczynski maintains, calls for a

54 The term trolley problem is a thought experiment, coined in 1976 by the English philosopher Philippa Foot which depicts the following hypothetical scenario: there’s a trolley car rushing down a track towards five workers. An observer of this situation is standing next to a switch. If he/she pulls the switch, it will divert the trolley to another track where there’s only one worker. In both cases, the trolley will kill someone, and the observer needs to decide whom to sacrifice. The trolley problem illustrates a common ethical dilemma when we need to choose what is the right thing to do. 55 Korbaczynski prefers to use the term automated and not autonomous for reasons clarified earlier. Automated implies a technology which reduces human intervention but does not replace humans. The reason for this, he explains, is that the technology is still under development and there is no autonomous vehicle designed yet (able to make decisions based on its own consciousness).

66 MSc Design Thesis: Jana Voykova closer look into the different stakeholders involved in this process and the social and economic impact of this shift, not only to the industry, but to ordinary citizens, as well.

‘Artificial Intelligence (AI) imposing itself as a common denominator for a number of automated systems offers promises of further improvements in productivity, efficiency, and reliability, is also suggested to become “one of the primary means to automate and aid interaction with information. In many cases, AI by implementing Machine Learning, becomes a proxy of information processing between humans and the rest of the world. One should therefore discuss which tasks may and should be taken over for human benefit when speaking of automated driving scenario’ (Korbaczynski, 2018, p. 14).

Analogously to my line of argumentation, Korbaczynski challenges the deterministic nature of technology which resides in various public spheres: ‘this view claims that the technology is the source of an independent force and power that can constitute the development of the society…. There is a possibility that a strong technocentric view may be one of the main obstacles to the development of a “people-first” design – the idea that by improving technology and changing physical space around people, their decisions and choices may be altered and reshaped to some extent. Accordingly, the technocentric view that is often visible among automotive industry, includes a certain belief that the technology has the ability to address not only strictly technological problems, but also address issues of society, economy or politics’ (Korbaczynski, 2018, p. 16).

Korbaczynski acknowledges the resistance to change that is common in social groups and organisations. This is a situation where the members of a society or an organisation are reluctant to embrace a technology or a new way of working because they don’t recognise its value immediately. It is often the case that ordinary people are unable to understand how a new technology functions and so the producers of automated vehicles need to take into account the mental models which stand behind human-machine interactions, and design their vehicles in a way where communication between the machine and the human flows seamlessly. This is connected with the engineering and interaction design side of the technology. As Korbaczynski points out though, the public discourse is too focused on the human factor of driving. It’s true that the human factor is the cause of a great percentage of the road accidents. As he rightly claims, we shouldn’t forget that faulty technology or products also cause accidents.

An interesting example Korbaczynski provides here (which is relevant not because it proves that technology doesn’t always perform well but because it reveals how companies often handle such situations), is the 1970s “Pinto case” when a faulty design of Ford Pinto caused the car engine to become extremely flammable even when the car was hit at low speed. Instead of reclaiming the production and fixing the issue, Ford did a cost-benefit estimation

67 MSc Design Thesis: Jana Voykova according to which it would cost them less to cover the damages and eventual death coverage instead of discontinuing the sales and fixing the problem.

‘Could that mean that legal and ethical principles have a significant gap, a grey zone, that could be used for the benefit of either side? The assumption made by Ford, that their actions were legal and equally ethical has caused a backlash and their project that was meant to save money, became even more costly for them’ (Korbaczynski, 2018, p. 37). This naturally raises the question whether we should worry about situations where an action is perfectly right from a legal but questionable from an ethical point of view.

The more troubling ethical query with autonomous vehicles is that we have to pre-program them to be able to make ethical decisions. The circumstances under which autonomous vehicles would really have to make an ethical decision might be rare, but we shouldn’t ignore the fact that artificial intelligence is not human intelligence and a machine needs to “know” how to act before an actual necessity to act arises (in a human driver scenario, the action is attributed to impulse and instinct, a rapid decision under the influence of a particular setting rather than a deliberate, predetermined choice).

As Korbaczynski puts it, ‘the ethics of technology must be developed simultaneously with the technology itself’ (Korbaczynski, 2018, p. 38). Unfortunately, the International Society of Automotive Engineers (SAE) is mostly comprised of engineers, ‘which entails that the knowledge and concepts SAE is producing express vision of an engineering character, than, e.g., a societal, cultural, or lifestyle vision’ (Korbaczynski, 2018, p.18).

Edmond Awad56 posits similar concerns in his analysis of the ethics of autonomous vehicles. ‘While ethicists, legal scholars, and moral philosophers’, he writes, ‘are capable of diagnosing moral hazards and identifying violations of laws and norms, they are not used to framing their expectations in a programmable way. On the other hand, engineers are not always capable of communicating the behavior of their systems using the same language that the ethicists and legal theorists use and understand’ (Awad, 2017, p.18). Awad identifies this gap between engineers and philosophers as one of the reasons for the lack of a comprehensive moral code for machines. In search for the optimal answer to the moral bafflements with autonomous vehicles, he co-designed the Moral Machine which made the backbone of his master thesis in Media Arts and Sciences at MIT in 2017.

56 Edmond Awad is a lecturer at the University of Exeter Business School (Department of Economics) and the Institute for Data Science and Artificial Intelligence and a former Postdoctoral Associate at MIT Media Lab (2017-2019).

68 MSc Design Thesis: Jana Voykova

The Moral Machine is an online platform which gives people the opportunity to judge 13 ethical dilemma scenarios. In some of the scenarios, the control of the car is in the hands of a human driver, in others the car is autonomous, and in a third part there is a shared control between a human and a machine (both of them being able to change the car’s trajectory). The purpose of the experiment is to gauge the public’s opinion on what ethical decisions we should embed in autonomous vehicles and examine how blame is to be distributed between humans and machines in an eventual accident with an autonomous vehicle.

The experiment was presented to 973 participants (US residents) and in each scenario, the choice was either to kill five and save one or kill one and save five (either pedestrians or passengers, animals and babies included) by taking a deliberate action or inaction (either to swerve or stay on the original trajectory of the car). In the different situations, important factors played a role, like the social status of the people involved (criminal, businessman, homeless person, athlete), their age (elderly versus young), health condition (fit versus fat), sex (female or male) and their abiding by the law (crossing on green light versus crossing on red light). The Moral Machine was further developed and is now available to people across continents who want to try and solve moral dilemmas with autonomous cars.

While I appreciate that this project gives ordinary citizens the chance to participate in the decision-making process of how to program autonomous vehicles and could also potentially serve the industry in the development of the technology, we are looking for an answer to a question which shouldn’t exist to begin with.

Focusing on liability as a solution for ethical dilemmas with autonomous vehicles is the right answer to the wrong question. Who gets to decide whether a homeless person is less deserving to live than an executive director? Or whether the life of three teenagers is more valuable than an elderly man’s or a dog’s (it’s interesting how we humans always think of our life as more valuable than an animal’s)? We would normally declare such decisions as appalling in any other context but when it comes to advancing technology, we seem to be willing to make a compromise. Contrary to what Awad, and other people, seem to argue, our taking liability and blame as a measure for achieving reflective equilibrium on ethical quandaries, only reinforces the worldview that profit and power supersede morality. This is an ideology which designers should condemn and impede, not help promote.

Awad’s pragmatic approach to the ethical limitations of autonomous vehicles was also adopted by Bryan Casey57 in his paper Amoral machines, or how roboticists can learn to stop worrying and love the law (2017). Casey argues, that ethical dilemmas, in the context

57 Bryan Casey is an affiliate scholar at the Stanford Machine Learning Group and legal fellow at the Center for Automotive Research at Stanford.

69 MSc Design Thesis: Jana Voykova of autonomous technologies, like self-driving vehicles, are illegitimate. He claims that ‘profit- maximising firms designing autonomous decision-making systems will be ‘less concerned with esoteric questions of right and wrong than with concrete questions of predictive legal liability’ (Casey, 2017, p. 1).

Casey maintains that, as relevant as such dilemmas might be to the world of robotics, in a practical situation where there is harm caused, it all boils down to liability. And liability, in his view, is governed by legal, not moral laws. According to Casey, it will be upon ordinary citizens, not engineers at tech and automotive giants to design a legal system which ensures that a “bad” robot has as much reason to behave ethically as a good one.

What he means is that legal principles are much more straightforward to understand and follow than ethical ones, especially since we haven’t agreed on what moral doctrine holds the ultimate answers to our moral perplexities. To put this into perspective, Casey provides two examples of what liability could look like in an autonomous vehicle accident. He imagines a situation where six teenagers are playing a ball game: one of them is the designated thrower and the rest of the players have to catch the ball. Suppose the ball gets out of the playground and onto the street with passing vehicles. The teenagers naturally go after the ball. However, a vehicle with minimised visibility is headed towards the teenagers and cannot stop in time to prevent collision. The vehicle faces the choice to either go straight on and strike the teenagers or swerve and overturn the vehicle with the one passenger inside.

If the law holds companies “strictly liable” for any damages their vehicles might cause, the manufacturer should pay the cost for the resulting harm. As per this legislation, the vehicle should sacrifice the one and save the many because this would be the economically optimal solution for the company, paying for one victim is cheaper than paying for more than one person.

In a parallel scenario, Casey assumes that jurisdiction recognises the legal defence of “contributory negligence”. This means that the company or manufacturer of the autonomous vehicle will not be held accountable if the accident was caused by the victim’s own negligence. Under these circumstances, the economically optimal solution would be the opposite — the vehicle should strike the five because they misjudged the situation going out on the street to fetch the ball and so the manufacturer will not be held accountable or charged any monetary compensation for the negligence of the victims.

Casey himself calls this kind of calculations dismal from an ethical standpoint, but this is apparently the best we can do in a situation where ethical dilemmas enter reality. I see where this reasoning comes from, however, Casey’s reflections are still deeply disquieting. First, the distinction between morality and liability is apparent, and regulation might be the only thing

70 MSc Design Thesis: Jana Voykova that will protect ordinary citizens in incidents involving technologies like autonomous vehicles; but we shouldn’t forget that we cannot divorce law from moral philosophy.

Why should we choose to ‘minimize liability’ and not ‘maximize morality’ to use Casey’s words? Just because it is the easy way out of this conundrum which we ourselves have created? The situations which Casey’s thought experiments and Awad’s Moral Machine are trying to resolve, give us a God’s view which we shouldn’t seek to have. Moreover, in trolley dilemma scenarios, we become remarkably selfish rather than rational when they concern us personally. We wouldn’t be as supportive of a liability precedence over an ethical consideration if the decision resulted in our own harm or the harm of a loved one. ‘Bonnefon et al. [8] also concluded that in theory, people support autonomous vehicles programmed to save others, but they would not want to drive or ride in one, thus posing a "social dilemma" for AVs’ (Ahmed, 2018, p. 63).

Second, Casey’s focus on liability and jurisdiction may be reasonable and easier to execute from a practical standpoint than hitting our heads against the wall trying to decide whom to sacrifice. However, if we let economic imperatives take precedence, we will simply be defending the financial interests of companies, enforcing cost-benefit analysis where human lives are affected (we are already doing this in a lot of situations, but this doesn’t mean it’s the right thing to do).

Third, it misses the whole point with ethical worries. Blame is not the primary ethical concern which puts us at unease here. After the harm is done, it should matter little whose fault it was or how much we should pay to compensate for somebody’s life. That is unless we want to continue living in a society where economics surpasses everything and literally everything has a price, as Michael Sandel holds in his book What money can’t buy: The moral limits of markets (2012).

Liability is not why ethical dilemmas with disruptive technologies like autonomous vehicles, are contestable. It is the question of whether we should strive for them at all. Ironically, Casey views incentivizing the worst of our robots to behave like the best of our philosophers as a democratic decision. How is his recommendation to follow the clear-cut path of economics rather than the equivocal principles of morality, a sign of democracy? Democracy for whom? The people with money? The ones in power? As idealistic as this position is, we shouldn’t twist the facts to make our decisions and actions more acceptable. As Korbaczynski argues, ‘while creating innovation and introducing new technologies is required for any industry, it does not translate to the positive impact among the society…. The car industry, as other industries is a business relying on profits – those with the best financial resources will receive the best possible product and service. To simplify, the higher the price of a car, the safer the product will be (Korbaczynski, 2018, p. 68).

71 MSc Design Thesis: Jana Voykova

Korbaczynski’s analysis of Google’s self-driving car project (which he compares with Daimler’s Smart Vision EQ), is worth mentioning here. Google started their self-driving division in 2009 which consequently spun off into a separate company — Waymo in 2016. Google’s interest and emergence on the automotive market is an event worth a closer scrutiny, because, among other things, they are traditionally a technological company, not an automotive one. Korbaczynski analyses Google’s self-driving project by looking into Waymo's Safety Report, issued in 2017.

Two aspects of his analysis are particularly intriguing and will serve for a further argumentation that I will introduce in the next section of this chapter: 1) in 2013, Google acquired the GPS navigation application Waze, which functions by sourcing information from drivers’ behaviour: ‘By analyzing users' behavior on a daily basis, Waze is currently able to connect drivers and riders based on pick-up and drop-off locations, current traffic pattern and expected commute time, giving Waymo which cooperates with Waze, crucial information about driving patterns and the behaviour of drivers during their regular trips’ (Korbaczynski, 2018, p. 55); 2)

‘It should be noted that the Waymo report provides far more technical details than documents provided by Daimler. In that sense it is much more difficult to extract the overall mobility vision presented by Waymo from the technological narrative’ (Korbaczynski, 2018, p. 56). Interestingly, Waymo’s report doesn’t include a discussion on the AI technology. Instead, they have used a direct comparison between the technology and the human brain or referred to the project as “building a safer driver for everyone”.

As Korbaczynski remarks, ‘Waymo chooses to anthropomorphise their vehicle to avoid possible debate on e.g., the AI ethics of vehicle automation’ (Korbaczynski, 2018, p. 61). I agree with Korbaczynski that this might be a strategic move because of the tense competition in the autonomous vehicles contest. On the other hand, though, Douglas Rushkoff is right that keeping a secret how something works in technology, is not an advantage, ‘it’s a security leak waiting to happen’ (Rushkoff, 2016, p. 195). ‘Nevertheless, by leaving out the discussion about core features of AI software that enables automated driving, Waymo chooses not to explain the functioning of a product which may have further consequences on their users’ (Korbaczynski, 2018, p. 61). I will analyse the anticipated motivations behind this business model in the following section.

72 MSc Design Thesis: Jana Voykova

4.4 Business ethics

Prices may be low but the costs are high. It was never really about efficiency anyway; industrialization was about restoring the power of those at the top by minimizing the value and price of human laborers…Of course industrialism wasn’t sold to us as the disempowerment of workers, but as the triumph of technology.

Douglas Rushkoff

Business ethics has a direct impact on the way we perceive ethics in technology and deserves special attention in a deliberation about the social and ethical implications of autonomous vehicles. While my outline of business ethics is neither sufficient nor complete, and some people might find it irrelevant for this thesis, I believe there are some interesting ideas from the field that could make the discussion of ethics in design and technology more constructive. Business ethics is a term that is usually obscured by ambiguity and is often perceived as an oxymoron58. According to Andrew Crane and Dirk Matten, business ethics can be largely seen as ‘starting where law ends’ (Crane and Matten, 2010, p. 546). The definition they provide in their book Business ethics (2010), reads:

‘Business ethics is the study of business situations, activities, and decisions where issues of right and wrong are addressed’ (Crane and Matten, 2010, p. 5).

It should be emphasised that right and wrong in this context mean morally right and wrong, not commercially, financially or strategically right and wrong. Crane and Matten accurately point out that business ethics is not only a concern of corporations but also governments, non-profit organisations, charities, and international organisations. They argue, and I agree, that there is a close connection between business ethics and law: ‘the law is essentially an institutionalisation or codification of ethics into specific social rules, regulations, and proscriptions’ (Crane and Matten, 2010, p. 5). However, law and business ethics are not equivalent.

As the authors write, ‘many morally contestable issues, whether in business or elsewhere, are not explicitly covered by the law. For example, just as there is no law preventing you from being unfaithful to your girlfriend or boyfriend (although this is perceived by many to be unethical), so there is no law in many countries preventing business from testing their

58 Oxymoron is a rhetorical figure which combines incongruous or contradictory terms.

73 MSc Design Thesis: Jana Voykova products on animals, selling landmines to oppressive regimes, or preventing their employees from joining a union’ (Crane and Matten, 2010, p. 7).

Conversely, law doesn’t always reflect moral judgment. For instance, the law dictates that we should drive either on the left or the right side of the road. Nonetheless, this regulates traffic but is not a moral or ethical decision as such. Crane and Matten rightly claim that business ethics is about the confluence of ethics and law, which is often related to questions posed as equivocal, or in other words, “grey area” questions. ‘What this suggests is that there simply may not be a definitive right answer to many business ethics problems’ (ibid.).

This is one of the reasons why, as Crane and Matten illustrate, topics like corporate social responsibility, lobbying, globalisation, and sustainability which are part of business ethics, are quite challenging to research and analyse, and this adds to the complexity of the term. They offer a prolific discussion on the power relations and mutual dependencies between the private (corporations) and the public (government institutions) sector and raise important questions about the meaning of the reciprocal power of companies and governments to influence each other through various practices (e.g. lobbying, donations, privatisation, bribery). As the authors point out, ‘the lack of a “global government” makes the “policing” of multinational corporations increasingly problematic’ (Crane and Matten, 2010, p. 492). ‘The increasingly political role taken up by corporations, the involvement of private actors in the regulation of business ethics, the weakening of the state in protecting our social, political and civil rights, etc.’ (ibid.), pose enormous challenges before business ethics.

The monopolies that both private companies and governments exercise on aspects of the society such as economy, legal system, labour market, education, are different in scale and type in different countries and are determined by the political regime and economic stability of the country in question. Governments, Crane and Matten posit, are defenders of public interests, however, they are also entities with interests of their own. Governments’ success is defined, to a great extent, on whether they manage to create a favourable economic situation for their electorate. This makes governments dependent on business. At the same time, governments sometimes compete with business in certain industries, whose needs may be better served by the resources of private actors, for example healthcare and the military. The sensitive relationship between public and private institutions leads to ambiguity about their accountability and legitimacy in a wider social context. This relationship and its blurry lines of authority become even more convoluted when globalisation and the business operations of companies across borders enter the discussion.

The advantage that businesses have over governments is most evident within research and development of new technologies (autonomous vehicles being one of them). The lack of expertise and resources on governments’ part is one of the factors that puts them in a

74 MSc Design Thesis: Jana Voykova subordinate position before private companies, and this directly reflects on areas that are traditionally conceived of as governments’ prerogative, such as the economy, labour force, education, scientific research. The dynamics of this dependency are embedded in the current economic and political system in the developed (Western) countries, namely capitalism. As some people define socialism as tyranny while others remember it as a time of social stability, so too whether capitalism is a good or a bad thing depends on whom one’s asking. Elizabeth Anderson59 identifies modern workplaces as private governments and compares the way companies function to communist dictatorships.

‘The economic system of the modern workplace is communist, because the government — that is, the establishment — owns all the assets, and the top of the establishment hierarchy designs the production plan, which subordinates execute’ (Anderson, 2017, p. 61). In this sense, corporations are private governments because 1) employees don’t own the production of their labour; 2) they don’t participate in the decision-making process; 3) the power today’s employers have in the professional sphere transcends the boundaries of employees’ private lives.

Richard Wolff60 makes similar analogy in his book Understanding Marxism (2019). Like Anderson, Wolff assumes an anti-capitalist position in an attempt to explain Marx’s ideas. ‘At the productive core of capitalism, in the relationship there between employer and employee, the latter produces surpluses appropriated by the former. In this exploitation, Marx locates a key obstacle that prevents capitalism from achieving its promised effects of liberty, equality, brotherhood and democracy…In capitalism, most workers are trapped into being either a wage-earner producing surplus for an employer or else a wage-earner serving an employer and living off the distribution of some other productive worker’s surplus. Freedom requires changing the system because otherwise you are forever trapped in it’ (Wolff, 2019, pp.30-31).

Confronting this parallel, as some people might get tempted to do, by referring to our freedom in a free market society to leave a disgraceful relationship, is obscene. As Andersson argues, to say that the individual freedom to exit a relationship means that authority doesn’t exist in it is ‘like saying that Mussolini was not a dictator, because Italians could emigrate’ (Anderson, 2017, p. 76). As a general rule, some people are in a more favourable position in the free market society than others, that is the whole point of capitalism.

59 Elizabeth Anderson is an American philosopher with background in political philosophy, economics and ethics. 60 Richard D. Wolff is Professor of Economics Emeritus at University of Massachusetts and a visiting Professor in the Graduate Program in International Affairs of the New School University in New York.

75 MSc Design Thesis: Jana Voykova

Whether one will be in a privileged position is greatly determined by, as under a communist rule, if one is supporting the system or opposing it. It all comes down to which side of the fence one is standing on. A closer look at the observations made by academics on capitalism are worthwhile in a thesis about the ethics of new technology because they provide an untraditional angle from which to approach this subject and lift the veil on some of the principles that stand behind it.

Michael Sandel argues that, in the past decades, economics has been disconnected from philosophy and we have drifted from ‘having a market economy to being a market society’ (Sandel, 2012, p. 23). The difference, as Sandel points out, is that a market economy is a tool for organising productive activity while a market society is a way of life in which everything is up for sale and markets penetrate areas of life where they don’t belong. In his book What money can’t buy: The moral limits of markets (2012), professor Sandel provides simple yet striking examples from public life, which we normally take for granted: paying off someone to stand in ticket lines for you, paying a fee for the opportunity to kill a wild animal, paying doctors an additional fee in order to have them at one’s disposal at all times.

The above examples are seemingly trivial and normal, and one might suggest that we shouldn’t worry about such predicaments all that much. I agree with Sandel, though, that we should worry or at the very minimum question these “services” for two reasons: 1) what happens with the people who cannot afford them?; 2) are we not degrading certain human values by treating human virtues as commodities? In other words, we should worry on the grounds of encouraging inequality and corrupting civic virtues which shouldn’t be governed by markets. Sandel explains our reluctance to engage in moral discussions within markets with the fact that markets don’t judge if some preferences are admirable or not.

‘If someone is willing to pay for sex or kidney, and a consenting adult is willing to sell, the only question the economist asks is “How much?” (Sandel, 2012, p. 29). However, as he eloquently puts it, ‘our embrace of markets has exacted a heavy price: it has drained public discourse of moral and civic energy, and contributed to the technocratic, managerial politics that afflicts many societies today’ (Sandel, 2012, p. 30).

It is not impossible to sell one’s child (in child pornography, for example), but this is still morally denounced. As might be expected, this happens much more often in poverty-stricken families than affluent ones. As Sandel writes, objections to a more stringent control over markets can be made on two grounds: libertarian and utilitarian. The former sees strict regulations of the market as an infringement on our individual freedom: by this logic, we shouldn’t be appalled at parents trading their children in child pornography as long as they (the parents) consent to it (exercise their free will) and another person is willing to pay for the “service”. But this sounds ludicrous!

76 MSc Design Thesis: Jana Voykova

The second principle against restoring morals within markets is connected with maximising social utility. We shouldn’t worry about a person selling their vote for the right amount of money as long as both seller and buyer are willing to make the transaction. In the end, both of them end up better-off and the utilitarian account of morality will be upheld. In both cases, though, I hope we can all agree that the “services” being traded cause our outrage and disapproval namely despite and precisely because of the parties’ consent to participate.

The first objection in such situations — the inequality conviction, is due to the fact that, the selling party’s consent may be coerced: poverty, drug addiction or another contingency which determines the choice. Here, I will bring back the surrogacy example which I presented in the introduction of my thesis. Surrogacy is a widely accepted practice in developing countries, where women are paid to bear a pregnancy, often by a Western couple who can afford it (the price the couple pays in developing countries is naturally substantially lower than what they would pay in their developed countries). But such exchanges have serious ethical, emotional and legal implications.

Giving a sterile family a chance to have children by means of surrogacy is a noble deed, indeed. But what happens when emotional bonds present themselves and the surrogate mother declines to give up her child (such situations are not hypothetical, they have happened in reality)? Are her instincts higher than the childless couple’s? Is the contract between the two parties legally enforceable either way? Is it morally permissible to use surrogacy as an economic incentive for women in poor countries and families?

The second objection — the corruption conviction, (provided there is no monetary or emotional dependency and the person is truly exercising their free will), goes to show that by putting a monetary value on our vote, body, children, relationships, behaviour, we compromise the way we intrinsically conceive of certain human values that shouldn’t be up for sale.

Another example which Sandel outlines in his book to illustrate why markets shouldn’t be allowed to roam in areas of life traditionally governed by non-market values, is the conversion between fines and fees. Apparently, in Israel a lot of parents showed up late to pick up their kids from day-care centres which resulted in teachers having to stay after work hours until the parents came. In order to tackle the problem, the day-care centres introduced a fine for late pickups. Instead of decreasing the number of parents who came late, this strategy had the opposite effect. It happened because the parents started viewing this as a fee for a service rather than a fine for intolerable behaviour.

77 MSc Design Thesis: Jana Voykova

When we incentivise something that we should be doing on principle without getting paid for it, we erode its value. Other such examples are offsetting carbon footprint by paying a fee for travelling by bus or plane or outsourcing (selling) a country’s carbon footprint to another country (other common practices today). Michael Sandel argues, legitimately so, that economists mistakenly assume that markets don’t taint the goods they exchange, and their profession doesn’t deal with moral questions as ‘their job is to explain people’s behaviour, not judge it’ (Sandel, 2012, p. 85). Recently, however, economists have started to change the way they view their subject and instead of thinking of economics as the study of allocation of material goods, they started viewing it as a ‘science of human behaviour’ (Sandel, 2012, p. 86).

This idea, Sandel maintains, is nonetheless promoted by both academic and vocational economists like Gary Becker, an economist at the University of Chicago. Shockingly, in Becker’s view, people use the same approach, namely cost and benefit analysis when making life-changing decisions like whom to marry, as they do with trivial choices like what brand of coffee to buy. Our putting a price tag on civic, environmental, social, and cultural behaviours, raises the uncomfortable question of whether there is anything at all which money cannot buy. As Michael Sandel argues, the original proponents of economics, like John Steward Mill, Adam Smith and Karl Marx never understood their subject as disconnected from moral and political philosophy. In recent decades, however, economists have forgotten this important dependency, and as a result, we have corrupted aspects of life which we should respect and protect on principle.

Michael Sandel calls this ideology market triumphalism. It appears that in recent years, it took even more crooked dimensions and morphed into the economic model that drives many technological innovators today. Harvard Business School professor emerita Shoshana Zuboff61 calls this system surveillance capitalism in her book The Age of Surveillance Capitalism (2019). Zuboff defines surveillance capitalism as ‘“a rogue mutation of capitalism marked by concentrations of wealth, knowledge, and power unprecedented in human history’ and ‘as significant a threat to human nature in the twenty-first century as industrial capitalism was to the natural world in the nineteenth and twentieth’ (Zuboff, 2019, p.9).

‘Industrial capitalism transformed nature’s raw materials into commodities, and surveillance capitalism lays its claims to the stuff of human nature for a new commodity invention. Now it is human nature that is scraped, torn, and taken for another century’s market project’

61 Shoshana Zuboff is a former faculty associate at the Berkman Klein Center for Internet and Society at Harvard Law School. She received her PhD in social psychology from Harvard University and her BA in philosophy from the University of Chicago. Zuboff received the Best Paper Award from the International Conference on Information Systems for her article “Big Other” from 2015. She is a contributor to the Frankfurter Allgemeine Zeitung and a columnist for BusinessWeek.com and Fast Company.

78 MSc Design Thesis: Jana Voykova

(Zuboff, 2019, p. 180). Human experience and personality are, as professor Zuboff argues, the raw material translated into behavioural surplus by the ubiquitous computing of machine intelligence for the economic profits of the proponents of the digital age. Smart homes, smart cities and smart machines are the digital apparatus (which Zuboff calls Big Other), that surveils, nudges, coaxes, obfuscates and herds human behaviour towards predictive outcomes, pretending to be catering to the needs of people, by improving their productivity and lifestyle.

The monumental research that Zuboff undertook and demonstrated in her book, reveals a regrettably unpopular explanation of the economic imperatives that drive technological progress today, and suggests how little ordinary citizens understand what is hidden behind those services, products and needs which are allegedly making life more convenient and us — more productive in the ever more demanding reality that we are living. Zuboff notes, and I agree, that personal information in this context, is the least of our concerns.

To quote Douglas Rushkoff, ‘privacy is the red herring…As anyone working with big data knows, the content of our phone calls and e-mails means nothing in comparison with the metadata around it. What time you make a phone call, its duration, the location from which you initiated it, the places you went while you talked, and so on, all mean much more to the computers attempting to understand who you are and what you are about to do next’ (Rushkoff, 2016, pp. 79-80).

According to professor Zuboff, surveillance capitalism’s birth took place in the early 2000s when Google launched their phenomenal search engine and were facing bankruptcy shortly after because providing a free service without taking anything from their customers proved to be an anchor dragging their business down. It didn’t take Google long to discover that altruism will not make profit and they needed to change their business strategy so as to justify the invested capital in the company and by extension — make their investors happy. They learned that the so-called digital exhaust, the residual data from our searches was in fact, the most valuable information they could collect (this is the metadata which contains information about micro emotions and psychology, often not even visible to oneself). The results were clear, the company maximised its profit in the next four years by a margin of 16, 690%62!

As time progressed, this business and financial model quickly replicated and became the driving force of all big tech giants in Silicon Valley (local and international companies naturally took after because as shown by Google, this was the path to definitive growth and

62 According to Google’s Annual Report for 2004, annual revenue went up from $19, 108 in 2000 to $3, 189, 223 in 2004. Shareholder equity went up from $27,234 to $2, 929, 056.

79 MSc Design Thesis: Jana Voykova revenue). Our outreach to regulation of these processes is completely mistaken. To quote professor Zuboff, debating about data ownership is like negotiating a four-hour child labour, it legitimises the extraction of data while we should really ask ourselves whether this data should exist in the first place.

As pointed out earlier, regulation might be our best bet in a situation of unimpeded private corporate power, however, the utopian ideal that governments would somehow protect us from this extraction of human behaviour and experience, is unrealistic. As evidenced earlier in this thesis, governments are just as interested in hoarding this data towards solidifying their authority and they have historically proven their pursuit of such ends. Crane and Matten rightly argue that ‘regulation is no longer the solitary prerogative of the government: it can be delegated to other parties’ (Crane and Matten, 2010, p. 494). This means that regulation might also be treated as a commodity to be traded and outsourced to the highest bidder.

Shoshana Zuboff draws a compelling parallel between surveillance capitalism in the West and the Chinese authoritarian regime in the East. People in the developed, “democratic” part of the world are quick to criticise the totalitarian rule of the Chinese government for introducing the so-called “social credit”63 system which is indeed a hideous social experiment. ‘As distinct as our politics and cultures’ (meaning Western democracy and Chinese authoritarian regime), ‘may be or have been, the emerging evidence of the Chinese social credit initiatives broadcasts the logic of surveillance capitalism and the instrumentarian power that it produces…In that sense, perhaps the most shocking element of the story is not the Chinese government’s agenda, but how similar it is to the path technology is taking elsewhere’ (Zuboff, 2019, pp. 729-730).

Totalitarianism and instrumentarianism, however, are two distinct philosophies, as Zuboff maintains. Some people mistakenly assume that surveillance capitalism is an Orwellian reality. ‘Although instrumentarianism and totalitarianism are distinct species, they each yearn toward totality, though in profoundly different ways. Totalitarianism seeks totality as a political condition and relies on violence to clear its path. Instrumentarianism seeks totality as a condition of market dominance, and it relies on its control over the division of learning

63 In 2014, the Chinese government started a project whose purpose was allegedly to ensure trust between the government and the Chinese citizens by implementing a system of scoring citizens on all levels of their life, from their personal relationships to their bank records and consumer habits. The social credit system is based on the ubiquitous surveillance capacity of today’s technologies which allows the government to track each citizen in a matter of seconds. Citizens are scored on “good” and “bad” behaviour and their rankings are sealed in a record which they cannot access or question. There is still no universal credit system implemented in the country but a network or local prototypes. Corporations will also be scored by a unified social credit code while citizens will be given an ID number.

80 MSc Design Thesis: Jana Voykova in society, enabled and enforced by Big Other, to clear its path. The result is the application of instrumentarian power to societal optimization for the sake of market objectives: a utopia of certainty’ (Zuboff, 2019, p. 739). In this wonderland of technological excellence and innovation, accessible to everyone, the digital absorbs the physical and blends with it under the principles of what Zuboff calls ‘economies of scale, scope and action’ (Zuboff, 2019, p. 522).

The rendition of data and behaviour happens on a global scale (attracting millions of users) and encompasses varieties of data (lifestyle choices, intellectual preferences, political beliefs etc.) in order to culminate into a modified (nudged) behaviour which happens in the real world. This is what Shoshana Zuboff defines as the translation of clickthrough to a footfall: the predictability of a desired behaviour in the digital realm to the predicted desired behaviour in the physical world.

Among the examples she provides are the popular augmented-reality game Pokémon Go, produced at Google, and Google Street View (a technology featured in Google Maps and Google Earth) which was initially a project supported by the CIA at a company called Keyhole which Google acquired in 2004. Pokémon Go was the instrument that predicted, herded and collected factual data about the intentions and behaviour of the players in a way that left them completely baffled about where the digital (fictional) ends and the physical (reality) begins.

Services like Pokémon Go and Google Maps seem to be fulfilling Google’s mission statement to ‘organise the world’s information and make it universally accessible and useful’ in a way that is, as professor Zuboff writes, ‘about us but not for us’ (Zuboff, 2019, p. 26). Google Nest, part of Google’s project about the so-called smart home, is another vivid example illustrated in Zuboff’s research.

While ‘making life easier’ by serving as our home security system and digital assistant (much like Amazon’s Alexa and Microsoft’s Cortana), Google Nest bypasses our awareness about the backstage operations that are taking place inside the product. The “mistake” which Google did by not disclosing to their customers that there was an inbuilt microphone in the system, testifies, yet again, to the asymmetries of knowledge, equality and power embedded in surveillance capitalism.

In 2019, Facebook CEO Mark Zuckerberg ambiguously announced at the annual Facebook developer conference that the future is private. What he meant by this was that Facebook was going to make sure their platform took greater care in their users’ privacy. What kind of privacy is there on a social media platform, though? It’s like claiming that cars have nothing to do with traffic. As Shoshana Zuboff puts it, privacy is not private, it is a collective

81 MSc Design Thesis: Jana Voykova challenge, it is a political issue. It cannot be entrusted to the same technology companies who are responsible for compromising people’s privacy in the first place.

To bring our use case — autonomous vehicles, into the perspective of this analysis, and return to Korbaczynski’s observations about Google’s self-driving project, autonomous vehicles, as long as they require vast amounts of data and sensors to “see” everything around them so as to function as expected, can be seen as a vital cog in the surveillance capitalism machine and a precious stone in the future smart (connected) city. The IT sector is disrupting the automotive industry, but automotive companies cannot simply stand on the side-lines and watch while their business is being torn to shreds. Either they join the lucrative race towards innovation, or they descend into oblivion. They followed the rabbit hole. When Google decided to embark on the self-driving car project in 2009, through its subsidiary Alphabet64, Google harnessed the laws of disruptive innovation65.

They found (or maybe fabricated?) an unarticulated customer need (autonomous transportation) and spun off the new endeavour in a new company (Waymo) so as not to harm their original business (well, it’s very hard to say what Google’s original business is since they are present in nearly every industry imaginable), all the while disrupting traditional automotive companies that were still catering to the evident needs of their existing customer base. Google’s long-term vision and business ingenuity combined with their lavish resources enabled them to successfully challenge a market they were not traditionally a part of. Why? To maximise their profit and more importantly, to expand their global outreach.

Of course, whether it was namely Google who disrupted the automotive industry is of little consequence now but I wouldn’t call it a coincidence that even though the idea of an autonomous car has a long history in the automotive industry, it wasn’t automotive curiosity but technological breakthroughs in AI and machine learning that enabled the realisation of the project 20 years ago, and Google did start investing in the concept after Stanford

64 Alphabet is Google’s parent company founded in 2015. As Avery Hartmans and Mary Meisenzahl write in an article published on Business Insider, the new structure of Google is quite confusing (maybe intentionally so). Alphabet is a collection of Google’s side projects, like the autonomous driving project Waymo, the healthcare research company Verily, the smart urban planning division Sidewalk Labs and the internet and TV service provider Fiber. Apparently, Google is on a conquest spree in all markets, industries and domains of civic life and for better or worse, they are not the only company with these ambitions. 65 Harvard Business School professor in economics Clayton Christensen made a breakthrough both in the academic and the business world with his book The Innovator’s Dilemma (1997). In it, he explained the reasons why successful companies often suffer the attack of emerging businesses, not being able to harness change because they are focusing their resources in satisfying existing customer needs while emerging markets undercut their business from below, foreseeing future needs.

82 MSc Design Thesis: Jana Voykova

University won DARPA’s Grand Challenge, and attracted Sebastian Thrun to join Google as the leader of their self-driving project (Google X) in 2007.

In an article from 2016, Sebastian Thrun and Eric Schmidt66 confidently summoned us to ‘stop freaking out about artificial intelligence’. They explained: ‘When we first worked on the AI behind self-driving cars, most experts were convinced they would never be safe enough for public roads. But the Google Self-Driving Car team had a crucial insight that differentiates AI from the way people learn. When driving, people mostly learn from their own mistakes. But they rarely learn from the mistakes of others. People collectively make the same mistakes over and over again. As a result, hundreds of thousands of people die worldwide every year in traffic collisions. AI evolves differently. When one of the self-driving cars makes an error, all of the self-driving cars learn from it.’67.

Venture capitalists and technologists boast around with their prodigy which is going to save lives and ‘free us from menial, repetitive, and mindless work’ as if they are innocent altruists ready to sacrifice huge financial investments in the name of the greater good. What repetitive, mindless work is that anyway? Are we all suddenly going to become ambitious Einsteins pursuing prominent careers in information technology, reading Nietzsche while being driven to work by our autonomous cars? The challenges which autonomous vehicles seek to resolve are political, not technological.

Further on, Thrun and Schmidt write: ‘For us, ultimately the hypothetical, long-term concerns are far outweighed by our excitement for the endless possibilities’. What possibilities are we rushing to and why? When the answers to these questions are “We don’t know, anything is possible” and “Because we can”, there is really not much room left for a reasonable debate on morals, is there? As Shoshana Zuboff emphasises, the essence of Thrun’s and Schmidt’s article is that ‘machines are not individuals and we should be more like machines’ (Zuboff, 2019, p. 767).

Donald Norman pleads in his book The design of future things (2007) that we need robotic systems which help humans, not automate and replace them. Unfortunately, it seems that cognitive scientists and their old-fashioned ways of thinking are not welcome in technology companies.

66 Eric Schmidt joined Google in 2001 in the capacity of CEO and served the post until 2011. He was a member of the Board of directors of Google’s parent company Alphabet which he stepped down from in 2019. 67 The quote is taken from Thrun’s and Schmidt’s article Let’s Stop Freaking Out About Artificial Intelligence published on the online publication Fortune in 2016.

83 MSc Design Thesis: Jana Voykova

In a TED talk from 2010 Sebastian Thrun pointed out that we barely talk about the 1.3 million fatal accidents which happen globally every year as a way to sell us the necessity of autonomous vehicles, cleverly appealing to our emotions. He didn’t mention other unpopular topics of conversation like the fact that, as of 2019, 785 million people remain without basic drinking water and 821 million were undernourished in 2017, the majority of which in Sub-Saharan Africa and Southern Asia68. Nor did he mention that widespread corruption in developing countries is one of the main reasons preventing them from getting “developed”, a fact that the “developed” countries seem to be, at the very least, ignoring.

Thrun slipped and said that robots don’t text while driving or get distracted as we, humans do. This apparently means that the only way for humans to stop multitasking while driving is to introduce machines, to basically outsmart us and live our lives for us. As professor Harari says, artificial intelligence has an immense potential in both positive and negative direction, however, it is difficult to predict which direction we will let it take. One of the problems with AI, as Harari points out, is that it provides the real possibility to translate economic inequality into biological inequality. The fact that today, philosophical questions are moving from the hypothetical world into reality and becoming practical questions of engineering (like autonomous transportation and bioengineering) requires that philosophers and ethicists also move along with them from the philosophy to the computer science departments69. Regrettably, these two camps are still on opposing sides.

Not surprisingly, another innovative project, which Google (more specifically Sidewalk Labs, a subsidiary of the company) embarked on, is the future smart city. As previously noted, autonomous vehicles are a key component of this project. In 2017, Sidewalk Labs submitted a proposal to Toronto Waterfront70 to realise the smart city project in a joint partnership.

Despite the futuristic visions about the city of Toronto, the vast collection of data from sensors placed throughout the city and the lack of transparency about the relationship between private and public institutions, sparked heated public debate. In return for its investment, Sidewalk Labs wanted a share of property taxes, development fees and increased value of city land that would normally go to the city, as cited in a 2019 article published by BBC.

68 The numbers are taken from UN’s Sustainable Development Goals Report 2019. 69 This quote was taken from a Q&A session with Professor Harari from January 2020 during the Newsweek Belgium event "The Future of Sapiens. 70 Toronto Waterfront is the agency responsible for the revitalisation of 800 hectares of brownfield lands on the waterfront into beautiful, sustainable mixed-use communities and dynamic public spaces. It was created in 2001 by the Governments of Canada and Ontario and the City of Toronto.

84 MSc Design Thesis: Jana Voykova

The Canadian Civil Liberties Association’s director Brenda McPhail argued that we need to be sceptical about technology companies’ belief that ‘if we have enough data, we can solve all our problems’. The author of the article — Jane Wakefield quoted the former Information and Privacy Commissioner Dr. Ann Cavoukian in her resignation letter: “I imagined us creating a smart city of privacy, as opposed to a smart city of surveillance”.

Dan Doctoroff, Sidewalk Lab’s head and former deputy mayor of New York, told the BBC that the project was “about creating healthier, safer, more convenient and more fun lives”. He forgot to specify smart, convenient and healthy for whom. ‘While the smart city concept is at a relatively early stage, thus far the discourse has failed to (or rather declined to) incorporate issues such as inequalities on the ground, and accumulation by dispossession of land and resources, that emerge out of sociotechnical complexities of urban development’ (Das, 2019, p.5).

This 21-century urban utopia appears to be the laboratory where everything is autonomous and smart, except the automated human pets. In our dystopian present-to-be-future, algorithms are cleaning our houses, driving our cars, choosing music for us, adjusting the temperature in our homes, turning on the lights, paying our bills, monitoring our health, telling us where to go, what to eat, how to dress, whom to like. What are we doing? We acknowledge the warning signs by posting an inspirational quote on social media and then everything is business as usual.

The popular mantra that this is innovation and we cannot slow it down, or stop it, is profane. Innovation is one of the euphemisms that the proponents of the digital economy use in order to, as Shoshana Zuboff calls it, bamboozle us and exert their ‘improved control of us’ (Zuboff, 2019, p. 181). After all, while ‘it may be possible to imagine something like the “internet of things” without surveillance capitalism, it is impossible to imagine surveillance capitalism without something like the “internet of things’ (Zuboff, 2019, p. 380).

The inevitabilism71 of the not so distant future connected world ‘is rarely discussed or critically evaluated. As in most accounts of the apparatus, questions of individual autonomy, moral reasoning, social norms and values, privacy, decision rights, politics, and law take the form of afterthoughts and genuflections that can be solved with the correct protocols or addressed with still more technology solutions’ (Zuboff, 2019, p. 418).

71 Shoshana Zuboff calls the inevitable, certain future of technological progress and innovation inevitabilism, the doctrine which present-day technologists and scientists fanatically embrace and protect against any form of impediment or criticism.

85 MSc Design Thesis: Jana Voykova

Professor Zuboff traces back the ideas behind surveillance capitalism to some of the ideas that came from the scientific and academic domains in the post-war period. The Harvard behaviourist and psychologist B.F. Skinner studied the reinforcement of behaviour and introduced the term “operant conditioning”, a type of learning process which modifies behaviour through reinforcement and punishment. He rejected the idea of freedom and ‘casted the notion of human dignity as an accident of self-serving narcissism. Skinner imagined a pervasive “technology of behaviour” that would one day enable the application of behaviour-modification methods across entire human populations (Zuboff, 2019, p. 600). At the time (1970s), Skinner’s concepts raised public outrage, today they are celebrated and emulated in the technology labs of Silicon Valley, MIT and Stanford University.

Skinner was greatly influenced by the beliefs of the early twentieth-century experimental psychologist Max Mayer. Meyer proposed the idea that psychologists should view humans as others (hence Zuboff calls the instrumentarian apparatus of surveillance capitalism Big Other) and study human behaviour from a distance. In this relation, humans are looked upon as objects of study, organisms among organisms whose individuality is of no importance as long as their observation and modification make contributions of social significance. Meyer hoped that this attitude would finally elevate behaviour psychology to the scientific status of physics, biology and mathematics. He posited that ‘freedom of action in the animal world signifies the same that is meant by accidents in the world of physics. Such accidents are simply phenomena for which there are insufficient information and understanding’ (Zuboff, 2019, p. 674).

‘Meyer argued that the future of the social sciences and of civilization itself rested on this shift from soul to other, inside to outside, lived experience to observable behaviour. The otherization of humanity was to be the road to a new kind of political liberation’ (Zuboff, 2019, p. 676). Self-determination, freedom and will are impediments to the objective scrutiny of the other one according to Skinner and Meyer. They are ‘the chief impediment to social progress’ (Zuboff, 2019, p. 681). ‘Skinner understood that the engineering of behaviour risked violating individual sensibilities and social norms, especially concerns about privacy. In order to allay these anxieties, he advised that observation must be unobtrusive, ideally remaining outside the awareness of the organism’ (Zuboff, 2019, p. 686).

Shoshana Zuboff recognises Skinner’s concepts in the work of Alex Pentland, Professor of Media Arts and Sciences at the MIT Media Lab and director of the Human Dynamics Lab at MIT. Pentland is a respected academic, entrepreneur and futurist, in whom Shoshana Zuboff sees a forerunner of what she calls ‘applied utopistics’72. Strangely enough, Pentland is highly

72 Shoshana Zuboff introduces the term “applied utopistics” to name the vision of 21st century scientists and entrepreneurs that technological evolution is the only way forward and technology is the single solution to

86 MSc Design Thesis: Jana Voykova esteemed by organisations like the UN73. Forbes appointed him as “one of the seven most powerful data scientists in the world” in 2012. His research is dedicated mainly to digital wearable technologies. Pentland is a distinct supporter of the Internet of Things and promiscuous connectivity which can be used to uncover aspects of human behaviour invisible to the external world. He participated in the development of Google Glass74 and is the founder of more than a dozen venture companies invested in the advancement of technology that monitors, renders and nudges human behaviour in the pursuit of technological excellence and maximised revenues.

Professor Zuboff sees Pentland’s work as a continuation of Skinner’s ideals with the exception that today, technology does give Pentland and his fellow futurists the instruments to manipulate and control social relations with the bonus feature of amassing handsome economic fortunes. “Great leaps in health care, transportation, energy, and safety are all possible,” Pentland writes, but he laments the obstacles to these achievements: “The main barriers are privacy concerns and the fact that we don’t yet have any consensus around the trade-offs between personal and social values.” Like Skinner, he is emphatic that these attachments to a bygone era of imperfect knowledge threaten to undermine the prospect of a perfectly engineered future society: “We cannot ignore the public goods that such a nervous system could provide.…”. Pentland avoids the question “Whose greater good?” How is the greater good determined when surveillance capitalism owns the machines and the means of behavioral modification? “Goodness” arrives already oriented toward the interests of the owners of the means of behavioral modification and the clients whose guaranteed outcomes they seek to achieve. The greater good is someone’s, but it may not be ours’ (Zuboff, 2019, p. 678).

In his utopia from 1948 Walden Two, B. F. Skinner envisioned a society governed not by political action but by behaviour modification based on reinforcement, punishment and rewarding. Walden Two is a community where scientific and technological progress are the instruments to design a predictive society of obedient, happy creatures, who enjoy high levels of efficiency and productivity and the margin for error is close to zero. This world reflected Skinner’s view that freedom equals ignorance and ‘further knowledge about human humanities’ problems. This utopian ideal (embedded in surveillance capitalism today) relies on the speed, scale and complexity of the ideas presented, and the absence of any lawful restrictions in order to instil a sense of ‘shock and awe, leaving onlookers dazed, uncertain, and helpless’ (Zuboff, 2019, p. 638). 73 Pentland is on the Board of the UN Foundations' Global Partnership for Sustainable Development Data, a member of advisory boards for the UN Secretary General and the American Bar Association, and previously for Google, AT&T, and Nissan. He is also a prominent advisor of the World Economic Forum. 74 Google Glass is a wearable technology developed in 2012 by Google which is a form of a digital assistant advertised as yet another “cutting-edge” product that would supposedly enable people to navigate their physical environment with ease and predictability. The technology was discontinued for end-customers in 2015 because of the public’s privacy concerns and is now only provided to business entities.

87 MSc Design Thesis: Jana Voykova behavior’ is a ‘new way of applying that knowledge to the design of cultural practices’ (Skinner, 2005, p.32). Skinner’s motivations were indeed convincing. He was worried about the inability of any existing political doctrine to cope with the pressing challenges of a planet with overconsumption, overpopulation and finite resources.

However, he found the solution in a morally dubious social experiment of submissive citizens, whose actions are constantly monitored, pre-determined, and any individuality is suppressed in the name of a well-functioning community. Freedom is indeed a malleable concept, but the way Skinner talks about it is ominous. In Beyond freedom and dignity (1971) he writes: ‘Whether we regard ourselves as explaining feelings or the behaviour said to be caused by feelings, we give very little attention to antecedent circumstances…We say a person is autonomous and so far as a science of behaviour is concerned, that means miraculous. The position is, of course, vulnerable. Autonomous man serves to explain only the things we are not yet able to explain in other ways…The task of a scientific analysis is to explain how the behaviour of a person as a physical system is related to the conditions under which the human species evolved and the conditions under which the individual lives’ (Skinner, 1971, pp.19-20).

Skinner claims that a person is never acting out of his personal beliefs, goals, intelligence and ethics, in other words — autonomously, but his/her behaviour is always conditioned on external factors. By monitoring and understanding those external stimuli, we can design a better culture. ‘Freedom, dignity, and value are major issues, and unfortunately they become more crucial as the power of a technology of behaviour becomes more nearly commensurate with the problems to be solved’ (Skinner, 1971, p. 27). Autonomous man is not easily changed; in fact, to the extent that he is autonomous, he is by definition not changeable at all. But the environment can be changed, and we are learning how to change it. The measures we use are those of physical and biological technology, but we use them in special ways to affect behaviour (Skinner, 1971, p. 101).

The similarities between Skinner and Pentland are startling. Pentland described his sociometer, a wearable sensor which measures face-to-face interactions between people as a reliable method for ‘learning the structure and dynamics of human communication networks’ and a means to ‘understanding human communication patterns studied in organizational behavior and social network analysis’ (Pentland and Choudhury, 2002, p. 1). Pentland and his PhD student Tanzeem Choudhury call the individuals whose biometric data is monitored through their sociometer, connectors. This is quite revealing of the little respect they seem to have for their subjects of study. They ambiguously identify their sociometer as ‘cheaper and more reliable than human-delivered questionnaires’ (Pentland and Choudhury, 2002, p. 2).

88 MSc Design Thesis: Jana Voykova

Pentland and Choudhury went on to explain the necessity to ‘know who is talking to whom, the frequency and duration of communication’. To record the identity of people in an interaction, we equip each person with an infra-red (IR) transceiver that sends out unique ID for the person and receives ID from other people in her proximity. We use microphones to detect conversations’ (ibid.) The sociometer collects the following type of data about each individual who wears it: 1) Information about people nearby; 2) Speech information; 3) Motion information. This means that the technology does not only collect data about the person who wears it but also about the people they meet, apparently regardless of their consent and knowledge.

Pentland and Choudhury did acknowledge the privacy concerns of the participants, stating ‘To protect the user’s privacy we agree only to extract speech features, e.g. energy, pitch duration, from the stored audio and never to process the content of the speech’ (Pentland and Choudhury, 2002, p. 3), as if they are doing people a favour by not infringing on their privacy.

At the 2016 Exponential Manufacturing Summit in Boston of Singularity University, Pentland suggested that although people were one of the biggest assets of organisations and corporations, they were the one factor that was always “messing things up”. “If people aren’t interacting correctly and information isn’t spreading correctly,” Pentland warns, “people make bad decisions.… What you’re trying to do is make a human machine symbiote, where the humans understand more about the network of interactions because of the computers, and the computers are able to understand more about how humans work.” (Zuboff, 2019, p. 668). The “progressive” visions of scientists like Pentland, sound as if our not being smart may be the single necessary condition for the future smart technologies to be possible.

All of these observations reflect neither a grand conspiracy against people nor a personal vendetta against Google or any other powerful corporation and claiming otherwise is insulting to the seminal work of intellectuals like Shoshana Zuboff, Douglas Rushkoff, Yuval Noah Harari and Michael Sandel. What Shoshana Zuboff calls surveillance capitalism is an ingenious business model which successfully monetises non-monetary assets and transforms them into an infinitely scalable reservoir. As Douglas Rushkoff points out, the people at Google are not the core problem, ‘they’re just the easy target’ (Rushkoff, 2016, p. 13).

‘Silicon Valley executives and venture capitalists are simply practicing capitalism as they learned it in business school and, for the most part, meeting their legal obligation to the shareholders of their companies. Sure, they are getting wealthier as the rest of us struggle, and yes, there’s collateral damage associated with the runaway growth of their companies and stocks. But they are as stuck in this predicament as anyone; many CEOs understand that meeting short-term growth targets is not in the best long-term interests of their companies

89 MSc Design Thesis: Jana Voykova or their customers, but they are themselves caught up in a winner-takes-all race for dominance against all the other digital behemoths. It’s grow or die’ (Rushkoff, 2016, p. 13).

Rushkoff sees capitalism’s weak spot as ‘a program that promotes growth above all else. We are caught in a growth trap. This is the problem with no name or face, the frustration so many feel. It is the logic driving the jobless recovery, the low-wage gig economy, the ruthlessness of Uber, and the privacy invasions of Facebook. It is the mechanism that undermines both businesses and investors, forcing them to compete against players with digitally inflated poker chips (Rushkoff, 2016, pp. 14-15).

Zuboff, Anderson and Wolff talk about capitalism as the clash between two opposing groups: employees and citizens on the one side, and power-holders: employers, capital owners and politicians, on the other. Rushkoff describes capitalism as the battle between humanity as a whole and its own growth syndrome which ‘has reached the limits of its ability to serve anyone, rich or poor, human or corporate’ (Rushkoff, 2016, p. 15). They are all referring to one and the same thing, though with different words and approaching it from different angles. Today’s economic model is a man-made phenomenon, ‘invented by particular human beings, at particular moments in history, with particular goals and agendas. By refusing to acknowledge this…we end up transacting and living at the mercy of the system… We are destined to repeat the same old mistakes. Only this time, thanks to the speed and scale on which digital business operates, our errors threaten to derail not only the innovative capacity of our industries but also the sustainability of our entire society’ (Rushkoff, 2016, pp. 16-17).

Naturally, it’s not enough to just acknowledge the fallout of our political and financial systems. We need to start looking for better alternative solutions by accepting that these new solutions, whatever they might be, won’t come to us at an exponential speed, just as there is no single quick fix for the social problems we are currently facing. ‘There’s only slow, incremental change, enacted consciously but differently by all sorts of people and institutions (Rushkoff, 2016, p.27).

Rushkoff believes that a possible solution is the designing of a financial model in which money is used as a transaction instrument to maximise value amongst people rather than as an extractive tool to create business monopolies and widen social disparity (as he exemplifies, this alternative model should be similar to the way medieval bazaars functioned, or the logic behind the commons, where responsibility and accountability are evenly distributed among all beneficiaries).

Much like Paulsen, Rushkoff convicts the modern perception that one’s job is the ultimate measure for one’s individual worth. As he puts it, ‘Something is standing in the way of our claiming the prosperity we have created. The toll collector whose job is replaced by an RFID

90 MSc Design Thesis: Jana Voykova

“E-ZPass” doesn’t reap the benefit of the new technology. When he can’t find a new job, we blame him for lacking the stamina and drive to retrain himself… And what new skill should he go learn? Even the experts and educators have little idea what gainful employment will look like just five years from now’ (Rushkoff, 2016, p. 109).

Even though we have automated so much labour and advanced technology as much as we have, it still feels wrong to give someone housing and food if they don’t have a job. This is counterintuitive. We have the means to provide each citizen a decent life but unless people clock in an eight-hour job, we behave as though they don’t deserve decent life.

Rushkoff proposes that we look back to the 1980s when technology was greatly seen as a means to democratise power and give ordinary people mechanisms to exchange value in a peer-to-peer network. He calls this alternative system digital distributism. ‘Distributed doesn’t simply mean decentralized; it’s not the principle through which alternative power centers emerge on the periphery of a system. Rather, when power is distributed, it is available throughout the network. It is everywhere at once. The same is true for capital in a distributed system, as well as value, energy, resources, companies, and people. Everything becomes more available to anyone’ (Rushkoff, 2016, pp. 432-433).

This doesn’t mean that individuals cannot own a company or profit from it, but unlike the way capitalism functions today, under distributism, growth doesn’t scale infinitely, and employees (citizens) are peers, not subordinates who have the right to share in the rewards by ownership stakes, for example. As evidenced in Rushkoff’s narrative, the fact that establishments like the bazaar, the commons and the crowdsourcing business model have only succeeded on a small scale or failed completely, is that we have historically used new methods to serve old principles.

Simply put, if we use new tools (as we currently seem to be doing with innovative technologies) to serve the old interests of contemporary “aristocrats” instead of addressing global societal and environmental issues, we cannot be surprised that what we ultimately end up doing is going around in (vicious) circles.

While Rushkoff’s observations are brilliant, logical and timely, they are relatively utopian: the radical changes he proposes are indeed achievable in theory but in practice, they are threatening the status quo which works as intended for those who designed it. This doesn’t imply that he lives under an illusion, he is well aware that the profound change we need is not going to come to us easily but in order to resolve a problem, we first need to know what it is and what is causing it. This is namely what Rushkoff is doing — explaining the principles behind the current social and political distress, one of them being ‘if the money stops fulfilling the needs of human beings, you change human beings to fit the needs of the money’

91 MSc Design Thesis: Jana Voykova

(Rushkoff, 2016, p. 276). Whether we will manage to dissolve and redesign these principles, though, is another master thesis.

As pointed out in the beginning of this chapter, my analysis of the social landscape around designers’ work and the factors which restrain them to demonstrate their social responsibility, as well as the social context around the design of autonomous vehicles, is fairly detailed but not exhaustive. However, I hope I have made clear where my critique of the development of this technology is rooted and why I see it as an outlet of designers’ social responsibility. Designers’ creative skills are often used to convince people that disruption and innovation are what we should require from our future and the fact that many designers are attracted to positions at prominent companies, which work with digital transformation, makes them just as responsible to properly arrange their priorities and look at their social surroundings holistically and critically.

All the same, this outlook on the design profession is quite grandiose and not all designers seek it. Honestly, not all designers have to. As I argued earlier, change can also happen on a smaller scale, but then this change will still be fragmented and ethical (social) design will still be doomed to be a sporadic volunteer endeavour, a luxury which not every designer can realistically afford. I believe Rushkoff is right, though, that ‘compromise is not failure; it’s incremental improvement in an imperfect situation’ (Rushkoff, 2016, p. 447).

As much as we like to entrust machines with the fate of our future, machines are created by people and people get inspired by other people, not machines. Therefore, I’d be more careful when describing humans as prone to error, flawed and having profound cognitive limitations. What’s more, technology isn’t developing itself, people advance technology and they should be able to slow it down or hit the brakes on it if that is necessary. Some things in life are indeed inevitable but technology isn’t one of them.

In the next chapter I will explain in more detail the different phases of my primary research and the rationale behind the design process I followed to apply speculative and critical design in my prototype exhibition. In the chapter that comes after that, I will analyse the collected outcomes of my research and discuss them in the light of my theoretical critique.

92 MSc Design Thesis: Jana Voykova

5. Research Process

The main phases that my research went through can be divided into three broad categories:

01. Literature review and analysis (desk research); 02. Primary research (the actual research which I conducted focusing more closely on finding an answer to my research questions); 03. Design process (creation of design artefacts to support the thesis text/ facilitate a room for discussion and reflection by utilising speculative and critical design).

Due to time limitations, the execution of these phases overlapped. The rest of this section is dedicated to a more detailed explanation of the research and design process.

5.1 Planning and concept development

The planning of my research started long before the actual start of the thesis semester. I chose to research the social and ethical implications of autonomous vehicles and discuss this topic in the context of designers’ profession already with the start of my education although in the beginning, my ideas were quite rough and sketchy. The planning of my project went through different iterations and phases, especially after I wrote my research plan. It took some deliberation around what aspect of the technology and its social impact to focus on: its implications on the labour market, law enforcement and regulation, security standards, relationship between machine and human. In the end, I decided not to focus on just one of these aspects but try and discuss critically the aspects which I find most problematic and important.

There are several reasons for this decision: 1) I was unable to engage important stakeholders; 2) my critique of the technology is not specifically directed at any particular aspect, I rather find problematic the consequences of the technology in broader terms and the motivations behind its development; 3) I couldn’t examine all social and ethical aspects because I didn’t have the time and my thesis would have become overwhelmingly long for a master thesis even if I did have the time. I firmly believe that the nature of my chosen topics requires a high-level perspective which I zoomed in and out of during the execution.

At already mentioned, I was planning to conduct a much wider research including various stakeholders related to the subject. I started contacting them well in advance so that I could

93 MSc Design Thesis: Jana Voykova plan ahead. I reached out to a number of automotive companies, automotive lobby organisations, technology start-ups who are developing the technology, research institutions, non-profit organisations. Unfortunately, none of them were willing/available to participate and inform my research for one reason or another. Eventually, this fact turned out to be beneficial for me as the engagement I was planning for initially, was not feasible for a 4- month project. I decided to narrow down the scope and dedicate my primary research on this question to just a selection of stakeholders, which were much easier to reach, namely ordinary citizens to give their opinion about the social and ethical implications of autonomous transportation.

I would say that the planning phase was ongoing until the end of the project because unforeseen circumstances presented themselves and I needed to be flexible with the execution. As far as my first research question is concerned, I was planning to talk to practicing designers since the beginning as they are in the best position to answer how they treat their social role and responsibility. In terms of literature review and writing, this phase was more intense during the first half of the thesis project, however, I was constantly revising what I had written and looked for relevant literature to include until the very end of the project. The designing of artefacts to support my thesis is explained in more detail in the Prototyping a virtual exhibition section below.

5.2 Interviewing and surveying

As previously clarified, I chose to do interviews and surveys in order to answer my research questions as these two methods allow the collection of qualitative and quantitative data fairly easily. I did 20 interviews with two types of groups: practicing designers and “ordinary” citizens. I contacted 25 designers out of which 10 were available for an interview. I started contacting designers early on in order to give them a few weeks to plan for the engagement I requested from them. I picked the potential participants from my personal and professional networks, online design communities, some of the interviewees were recommended to me from the people I had already interviewed. I contacted designers from different design disciplines: service design, UX design, illustration, graphic design, because I wanted to collect more varied opinions. I also reached out to two designers who had law education because law and ethics are closely connected, and I was interested to see how designers of mixed majors interpret the question of social responsibility and ethics. Not surprisingly, it was much easier to recruit people for a conversation on autonomous vehicles: I reached out to 11 people, out of whom 10 confirmed their interest to participate. In terms of targeted background, I turned to people who worked either in

94 MSc Design Thesis: Jana Voykova automotive/automobile systems development or software engineering/IT/people with other professional background who were just interested in the topic.

I conducted the majority of the interviews in February and March. I provided a consent form (two consent forms, adjusted for the two research questions; they can be found under Appendix B. Interview/survey guides) to all participants so that they could understand what the purpose of the interview was, the format, the timeline and how the input provided was going to be used. The majority of the interviews were conducted through an online conference software, some of them I conducted on the phone, another part I conducted face- to-face. I audio-recorded the interviews (obtaining permission) so that I could have them as a reference during my analysis. In terms of demographics, most of the participants from the two groups are Bulgarian, the rest are members of different countries in Europe and Latin America. I have promised anonymity to all participants, however, they have agreed to be quoted in my thesis (with their answers anonymised).

When it comes to the survey phase, I started collecting answers for the survey on autonomous vehicles already during the summer (before the last semester started) so that I could collect more answers. I reached out to car forums in different countries (Bulgaria, UK, US, Germany), as well as my personal and professional networks. Some of the people who already filled in the survey, distributed it to other people whom they thought were suitable respondents. Unfortunately, I missed to ask the respondents to this survey to specify their location. I corrected this mistake in my second survey. I introduced the second survey, for which I recruited only designers, fairly late in the research process, however, I collected a lot of answers relatively quickly. Quite a few designers were intrigued by the topic and expressed an interest to read my thesis when it was finished. A lot of the designers I know personally helped me by sharing the survey with their professional networks.

The questions which I asked in the interviews and the surveys can be found under Appendix B. Interview/survey guides of the Appendices section. Each conversation had different duration and dynamic, although I followed one and the same interview structure. In the course of the interview phase, I slightly adjusted the questions judging from the reactions of the first participants. I focused on asking questions which addressed my research questions as well as important elements of the topics at hand. I wrote the questions as objectively as I could so as not to embed my personal biases in the topics. I think I managed to do that because some participants asked me explicitly about my personal stand on these questions which shows that it wasn’t clear from the questions I asked. The tables below describe anonymously the demographics of the participants in the interviews.

95 MSc Design Thesis: Jana Voykova

Table 3. Interviewees: design practitioners

NATIONALITY DESIGN PROFILE SEX

service design/ 01. India male mechanical engineering

02. Finland service design/law female

03. Bulgaria illustration male

04. Germany design strategy/ service design female

05. Colombia service design/law female

06. Bulgaria illustration female

07. Bulgaria graphic/ brand design male

08. Bulgaria UX design male

09. Bulgaria UX design female

10. Colombia service design/industrial design female

Table 4. Interviewees: autonomous transportation

NATIONALITY PROFESSIONAL FOCUS SEX

01. Bulgaria human resources/IT female

02. Bulgaria business relations/IT male

03. Bulgaria bid management/IT female

04. Bulgaria IT/automotive software development male

05. Bulgaria IT/software support male

06. Bulgaria IT/automotive software support male

07. Bulgaria product development/ automotive male

08. UK IT consulting male

09. Sweden automotive female

10. Argentina software engineering male

96 MSc Design Thesis: Jana Voykova

5.3 Reiteration of second research question

In the course of my research, I realised that my second research question, namely “What are the ethical and social limitations of autonomous transportation and how can design be used to aid the public understanding about important social phenomena?” was too broad for me to be able to answer in the course of four months. Additionally, it was not feasible to search for this answer, by gathering opinions of citizens and non-experts, not all of whom had the required expertise to provide a valuable input on the various expressions of this social impact. This is why I restated my research question:

How do citizens perceive the social impact of autonomous vehicles and can design aid the public understanding of important social phenomena?

I was going to answer the second part of the question through the prototype exhibition in which I shared my design artefacts with a selected group of people. The reiteration of my second research question helped me to back up my assumption that the constructive social conversation is often hijacked by omitting salient parts of the (digital) transformation process (which autonomous transportation is related to). Directing my research question to people’s perception of the social impact of the technology helped me to assess their proactive engagement in this conversation and their interpretation of the information available in the social space. The outcomes of my interviews and surveys are analysed in the Research outcomes and Discussion chapter of this thesis.

5.4 Prototyping an exhibition

I started the exhibition preparation (search for a venue, think about marketing the event, order frames for my posters, design the actual artefacts, find a printer agency to print my designs, etc.) already in the very beginning of my thesis project so that I could give myself time for any contingencies. Overall, my idea about the exhibition and the artefacts I was going to display, did not change radically throughout the research although I refined some details around it in the course of the project.

I was planning to host the exhibition in mid-April 2020 in Sofia, Bulgaria. For better or worse, however, the Corona virus pandemic currently looming in Europe, and the subsequent social lockdown prevented me from realising an exhibition in the traditional sense. As social gatherings were suspended, I came up with a workaround and set up a

97 MSc Design Thesis: Jana Voykova virtual exhibition instead. I then shared it with a group of people and collected their feedback. I knew that the effect was not going to be as resounding, even though technology enables online communication, nothing can replace a physical encounter with an object or a person. But circumstances were outside of my control at the time. The gathered insights from the virtual exhibition are analysed in the Research outcomes and Discussion chapter of this thesis.

The table below highlights the demographics of the selected participants in my prototype exhibition.

Table 5. Prototype exhibition: focus group

NATIONALITY PROFESSIONAL FOCUS SEX

01. Bulgaria philosophy/religion female

02. Bulgaria law female

03. Russia ethics female

04. Bulgaria fashion design female

05. Bulgaria IT/project management male

06. Bulgaria IT/project management female

07. Sweden IT support male

08. Sweden IT security student male

09. Bulgaria architect female

10. Spain psychology female

Instead of narrowing the focus to autonomous transportation as such, I focused on the concept of smart cities and the social division which technological evolution is already causing in a certain degree. The reason for this, as explained previously, is that my concern with disruptive technology and autonomous vehicles is not so much the shortcomings of the technology as such but how and why we design it the way we do and the importance we attribute to it: instead of augmenting and improving human behaviour intellectually, we use it in a way that attacks and exploits our vulnerabilities, and instead of addressing social, political and environmental anomalies by means of technology, we use it in a way that accelerates social division and compels us to adopt an egocentric God-like view. Another key aspect of my concept with smart cities was to question the seemingly widespread view that

98 MSc Design Thesis: Jana Voykova the best way to fight urbanisation is by introducing innovative technologies rather than addressing the political issues which are essentially the root cause behind it.

Using speculative and critical design, I was aiming to bring up my critical perspective on the question and inspire reflection on what this means for important facets of life like morality, freedom of choice, individuality, power, equality. It was a way to demonstrate to people the non-conventional side of design as an instrument to advocate for more ethical behaviour on a societal level as opposed to design being used to advertise, sell and maximise profits. By creating a speculation about the future and bringing forward painful aspects of contemporary society, I was hoping to inspire people to start questioning the course of human evolution and the value systems that we have normalised, hopefully convincing them to start making more deliberate decisions in their capacity as citizens and consumers.

I used primarily graphic design and typography for the design of my artefacts. Synne Skjulstad and Margaret Rynning of Oslo School of Arts, Communication & Technology, introduced an interesting initiative which they called speculative graphic design at a conference held in 2015 in Mumbai. Their goal was to teach graphic design as an ‘agenda- setting discipline’ as opposed to the commercial, business-oriented graphic design that is taught in most art schools. Skjulstad and Rynning see speculative graphic design as a way to enrich design education and empower students as future designers and citizens. I also believe that the image of graphic design, as purely visual art, should be changed and utilising graphic design in my thesis was a way to do that. Whether something looks nice, doesn’t mean that it doesn’t communicate a meaningful message and vice versa. As Skjulstad and Rynning point out, ‘graphic design still tends to be understood as predominantly serving client’s needs, and as expressing what the client pays for [Hollis 1997]. However, graphic design may also be understood not just as a craft but also in terms of intellectual problem solving [Canaan 2003] (Skjulstad and Rynning, 2015, p.4).

Another reason I used graphic design is that graphic design was what inspired me to want to become a designer, it is a discipline with a long tradition and I also think graphic design can make many contributions to service design, because these two design disciplines often lose the connection with each other.

My virtual exhibition and the artefacts are black and white for many reasons: 1) the absence of colour does not distract from the message; 2) it successfully translates my concept of the contradiction between a world of winners and losers, it also alludes to the concept of ethics — the grey areas in between white and black; 3) this is a favourite colour combination of mine. I designed most of my artefacts in a square format because squares are perfect shapes, so they fit in well with my pursuit of utopian ideals.

99 MSc Design Thesis: Jana Voykova

The components of my prototype exhibition are:

01. Eight typographic posters featuring quotes which I took from my research and which I think are worth reflecting on. Two of the messages are authored by me. They focus on questions like the way we treat technology, inequality between different groups of society, business and economic models. The posters are essentially a critique of today’s value system of developed nations and I was hoping that these messages would make people reflect on what was written on them, initiate a debate and spark interest about where they came from and why they communicate what they communicate. I had a very clear vision for the design of some of the posters but for others I explored several iterations. I chose a group of typefaces which I used throughout my artefacts. Some of the working versions for some of the posters can be found below.

Figure 5. The following four images show some of the earlier versions of some of the posters. The finished designs can be found in the virtual exhibition.

100 MSc Design Thesis: Jana Voykova

101 MSc Design Thesis: Jana Voykova

Figure 6. When the printed posters were ready, I set them up in frames.

102 MSc Design Thesis: Jana Voykova

103 MSc Design Thesis: Jana Voykova

02. The second element of my prototype exhibition is an instruction manual in the form of a short brochure which is part of my speculation and represents a future where machines are conscious and autonomous. This is an inflated and a rather far-fetched fictional scenario but still relevant. It is an attempt to mock human nature and chastise our worst instincts which we often exploit with the way we design technology today. The brochure contains eight machine laws which instruct machines how they can manipulate humans successfully. They are meant to question the importance we attribute to technology and also mock our inclination to give in to our primitive instincts. I included a Machine Manifesto which machines supposedly follow in their pursuit of superiority over humans so as to strengthen the message.

Figure 7. These are two of the spreads of the finished brochure.

104 MSc Design Thesis: Jana Voykova

03. 15 rough sketches depicting a future dystopian scenario where the social division between low-skill/low-pay and high-skill/high pay professions has reached a peak and has become indoctrinated. The scenario aims to question the seemingly common understanding that urbanisation can only be fought and resolved by introducing more technology in overpopulated cities instead of providing the same level of subsistence opportunities in the areas around cities. It also aims to criticise the normalisation of the fluctuations in the labour market as a consequence of automation. The speculation is intentionally strongly exaggerated so as to provoke, unsettle and spark reflection and conversation on this topic but the concept is believable and feasible so as not to drift too much from reality.

At first, I was not sure what format to use to represent the dystopian scenario and what aspect of the thesis theme to show. I was considering making a video or designing a city model out of wood or something like that, but I was still not sure what this scenario would depict. Another idea I had was to 3D print the elements of the scenario, but these ideas were connected with investing more resources and time in the realisation. I was also considering using photos but eventually, I decided to do sketches because this gave me most flexibility with the design of the story and was easiest to execute. I was concerned about the quality of my sketches because I cannot draw well but this was not the point of the artefact. That is why I focused on what I was saying, not so much how beautifully I said it (which is a main principle of

105 MSc Design Thesis: Jana Voykova

speculative design). The sketches themselves have an unfinished look, they represent a general idea but not in a detailed manner and not in a strictly coherent story. This is intentional because while the dystopian future I describe is believable and it is already a reality to some extent, certain details in it are open for interpretation. This reflects my assertion that the future is not pre-determined and can take a number of different directions. In that sense, the viewers can reflect on what those directions might be, and they don’t necessarily have to be dystopian.

Figure 8. These are some of my scribbles while I was trying to decide what to represent with the sketches of the dystopian present-future.

04. In my speculative dystopian future, I introduced a wearable technology (biometric bracelet), which I called the sub-meter (allusion to submitter). It was essentially a device which the higher class of the smart city (the winners) uses in order to exert control over the losers (the people who are reluctant to embrace technology and innovation). I decided to design a prototype for this technology in order to make it

106 MSc Design Thesis: Jana Voykova

more tangible. The sub-meter consists of two elements: a bracelet which people wear on their hands and which records their biometric data, and a palm implant, connected with a wireless signal to the bracelet. The idea is to show how innovative wearable technologies (which are an important part of the vision for the smart future), can be used with evil intent. I used one of my old belts, couple of pins and an innertube repair patch to design the prototype. At first, I designed it so that it had the name of an imaginary technology company but then I gave up this idea because this was irrelevant. I was not planning for this wearable device when I started drawing the sketches, but the idea came through in the sketching process.

Figure 9. I used one of my old belts to design the prototype of the biometric sub-meter.

05. I wanted to display the name of my exhibition in a 3D format, hoping that this way, it would be more perceptive for the audience. I explored materials and prices for the cut-outs already in February (the start of my project) and it turned out that the most viable solution in my case was to have the letters cut out of styrofoam (for the size I wanted this material was the cheapest). At first, I was thinking of naming the exhibition #urbanised but this was representing only a part of the problem I aimed to bring up. This is why I changed the name to #submission. This expresses the concept of the exhibition in a good way, suggesting that today, it is extremely easy to trick people into believing that something is beneficial for them which might not necessarily be the case. I believe that most people hardly question what values are imposed on them, accidentally or deliberately and submit to social pressure too easily, especially if the ideas in question have been normalised.

107 MSc Design Thesis: Jana Voykova

I ordered the cut-outs at a local construction materials workshop. When the letters were ready, I used black spray and car paint to paint them. It turned out that the spray had an oxidising effect on the styrofoam which came through some time after the spraying was done (hence some of the letters got slightly damaged). This is why I used black paint for the rest of the letters. Below are some photos which show my process.

Figure 10. The following four images show the process of painting of the styrofoam letters.

108 MSc Design Thesis: Jana Voykova

109 MSc Design Thesis: Jana Voykova

6. Research outcomes and Discussion

Optimism is endemic, meaning that it is unnatural for designers to think about the implications of their (technological) products: technology is good; products are good; and the future (through technological products) will therefore also be good!

James Auger

In this chapter I will summarise the outcomes of my research and discuss their implications for previous and future research. Since my thesis is dedicated to the examination of two fairly distinct questions, I am going to analyse the outcomes of the research questions separately and then try to unite them and articulate once again where I see the convergence between them. I will do that in the Discussion section of this chapter.

6.1 Research question #1: the social responsibility of designers

In this section, I am going to analyse my interviews and the results of my survey on the social responsibility of designers and the social importance of design. In doing that, I will answer my first research question: how do practicing designers perceive their social responsibility and how can they demonstrate it in their practice?

6.1.1 Interviews analysis

The interviews I conducted were very insightful and aided my understanding of how design practitioners view their profession and what challenges they face in their daily work. Since I only talked to 10 designers, my analysis of the results is qualitative. These interviews and the collected insights are not at all an objective representation of what the majority of design professionals think, especially given the fact that the social context which designers create in, looks quite differently in different countries. This is something that I noticed in the conversations as I interviewed people from different locations and backgrounds. Some of the designers were employed at a company, others were freelance designers or entrepreneurs. Not surprisingly, the former had less freedom to determine what projects to take up than the latter which directly affects their capacity to always demonstrate ethical behaviour in their

110 MSc Design Thesis: Jana Voykova work. In support of my assumption, none of the designers was hesitant about what the social importance of their profession was, and they were all aware of the tremendous impact their work has on people. Four designers said that their social responsibility is not determined by the fact that they are designers but by the fact that they are human beings.

As argued in my theoretical overview, designers are not the only people who should abide by ethics codes and being attentive to how one’s work affects the people who come in contact with it, is more closely related to designers’ principles as human beings, not their responsibility towards the profession. However, depending on the design profile of different designers (graphic design, service design, UX design etc.), I concluded that their social responsibility and importance would scale differently. For example, service designers’ work begins with the central idea to drive positive change and look at a problem from a holistic lens, while illustrators are more drawn towards the artistic side of the profession, not so much the impact it has on people.

Interestingly, one designer did not think the specific profile was indicative of how much designers could demonstrate their social responsibility. She said that how a designer arranges their priorities, was more important than their profile and this is indeed a valid point. As anticipated, the majority of the designers I talked to, made a clear distinction between socially important projects and business ones, and expressed the view that business is driven by profit to a great extent and not by doing good which sometimes results in compromises on the ethical side of the profession.

That said, one of the designers made an interesting observation that even though she was working primarily with business clients, the very focus of some of the projects she worked on was addressing social causes. It’s generally true that the fact that a business stakeholder is assigning a project doesn’t automatically make it unethical or meaningless. She said that making good does not always need to be related to grand projects like eradicating global poverty, it can be a humbler endeavour like teaching kids something useful. I agree, business is not a synonym of unethical activity. However, as pointed out by the majority of the designers, businesses’ primary focus is generating revenue and social design isn’t normally seen as a lucrative activity. Therefore, it is something that a select few can afford doing.

Three designers said that they have to lead a never-ending battle to educate others about the benefits of designing ethically. As discussed in my theoretical overview, this is related to the general lack of incentive in doing social design, it is almost always a volunteer experience, and this makes it less appealing for people who have to think about economy and making a living, not simply doing noble design. Several designers shared the view that, in order for a designer not to end up in a situation where they would be faced with a choice between working on a morally dubious project or else quit the job, they would have to set the tone

111 MSc Design Thesis: Jana Voykova right from the very beginning and choose their places of employment based on the company culture and the business the company runs. If a designer joins an advertising agency for example, they cannot be surprised afterwards that their work is sometimes convincing people that smoking is a good habit.

Contrary to my assertion that today, a good majority of the needs in society are artificially created and stem from business purposes rather than a social drive, some of the designers I interviewed were of the opinion that citizens themselves dictate those needs and designers are not in a position to manipulate citizens’ perception of existing needs through their work. What they meant was, that, unless people see value in a product or a service, they will not accept it, buy it, or use it.

This is naturally true. What I don’t entirely agree with, though, is that the needs which a product or a service satisfies, are always or almost always a reflection of what people truly want or need, as emphasised throughout my theoretical discussion. I think that, especially today, a lot of the potential needs are manipulated and distorted, and this is where the critical thinking of the designer should appear.

At the same time, I do agree that some of the products and services which are being provided to the masses without people asking for them explicitly, do bear real value and do make their life more meaningful. In this regard, it is true that the market itself regulates the success or failure of a product. But thinking critically about these processes, as Victor Papanek communicated in his book Design for the Real World, goes far beyond whether people like and use a particular product. It assesses these processes comprehensively and examines the long-term impact of those products as well as the motivations behind them. On that note, three of the designers said that there needs to be a better utilisation of scientific research in the process of defining and addressing needs in businesses so as to avoid launching a product that has no real practical value. They thought that scientific research is rarely used to back up a need for a certain product and sometimes companies design a product just because they think it’s cool.

One of the designers mentioned imposter syndrome as something that scared her in the profession, which is a concern of many people in the creative industry. The fact that artists fear failure can sometimes mislead them to believe that their work is not as important or the gravity of what they do is not as significant.

Different designers had different opinion about technology, some of them were more optimistic about the increasing automation in a lot of spheres of life, others were more critical in their reflections. One designer said that technology is a “white-world concern”. As

112 MSc Design Thesis: Jana Voykova far as ethical design and demonstrating responsibility go, the majority of the interviewees said that choosing to work ethically happens on an individual level. Still, some of them said that creating a designer’s association and swearing to a designers’ ethics code would aid their capacity to demonstrate social responsibility. I believe this statement backs up my argumentation that the deficiencies in the current global socio-technical context: the view that economic gains surpass ethical restraint, personal dependencies, as well as political vulnerabilities, often restrict designers in defending their social responsibility.

Sadly, there are already design unions and associations in a lot of countries where design is seemingly highly esteemed but not much change has been done to really empower designers to be more resilient to hierarchy in corporations, resistance and undervaluing of their expertise, for instance. That said, some designers were optimistic that citizens, public sector and businesses were slowly starting to respect designers’ capabilities more and include them more in their operations. But as they pointed out, this is a long and slow process which will take years. Two designers pointed out that designers need the support of decision-makers and people in charge when addressing social problems, but more often than not, it is only designers who show up at conferences to discuss such problems. Two other designers said that even though many companies give the impression that they care about embedding design in their operation models, they don’t actually do it but rather pretend that they do.

I had an interesting conversation with one illustrator who had worked in graphic design but had quit his job because design was not fulfilling and satisfactory for him. He identified himself as an illustrator rather than a designer and said that illustration gave him the means to express himself in a way design never did. He talked about the motivations behind this decision and the red tape that exists in corporate settings where different departments work on a solution together and each has their own interests and limitations. This restricts the designer to fully utilise their skills and do what they know how to do best.

As he put it, if a designer is continuously seen as a mere tool to execute the technical part of a design with no regard to their artistry and intellectual capacity, they naturally lose interest in the job and see no meaning in trying to push for change. Starting an independent practice or going into a more artistic profile like illustration, might indeed be a way for designers to eliminate the frustration which comes from working in a rigid corporate structure but on the other hand, corporations need designers to push for this change and if designers are not there to do it, this change will never come. As Mike Monteiro holds, companies like Facebook, Twitter and Uber need designers who care about the impacts of the designs these companies create and are not afraid to advocate for the people who will come in contact with these designs.

113 MSc Design Thesis: Jana Voykova

As most of the designers I interviewed said, designers are involved in the process of finding and resolving needs in society, but they don’t have a deciding voice. Their skills are rather used instrumentally. Two of the designers stated that designers are not taught properly during their education that they have tremendous power to impact society and the profession is not presented in this way right from the start. This is why design students go out of college to take a job where they don’t think they even have the expertise to stand up for people’s rights if it comes to that. There are naturally differences between different programmes and universities but generally, the university curriculum (especially in art schools) needs to shift their focus from teaching designers only technical skills like layout design and proper use of colour to also educating them about the impact their work will have on people. As one designer pointed out, designers need to be involved more in the strategic phase of an organisation’s work process. It is very hard to advocate for ethical design or real human-centred design from the position of a person who is not taken seriously and is only said to be good at cosmetics.

With service design we do see this shift but when we put service design into this same business context, it eats it up or transforms it into political correctness. Hopefully this will change, it is going to be a long process and it is going to look differently in different parts of the world. Realistically, as one of the designers argued, changing the system at its core is probably too optimistic and a rather utopian ideal.

I had two particularly interesting conversations with two designers who also have education in law. I targeted lawyers in other phases of my research as well because the connection between ethics and law is crucial and lawyers’ take on the topics of my research is very relevant, especially if they work in design. One of the lawyers-designers said that in future, there will be mixed majors, designers have to have a mixed background, for example if a service designer is to work in healthcare, it would really help them to understand how healthcare works, not just work as a consultant there but as a person with solid knowledge relevant to the field they are going into. The same goes for automotive. Having an expertise in both design and automotive can be crucial to really achieving systemic change in the automotive industry.

Following are extracts from my interviews which I believe are interesting to read in a “raw” format. They are direct quotes from each interview. I have anonymised them although in some of them, one can infer certain details (for example if the quote was taken from a conversation with a designer with law education or an illustrator).

114 MSc Design Thesis: Jana Voykova

Interview #1.

Who you are is more important than what you can bring to the table. I always start the presentation (when working on projects with other experts) not with what I did but with who I am. Because they can easily consider you are a fresher, or a junior and then the gravity of what you’re saying reduces. Our task is to first set the tone right. As service designers, our role is beyond just making things visually good…. we have a lot of things to change or to have a better impact in the world, but it always starts with a better communication, so people start listening.

Hierarchies definitely exist, in bigger companies…but I happened to be in circles where they were willing to listen. But again, if you are trying to say something and if you’re not portraying yourself well in front of the crowd, then nobody is going to listen to you. You have to be very specific and very confident and you have to appear that you know your game.

How can designers solidify their social position and responsibility when met with resistance?

Depends on the context and the group of people that you’re talking to. When they are corporate it’s always good to show them numbers. Resistance is always on couple of things: it’s either they really don’t like your idea or what you’re saying, or they don’t understand but they don’t want to admit it… so it’s important also to understand why there is resistance in the first place because there could be a possibility that you are also wrong. It’s okay to be wrong because you come from super social innovation background and maybe they come from a different background…money, money, money but then we have to see the common ground and work on it.

The phase of user research and doing field research, going through articles, plays a very important part because often times what happens in bigger companies sometimes, they do things because they just feel like doing it, they’re like yeah, this is cool, let’s do it. But there should be something to back it up, like there should be research which has been done which could possibly say if there is this need and then test it with users. Defining needs is a very tricky question because it changes with context. Sometimes in bigger companies you can easily see that the things are usually like you stop hearing the things which are not going to help you…User research is very important so that you base the needs on research. I’ve seen anthropologists do it (user research) very well, also people who have background from psychology, sciences related to that, if they also know the design thinking, they are able to do it better than anybody else because they have the previous studies which makes them more valuable when it comes to outputs…Sometimes it happens that you don’t focus on the need but come up with something which makes sense, like this very classic example of Henry Ford: “If you would have asked people what they needed, they would have said faster horses”.

115 MSc Design Thesis: Jana Voykova

Topics like the ethics of design or how to bring equality, how to design for the future, how to design sustainably, are amazing. The problem is these are always in the circle of design, so you are not involving all other people who could possibly benefit and change things but you are just making designers… like, there is a rage in designers to change but it’s just us, the people who are coming for discussions are just us, what’s the point? It’s not going to change anything drastically unless we bring everybody together in this. And I think there is lack of that, maybe I don’t know much about it, but from where I stand, whatever design events I go to, it’s just designers discussing. We know climate change is happening, we know there is tremendous rage between communities, we know immigration sucks, the process of it, but unless we are able to get that same kind of feeling and everybody comes on the same page, then it doesn’t makes sense to have such conferences.

Design thinking should be taught to every professional. It should not be limited to the elites who call themselves designers, but it should be simple thinking that makes sense. Then the focus is really about bringing value. I’m not against structures and processes and sprints, of course they are really good. But I think we are not taught to think, and thinking should be taught everywhere.

Interview #2.

Ideally, designers should be creating new solutions because they have the tools for it and the ability to think big. I strongly believe that you can use design to solve really big nasty problems of the world and I think designers should carry that responsibility to do that. In a broader sense designers’ responsibility is to create positive social impact and that can of course be in many different ways, my focus is women’s rights and gender equality but in a broader sense it is about positive social impact.

Designers’ responsibility comes in all phases of the design process but maybe most in the phase that you are gathering insights and taking into consideration all the possible views. I’ve been actually quite surprised in some of my work that I’ve been doing with NGOs for example. They don’t really, I don’t know if this is the case generally, they don’t really use science, there is so much research available, the topics have been researched for example in social science. They just say this feels good. They don’t base their work on research and it’s a shame because there is so much good information available.

I’m an entrepreneur at the moment because I haven’t found a way to really bring this wider perspective into a company so I think it can be limiting and after talking to many designers they say that they would like to have more impact in the world but they are not able to do in their positions. I don’t know what the solution for that would be, maybe the company should be more aware and somehow doing more good projects or the designers should actively promote and suggest projects and bring their own ideas on the table.

116 MSc Design Thesis: Jana Voykova

I don’t know if it’s hierarchy or there are always these commercial limitations, but you are supposed to then, you have this business project that you are working on and it has its limitations, timeline and how long you can actually spend on solving the problem. So, I think there is resistance, but I am optimistic that businesspeople would like to have more positive social impact, but the reality comes in the way, budgets are limiting. But I don’t think this should be stopping the work.

Is there anything about the profession that scares or irritates you?

The recent development that design is becoming a buzzword, companies and people are saying they are doing it, they are not actually doing it, and it’s not like there is a right way to do it but I think as a minimum it should be based on some kind of research or insight and then really go into the process somehow. Designers should be able to articulate the value of design clearly.

Somehow, we are still thinking too small. We should be able to somehow expand the horizon and I know that there are some really good companies globally but it’s like they are doing good projects in some developing countries, but not looking their own societies, there are so many social and environmental issues that would need definitely work. I think the challenge is to be courageous maybe but also how to find models how to finance the work, it always comes down to money, and how do you make sustainable models to focus on doing good which is not always generating money.

Interview #3.

Working as a UX designer, I’d be more careful when defining my role as socially oriented, I think there are other types of design that are more socially important. My work is different, when we say social responsibility, I envision IDEO’s projects, this type of design.

Designers’ social responsibility comes already in the discovery phase of the project they work on, when they find out what they are doing, why they are doing it and the scale of it, your responsibility should come forward, otherwise you don’t know what you are doing, why and for whom and sometimes if you realise how big a thing you’re designing, you might get scared.

As far as designers can influence how society evolves, I believe this has to do with behavioural change, because of the fact that the product you are designing, affects behaviour, in this sense they do. I sometimes wonder if there is a way that a digital product could affect the behaviour of patients with psychic instability, I have searched for this type of products, but I haven’t found any. On behavioural level, design can influence how society evolves but in idealistic terms or more ambitious aims...not that behaviour change is not ambitious, small steps can lead to something bigger.

117 MSc Design Thesis: Jana Voykova

If I faced a situation where the project that I work on is taking an unethical direction, I’m not sure how I’d react, I hope I will leave the job. On the other hand, if designers have the power to influence or change this, things need to change on a system level, the model of functioning of the whole society needs to change. It’s a matter of utopia maybe.

It’s a constant battle between designers and business when it comes to addressing needs, the stakeholder makes the decision, it’s not the designer, it again boils down to the model of operation of businesses which is generating profit.

Interview #4.

Technological progress and the ubiquitous access to information are what enables designers to drive social change more easily. If they create a new solution to an existing problem and they share it with their network, other people take up and develop the idea further. People with good ideas will be most valued in the future, the development of technology will enable designers to do things more quickly with less resources, it already has but this tendency will continue… As technologies develop, we develop mentally along with them.

When designers meet resistance to their expertise, they should first talk to their superiors and communicate to them where they are coming from and why, if this doesn’t change things, then they should naturally look for employment elsewhere. If they are not in a position to do this and circumstances don’t allow them to quit (if they have children, loans, personal contingencies), they will have to accept the situation and not complain about it.

One’s principles are one’s boss. You just have to create your own rules and principles and when a client wants something that goes against those principles, the designer is in a position to refuse to go further. Society (ordinary people, users) determines what needs there are but corporations are the ones who realise potential solutions to address those needs.

118 MSc Design Thesis: Jana Voykova

Interview #5.

I like UX and web design because I like the web as a space, it allows you to connect to people who are at the other end of the world and exchange ideas with them. I like coding, I like writing code because it’s a very logical task.

The social responsibility of designers pertains to everything they do, be it illustration, creating interfaces, designing user experience design. Designers are first and foremost humans, and we all bear a social responsibility. We don’t live in a vacuum or another planet, we are not unicorns or whatever it was they called us.

When I am designing an interface, I sometimes have to choose between two options: design it the right way, this doesn’t happen often, but I’ve had such situations, or design it so as to deceive the user. For example, ads in mobile applications. Let’s say the ad is a video and the user wants to close the ad, there is a close button there with a dialog window that says Close video or Resume. Here, users are tricked visually, because normally Resume video is a blue button which the brain perceives as something they should press. However, if they want to close the video, the Close button is a typographic button with a tiny, almost invisible typeface…I had a situation where I was working on an ad campaign for a start-up application and I was asked to design the ad Close button and the subscription conditions with a really tiny and almost invisible typeface.

Who sets these requirements?

It’s a matter of intentions, whether one would like to deceive the user or not. As long as responsibility is concerned, it is the designer, the developer and the product owner who bear responsibility. It’s a matter of luck whether one would work in a team where everyone says no, this is not right, and we will not do it like that. But it should be clear that this will affect the conversion rate…The product owner owns the decision. I confronted them about it and I’m not sure if you want to hear the answer. The answer was that the purpose was not that the users would read the text.

What was the purpose of the text?

The purpose of the text is to inform the user about the conditions of the product. I perceive myself as an advocate of the user so I went on and increased the type size, nobody noticed….I think that sometimes in such situations they (decision-makers) test how far one would go…Start- ups generally adopt such practices from Silicon Valley, observe and copy practices from companies that have heavy user databases like Facebook. They don’t have a policy, for them, the driving force is, sadly, to generate revenue in order to satisfy their investors.

119 MSc Design Thesis: Jana Voykova

In my experience as a designer, it’s always been a fierce battle to advocate for ethical design.

It’s strange how, in the beginning when launching a product, designers are very important. However, as income starts coming in, the designer is gradually pushed to the periphery. If I am to talk from my experience, designers are just hands, there is a stereotype in society that designers make pretty things. I don’t know how this will change, designers need to unite their forces, there should be a designer’s union or something like that which can protect designers. It’s very idealistic, I don’t know. For me personally, the solution is to quit if something is not done right. Not every designer can afford to work at an in-house agency where they have the final say.

Interview #6.

Social responsibility is huge, when you design anything, you are giving the user tools to live, a way to behave in every single aspect of their life... Everything related to design, is related to ethics. In my field, I am not only working with service design, I am also working with legal design, and that’s really specific about ethics, because I am touching legal touchpoints. If I design a legal touchpoint, only thinking about one of the parties of the experience, for instance in a legal contract of a bank, if I only have in mind the bank but I don’t have in mind the user and I design it thinking about the behaviour of the user is going to be to sign it because I design it with that purpose, you’ve seen the proper words, to push him, to nudge him to sign it, the ethics here is really, really problematic. So, when I do the interviews to hire people in my company, I think that component is really, really relevant. Because I can teach how to do service design, I can teach how to do legal design, I can teach how to do design, but the ethic part, that’s part of you, when you were a child, I think. That’s part of being a human being. I need to have people in my team that really know where the impact of every single touchpoint is, they are building for the final user and for the corporations that are trying to design better services. If I feel that the touchpoints that I am building are going to change the behaviour of the user against their will, I am not going to do it and I have stopped projects because of this.

My country is a really conservative context for design. If I’m not talking from my entrepreneurial perspective, if I’m talking from their perspective, from the corporation’s perspective, some of my clients are really Avant- guard. When I talk as a designer, they are like Oh my Gosh, the designer is saying something, let’s hear them. Actually, in finance industry they are like oh, let’s hear them, they know stuff. But in the legal industry, lawyers, they are really driven by the competition. Right now, I’m going to a legal firm and I think they hire me because they want to be the first ones, not because they really believe in the final outcome. I’m trying to change that but it’s hard if I’m not there with them to change it.

Of course designers influence how society evolves, we are doing it every single day of our lives, I think that every single project or, not even a project, we don’t need a project to change the evolution of the planet or life, I think that when we talk in a political conversation, we are already

120 MSc Design Thesis: Jana Voykova doing it. Designers have the mindset and the tools to change things. Ethics are important for life but what if we just talk about ethics and we do nothing about ethics. But as a designer, you can do a lot. A lot.

Who determines what are the needs of society and how they should be addressed?

I think this is a really interesting question and I know where you’re going with it. I think sometimes designers could decide that, sometimes we can do this. For example, if we say, you really need this kimono, it’s blue, it goes well with your eyes…Sometimes, we could be unethical with that approach, but I think the real one that should design what they really need, should be the user. But the user does not always know what they need. So sometimes the designer could identify the need, but the ethical part is really crucial. We need to design things that people really need, not only things we think they need.

Is there anything that scares you in the profession?

There are people who just read a book or take a two-week class, and now they are design thinkers, and that’s scary, that’s so unprofessional. They go and lead a workshop and then they just deliver an excel with a lot of stupid ideas that are not relevant, and the company does not have the ability to convert these ideas to real life and at the end when a real designer goes there, they are like: no, no, we don’t believe in that because some design thinker came here and did our post-its. No, he was no designer. I think that’s huge in the industry because it could be a wave, a fashion wave. Actually, it happened with design thinking a lot. When I came to companies, they said, you should say that you are an innovation consultant. I decided not to, I decided to work with service design and stick with that. I think that design thinking shouldn’t be a method, design thinking is a mindset.

One of the things that I’d love to do, to be involved in law design, how I could be part of the designing of the law process, because at the end who is going to be the user of the law? The citizen. But the first thing that I do a lot is to design the explanation of the law, but it’s like the final step, I want to be part of the before step. Like, actually, do we need that law? Like there is a lot. A lot. And when they do it, I think they do it because they want to get a salary. But they don’t really understand what’s the impact, they don’t have the system thinking that the designer has…in my company now we have a lab and we are sitting down and having a conversation about how we can prototype a law before the law is launched. For example, there is a law that says that you cannot put a ticket on somebody here that is driving a car or a motorcycle unless you can see who is driving in that motorcycle. So, if somebody kills somebody in the motorcycle, and you cannot know who the driver is but you have the registration plate of the motorcycle, but the guy has a helmet. So, of course you’re never going to know who is driving. Because there is another law that says that you should have a helmet. So, there are two laws that go against each

121 MSc Design Thesis: Jana Voykova other. But they don’t know because the lawyer who said that law, doesn’t have a motorcycle, and he doesn’t use a helmet.

I think that from my perspective, being a lawyer designer has been really useful and has given me a lot of advantage. I think that the future design will be mixed majors, you are a designer but a designer of policy, designer of health…we need to understand the structure that we are going into.

Interview #7.

Being a designer is connected with bearing a social responsibility in different measure, depending on the essence of one’s work. Not everyone would engage in socially important causes, I think that designers do have the opportunity to bear a social responsibility, however, it is a matter of choice whether they do. I personally am trying to be mindful of what is happening around me and inform myself on what the adequate solution is for each project I take on and how the people who are going to use my product, are going to be affected.

Why do you think some designers choose not to demonstrate their responsibility?

It’s hard for a visual designer to demonstrate a social responsibility, unlike a service designer for example. We all know that we won’t save the world with one poster. Maybe designers don’t want to take themselves too seriously.

People have strong opinions about design because it is visual art. Everyone has their taste and opinion and the right to them, be it good or bad. People have opinions they like to express, when they see something, they first perceive it visually and judge if it looks nice, rather than think about how it works.

Designers can influence the way societies evolve in different ways depending on the profile they have. As far as illustration is concerned, there are many different types of illustrations. The first type of illustrations people come across is the ones from children’s books. An illustrator designing children’s books can choose to focus on topics that have not been presented to children before, for example emotional intelligence. They can represent them in a way that is easy for children to understand. If, however an illustrator works on illustrations for mobile applications and web, then they probably don’t have the same ability to make decisions and influence their work. There is a limit they will eventually reach and then they’d need to collaborate with people from another design discipline or another area of expertise, for example healthcare. When several professions interact with one another, the impact is much bigger and meaningful.

Interview #8.

122 MSc Design Thesis: Jana Voykova

Design is a combination of functionality and emotion. The visual part of the communication in design is half of the truth, if it doesn’t work, it’s not design, it’s just a pretty object. These two aspects of design are part of my thinking and the way I function in life in general, for example the way I arrange ingredients when cooking. They are arranged beautifully but simultaneously in such an order so as to make the process run as smoothly as possible. I think this is one of the best things in our profession, we combine pleasure with utility.

When I read that (meaning designers’ social responsibility), the first thing that came to my mind was couple of prints that I saw recently which were advertising cigarettes and which made me angry. I understood what they communicated, why they was there, what their impact was going to be and what business need drove them in the first place, but I was irritated because I knew they were going to influence people….People may not realise the way it affects them but they will definitely feel it on a subconscious level. The image backed up by words, channels the emotion in a way, it’s very selective, it’s as if it almost represents a next level of evolution in one’s existence, while this product kills. This was one of the things that came to my mind when I read social responsibility. It’s a matter of personal choice, being a designer, one chooses whether to work on projects they don’t believe in and know are not good for people, such as cigarettes, alcohol, gaming, even social media.

We need to be aware of the impact our profession has on people. Design is not at all the most important profession, what designers do is to a great extent a form of optimisation, giving a product or a business an additional edge. We should be aware that we primarily work with people’s intuition, this can be interpreted as a form of manipulation or as nudging. The problem is that a lot of those decisions are taken on a business level. If a designer works for a business whose focus is to exploit and manipulate, they have to accept that, as a professional, what’s required from you is to do what’s necessary for the business to be successful. At the end of the day, there are a lot of things in life which are in the grey area, from a moral perspective, but we live in a free society and so people have to make this choice on their own.

The most satisfying aspect of my profession is to create something that people use, that is aesthetically pleasing and gives people satisfaction and confidence. Satisfaction comes from the fact that one’s work is meaningful and gave meaning to another person.

Our work is not individual, we always work in collaboration with other people and support their work. What matters for each of us, is for us to be open to the world and seek to understand how it works, because then, using our designerly way of thinking and approach to solve problems which haven’t been resolved yet or address solutions that are not effective. This happens when we get out of the things that occupy our minds daily, like what palette to use, at least in visual design. As far as service design is concerned, I don’t have sufficient knowledge about what parameters apply there, at the end of the day, the market and the end users decide whether

123 MSc Design Thesis: Jana Voykova something will succeed. You might have invested 300 hours in it, and it might still have been doomed to failure from the very beginning.

The beauty and the danger of the free society is that individuals have to make the choice themselves. A lot of the techniques for manipulation and influence are magnetic, for example social media and mobile games. As designers, we make the choice in the very beginning, if we work in the gaming industry, we cannot be surprised about the principles and techniques applied after, when a project is assigned to us.

Interview #9.

I think it’s more human than in a career. Last year people were really angry about some aspects of the government and we had a series of marches, we got out and walked with our signs, and screaming, we wanted to have access to health, not only me because I am very privileged, I have my savings, I studied, my father gave me this opportunity to have everything and I am very privileged in my life. But other people like farmers and people who live in rural areas far from the cities, they don’t have access to health or insurance or even security. The aspect of social is in the human, not in the career, it affects your life, the health you can access, the food you can buy, the taxes you have to pay. But in design I think it’s more important because in theory, in the academic books, in design thinking methods, in service design methods, you always talk about persons, humans so I think it’s more important. I think it’s more related to people, in design you talk about designing for others, so I think others is not only make chairs or services, you have to design their future. You can make a change from the area you know, that is design. You need to work for a living, and you need a salary. Sometimes this salary and this work is from the industry that not always is taking care of the people. You can always dedicate 20% of your time to make it social, or to think about how you can make it possible to give something to the society…You can find two types of designers: you can find social designers, only focused on helping people and graphic designers, who work in branding studios, who work in agencies but the social impact is not mixed. I don’t know why exactly but I think it’s because you have to earn money and sometimes social is not perceived as you can earn from….I think even universities and academy separate the design industry and social design, I think that’s the problem with social design because all the industry separate it. Now I think many industries try to mix it, for example Scandinavia. For example, Scandinavian design and sustainability try to stay in the same industry. Because we have to take care of people and resources, so we need to understand the project with different perspective, not only the industry, or money, we have to understand the social aspects of the project.

Social design is behaviour, it’s how you act around a context and change around the context, even change the context with your behaviour.

124 MSc Design Thesis: Jana Voykova

I think we can make it (influence how society evolves), we can change but in our academic environment, in universities, they don’t teach you, you can make it, you can change. For example, physicians and doctors, nurses, they always know they have a social impact, but we don’t really know about it, I think this is a university and academic problem. I don’t even know I have that power to change something, I think it was I can only make beautiful things, products, create a blueprint for a service but I don’t even think that I have a great power to change. Some professions make it clearer to their students and workers. I think it’s because of the environment of design, it’s always thinking about other things, not social. I think we have the power, but we don’t know it.

I don’t think politicians, or the governments are thinking, oh how can designers help us. They are thinking about how other politicians or economists or physicians can help, but they are not thinking about design. I think that’s the problem, when something happens in the context where you have to include other areas of knowledge, you don’t think in design.

Yes, I think the industry is now realising that design can change things, they are taking decisions in the places where the consumer is, I think the industry is realising they need designers because designers understand the environment in a different way and it’s a really good thing to happen, I think it’s going to take time to make it normal, all the industries have to have a designer, for example in medical centres or hospitals you don’t see a designer, you only see a graphic designer to make advertising or some images for the hospital but they don’t think about the solution. You see designers making products or images, so I think the problem is you don’t have the access to all industries.

Of course, the government determines needs in society, but I think again, business clients think behind the best, they don’t think about other needs, not the real needs, or what causes the real need, what’s behind the need. It’s not only that people don’t have access to health, it’s why. They don’t think in the 360 aspect of the need. Design is really good about this; design can make it possible to understand the 360 view of the problem. So, I think the governments see the needs because of the numbers and data, but they don’t know, you can see the data, but you can’t see the why behind the data. This is one of the biggest problems of governments because they don’t see why something is happening and go to the root of the problem. It’s like extinguishing a fire, you see the need, but you have to see what caused the fire. You can see it more like a system. It’s not just one need but a system, if you change something, the system can organise to fix the need.

Are designers involved enough in this process of finding and addressing needs?

No, in my context, I think it’s going to take time to be, as designers, more involved in these processes to understand the needs…In my dreams I’d like to have a government who includes service designers and behaviour economists around the decisions. I know they have economical

125 MSc Design Thesis: Jana Voykova resources, they have anthropologists, sociologists, phycologists but I don’t think they have designers. In the future I think this is going to happen. Maybe in other countries, Scandinavia is really good example, IDEO have this: service designers, experience designers, even graphic designers who have different mindset, and start to work in different industries, inside the industries, not as consultants.

What scares me is that people don’t value our work. They don’t see our effort around creating something. Sometimes I think that people see our work as not that difficult. This is one of the things that scares me because it’s difficult to change this mindset of people who think design is easy or design is not that hard, or you don’t have to pay a lot for design…We have to educate people about our profession. We have this perception of what lawyers do, what physicians do, what chefs do but designers…it is not really, clear. It’s not really transparent. People have this view of designers making products and beautiful things, when we have a brief, people say: “make it beautiful”, no, I have to think what I need to communicate, it’s not making it beautiful, beautiful for whom? For what? Who is going to use it? So, I think it’s one of the biggest problems about the perception of our work.

Interview #10.

I always had a thing for aesthetics I would say. I always liked nice-looking things already as a child and then when I got older, I guess at some point I started to realise that you can do crazy things with designing. At some point something changed, and I thought oh, actually design is not just a thing, it’s a tool, it’s something you can use to do something else with. I thought that was really a cool thing, so I started to look into this, back then it was pretty much a niche, using design actually for a purpose and that’s how I got into it, I guess. First of all, I do think that designers have a social responsibility but not because they are designers, I think everyone has a social responsibility, basically everyone who has the power to create something. In my opinion designers are creators, I think creators is sometimes a better title, I may not be a designer, but I am a creator of things. And I think as long as you make something that is considered to be used by others, I think you do have a responsibility.

I got really into design thinking which is based on a coincidence because I stumbled upon it on a conference and then I found so many things of service design in it and I was like, oh, this is interesting so I looked at it further and then I think it shaped my thinking of design processes a lot and now I am actually thinking that the responsibility starts with where you are trying to find the problem that you want to solve. I don’t design anything that doesn’t serve a need or doesn’t fix a problem. I think it definitely starts at the very beginning where you actually identify a need, that’s where the responsibility comes in because you are responsible to find the right need.

No, I do think that is dependent on many things such as capitalism, probably. If it was up to me, I would offer everything for free all the time just to help people, but this is not how I can pay my

126 MSc Design Thesis: Jana Voykova rent, so I don’t. But I think you always have to find balance between social responsibility and the circumstances that you put the design in. It is dependent on economics, for sure, financial stuff, sometimes you are tied to contracts, monetary budgets. So, I think as a designer you should definitely start with the social responsibility, but I guess it will be cut here and there. For example, in my job I also do educate clients, so if a client is not aware of some things, I have to do to be socially responsible, like for example let’s say human centricity, that’s a part of the social responsibility I think, and the client doesn’t want that because they don’t budget for that, then I educate them and actually say, ok, if you want this to be a sustainable thing that lasts for a while, you have to do this….I guess it depends on how seriously you take this and I do take it very seriously so I think part of my job is to also educate others about this responsibility.

How can designers advocate for ethical design?

I do think talking about it is a big deal, not neglecting it, making it something that people talk about. Like I said, I talk about it with my clients, I do tell them about it. Most of the time they are curious and listen, but that’s also because I work in a field that’s not much profit driven. I don’t have many corporate clients. But I can imagine that sometimes people want to cut it short. At the same time, I think that you should always, I like the sentence: To lead by example, so you actually show people that it can be done, that inspires people to do the same. I think it works the best that way because then you actually don’t have to educate them by telling them, they would just see and realise that it works. But you can actually make a difference if you just push through and stand by your values. I guess it’s a tough job to stay with your values, I’m not perfect with that either but if you make it a priority then I think it’s always possible in some way, it may be limited but it’s possible in some way.

Do you think that designers can steer how societies evolve?

I guess they should. And I guess they do. At the same time, it also depends a lot on the people who are in power, meaning the people who have money. Because obviously you need people to buy your designs and it will be the people who can actually afford that….I think they do in some ways, for example, at some point someone decided that there should be toilets for men and women so now there are toilets for men and women everywhere. I guess that person made a big decision about society. That’s a design decision that’s been made.

What gives you satisfaction in your work?

Actually, two things; the process of doing it but also when it’s successful at the end. When something turns out to be very useful, I find that very satisfying, to know that I have helped creating something that somebody finds very useful, I find that very cool.

127 MSc Design Thesis: Jana Voykova

I believe that a very good designer is someone who is able to change perspective. I think something that should be brought forward more is accessibility. I think a lot of things are designed for many people but then only a few can actually use it or have it accessible to them. For example, also a Corona virus example, schools are closed right now, and teachers started to do stuff online. Well, not every kid has a computer, not every family is able to afford a computer or use a computer in that way. That’s to me an accessibility problem and I think it happens a lot, with many things because people are not able to shift perspective when they design it. I think that happens a lot and I think that’s something that should be put more attention on.

How do you think the design profession will evolve in the future?

I hope actually that it will become more and more human and planet-centred actually. I would like for design to take more into account further than the people. I don’t know if that’s going to happen, but I think it must be a logical consequence to some extent. I guess it’s difficult and I think it evolves very fast and very slow at the same time. A lot of minor things happen so quickly but at the same time, it’s so slow. I do think that designers are more seen now, I do think that people get more what a designer is and what they do…Everyone is creative, everyone can do whatever, they just think they’re not. People grow up thinking I wasn’t born creative, that’s bullshit, I went to school and learned how to be creative, that’s it. I wasn’t born creative either. It’s something you can train and get better at.

6.1.2 Survey analysis

The survey which I shared with designers in order to research how they viewed their profession in terms of the social and ethical aspects of it, yielded very similar results to the insights I gathered during my interviews. 61 designers took participation in the survey, that is why the data gathered should be treated as a sample, not as a definitive and truthful representation of the opinions of the global design community.

The questions I asked in the survey were similar to those I asked in my interviews. They focused on whether designers perceived their profession as socially responsible, if they thought that designers can steer the evolution of society, what their take on the relationship between ethics and business (economic growth) is, if ethics is necessary in design education and what challenges they deemed worthy of designers’ attention today. The respondents represented a range of locations — mainly in Europe (UK, Sweden, Bulgaria, Germany, France, Finland, Spain, Portugal), but also Canada and US, and individual respondents from Egypt, Belgium, Wales, Scotland, Czech Republic, Switzerland, Macedonia, Turkey, India, Indonesia. The majority of the participants are female, 66% and 34% male, in the ages

128 MSc Design Thesis: Jana Voykova between 22 and 51 years old. 95% of them defined the design profession as socially responsible and designers as bearing responsibility towards the society.

Interestingly, one designer was of the opinion that designers do not bear a social responsibility, another wrote that it depends on the power they are given while a third responded that it’s 50/50, which I interpret as depending on the situation. The majority of the designers, 93%, answered that designers do have the power to influence how societies evolve, but 61% indicated that even so, they don’t have a deciding voice when it comes to making decisions pertaining to the needs of society. These results support my findings from the interviews, as well. One designer argued that governments have the power to steer society and designers don’t generally create the content but the visuals.

Two other people also shared the opinion that designers don’t have the power to steer the evolution of society. When asked about who determines what needs there are in society and how they should be addressed, a good majority, 74% attributed this decision to the private sector. Fairly high percentages (above 50%) were given to public sector, academic and scientific communities and ordinary citizens while 24 people indicated that all of the proposed groups (private, public sector, citizens and academic research community) define society’s needs together.

As already mentioned, the majority of the respondents, 61% indicated that designers were not involved enough in the process of defining needs in society and did not have a deciding voice. This result coincides with what I found during my interviews. Most of the designers I interviewed did not think that other professionals and people in society perceive them as experts with a reliable and relevant opinion and skills, other than cosmetic ones. Three people answered that, compared to previous times, designers today are listened to more and some companies do value designers more, but designers still don’t have a deciding voice in most situations.

When asked about what kind of projects they would choose to engage in if they had the freedom to make the choice (profit-driven, socially important, both or neither), 62% answered they would get involved both in projects which are focused on generating profit and such that address social causes. One person answered that they would ideally engage in lucrative projects which addressed social causes. Another interesting answer to this question was that, unfortunately, most lucrative projects are mostly exploitative and if we all were willing to invest more money and effort into people’s well-being rather than exploit them, this question wouldn’t be relevant.

Surprisingly for me, a good majority — 72% of the respondents didn’t see business (economic growth) and ethical behaviour as mutually exclusive, while 13% did think they

129 MSc Design Thesis: Jana Voykova were not compatible. One person answered that ethics and economic growth are not necessarily mutually exclusive; however, some companies are just picturing themselves as ethical and mindful of sustainable behaviour just to appeal to consumers (what I identified as greenwashing earlier in my thesis).

Interestingly, one person thought that economic growth and ethical behaviour are not mutually exclusive, in the sense that growth could happen ethically. Nevertheless, he/she saw profit-driven behaviour and ethical behaviour as incompatible. These results back up my findings from the interviews and circle back to the discussion from my theoretical background on business ethics. In theory, economic growth and ethical behaviour should have no problem to co-exist and be successful alongside each other. In practice though, it often seems that we cannot reconcile the two and upholding the principles of the one usually harms the other.

When asked how they would react if they found out that the company/project they worked for, was involved in ethically dubious behaviour, 69% of the participants indicated that they would confront their employer while 33% would quit the job. Some of the respondents answered that they would weigh the pros and cons of the situation and not take actions if they didn’t have a backup option for getting another job, for example. One person answered that in a perfect world, it would be great if everyone was 100% ethical 100% of the time, but in reality, when people have contingencies, this is not always possible.

Another respondent also gave an interesting interpretation of this hypothetical situation. They wrote that if the behaviour is illegal, they would turn to the authorities; if it’s related to intellectual property infringement, they would confront their superiors and seek legal advice, if the work is legal but they (the designer) don’t personally agree with it, they would still do it. Ironically, one person indicated that they wouldn’t do anything in this situation. The same person didn’t think that ethics should be included in design education. I would be curious to know what the motivations behind these answers are, but I should say that I admire him/her for being upfront. Sometimes we tend to hide our real opinion because it might not be accepted well publicly, even in surveys which are anonymous.

90% of the respondents supported the claim that ethics should be part of design education while four people didn’t think that was necessary. For one person this was dependent on whether the work involved ethical dilemmas and another person answered that ethics could be defined in numerous ways and was open for interpretation, so we have to agree on what kind of ethics we mean. This is to an extent true, as previously stated, ethics is about the so- called grey area questions in life and there is seldom a yes/no answer to moral quandaries. Still, I firmly believe that ethics should make it to the design curriculum and more

130 MSc Design Thesis: Jana Voykova importantly to the study plans of the STEM (science, technology, engineering, mathematics) programmes.

The last question I asked was what challenge/aspect/phenomenon in society was worthy of designers’ attention today according to the participants in the survey. Human rights, education, privacy, healthcare, inequality, sustainability, poverty were popular answers. Other interesting suggestions were food disorders and the lack of unbiased data in research that leads to biased designs. Among the other answers to this question were addiction, fake news, automation and the less need for human labour, planned obsolescence, urban life, urbanisation, bridging the gap between profit and ethics, healthcare, and changing people’s perception of what designers do and what problems they could help resolve.

To summarise my findings from the interviews and the survey, and answer my first research question, it seems that designers generally know how much social responsibility their profession bears and realise the consequences of their work for people. Still, external factors like their employment relationships, their personal dependencies, the way the profession is perceived in society, the goals of the projects they normally work on, restrict the extent to which they can justify and demonstrate their social responsibility.

In order for them to overcome those limitations, they need to make a personal choice about what companies/projects/teams they partner with. They should also educate people about the impact of their work and try to stand up for the potential users of their designs by communicating their concerns and understanding why a design is being designed to be potentially harmful, if it is.

A lot of designers don’t see economic growth as incompatible with ethical behaviour, but they do see that generating profit and doing social good are, to a great degree, on opposing sides and social design is still not attractive for companies because it doesn’t provide revenue. Although the academic community can try to educate designers to better grasp the social and ethical aspects of the profession, the limitations explained above pertain mainly to the greater social context designers create in. Therefore, in order to achieve a radical and systemic change, we need to alter processes, practices and assumptions about ethical behaviour and social design on a wider societal level. This is namely what Devon and Poel’s ethics paradigm is about — changing the social code to enable ethical design more effectively and translate individual ethics into collective ethics. Designers do see that this process is already underway to some extent, however, there is a risk that the so called sustainability and ethical awareness is sometimes just a form of polishing one’s image and therefore, we need to analyse such endeavours carefully and critically.

131 MSc Design Thesis: Jana Voykova

Figure 11. This infographic summarises the outcomes of my survey on the social importance of design.

132 MSc Design Thesis: Jana Voykova

6.2 Research question #2: social and ethical implications of autonomous transportation

With my second research question, I was trying to find an answer to how citizens perceive the social impact of autonomous vehicles and how design can aid the public understanding of important social phenomena. I will first discuss the insights I collected from my interviews and then look at the results from my survey. Then I will provide a summary of the insights from my prototype exhibition in which I used speculative and critical design to raise the public awareness to a possible future outcome of technological progress which depicts a dystopian scenario. This will ultimately help me to find out if/how design can aid people’s understanding of social events which are not discussed openly enough. I will conclude this chapter by offering a discussion on the yielded outcomes from both research questions, thus trying to find a common ground between them.

My assumption regarding my second research question was that disruptive technology (autonomous transportation) is not the universal remedy for all problems in modern cities and it bears shortcomings which are not paid enough attention to in the social space. This might be presenting a skewed version of the technology and its impact for society which, in its turn, misleads people to believe that the benefits of new technology always outweigh the risks. Consequently, people disengage from this conversation because they accept technology as a pre-determined future which they cannot and don’t want to influence in any way as there is no better alternative.

6.2.1 Interviews analysis

My interviews (I prefer to call them conversations) with people about the social impact of autonomous vehicles were just as fascinating as my conversations with designers on the social role of their profession. As noted earlier, I only interviewed 10 people, the majority of whom are Bulgarians. This means that the gathered information should be rather looked at as a qualitative sample. All of the interviewees were interested in the topic in one way or another (except for one), however, some of them were more knowledgeable because they either work in the industry or their job is somehow connected with the development of the technology. I asked them questions which can be grouped in two categories: 1) questions which have to do with their personal interest in the topic; 2) questions which reveal their opinion on different aspects of the potential social effects of the technology, like labour force, the collaboration between governments and private players in the development of the

133 MSc Design Thesis: Jana Voykova technology, its ethical and legal implications. I also presented them with some previous research done in the field to see how they interpret the results in the shadow of their personal opinions.

Out of 10 people, only one was explicitly against the technology but not in the sense that if she could, she would prevent its development. Instead, her concern was that the technology could potentially eliminate traditional driving at some point and leave no choice to people like her who actually like to drive. Another person was also unsupportive of the technology because he likes driving and stated that if he wanted just transportation, he would ride the bus. However, both of them believe that the technology can be developed in a way that takes into account the risks around it and they don’t mind it being developed for people who need/want it, as the potential positive effects are significant. All of the interviewees said that people don’t know enough about the technology in order to form a reliable opinion on it, a lot of them said that people don’t have relevant information on any topic whatsoever and especially when it comes to technology, people normally base their opinions on negative or positive bias from the press.

This is the reason they identified for the negative results I presented them with from a survey done in the US in 2018 where 55% of the surveyed didn’t think autonomous vehicles would minimise the percentage of road fatalities. Two people said that even those who work in the industry sometimes don’t know for certain what will happen because the technology is still new and under development and it is difficult to predict how exactly it will evolve. Another person said that when it comes to autonomous vehicles, people’s opinion is usually determined by whether they like to drive or not which makes sense although other participants, who are car enthusiasts, are surprisingly supportive of the technology. She made an interesting observation that people don’t think about bettering of society, rather they base their perspective on egoistic factors, nobody is that engaged in saving the world from hunger or poverty or thinks about grandiose social plans so improvement of society as a whole isn’t determinative for their vote.

This is an interesting observation and I’d argue that in a lot of instances it’s true. However, I do believe that there are people who do base their preference and opinion on the bettering of the society as a whole. She noted that the information pollution today is enormous and beyond the ability of a human to comprehend it and judge if it’s trustworthy or not, even reliable sources present information in a biased way. On that note, the majority of the people I interviewed said that many people don’t trust the technology because it is unknown, it hasn’t proven its positive impact and ability to perform better than human drivers yet. This was something that Korbaczynski also mentioned in his analysis of the technology as an obstacle for its successful adoption.

134 MSc Design Thesis: Jana Voykova

One participant in the interviews said that sceptics who are sceptics from the get go will probably not be convinced even when the technology has proven its ability to perform better than humans, this is how the human brain works, if a person doesn’t believe in something, it doesn’t really matter what evidence the proponents of the technology will present them with. This might be true in some degree, however, I do think even sceptics can change their mind, otherwise, it would be meaningless for people to try and have a conversation about something with people of the opposite opinion, which isn’t the case in most situations, that’s the sole purpose of having a productive and meaningful conversation. Another person mentioned the so-called backfire effect. This essentially means that if a person is negatively biased towards something and is presented with data which disproves their opinion, instead of convincing them, it has the opposite effect — they become even more prejudiced.

One person was of the opinion that even when the technology has proven itself and has performed error-free for 10 years, except for one time, people would still pick up on this one negative case and this will sway their opinion. The majority of the people I talked to said that usually, media doesn’t portray the technology in an objective manner, press focuses on the negative aspects of it and presents biased data in order to make sensations and grow their audience. On the other hand, some participants said that even the companies who develop the technology are very selective with the information and details they disclose, either because they want to keep their competitive edge or because they are themselves not sure if they are on the right track since this is something that has not been done before.

One interviewee rightly pointed out that none of the relevant questions around the technology is being brought up in the social space, like the impact on safety, traffic control, moral and legal aspects in case of an accident, security, privacy. Most of the people saw this as normal since some of these questions, being controversial, are bad publicity and marketing doesn’t work like that. As one of them noted, in order to sell, you don’t emphasise the grey areas or vulnerable sides of a product.

However, even the more sceptic people believed that the companies which are working on the technology, do take these sensitive aspects into serious consideration and will make an effort to design the technology to be as safe as possible and test it well before they launch it on the market. In terms of benefits, the majority of the interviewees saw better road safety and the ability of the technology to perform better than a human as a clear advantage, especially for countries like Bulgaria where “everybody drives as they please”, as one person put it. One person said that for companies, the motivation to invest in it is clear — a new revenue source, however, he pointed out that this doesn’t make the technology less valuable and useful.

135 MSc Design Thesis: Jana Voykova

Another person saw a great benefit in the technology allowing him to release control on long distance driving. He commutes to work every day and he said he would like to be able to use the time doing something else instead. This was a factor which four other interviewees also mentioned, however, one of them did say that this time can also be used unproductively.

According to one person, the technology would bring huge benefits in financial and environmental terms. She saw the adoption of autonomous traffic as a way for companies to reduce labour cost in truck transport. She also said that there is a huge shortage in truck drivers all over Europe (that is also relevant in the US), and it seems that in the future, even fewer people would want to become truck drivers so this could potentially address this pain point. She also mentioned the optimisation of traffic which would reduce emissions in the sense that currently a lot of trucks drive half-empty. Autonomous cars being connected, could “communicate” in real-time and optimise their routes. The connectivity between cars as a way to optimise traffic was mentioned by four other people too. One of the interviewees mentioned the potential benefit for disabled groups in the society.

Two people pointed out energy efficiency as a consequence of more optimised energy- consumption processes in autonomous cars. Two people noted that owning a car is becoming more and more expensive and car sharing, and subsequently automated vehicles are a natural next step in the evolution of transportation. As far as risks with the technology go, several people said that the technology is still in its infancy and cannot yet perform as currently advertised, they said that ad-hoc events are something the technology is still yet to be trained on. Two people identified the negative impact on the labour market as a risk, but they saw it as a short-term issue as the people whose jobs get obsolesced because of the technology, will re-skill to other jobs. One person expressed the view that technology (in general) does defect sometimes, and this is one of its downsides. One of the respondents to my survey gave the same answer. He wrote that his experience working as an IT specialist shows that sometimes technology breaks out of nowhere for no apparent reason so it shouldn’t be over-trusted to always perform better than humans.

Eight people mentioned security vulnerabilities as a risk which needs to be taken seriously. One of them saw the security aspect as much more productive to focus on than ethical and moral dilemmas. Two people (whose job is connected to the development of autonomous vehicles) stated that in case of an emergency or a situation where there is a predicament and the technology “gets stuck” for some reason, it is being developed to shut down and signal to the rest of the cars in the fleet that there is a defect in the system, thus mitigating the risk in real-time. Three people identified posing the ethical and moral considerations as whom to sacrifice as inherently wrong, ridiculous or creepy. This question shouldn’t exist, they posited.

136 MSc Design Thesis: Jana Voykova

According to them, we should instead design the system to be as secure and hack-resilient as possible. Most of the interviewees didn’t think that we could eliminate accidents 100% but they did say that AI has much quicker and rational reactions so the result will always be better if machines drive rather than humans. Three participants expressed their unease and lack of expertise to answer the moral/ethics question. In this sense, they supported the view that the practical solution will be focusing on liability and doing a cost-benefit analysis, rather than judging if a young person or an elderly citizen should take the hit in a potential crash.

Interestingly, one of the interviewees imagined the ethical dilemma issue and the potential to buy a safer vehicle to be possibly treated as an upgrade feature, something that exists with traditional cars. As he put it, ‘ultrarich people can afford to pay more in order to get a better and safer car’. Whether this principle would be translated to autonomous vehicles and their ability to make moral decisions, is yet to be seen.

Another person pointed out that countries definitely need to adopt a common principle regarding how to handle ethical dilemmas. He said that even today, laws and the notion of how much a human life is worth differ from country to country (as is the case with death penalty or euthanasia, for example), but we need an unbiased AI (as he put it, easier said than done) when it comes to making an ethical decision. He did specify that taking such decisions gives humans a God’s view and so this question should be addressed with great care. Another person said that giving ordinary citizens the power to judge ethical dilemmas might rather lead to unnecessary panic than a productive solution because people don’t have the expertise and they trust negative bias too much.

Six people argued that ethical decisions should be taken on a higher governmental level, by a team of experts, not ordinary people. Three of them said that there should be a global framework adopted everywhere and this decision should be unified throughout countries. As one of them put it, it would be ridiculous, if a person wants to ride in an autonomous car from Bulgaria to India for example, to have to “harm” different classes of people (based on their sex, age, social status) in the different countries they pass through. This is why some of the participants saw the Moral Machine experiment as completely wrong to approach this question. Two people suggested that human error doesn’t just mean human intuition and collisions happen because of deliberate mistakes: because inexperienced, drunk or drugged people drive and because the roads are congested. Another person claimed that as far as machines go, if we embed moral judgments in them, they will always be biased, because humans are biased so even if we leave this decision to humans, it will not be 100% right.

137 MSc Design Thesis: Jana Voykova

Asked about what role governments should play in the development of the technology, the majority of the interviewees said that government’s role is mainly related to regulating the security, privacy, ethical and legal aspects of the technology and improving the road infrastructure in order to accommodate for these cars. As they argued, an autonomous car would stop and wonder what is going on if it is thrown in bad road conditions lacking signs and road markings. Several people argued that governments simply lack the knowledge required to participate in the development of the technology and they could support the technology by not standing in the way of private players.

Instead, governments could aid this process by reducing taxes for potential buyers of autonomous vehicles and by sharing more information on the technology with the general public so as to gain people’s trust in it. One person said that governments can restrict the building of parking spaces so that housing buildings could be built instead and squeeze more people in the city. An interesting observation was that governments, as far as their mandate goes, probably don’t have much of an incentive to promote the technology because they don’t know if it will become a reality in the timeframes of their mandate. Two people expressed the opinion that governments, apart from actively participating in regulation of the technology, they should also have the final say whether the technology is indeed the best solution. In this sense, the technology should, first and foremost, be seen as a means to improve social life, not drive profit for the companies that develop it. This is an extremely relevant position, but whether it will happen in reality, remains yet to be seen.

On the question whether they would buy an autonomous car if cost was no factor, five people said they would if the infrastructure is suitable for the technology and it is proven safer than traditional driving, one person said they would buy if the car had two regimes giving him the option to choose whether to drive or ride75, four people said they wouldn’t buy an autonomous car even if they could afford it: one of them doesn’t drive and has no intention to drive in future, but would use autonomous public transport, one would like to have a car only when she needs it so she would rather invest in renting an autonomous car instead of

75 This reminded me of Toyota’s approach to the technology. Their automated car (they also prefer to use this term as opposed to autonomous) has two driving modes: a Guardian and a Chauffeur mode. The former is rather connected with assisted driving, the driver has the control, but systems like Lane Departure Alert (LDA) and Automatic Emergency Braking (AEB) can help the driver prevent a crash; the Chauffeur capability is a measure of the degree to which the vehicle takes primary responsibility for driving, relieving the human driver of some or all driving tasks. The higher the degree of the capability, the more complete the level of automation. Toyota will present their level 4-automated vehicles later this summer in Tokyo’s Odaiba district and give citizens who are interested to test the technology, the opportunity to do that. On their news page, Toyota Research Institute (TRI) emphasise that their automated vehicle aims to ‘amplify human performance behind the wheel, not replace it’. This might be a good strategy to quash public scepticism. Information is taken from TRI’s whitepaper Automated Driving at Toyota: Vision, Strategy and Development. Whether TRI envision that gradually, the Guardian mode will be eliminated, does not become clear from the report.

138 MSc Design Thesis: Jana Voykova buying one, another person said she wouldn’t buy no matter what and yet another person would prefer to have an electric car but not autonomous.

One person said that with time, generations would probably not even want to take training and driving will not be even desired as there is a tendency for young generations not to own things but to share (I should point out that what we refer to when we say share a service is not quite sharing, it’s just paying someone so that you can use their asset/product). Another person said that maybe owning a car will become a luxury as it was back in the days and this way, owning a vehicle will not be a commodity anymore.

Asked whether autonomous transportation should become the norm and if both types of driving (manual and automated) could co-exist, all interviewees said that human-driven and autonomous cars can definitely co-exist and probably will for a long time as the technology is being trained to function alongside human drivers and there will be severe resistance from people if the government decides to prohibit traditional driving76. If traditional driving becomes prohibited or naturally abolished was a matter of at least 40-50 years, some of them said. One person said that people need to be able to choose whether or not to drive and this shouldn’t ever be a decision made for them.

Four people said that it is going to be difficult for autonomous cars to drive alongside human drivers simply because rules apply for everyone in an all-machine scenario which is not the case with all-human drivers where the unpredictability rate is high. One person said that traffic can be divided so that public transportation only drives in one specific area and other traffic doesn’t interfere with it because most accidents happen where these two types of traffic intersect.

All people that I interviewed saw job loss as a legitimate concern, however, most of them thought we will be able to tackle this problem as we have done before, this was the result of every other industrial revolution and we managed to find a solution and re-skill people before so there isn’t any reason why we won’t be able to do the same today. Two of the participants said that it makes sense for new jobs to be opened in the same sector, in the production, sales or maintenance of the autonomous fleets.

One person referred to the debates happening in a lot of countries to introduce a universal basic income as a measure to handle this issue. Interestingly, two people said they didn’t

76 A 2018 survey of the car parts manufacturer in the US CARiD showed that 71% of the 1034 respondents they surveyed said they would miss driving if it became abolished at some point, while 75% of them said they would still choose to drive if they had the right to make a choice. As stated in the survey results, ‘people’s personal preference for driving would need to be changed, rather than economic or infrastructural changes, if autonomous vehicles are to become as widespread as some have predicted’.

139 MSc Design Thesis: Jana Voykova think AI would ever transcend human capability because human intelligence cannot be synthesised and reproduced in any way, one of them noted that probably with quantum computing this will be possible but human intelligence and creativity as such would never become transferrable to machines. Another person said that ‘automation and robotisation are not done because they are cool but because they are cost efficient. He said that people should stop seeing this as a grand scheme against humanity but as an economically viable solution. This is naturally true, but I do think we need to ask: economically viable for whom?

Asked whether data privacy was bothering them, the majority of the people I interviewed didn’t see this as a major concern simply because we are already giving our data in order to use many current technologies so this will not be any different with autonomous vehicles. Some people said that this is a trade-off for the technology to function as expected, others said that since we already give away our data, the question is how it is going to be processed and secured, not if we should allow its collection and accumulation in the first place. One person said that this is part of how business is done today, according to another, it’s a concern as long as guarding it goes but she was optimistic that we will have the foresight to protect it as we have shown we tackle progress well. One person said that data collection in terms of quantity will not be more than before, but for developing countries (once the technology reaches them) there will be no way to cheat anymore because of the technology’s ability to “see” everything.

It’s common that today, in poorer countries where people have a peculiar mentality (not in a positive sense), some people take advantage of the lack of control and penalties when they disobey the law in one way or another. With cameras everywhere this will not be possible anymore. She said that it’s funny how we are becoming more and more stupid and unable to use our cognitive abilities as a result of technological progress, the example she gave was Google Maps. It’s so easy to become dependent on technologies like that which show you every little detail of your journey and you practically forget how to think for yourself, it’s more difficult to find a solution which is not obvious when you don’t rely on your own cognitive abilities. One person said that he doesn’t have anything to hide and data was necessary for the machine to operate so data collection was not bothering him.

I asked what other measures we can implement as an alternative solution before we introduce autonomous vehicles so as to handle the challenges in big cities like pollution, congestion, traffic fatalities. Two people answered that we definitely need better and longer training, starting at 15 for example. One of them made a comparison with Finland’s driving courses which are designed to prepare drivers for situations where the car is taken out of control or technically more difficult to steer (for example lacking driver assistance systems), so as to teach them how to drive in less than ideal conditions. Another person said that it’s difficult to address driver distraction because this is just human nature. Also, cars are

140 MSc Design Thesis: Jana Voykova becoming less of a challenge to drive, systems become more sophisticated, requiring less attention and so people naturally have started trusting technology too much. In this sense, they have become accustomed to not being careful enough because there are 2nd layer of safety technologies in the car. Another person said that we could potentially design the vehicles so that they “detect” when a person has drunk and if the amount of alcohol in their organism is above the norm, then the vehicle won’t start. Even so, though, he noted that humans will never have the precision of machines.

Two people said that other measures do exist already, but in countries like Bulgaria, laws are not followed, regulation is weak and road infrastructure and drivers’ mentality are not even worth discussing. One of them said: “there is still a lot to be done on a local level, but things are not objective, when it comes to pollution, all cars are put in the same category, we also cannot fight corruption with autonomous vehicles”.

Two other people also mentioned corruption and political disengagement as areas of improvement. One suggested that people should be encouraged to use bikes and other means of transportation more often but then the right infrastructure needs to be in place. On the question who determines whether this technology is needed and why we need it, one person said that it’s statistically safer and the private sector determines but this doesn’t mean the solution is not good only because private players provide it.

Another one said that in a free market society, supposedly people decide but in reality, it’s the private sector. According to one person, this is not a deliberate decision, but consumers do demand it, they need more driver assistance systems to help them. Two other people said that this was economically efficient solution, eliminating labour cost. Several people said it was a natural evolution and the urban landscape demanded it. One person was of the opinion that it was private companies and scientists who make their case, but governments should decide if the benefits outweigh the risks.

Another opinion on this question was that in a free market society product development is not always driven by specific needs and it’s more important to decide on how public and private sector collaborate instead of debating who has decided. When asked if they saw new technologies like autonomous vehicles as a factor widening the divide between developed and developing countries, the majority of the interviewees said they didn’t see this as that big a problem because it has always been the case that poorer countries take advantage of new inventions at a later stage. One person said that today, with the faster pace of adoption of technology, this process will take even less time. Another person pointed out that this could indeed open up a discussion on economic disparity as a result of technology and the inability of poorer countries to benefit from new technology right away.

141 MSc Design Thesis: Jana Voykova

6.2.2 Survey analysis

I surveyed 101 people to find out what their opinion is on the social impact of autonomous vehicles. As mentioned earlier, I didn’t ask them to explicitly specify where they are based, but I shared my survey with people located mainly in Bulgaria, UK, US, Sweden, Germany, and individual respondents from Belgium, Israel, Austria, Argentina and Indonesia. The questions I asked were similar or the same to the ones I asked in my interviews.

Interestingly, in the beginning, when my survey was answered mainly by people in car communities and forums, the result suggested that the respondents were strictly against the technology. As more and more people started to answer though, the result slightly changed. 65% of the respondents in the survey are male, 35% female, between the ages 23 and 72. Asked whether they would consider changing their current vehicle with an autonomous one, given the technology’s potential positive effects on the society and environment, 45% said they wouldn’t, while 24% said they would. 29% said they were not sure.

49% responded that they liked driving as opposed to being a passenger as the motivation behind their answer (either affirmative or negative), 33% indicated the passive role of the driver, 34% trusted the technology to perform better than humans, 16% selected the price as a factor. Relatively low percentage, only 24% indicated security vulnerabilities as the reason, 38% were motivated by the positive social impact of the technology in terms of minimised accidents, optimised space and reduced pollution. Individual answers were attributed to using time more efficiently. Less than half of the respondents, 38% were of the opinion that the benefits of the technology do outweigh the risks, while 22% didn’t support this claim, 40% were not sure.

A majority of 55%, indicated that autonomous vehicles shouldn’t become the norm in the future. The reasons for this varied. Some people wrote they didn’t think the technology could replace humans 100%, others indicated the technology would not resolve bad infrastructure like potholes and uneven roads. Some people said they liked driving and if they just needed transportation, they would take a taxi/train/bus. Quite a few people suggested that autonomous cars should have two regimes, or both modes should be allowed to pick between. They saw driving as much more than transportation but would appreciate the opportunity to choose not to drive if they had a drink or were tired. One person wrote that some things should remain non-digital.

On the plus side, the majority indicated that autonomous cars will improve road safety and minimise accidents, a few people mentioned the technology was the future and it was only

142 MSc Design Thesis: Jana Voykova natural, some preferred not to answer because they felt they didn’t have enough information to express a reliable opinion, some people pointed to ecological reasons and eliminating the lack of truck drivers. A good majority of the respondents, 60% said that governments should have a supporting role in the development of the technology, the aforementioned CARiD survey in the US in 2018 showed the same result — 66% of the 1034 respondents said that the US government should be involved in regulating the technology.

25% of the respondents to my survey expressed the view that the government should restrict the technology. Interestingly, another 2018 survey in the US conducted by the Public Policy Polling organisation found that 75% of the 2374 US voters wanted Congress to apply the brakes to driverless cars being tested on the roads until the technology is proven safe enough. That said, the question how exactly the technology will be proven safe enough in a lab environment remains unanswered. 15% of the respondents to my survey preferred a passive role of the government and 14% said governments should have no role in this process at all. When it comes to liability, 61% said that the manufacturer should take the blame in a potential accident, 30% said that the development team (who designed the vehicle) should be held accountable, 35% indicated it should be the owner of the vehicle, 5% said there was nobody to blame as there is no driver.

Individual answers were given in support of a case-by-case judgment and looking at the cause of the accident, it may be so that a component of the system defected or there was another reason behind the accident so each case should be assessed on its own merits. When asked what alternative measures can be taken to address the problems which autonomous vehicles would potentially resolve (minimise road fatalities, address pollution, optimise space, free up time for more productive activities), the answers provided were similar to the ones I collected in my interviews: improve road infrastructure, better training, stricter punishment for disobeying traffic rules, better public transportation, shared riding, promoting sharing rather than owning a vehicle by introducing higher taxes for people who own a car, provide better conditions for bike riding.

Couple of people answered that traffic could be divided to be autonomous in designated areas and non-autonomous in other areas. Three people suggested that cars be equipped with sensors so as to be interconnected and able to detect defects or mistakes automatically and signal to one another about potential incidents. Only one person saw the idea of attracting people to big cities as a potential area of improvement: they answered that job opportunities could be made better in the smaller towns where people can walk to work as opposed to having to move to big cities where salaries are better, and the quality of life is higher. Several people mentioned prohibiting mobile devices in the car so as to keep drivers’ attention on the road.

143 MSc Design Thesis: Jana Voykova

To conclude the results of my interviews and survey on the social impact of autonomous vehicles, I believe that my predictions and assumptions were both confirmed and rejected at the same time. They were confirmed to the extent that all people I talked to and some of the survey respondents, did think that there is insufficient information in the social space around the technology and more attention should be paid to addressing the controversial topics around it so that citizens can understand it better and not base their opinions on negative bias. My claim that we shouldn’t even try to resolve ethical dilemmas was partially confirmed, a good majority was not comfortable making such a choice because this puts us in a power position which we shouldn’t have in the first place. However, my assertions were also rejected in the sense that the majority of the people I interviewed and surveyed did support the development of the technology, regardless of its shortcomings since the potential benefits would outweigh the risks and they saw this as a worthwhile compromise to make.

I would argue though, that the way people expressed themselves, while being rational and objective, and my attempts to make them think deeper about underlying factors like the importance and inevitability we attribute to technology, the normalisation of trade-offs in order to receive access to technology, the futuristic visions about smart cities and the way we understand life in urban areas, all made me believe that while people see the root problems, they generally don’t have the habit of questioning already normalised practices in society and looking for solutions elsewhere. I will expand on this in the Discussion chapter.

As a whole, my research shows that people are curious about the technology and even though it’s yet to be proven safer, they are willing to bet on it because of its potential to ease urban life and prevent accidents. Nevertheless, they seem to be unwilling to completely give up traditional driving and would prefer to have the choice between to drive themselves or be driven. As far as I know, only Toyota is currently developing the technology to provide two regimes and publicly advertising it as a semi-autonomous fleet so they might have an advantage on the market. Even though some companies are partnering and collaborating with each other in the development of the technology (like Daimler and BMW), it is still not clear how exactly they will handle issuing patents, agreeing on a common liability and ethical framework, as well as handling of security standardisation and legislation. These are vital questions which will determine the success or failure of the technology and they aren’t addressed adequately either by private or by public stakeholders.

144 MSc Design Thesis: Jana Voykova

Figure 12. This infographic summarises the results from my survey on the social impact of autonomous vehicles.

145 MSc Design Thesis: Jana Voykova

6.3 Virtual exhibition feedback

In order to verify the validity of my claim that design can and should steer the public attention to a more critical outlook on the social landscape we live in, I prototyped an exhibition in which I presented a highly critical and speculative perspective on the kind of future that we might be designing as a result of our current obsession with progressing technology almost at all costs. It was a way for me to test how design can facilitate a room for reflection and inspire people to look at technology from a different angle. The artefacts I displayed were intentionally provoking and some of the concepts were purposely overblown so as to initiate a discussion around the messages which they attempted to communicate.

For reasons already explained, I was prevented from doing an actual exhibition, so I organised it as a virtual experience and shared it with a focus group of 10 people. I gave them time to look at it and think about the associations it raised and collected their feedback afterwards. Judging by their insights, I conclude that my speculative and highly critical approach is not equally perceived by different people, they either loved it or disliked and didn’t quite agree with the approach I chose. I wouldn’t say this is surprising because the perspective I am communicating is quite adamant, so it naturally inspires strong reactions and different people have different perception of technology, which they normally base on their own experience and life dynamic. The majority of them understood my ideas quite well, but some people were confused about some aspects of the exhibition and I had to explain to them what I meant with these artefacts.

All of the people I shared my exhibition with, gave me very positive feedback about the technical execution of the idea and the visual aspects of the artefacts, but some people didn’t like that I depicted a dystopian, negative scenario. They did see what my motivations were and understood what I tried to say but as one of them put it, the exhibition and the concepts it communicates are interesting but abstract. Another person perceived the purpose to be rather to scare people, than make them reflect. She did say that these are very relevant questions which more or less everybody is asking themselves today, but technology does have a positive side as well and sometimes an initially good idea just turns out badly. This is of course true, technology, like everything else has a dark and a bright side. But do we sweep the yin under the rug because of the yang? I don’t think we should.

Another person didn’t like the fact that I called my exhibition #submission because this lifts up negative connotations which can sometimes have a repulsive effect on people.

146 MSc Design Thesis: Jana Voykova

Four people said they liked the contrast between black and white both in the artefacts and the black and white photographs showcasing the exhibition. Two people said that the monochromatic theme channels the good vs. bad dichotomy while reflecting my concept about the clash between winners and losers in a good way. One of them pointed out though, that my approach was quite polar, not taking into consideration the grey zones in between the use of technology. She perceived it rather as a prophecy than an undesirable eventuality.

Another person said that the exhibition reminded him of Huxley’s Brave New World. One of the posters made a strong impression on him because of its message and its position in the exhibition — a poster which says: We are all brainwashed, however, some forms of brainwash are cleaner than others. He said that all messages in the posters were strong but the way I positioned this one was, as he put it “sneaky”. He said: “I was reading the posters and then suddenly my eyes were drawn up to this poster which says: “think again, this all might just be brainwash”. He thought this was cool because potentially, all messages/opinions can be seen as a form of propaganda, so this poster was drawing attention to this, to be more critical toward what one’s viewing/reading and not take it as a 100% truth but try to find the truth on their own. He also liked the idea to display the title in a 3D format along with the artefacts.

Another person was most impressed with the user manual, she said that turning the human- machine relationship upside down was very nice (instead of humans using machines, the manual depicts a situation where machines use humans). One person found it interesting that I wrote a machine manifesto, she said that this approach, which is also used in religion and art movements, bears a sense of utopia. Two people liked the fact that I questioned something that has been normalised so much, one of them said she never looked at technology from this angle before and it struck her how the progress of technology indeed has such a tremendous impact on the evolution of different societies. She was mostly impressed not with the speculation that such a stringent and unpleasant divide can happen in future but with the fact that this gap between classes (often because of technology) is already a reality in a lot of places. What she found interesting was how the contrast between the sketches and the digitally printed posters represents the contrast between manual and machine labour. I must admit that this was rather a happy coincidence, not something I deliberately wanted to portray by mixing digitally and hand-drawn artefacts.

One person said that the division of labour/classes of people has persisted throughout time in one way or another and is defined by the value systems of the time in question, so this is not so bothering, it just depends on what skills are valued and how. Another person said that the phenomena which I am portraying are apparent even today, although in a milder fashion. He said that he liked the subject I was bringing up with my exhibition because it suggests that technology will be incorporated in our lives to an extent most people won’t (or

147 MSc Design Thesis: Jana Voykova can’t) envision at all. He defined himself as a “sci-fi dork” who appreciated that I was bringing to the spotlight a techno-totalitarian reality of a population that is approaching, as he put it “almost transhumanistic levels of technological integration”.

According to one person, morality doesn’t reflect universally valid principles equal for everyone and much like formal power, it is forced on people. Instead, it imposes the values of a particular group in power, like the church or the state, for example. She said that the division between people might even be necessary for the world to function although she did say that the current world order makes people “slaves of the social system they live in”. Two people found the biometric bracelet unsettling because it represents the idea of extreme control exercised over someone, not just remote monitoring and manipulation but physically visible control.

One person liked the fact that the exhibition is quite philosophical and idealistic. She said that I managed to communicate a lot by using fairly simple tools and stylised designs. Typography, she said, is usually used with marketing purposes but I use it to achieve the opposite effect. She was also intrigued by the method I used, namely speculative and critical design and was happy to find out more about what is means and how it’s applied. Another person said that while she thinks that humanity’s survival is dependent on people’s mental and spiritual elevation, she definitely sees technology as beneficial for our future. She agreed that we are becoming lazier and more dependent as a result of relying too much on technology, but this is just a feature of the flipside of technology and like everything else, it also has positive and negative sides. An interesting observation she made was that we should probably be more suspicious of people who want to portray themselves as too righteous and always following their ethic principles. What she meant was, that nobody is capable of remaining completely impartial when something affects them or the people they love, no matter how principled they normally are.

When asked what they thought about using design to bring up a subject matter like this, no one found it odd or inappropriate, on the contrary. Two people said that meaningful art is provocative, and every art has to have a moral foundation which defines it, in other words, design and art need to have substance. One person said that this was just a different way to use design, so she didn’t find it strange. Two people mentioned that paying attention to how the message is presented visually, strengthens the message itself and compels people to think more carefully about what they are observing. One person said that design and literature are usually best suited to communicate this type of messages, this is basically their sole purpose. Interestingly, one person said that people, as most mammals rely on visual communication when relaying information about our surroundings and so, graphic design will remain one of the most powerful tools to “shape public opinion” or in other words, “brainwash”, as he put it.

148 MSc Design Thesis: Jana Voykova

He mentioned that images can indeed say a thousand words but not entirely because sometimes, when we express abstract concepts, visuals might not be enough to communicate our ideas. Therefore, we need to back them up with language or text, for a better precision. For that reason, he found the way I presented my concept very suitable from a design perspective.

From the feedback I gathered on my prototype exhibition, I infer that my assumption about design’s capacity to serve as an instrument to draw attention to socially important phenomena, was confirmed. Even though different people had different associations and perceived the artefacts somewhat differently, my ideas and the messages I tried to put forward did raise strong reactions in my limited audience. It made people reflect and got them interested in a conversation around why I was doing what I was doing and what I wanted to achieve. This shows that design can indeed be used to drive social innovation, but the question of on how big a scale design can do that just on its own, remains unanswered.

149 MSc Design Thesis: Jana Voykova

6.4 Limitations of research outcomes

The purpose of this research, as indicated earlier, is not to supply statistical data based on hard evidence or make irrefutable statements. Among other things, my research is based primarily on gauging opinions and opinions are not scientific data. It rather aims to produce academic material which could aid the future research in the respective field. Nevertheless, it is important to take into consideration the research limitations and the constraints of the generated outcomes. Although I managed to collect a variety of insights through my research, they are not conclusive in any way. Therefore, they should rather be assessed for their qualitative character.

The majority of the people I engaged have good education, relatively high standards of living and easy access to technology. I would assume that people who are in less privileged positions and who have limited access to innovations would show a different outlook on the topics discussed (although, as previously established, people who occupy more unstable positions and whose job/life is not reliant on technology, tend to be more likely to have a negative opinion about it). It should also be noted that a large portion of the collected insights are from Bulgarian residents. Another important limitation to mention is that doing an online exhibition prevented me from being able to observe the immediate reactions of people, it did allow the audience to review the artefacts in peace, but it also gave me less control as a mediator between them and the artefacts.

In a real exhibition setting, I would have been able to invite a broader audience, possibly people who are complete strangers to me so it would have been less staged in a way. This was what I was planning for initially. I organised the exhibition review and feedback as a personal experience instead of gathering people in a workshop format because I wanted to give them time to go through the artefacts with their own pace and reflect on them in peace. One participant said that the virtual set-up was even better because there is a lot to see and read so it would have been more rushed in a traditional exhibition and maybe people would not have given the artefacts enough attention and reflection.

The methods I applied in my research also come with their limitations, approaching the research with a highly critical mindset, detracts from the objectivity of the research although I did try to analyse the outcomes and perspectives I gathered as objectively as I could. As I explained in the beginning of my thesis, the risk with speculative and critical design is that the deliberation and conversation it invokes, stays confined to the boundaries of the exhibition experience and even though design can indeed shape opinions, it’s very hard to actually make people act upon what they have seen or discussed after the conversation is

150 MSc Design Thesis: Jana Voykova over. Habits and prejudices are hard to crack so we need a more systemic change to enable altering of views and behaviours. Of course, this is no reason to give up trying to bring up painful conversations in whatever format available and suitable.

6.5 Discussion

How is my research important and why does it matter? While there are limitations to it, which I already acknowledged on several occasions, I believe that, by trying to bring together two fairly diverse topics, my thesis makes an interesting academic material. The existing academic research on designers’ social responsibility focuses mainly on questioning designers’ role in promoting consumerism but it doesn’t consider the underlying factors for this mistaken interpretation of the profession or any other outlets of designers’ social engagement which condition their work and behaviour. By hearing the perspective of practicing designers as opposed to just reviewing literature on the topic and offering my personal interpretation of what being socially responsible means, my research tries to put ethical design in a more practical frame.

Additionally, analysing the development of autonomous transportation as a use case adds a different dimension to the research on designer’s social responsibility and extends its boundaries to premises that are not traditionally thought of as designers’ province. As already established, major changes are needed in the wider social landscape so that ethics can have a more prominent place in design (and technology). This is connected with amending how the design profession is perceived by other experts and spheres of live, with critical evaluation of the values that are instructed in society, with reforms in the design education and the socio-technical order and just as importantly — political and economic restructuring.

As evidenced in my research, the design community has taken the first steps to realising this change: with service design, with focusing on the ethical aspects of the profession, with trying to infuse design thinking into both private and public sector, however, there is still a long way to go and an existing risk that, unless we manage to actually change political, economic and business structures, these steps might be assimilated and turned into political correctness at best, or become just a trend which, sooner or later will pass.

As I argued earlier in my thesis, social responsibility and ethical behaviour are not solely designers’ or ethicists’ obligation. It’s everyone’s duty to uphold morality but the nature of designers’ work (and artists in general), makes it easier for them to advocate for this change of perspective more effectively than a bank clerk, for example.

151 MSc Design Thesis: Jana Voykova

Individual efforts towards ethical behaviour are necessary but they don’t automatically decode how society at large behaves and perceives ethics, and that is why we need a social ethics framework which encompasses social relations in their entirety.

The ostensibly popular vision that “if I don’t do it, somebody else will”, is the primrose path that lead us here. If we reprogram the way ethics is perceived in social relations and affairs globally, then designers and all other professionals won’t be obstructed by personal contingencies to demonstrate their social responsibility. Nevertheless, external factors and dependencies are no excuse for unethical decisions and actions.

In one of my interviews, a designer told me that she once tried to dissuade a young person from studying design, not because being a designer is shameful but because the profession is not given the appreciation it deserves. This, I think is one of the main reasons people have difficulty understanding how the topics of my thesis relate to each other: they don’t see design as suited to dispute technological challenges, let alone political and social ones.

I did argue earlier that not all designers are obliged to devote themselves to do socially important work or engage in political conversations; not because I believe that some designers shouldn’t, but because I cannot force them to think they should. As Mike Monteiro puts it, though, ‘to say that designers shouldn’t get political isn’t just naive, it’s revisionist’ (Monteiro, 2019, p. 418). What he means by political, is that designers should care about who one’s work is affecting and how, whom it’s hurting or discriminating, who is making the decisions and who is left out of them, what places they choose to go to work, what causes they support and what projects they criticise. Whether they do it by drawing illustrations, designing workspaces and applications or by creating customer journey maps, is irrelevant.

Being socially responsible doesn’t just mean refusing to put one’s creative skills to making a potentially harmful or morally dubious product appealing for the audience, it means publicly exposing and condemning these potential harms and behaviours. Social responsibility is concerned with critical scrutiny of the way we define civic virtues and social practices. In an overwhelmingly rapid technological and social change, social responsibility requires careful revision of how we use innovation and progress and what we develop them to do because today, they impact all spheres of public and private life. I completely agree with what one of the participants in my interviews said, namely that we shouldn’t entrust ordinary people with important decisions because their opinions are too often based on negative bias and they simply lack the necessary expertise. However, we cannot leave them out entirely of this decision, either.

152 MSc Design Thesis: Jana Voykova

Governments, as representatives of the general population are indeed, best suited to have the final say and, as two of the people I interviewed argued, governments should decide whether the benefits of a new technology (in this case autonomous vehicles) offset the risks. In more egalitarian and democratic societies, governments do demonstrate (at least on the surface) their public responsibility better than their counterparts do in poverty- and corruption- ridden countries.

But lobbying and private sector dominance over public institutions do exist in developed countries as well. Therefore, governments in democratic societies aren’t exempt from exerting control (by taking citizens’ interest as a guiding principle) over the way private companies deliver the benefits of new technology to the society. Most of the participants in my interviews didn’t see technological progress as problematic as far as the widening divide between countries goes. As they argued, it has always been the case that the benefits of scientific and technological developments become a reality in developed countries first and with time, developing countries gradually start taking advantage of new inventions, too. I understand their point of view and their claim is relatively legitimate.

With time, developing countries will probably get access to the same benefits. What is troubling here at least in my view, is the latent problems which have historically made developing countries inferior in almost all respects and our claiming that we are doing our best to promote ethics and sustainability when we clearly aren’t. When I expressed my concerns that technologies like autonomous vehicles might give developed nations the power to control politically and economically weak countries on a whole new level, one of the people I interviewed, said that he never thought about that. Sure, more educated people who were born in poorer countries can and have all the right to emigrate to politically more stable locations and find better life there. This is what Gillian Brock77 calls “brain drain”.

77 Gillian Brock is professor of philosophy at the University of Auckland, New Zealand. Her research focuses on global and international justice, ethics, political and social philosophy. In her book Global Justice: A cosmopolitan account (2009), she argues that immigration can have disastrous effects for the countries which people exit because it deprives them of knowledge, skills and expertise and this has lasting ramifications for their economy and social politics. She gives an example with the unequal distribution of healthcare workers globally and the fact that developing countries like the Philippines for instance, often “export” trained healthcare personnel to developed countries. This is a problem not so much because, as Brock argues, developing countries have invested resources in the training of individuals who are then benefitting developed countries (because the opposite is also true, many people from developing countries get training abroad and come back to their developing countries so theoretically, this benefits the developing country but the resources invested by the developed country are lost for them). The bigger problem with immigration as I see it, is that 1) the majority of the educated and skilled people leave so the majority of those who stay are not in a financially or intellectually strong position to fight for political change; 2) instead of eliminating corruption in the developing world, it exacerbates and endorses corruption while dooming those poorer countries to never become developed. These are political problems and private companies shouldn’t be the ones resolving the uneven distribution of educated individuals between countries, be it in their own interest (outsourcing businesses to developing countries cuts a

153 MSc Design Thesis: Jana Voykova

As she argues, though, immigration may improve the life of those who leave but it may also worsen the life of those who don’t. It’s like leaving a sinking ship: the ones who manage to save themselves may not have caused the leak, but they aren’t bettering the chances of survival for those who stay onboard, either. Of course, I’m not claiming that it’s developed countries’ fault that many developing countries are corrupt to the bone, but it does seem like the supposed efforts of the international community to fight global injustice, corruption and poverty have been futile despite the huge capital and manpower resources invested in this project. As evidenced in WEF’s Risk Report for 2020, in spite of the global awareness of technological, social and environmental disruption, there is still inadequate global cooperation to tackling these challenges collectively. It seems this is not because we lack resources, but rather because we use these resources to fill in the wrong holes. This is something which people in privileged positions don’t necessarily see as their fault or duty to repair but realistically, we are all accomplices in the distribution of global (in)justice.

My research shows that people do see disruptive technologies as a cause for social disruptions, like potential job loss, but they either rely too much that, as with previous industrial revolutions, we will be able to re-train and re-skill those whose jobs become automated, or they don’t acknowledge the role of technology in these processes at all. It’s true that people are generally adaptable, and we have indeed previously managed to handle job loss well enough, but while we are pushing the envelope with each new revolution, very few people seem to be asking if we are pushing it in the right direction and for the right reasons. As Douglas Rushkoff argues, ‘the labour force isn’t simply replaceable, it’s in constant flux, perpetually changing and responsible for its own training and care’ (Rushkoff, 2016, p. 98). This, in my view, isn’t a sign of democracy and progress but of economic and social instability.

Then again, it’s hard for a person who has a seemingly secure job and a good quality of life to question the effect of technological progress on the fates of less adaptable and skilful classes. These cases may indeed be outliers and the way automation affects the labour market does vary across countries and industries, but this doesn’t mean that those outliers don’t exist and shouldn’t matter. For one, if the outliers were, in fact, we or a member of our family, we would have had a radically different outlook on this question.

Another thing that struck me during my research although I did predict this outcome, was that a lot of people accept the privacy trade-offs which we have to do in order to use technology, as a fair bargain. They are indeed aware that this is too important and serious to take lightly, so they point to the necessity for regulation of new technologies. But privacy is

lot of operational cost for corporations). Consequently, it opens new jobs in poorer countries, but this only helps the more adaptable and skilled population of those countries and doesn’t resolve their political struggles.

154 MSc Design Thesis: Jana Voykova not an arbitrary question. As Shoshana Zuboff argues, it’s not about having nothing to hide or having nothing against one’s innermost personality exposed and analysed through a magnifying glass just so that we can live a more comfortable life. Contrary to what we have been led to believe, these trade-offs may in fact be, as Zuboff’s research shows 1) not really necessary for the services we supposedly need to operate as they should; 2) in the name of someone’s greater good but not ours. Additional layers of security and encryption should not be about trying to preserve our privacy. Privacy trespassing shouldn’t exist at all: neither in our email accounts and mobile phones, nor in our cars and cities, and falling back on the government’s power to restrict and regulate this, isn’t going to protects us, it’s just going to legitimise abnormal and morally objectionable practices. Technology shouldn’t be an either…or choice but a both…and one.

Several people resisted my critique of technology’s pervasiveness in social life by stating that humanity has gone as far as it has thanks to scientific curiosity. I’m not sure if it’s because I didn’t manage to communicate this properly or because of something else, but I feel like they missed my point. I never argued that technology and scientific research are inherently bad. What I criticise is the way we push them to extremes and drain them completely of moral judgment. This view doesn’t impede progress, it assures that progress is designed to serve the sustaining interest of people and planet in their entirety. As Kobylarek rightly points out, the absence of moral character in scientific research leads to ‘scientific pathology’ — ‘ethical abuse and in extreme cases law-breaking’. (Kobylarek, 2019, p. 9).

The questions that I brought up in my thesis are quite complex and multifaceted and designers cannot and shouldn’t solve them on their own. There is no pre-determined universal fix to them, either and this is one of the main reasons I argue that we should stop viewing technology as a formula for success in all areas of life. We might be thinking that the ends justify the means but what essentially happens is that technological and scientific progress create a fortune bubble for a great deal of the world’s population while making the survival of the rest of it a lot harder. Many people seem to be professing utilitarian views and this is probably one reason why they embrace technologies like autonomous vehicles without much reservation. I don’t think utilitarianism is the road to fairness, prosperity and grace, though because utilitarianism, among other things, is a hypocritical account for morality. We need to dig deeper and ask ourselves whether what has come to be accepted by the many is indeed in the best interest of all: writers, journalists, philosophers, sociologists do it, I don’t see why designers and artists shouldn’t.

155 MSc Design Thesis: Jana Voykova

7. Conclusion and Future work

In our culture, speaking about “the future” is a way of saying things about the present ⏤ critical, utopian, projective, pragmatic and/or simply unspeakable things ⏤ but too often it is an alibi for saying nothing at all. “The future” is that place where skateboarders hover and ambient fields of graphical user-interfaces are slightly more elysian; it is a rhetorical sink where half-baked marketing plans usurp the place where actual ideas are supposed to go.

Benjamin Bratton

In this thesis I tried to find a bridge between design, ethics and technology by examining how designers understand their profession and its social significance on the one hand, and how people perceive disruptive technology (autonomous transportation) in terms of its social impact, on the other. By analysing the implications of autonomous transportation for society, I tried to contextualise what I mean by social responsibility of designers. The outcomes of my research show that designers generally realise the enormous social importance of their profession, but they are often not in a position to advocate effectively for a truly human and planet-centred, ethical design because of deficiencies in our social fabric.

I hope I’ve managed to shed light on some of these deficiencies. As far as technology is concerned, my research concluded that people are aware of the gravity of the social implications of disruptive technology (autonomous transportation) but they often lack enough information to be able to make an informed decision about these implications. It also suggests that people generally tend to perceive technology as a net-positive phenomenon whose downsides are almost always justified by its potential benefits, either because of the way technology is portrayed in the social space or because they, for one reason or another, don’t think beyond the visible layers of technological progress and don’t usually question already accepted beliefs and behaviours.

Given the limitations of my research, further examination of the topics at hand would need to 1) zoom in on all or some of the aspects and consequences of the development of autonomous vehicles, brought up in this thesis; 2) alternatively, focus on the more technical side of the technology and pay more attention to critical elements like security, interaction design, regulatory standards as these are factors which determine the successful adoption of the technology and people’s perception of its social impact; 3) involve more stakeholders in the research in order to draw more objective conclusions, e.g. policy makers, journalists, researchers, auto manufacturers, ethicists, lawyers, technology companies; 4) engage more people (designers and other professionals) from various countries in order to explore further

156 MSc Design Thesis: Jana Voykova how social responsibility scales across countries, disciplines, ages and backgrounds, possibly involve employers, non-profit organisations and educators to track how social responsibility and ethical behaviour are perceived on their side of the equation; 5) use other methods and analyse the yielded results more systematically (embedding philosophical, speculative and critical approaches in the research as I did, facilitates an interesting discussion on the chosen topics but it also makes the thesis more subjective); 6) focus more closely on the alternative solutions to the problems which I outlined as the primary obstacles before more ethical design and the necessity of technologies like autonomous transportation. This way, instead of emphasising the probable negative outcomes, the public attention will be drawn to possible solutions which in its turn, might make them more engaged.

As the topics I combined in this thesis are each challenging to research, it would make sense to undertake their further examination as part of a PhD thesis or a professional research project which can be extended to a longer period of time, thus supplying a more complete contribution to the existing research on these topics.

This may not be the message that immediately comes across from my deliberations so far, but I do think that technology is exceptional on many levels. I am old enough to remember what it was like to listen to music on cassette tapes or have to go to the public library to search for literature, and these are just the simpler examples of what technology has enabled us to do. It’s ironic but honestly, if it wasn’t for technology, I wouldn’t have been able to involve as many people in my research as I did, let alone people who live on the other side of the globe. In fact, the execution of this thesis would have been dramatically different without the progress of technology. I do acknowledge and appreciate that but technology and the possibilities it gives, are still a privilege which a lot of people take as a given. We shouldn’t forget that.

On the whole, ethics is not something that we should discuss solely in the context of the design of disruptive technology like autonomous transportation. We can (should) talk about ethics in every single aspect of life, big and small: education, healthcare, political system, sports, journalism, music, even criminal justice. What ethics can contribute with, to both design and technology, however, is bring back moral restraint and humility to these two fields and inspire them to rather focus on asking the right questions, instead of jumping to what could in fact, turn out to be the wrong answers.

In the beginning of my thesis I stated that technology is not the panacea which is going to save us from our wicked problems. Neither is design. There is no silver bullet to the challenges we face in modern societies. Because of their complexity, we need to approach them holistically and search for the solutions beyond our advantageous positions. We tend to look at our parents’ and grandparents’ youth with a sense of mockery and think of their

157 MSc Design Thesis: Jana Voykova beliefs as dated and elementary, but they have lived in times of scarcity. Today we, at least in the more “developed” world, live in times of seemingly boundless abundance. Today, ethics seems to mostly be an abstraction and integrity — just a word in the dictionary.

The way we perceive ethics in our social life teaches us that we can cut corners and achieve things without much effort; that we shouldn’t strive for self-improvement, that being responsible is optional and most importantly — that rules don’t apply for everyone. In addition, the way we design technology teaches us to be self-centred and conceited. It impedes our critical judgment and misleads us that the future is already decided, and we cannot do anything to alter it and design our own future. The future is, in actual fact, the present. The decisions we make today define what tomorrow will be. Without foresight and clear vision, though, we risk shaping our future in a detrimental way. If we keep crossing lines, we may reach a tipping point when it won’t matter anymore who decided what, when and why.

The fact that my thesis is so political, is not a coincidence. Design and ethics are political. So is technology. I realise that many of my statements and arguments might be infuriating both for people who are actively doing something as opposed to talking about ethics in design and technology, and the ones who want to advocate more effectively for ethical design but who are nonetheless prevented to do so because of factors they cannot directly control. The former might be annoyed because my approach can be understood as too demanding and extravagant. The latter might think that I am too short-sighted because not everyone can afford to be as idealistic. These are fair arguments, but I hope I have managed to explain why I think we should be more honest when we discuss ethical and responsible behaviour and what are some of the reasons why we often fail to do so.

What I see as key takeaways of this thesis is this:

01. Design is not just an instrument to make the world more visually appealing, it’s a powerful tool to drive constructive change and look at society critically; 02. Ethical design (design ethics) is neither a business metric nor a social trend, it’s a mindset which should be ingrained in all design activity; 03. Ethical restraint is not about obstructing progress, it’s about sustainable progress; 04. Technology is indeed amazing, but it can sometimes be used amazingly incorrectly; 05. The future is unknown, it can take many possible directions but not each one of them is necessarily desirable so we shouldn’t be complacent about the changes that beset our societies.

158 MSc Design Thesis: Jana Voykova

I may be over-trusting human nature, ability and willingness to self-correct, but I firmly believe that we won’t find the solution by discouraging ourselves to preserve our autonomy and self-critique, using new technologies to applaud this inclination and make it easier to achieve. The ability to think critically, uphold morality and strive for intellectual maturity are not vices as we seem to be assuming. They are capabilities and values we should be guarding passionately and educating others to do the same. As Michael Sandel cleverly put it ‘changing out nature to fit the world, rather than the other way around, is actually the deepest form of disempowerment’ (Sandel, 2007, p.97). Progress by any means necessary is not the way to fight human imperfection. It’s the definition of disempowerment and egocentricity.

We helplessly try to convince ourselves that big changes start small but sometimes small changes remain small, or never come. While people in the more developed part of the world may be living in modern democracies, some of them are still kings and others are still wretches. After all, in order for the kings to remain kings, the wretches need to remain wretches but while the wretches don’t really need the kings, the kings need the wretches in order to secure their position. This is how the world works. It isn’t broken. We designed it that way. The good news is that we can redesign it. The bad news is, by the time we actually do, it may be too late.

159 MSc Design Thesis: Jana Voykova

References

Ahmed, M. S. (2018). Autonomous Vehicles - Engineers Perspective. International Journal of Science and Engineering Investigations vol. 7, issue 79.

Aldridge, A. and Levine, K. (2001). Why survey? In: Surveying the world: principles and practice in survey research. Philadelphia: Open University Press.

Alphabet. G is for Google. [online]. Available at: https://abc.xyz/. [Accessed 23 April 2020].

Anderson, E. (2017). Private Government. New Jersey: Princeton University Press.

Anderson, E. Elisabeth Anderson. [online]. Available at: https://lsa.umich.edu/philosophy/people/faculty/elizabeth-anderson.html. [Accessed 16 May 2020].

Auger, J. (2013). Speculative Design: Crafting the Speculation. Digital Creativity, 2013.

Awad, E. (2017). MORAL MACHINE: Perception of Moral Judgment Made by Machines. Media Arts and Sciences at the Massachusetts Institute of Technology.

Bannon, L. J. and Ehn, P. (2012). Design Matters in Participatory Design. Routledge International Handbook of Participatory Design Routledge.

Basnayaka, C. M. and Jayakody, D. N. (2020). Internet of Things for Smart Cities. Sri Lanka Technological Campus.

160 MSc Design Thesis: Jana Voykova

Beard, M. and Longstaff, S. (2018). Ethical by design: Principles for good technology. Sydney: The Ethics Centre.

Beck, G. (2019). Ep 60 | 5G and AI Everywhere: 2030 Will Be a New World | Jeff Brown | The Glenn Beck Podcast. [online]. Available at: https://www.glennbeck.com/st/the_glenn_beck_podcast. [Accessed on 14 February 2020].

Berman, D. (2009). Do good design: how designers can change the world. Berkley: New Riders, an imprint of Peachpit.

BOOA. (2020). Basinkomst, är det möjligt? | Roland Paulsen. [YouTube video]. Available at: https://www.youtube.com/watch?v=7EeMOrn6jlo&feature=youtu.be. [Accessed 28 April 2020].

Botsman, R. About. [online]. Available at: https://rachelbotsman.com/about/. [Accessed 22 February 2020].

Bowles, C. (2018). Future Ethics. UK: NowNext Press.

Bratton, B. [online]. Available at: http://www.bratton.info/. [Accessed 28 February 2020].

Bratton, B. (2016). On Speculative Design. [Publitas]. Available at: https://view.publitas.com/p222-9556/benjamin-bratton-on-speculative-design/page/1. [Accessed: 11 May 2020].

Brock, G. (2009). Immigration. In: Global Justice: a cosmopolitan account. New York: Oxford University Press.

Brock, G. Professor Gillian Brock. [online]. Available at: http://www.arts.auckland.ac.nz/people/gbro064. [Accessed 16 May 2020].

161 MSc Design Thesis: Jana Voykova

Cadwalladr, C. (2019). The Great Hack: the film that goes behind the scenes of the Facebook data scandal. [The Guardian]. Available at: https://www.theguardian.com/uk- news/2019/jul/20/the-great-hack-cambridge-analytica-scandal-facebook-netflix. [Accessed 1 May 2020].

Casey, B. (2017). Amoral Machines, or: how roboticists can learn to stop worrying and love the law. Northwestern University Law Review Vol. 112.

Casey, JP. (2018). CARiD survey suggests US public not ready for autonomous vehicles. [Road Traffic Technology]. Available at: https://www.roadtraffic- technology.com/news/carid-survey-suggests-us-public-not-ready-autonomous-vehicles/. [Accessed 15 February 2020].

Christensen, C. (1997). The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail. Boston, Massachusetts: Harvard Business School Press.

Crane, A. and Matten, D. (2010). Business Ethics: managing corporate citizenship and sustainability in the age of globalization, 3rd ed. New York: Oxford University Press.

Credit Suisse. (2019). Global wealth report 2019. Switzerland: Credit Suisse Research Institute.

Das, D. (2019). Smart City. The Wiley Blackwell Encyclopedia of Urban and Regional Studies. John Wiley & Sons Ltd.

DARPA. The Grand Challenge. [online]. Available at: https://www.darpa.mil/about- us/timeline/-grand-challenge-for-autonomous-vehicles. [Accessed 10 March 2020].

Dellinger, AJ. (2019). Facebook's New Privacy Focus Will Have To Win Over A Lot People. [Forbes]. Available at: https://www.forbes.com/sites/ajdellinger/2019/04/30/facebooks-new-privacy-focus-will- have-to-win-over-a-lot-people/#16ed03741e94. [Accessed 17 March 2020].

162 MSc Design Thesis: Jana Voykova

Devon, R. and Poel, I. (2004). Design Ethics: The Social Ethics Paradigm. International Journal of Engineering Education. Vol. 20, No. 3, pp. 461-469, 2004.

Dodel, M. and Mesch, G. (2020). Perceptions about the impact of automation in the workplace. Information, Communication & Society.

Dunne, A. and Raby, F. (2013). Speculative everything: design, fiction, and social dreaming. Cambridge Massachusetts and London England: MIT Press.

Durrant, A., Vines, J., Wallace, J. and Yee, J. S. R. (2017). Research Through Design: Twenty-First Century Makers and Materialities. MIT DesignIssues: Volume 33, Number 3 Summer, 2017.

DW Documentary. (2017). Will robots steal our jobs? - The future of work (1/2) | DW Documentary. [YouTube video]. Available at: https://www.youtube.com/watch?v=NGUbboTjT18. [Accessed 17 February 2020].

Falbe, T., Andersen, K. and Frederiksen, M. M. (2020). The Ethical Design Handbook. Germany: Smashing Media AG.

Frayling, C. (1993). Research in Arts and Design. UK Royal College of Art Research Papers. Volume 1 Number 1 1993/4.

Friedman, M. (1970). The Social Responsibility of Business Is to Increase Its Profits. New York: The New York Times Magazine.

Friedman, M. (2002). Monopoly and the social responsibility of business and labour. In: Capitalism and Freedom. Chicago: The University of Chicago Press.

Gonella, F. (2019). The Smart Narrative of a Smart City. Frontiers in Sustainable Cities Volume 1 Article 9.

163 MSc Design Thesis: Jana Voykova

Gonella, F. GONELLA Francesco. [online]. Available at: https://www.unive.it/data/persone/5592908/curriculum. [Accessed 16 May 2020].

Google. (2005). Annual Report 2004. Mountain View, CA: Google Inc.

Greenwald, G., MacAskill, E. and Poitras, L. (2013). Edward Snowden: the whistleblower behind the NSA surveillance revelations. [The Guardian]. Available at: https://www.theguardian.com/world/2013/jun/09/edward-snowden-nsa-whistleblower- surveillance. [Accessed 15 May 2020].

Harari, Y. N. (2018). 21 Lessons for the 21st Century. UK: Penguin Random House.

Harari, Y. N. (2020). Yuval Noah Harari & Newsweek Belgium: "The Future of Sapiens". [YouTube video]. Available at: https://www.youtube.com/watch?v=l7_4KXVvgyM. [Accessed 26 April 2020].

Hartmans, A. and Meisenzahl, M. (2020) All the companies and divisions under Google's parent company, Alphabet, which just made yet another shake-up to its structure. [Business Insider]. Available at: https://www.businessinsider.com/alphabet-google- company-list-2017-4. [Accessed 23 April 2020].

Hellsten, S. and Mallin, C. (2006). Are ‘Ethical’ or ‘Socially Responsible’ Investments Socially Responsible? Journal of Business Ethics (2006) 66: pp. 393–406.

Johnson Controls. About Us. [online]. Available at: https://www.johnsoncontrols.com/about-us/our-company. [Accessed 10 February 2020].

Johannessen, L. K., Keitsch, M. M. and Pettersen, I. N. (2019). Speculative and Critical Design — Features, Methods, and Practices in Proceedings of the 22nd International Conference on Engineering Design (ICED19), Delft, The Netherlands, 5-8 August 2019.

164 MSc Design Thesis: Jana Voykova

Kobylarek, A. (2019). Social responsibility of science. Journal of Education Culture and Society No. 2_2019.

Korbaczynski, K. (2018). Social and ethical implications of Automated Vehicles. Understanding virtualization and dematerialization of the human driver in Smart Mobility scenarios. University of Bergen Department of Linguistic, Literary and Aesthetic Studies.

Kurzweil, R. (2005). The Singularity is Near: When humans transcend biology. New York: The Viking Press.

Kymlicka, W. (2002). Contemporary Political Philosophy: an introduction, 2nd ed. New York: Oxford University Press.

Laufer, W. (2003). Social Accountability and Corporate Greenwashing. Journal of Business Ethics 43: pp. 253–261, 2003.

Lefton, A. (2019). As a designer, I refuse to call people “users”. [Medium]. Available at: https://medium.com/s/user-friendly/why-im-done-saying-user-user-experience-and-ux-in- 2019-4fdfc6b7de23. [Accessed 02 March 2020].

Littler, J. Professor Jo Littler. [online]. Available at: https://www.city.ac.uk/people/academics/jo-littler. [Accessed 4 May 2020].

Littler, J. (2018). Against Meritocracy: culture, power and myths of mobility. New York: Routledge, an imprint of the Taylor & Francis Group.

Lomax, J. (2020). What is behaviour change? Towards a working typology. Technical Report (Briefing Report 4, 2020).

165 MSc Design Thesis: Jana Voykova

Maeda, J. John Maeda. [online]. Available at: https://www.media.mit.edu/people/maeda/overview/. [Accessed 12 April 2020].

Maeda, J. (2016). John Maeda on Design, Business, and Inclusion. [LinkedIn Learning]. Available at: https://www.linkedin.com/learning/john-maeda-on-design-business-and- inclusion/welcome. [Accessed 21 April 2020].

Manzini, E. (2015). Design when everybody designs: an introduction to design for social innovation. Massachusetts: MIT Press.

Maurer, M., Gerdes, C., Lenz, B. and Winner, H. (2015). Autonomous Driving: technical, legal and social aspects. Ladenburg: Daimler und Benz-Stiftung.

Mitrovic, I. (2016). Speculative – Post-Design Practice or New Utopia? Zagreb: Ministry of Culture of the Republic of Croatia & Croatian Designers Association.

Monteiro, M. (2019). Ruined by Design: how designers destroyed the world, and what we can do to fix it. Great Britain: Amazon.

Naughton, J. (2017). The rebirth of Google Glass shows the merit of failure. [The Guardian]. Available at: https://www.theguardian.com/commentisfree/2017/jul/23/the- return-of-google-glass-surprising-merit-in-failure-enterprise-edition. [Accessed 3 April 2020].

Nini, P. (2004). In Search of Ethics in Graphic Design. [AIGA]. Available at: https://www.aiga.org/in-search-of-ethics-in-graphic-design. [Accessed on 22 March 2020].

Norman, D. (2002). The design of everyday things. New York: Currency and Doubleday.

Norman, D. (2007). The design of future things. New York: Basic Books.

166 MSc Design Thesis: Jana Voykova

Oppermann, A. Artificial Intelligence vs. Machine Learning vs. Deep Learning. DeepLearning Academy. [online]. Available at: https://www.deeplearning- academy.com/p/ai-wiki-machine-learning-vs-deep-learning. [Accessed on 12 March 2020].

Pachamama Alliance. [online]. Available at: https://www.pachamama.org/social- justice/social-responsibility-and-ethics. [Accessed 05 March 2020].

Papanek Foundation. The Victor J. Papanek Foundation. [online]. Available at: https://papanek.org/about/. [Accessed 5 May 2020].

Papanek, V. (1971). Design for the Real World, 2nd ed. Chicago: Academy Chicago Publishers.

Pasman, G. (2016). Design Fiction as a service design approach. ServDes. 2016 Fifth Service Design and Innovation conference, pp. 511-515.

Pentland, A. Alex 'Sandy' Pentland Human Dynamics. [online]. Available at: https://www.media.mit.edu/people/sandy/overview/. [Accessed 17 April 2020].

Pentland, A. and Choudhury, T. (2002). The Sociometer: A Wearable Device for Understanding Human Networks. Cambridge Massachusetts: MIT Press.

Perkins Coie and AUVSI. (2019). Autonomous Vehicles Survey Report 2019. [online]. Available at: https://www.perkinscoie.com/images/content/2/1/v3/216738/2019- Autonomous-Vehicles-Survey-Report-v.3.pdf. [Accessed 15 February 2020].

Portigal, S. (2013). Interviewing People: How to uncover compelling insights. New York: Rosenfeld Media, LLC.

Prusynski, M. (2005). Design Unplugged: Design, Activism, and Social Responsibility. University of Idaho.

167 MSc Design Thesis: Jana Voykova

Purpose. (2018). Ethics & Tech: Panel. [online]. Available at: http://purpose.do/ethics- tech-panel/ [Accessed 25 February 2020].

Rawls, J. (1971). Justice as Fairness. In: A theory of justice, revised edition. Massachusetts: Harvard University Press.

Rushkoff, D. (2016). Throwing rocks at the Google bus: how growth became the enemy of prosperity. New York: Penguin Publishing Group.

Rushkoff, D. (2019). Team human. New York: W. W. Norton & Company.

Sandel, M. (2007). The case against perfection: Ethics in the age of genetic engineering. Cambridge, Massachusetts, and London, England: The Belknap Press of Harvard University Press.

Sandel, M. (2009). Justice: What’s the right thing to do? New York: Farrar, Straus and Giroux.

Sandel, M. (2012). What money can’t buy: The moral limits of markets. Bristol: Allen Lane, an imprint of Penguin Books.

Schwab, K. (2016). The Fourth Industrial Revolution. Switzerland: World Economic Forum.

Scott, D. (2012). Designing for social change: social responsibility and the graphic designer. Malmö högskola.

Singh, Kh. T. and Singh, M. S. (2013). Ethics in Corporate Social Responsibility. IOSR Journal of Business and Management Volume 9, Issue 2, pp. 16-21.

168 MSc Design Thesis: Jana Voykova

Singularity University. About. [online]. Available at: https://su.org/about/. [Accessed 20 February 2020].

Singularity University Summits. (2017). Artificial Intelligence | Maarten Steinbuch | SingularityU Germany Summit 2017. [YouTube video]. Available at: https://www.youtube.com/watch?v=hnyOzr3Pw7s. [Accessed 15 February 2020].

Skinner, B. F. (1971). Beyond Freedom and Dignity. Victoria, Australia: Penguin Books.

Skinner, B. F. (2005). Walden Two, 3rd ed. Indianapolis: Hackett Publishing Company, Inc.

Skjulstad, S. and Rynning, M. (2015). Graphic Design Speculations: Teaching visual identity for water sustainability within a speculative design framework. Cumulus Mumbai 2015.

Spiekermann, E. bio. [online]. Available at: https://spiekermann.com/en/sample-page/. [Accessed 7 March 2020].

Stanford Encyclopedia of Philosophy. Philippa Foot. [online]. Available at: https://plato.stanford.edu/entries/philippa-foot/. [Accessed 10 April 2020].

Stappers, P. and Giaccardi, E. (2017). Research through Design. In: The Encyclopedia of Human-Computer Interaction, 2nd ed. [online]. Available at: https://www.interaction- design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd- ed/research-through-design [Accessed 11 March 2020].

Steinbuch, M. [online]. Maarten Steinbuch. Available at: https://www.tue.nl/en/research/researchers/maarten-steinbuch/. [Accessed 2 March 2020].

169 MSc Design Thesis: Jana Voykova

Thrun, S. (2011). TEDx Brussels 2010 - Sebastian Thrun - Rethinking the Automobile. [YouTube video]. Available at: https://www.youtube.com/watch?v=r_T-X4N7hVQ [Accessed 23 April 2020].

Thrun, S. and Schmidt, E. (2016). Let’s stop freaking out about Artificial Intelligence. [Fortune]. Available at: https://fortune.com/2016/06/28/artificial-intelligence-potential/ [Accessed 23 April 2020].

Toronto Waterfront. About us. [online]. Available at: https://www.waterfrontoronto.ca/nbe/portal/waterfront/Home. [Accessed 8 April 2020].

Toyota Research Institute. Automated Driving at Toyota: Vision, Strategy and Development. [online]. Available at: https://automatedtoyota.com/why-toyota-is- developing-automated-driving-technology/. [Accessed on 20 March 2020].

Travis, D. and Hodgson, P. (2019). How to approach desk research. In: Think like a UX researcher: how to observe users, influence design and shape business strategy. New York: CRC Press, an imprint of Taylor & Francis Group.

Uher, J. (2016). What is Behaviour? And (when) is Language Behaviour? A Metatheoretical Definition. Journal for the Theory of Social Behaviour (2016) 46: pp. 475- 501.

United Nations. (2019). The Sustainable Development Goals Report 2019. New York: United Nations.

Verma, S. (2018). BBC Documentary - Hyper Evolution: Rise Of The Robots. [YouTube video]. Available at: https://www.youtube.com/watch?v=9SA2WLDB65A [Accessed 25 February 2020].

Vulpian, A. (2008). Towards the Third Modernity: extracts. UK: Triarchy Press.

170 MSc Design Thesis: Jana Voykova

Wakefield, J. (2019). The Google city that has angered Toronto. [BBC News]. Available at: https://www.bbc.com/news/technology-47815344. [Accessed 23 April 2020].

Whitworth, B. and Adnan, A. (2013). The social design of technical systems: building technologies for communities. Denmark: The Interaction Design Foundation.

Wolff, R. (2019). Understanding Marxism. US: Democracy at Work.

Wolff, R. About Richard D. Wolff. [online]. Available at: https://www.rdwolff.com/about. [Accessed 16 May 2020].

World Economic Forum. (2017). The Smart City Revolution: Where's It All Going? [YouTube video]. Available at: https://www.youtube.com/watch?v=mXtpdby9JSQ&t=1476s. [Accessed 7 February 2020].

World Economic Forum. (2020). Jobs of Tomorrow: Mapping Opportunity in the New Economy. Switzerland: World Economic Forum.

World Economic Forum. (2020). Net-Zero Challenge: Fast-Forward to Decisive Climate Action Report. Switzerland: World Economic Forum.

World Economic Forum. (2020). The Future of the Last-Mile Ecosystem. Switzerland: World Economic Forum.

World Economic Forum. (2020). The Global Risks Report 2020. Switzerland: World Economic Forum.

World Health Organisation. (2018). Global Status Report on Road Safety 2018. Geneva: World Health Organization.

171 MSc Design Thesis: Jana Voykova

World Population Review. Middle Income Countries 2020. [online]. Available at: https://worldpopulationreview.com/countries/middle-income-countries/. [Accessed 21 February 2020].

Zimmerman, J., Forlizzi, J. and Evenson, S. (2007). Research Through Design as a Method for Interaction Design Research in HCI. Human-Computer Interaction Institute. Paper 41.

Zimmerman, J., Stolterman, E. and Forlizzi, J. (2010). An Analysis and Critique of Research through Design: towards a formalization of a research approach. DIS 2010, August 16-20, 2010, Aarhus Denmark.

Zuboff, S. (2019). The age of surveillance capitalism. New York: Hachette Book Group.

172 MSc Design Thesis: Jana Voykova

Acknowledgments

This thesis is a product of my encounters and interactions with some remarkable people.

I don’t know them personally, but I am utterly grateful for the work of all the people, scholars and academics I have cited in this master thesis (even the ones I criticise and do not agree with). Their efforts helped me to put my critique in order and try to look at the subjects of my research more objectively and rationally.

I am indebted to many people with whom I had a lot of insightful conversations on the topics of this thesis, especially Zhivka Koleva who brought in key insights that aided my argumentation. Two amazing designers and individuals who supported me immensely deserve special acknowledgment — Petia Koleva for always being my second pair of eyes and understanding my frustration, and Sarah Kathrin Glaßner for showing me that there is another approach to a problem and another side to the solution.

I am dearly grateful to each person who gave me of their personal time (some of them without even knowing me) without getting anything in return: to do an interview or express their opinion by filling in a survey or welcoming my point of view in my prototype exhibition. I have not mentioned their names because I promised them anonymity, but they know who they are.

Most of all, I am appreciative of my family, for allowing me to be an idealist in an unideal world. I owe my privileged position solely to them and if it wasn’t for them, I wouldn’t even have had the opportunity to start, let alone finish my education. If I graduate successfully, they would probably have just as much right to claim the academic title.

173 MSc Design Thesis: Jana Voykova

Appendices

A. The Ethics Centre: Principles for good technology

In their paper Ethical by Design: Principles for good technology, The Ethics Centre outline a framework of eight principles which technology designers should follow in order to make sure that the designs and technologies they create are ethical and mindful of the impact they have on people. These principles are explained in detail in the paper, however, for the sake of brevity, here I provide a summary of what they are, what they mean and stand for:

PR00. Governing principle: Ought before can

The fact that we can design something, doesn’t mean that we should. We should make sure that what we build helps us to create the best possible world we can. As Matt Bear and Simon Longstaff put it ‘Adherence to ethical principles should trump all other opportunities and concerns. No technology is useful or ground-breaking enough to justify ignoring ethical concerns’ (Beard and Longstaff, 2018, p. 61).

PR01. Non-instrumentalism

We shouldn’t design things that treat people as mere instruments to our ends. Certain things in life have intrinsic value and therefore, shouldn’t be treated as cogs in the wheel. Our creations should benefit people, ecosystems, political communities, human relationships, in other words things we should respect for what they are, not what they can bring us. This is what Michael Sandel also argues in his analysis of how markets penetrate areas of life which should be valued for their moral worth, not their market price.

THE RULES OF THUMB:

Don’t reduce people to the status of mere “things” — technology should not treat intrinsic goods as transactional and instrumental. It shouldn’t make trade-offs if they are going to harm things of intrinsic worth;

Sacrificial tech — we should weigh a technology’s value against the value of intrinsic goods and the latter should always override the former;

174 MSc Design Thesis: Jana Voykova

People aren’t parts — Things with intrinsic value aren’t part of technology, they are part of the purpose for which technology exists;

Don’t manipulate — our designs shouldn’t exploit or manipulate people as people are not means to anyone else’s ends;

Promote dignity — we shouldn’t help people disrespect their own dignity, even if this means limiting their liberty, for example, we shouldn’t allow people to sell themselves into slavery even if they are willing to do so.

PR02. Self-determination

People are autonomous beings. We shouldn’t design technology to limit our ability to choose for ourselves. This doesn’t mean that the freedom we give people should be limitless, either because this can have a paralysing effect. Our designs should support humans in their choices, not restrict them.

THE RULES OF THUMB:

Non-user immunity — our designs shouldn’t treat people as collateral damage, for example there are ethical questions to be answered about technologies which change how people use physical spaces or include non-users within a network without their consent;

User sovereignty — the users of our designs and technologies should be aware of what using them involves and should have unalienable rights to the products of their use, like data, artwork, intellectual property. They should have the right to ownership of these assets;

Nudging — nudges are design choices that consciously influence how people act. This is a sensitive topic because nudging, done even with noble purposes, can be seen as an infringement on our self-determination. However, there are ways to nudge ethically. We should nudge toward rationality, for example nudging against biases that cause discrimination. We should also nudge toward virtue, for example aiding people to become better versions of themselves and help each other more. When nudging, we should take into consideration the long-term consequences of this nudge, but in order to do that we need to be aware of “the plan of life”, or the goals of the particular person or group we are trying to nudge.

175 MSc Design Thesis: Jana Voykova

PR03. Responsibility

We should anticipate the potential harmful or unproductive uses of our designs. Designers have responsibility towards the people they design for and so they should be mindful of the intended purpose of their design, the affordances it has (what it allows/restricts the users to do with it), and the repurposing and values embedded in their designs. Designers cannot be held accountable for all possible implications of their designs, though. The responsibility of the design and use of something is shared between the people who design it and the ones who use it. There may be a market demand for something, but designers still have a duty to say “no” if the requested product is going to have foreseeable unethical implications.

RULES OF THUMB:

Update and recall — the group of people designing a technology has the obligation to update or recall it in case they discover that it might defect or cause unanticipated harm;

Rights custodianship — when people are required to hand over some element of their moral rights in order to use a technology/design, the people they are handing it over to, have an obligation to guard them as their own rights;

Transparent provenance — the complete history of use of artefacts and devices should be available to its current owners/custodians or users;

Manage repurposing — Designers should ensure they have taken all reasonable, ethically permissible steps to minimise harmful repurposing. This may mean to design the technology in ways that ‘nudge’ against abuse or restrict the extent to which design affords repurposing.

PR04. Net-benefit

We should design technology that makes a positive contribution to the world. Even if it does more good than bad, ethical design requires us to reduce the negative effects as much as possible. Defining what good/beneficial means can be challenging. It should be measured against the following criteria:

• Human wellbeing

• Individual freedom

• Improvement of the moral character of people

• The common good

• Inspiring wonder and hope for the future

• Access to basic needs

176 MSc Design Thesis: Jana Voykova

• Protection of human rights and dignity

• The state of the environment

In that sense, the fact that a technology brings utility or benefit, is not enough if some or all of the other principles of ethical design are ignored.

RULES OF THUMB:

Acknowledge opportunity costs — time and resources spent on developing one technology may not be equal to the time and resources spent on another. We should do the most good we can, we should dedicate energy to develop technologies in proportion to the potential benefits each development offers;

Anticipate side-effect — we should be mindful of foreseeable side-effects of the technologies we design and should aim to restrict or eliminate them;

Minimise harm — potential harms must be addressed even if this means investing more resources;

The ends don’t justify the means — we should design technology in ways that it does the right thing in the right way. Unless a technology satisfies all of these ethical design principles, it shouldn’t be designed.

PR05. Fairness

Different designs can reflect biases, support the status quo and generate blind spots which results in different groups of people being treated differently. We should be able to judge different cases on their own and justify why there are differences in the ways our designs treat different people.

RULES OF THUMB:

Treat like cases alike — we should ensure that the basis for distinctions is non-arbitrary and based on real and significant differences between cases;

Treat different cases differently — we should follow the principle of impartiality, in other words, we should not make false equivalences between cases. Edge cases are discriminatory;

Natural justice — if someone has objections toward a design/technology, they should be given due process and impartial assessment of their objections;

177 MSc Design Thesis: Jana Voykova

Minimise moral hazard — moral hazard implies a situation where the person who invests in something, leaves the potential suffering/harm to the other people involved if the investment fails. Technologies might generate advantage to one group by generating risks to another. Benefit and risk should be borne by one and the same group of beneficiaries;

Diversity in design — avoiding unconscious biases in the design of something means involving diverse teams in the design process. Diversity in that sense means diversity in gender, sex, race, class, culture, ethical profile.

PR06. Accessibility

Sometimes our designs can exclude people who are outside of our target group. Therefore, we should design in a way that enables any person with the right and need to use our design/technology, to do so. We should be mindful of designs which deliberately or accidentally exclude people who might benefit from it.

RULES OF THUMB:

No edge cases — we should design with regard to who would be likely to use our designs, not only who we want to use them;

Close the gap — if there are inescapable accessibility differences, active steps should be taken to close the gap as much as possible;

Knowledgeable use — access doesn’t simply mean physical access. People should be informed of how they can use a technology, what consequences it has for them, and what implications it may result into;

Show your work — information around the design process, intended purposes, potential harms and implications of a technology/product should be publicly available;

Succeed slow — products shouldn’t be designed following the “fail fast” principle, their release on the market should happen after proper testing, bug-fixing, and assessment has taken place, not before.

178 MSc Design Thesis: Jana Voykova

PR07. Purpose

Design is intended to serve a specific purpose and solve a specific problem. Good design serves an ethical purpose and does so in effective and efficient ways. This means that when designing, we need to be aware of our ability to address the problem we are trying to address and be honest about the limitations of our solutions.

RULES OF THUMB:

Legitimate purpose — technology must aim to contribute to the common good of humanity, if it’s aimed to fulfil evil purposes or even neutrality, it’s not justifiable no matter how responsibly it has been designed;

Clarity of purpose — every technology should be created clearly communicating its purpose and characteristics;

Honesty of purpose — we should present the technologies we create in an objective manner, we shouldn’t oversell them or taint their ability to serve the intended purposes;

Principled effectiveness — technology should be able to achieve its purpose as effective as possible without harming the other ethical principles in this framework;

Principled efficiency — technologies should be able to fulfil their purpose as efficiently as possible within the constraints of the other means.

As Matt Beard and Simon Longstaff point out, this framework is aimed to make considerable contributions to achieving ethical design and technology, however, it’s not a silver bullet. They based these principles on a reflective equilibrium combining the laws and principles of several ethical theories. Just as there is no single theory which contains the complete and utmost objective reflection of morality, so no theory can be tarnished as 100% morally wrong. ‘Not all the issues facing technology today stem from technology itself…This is important because unless we recognise the broader ethical issues at play – matters of social justice, economics, governance and so on – technology companies will still face issues in securing legitimacy and trust, and their activities will be, at least to some extent, at odds with the purpose, values and principles outlined here’ (Beard and Longstaff, 2018, p. 121).

179 MSc Design Thesis: Jana Voykova

B. Interview/survey guides

INTERVIEW GUIDE: DESIGNERS

Purpose/Goal of interview:

Understand how designers perceive their profession, what designers’ social responsibility is and how it can be demonstrated in their practice.

Questions:

1. How did you decide to become a designer? What appealed to you in the profession? 2. What do you think about when you hear social responsibility of designers? 3. Do you think designers do bear a social responsibility and if so, what does this mean for you? 4. Where in the design process does this social responsibility come? 5. What can designers do to drive it forward? 6. Do you think that designers have the power to influence how society is evolving? 7. How can they claim and demonstrate this power? 8. Are they given the opportunity to demonstrate it? 9. Who determines what people’s needs are (when creating a solution/service/product and how they should be addressed? 10. How involved do you think designers are in this process? 11. What gives you satisfaction in your work? 12. What scares you about your profession? 13. What, in your opinion, are the biggest challenges before the design profession currently? 14. How do you see the design profession evolving in the foreseeable future?

SURVEY GUIDE: The Social importance of designers Consent Form

180 MSc Design Thesis: Jana Voykova

INTERVIEW GUIDE: AUTONOMOUS TRANSPORTATION

Purpose/Goal of interview:

Research how people understand the social impact of autonomous transportation and find out if they explicitly search for information on the topic. I introduced some previous research done in the field to see if it affects people’s preconceived opinion.

Questions:

1. How much do you know about the progress of development of autonomous vehicles? 2. What do you think about this technology? 3. Where do you take information from? 4. Do you think that the general public knows enough about autonomous transportation so that they can form a reliable opinion about it? 5. Is there anything around autonomous transportation that is not discussed in the public space and you think it should? 6. What are the benefits of autonomous transportation? 7. What are the downsides? 8. Why do we need it? 9. Who (should) determine(s) whether we need it? 10. Are there any other measures we can implement to address the problems which autonomous vehicles will potentially resolve once introduced in the society (minimise the number of road accidents, optimise space and traffic, reduce air pollution)? If so, what are they? 11. Do you think autonomous vehicles can co-exist and perform as expected alongside vehicles driven by humans? 12. What should be the role of governments in the development of this technology? 13. Would you buy an autonomous vehicle even if cost was no issue? 14. What do you think about autonomous transportation displacing professional drivers? Is that a legitimate concern? 15. In order to perform as expected, autonomous vehicles need to collect a lot of data not only about individual cars or people but also about the surrounding environment. What do you think about that? Does this bother you? 16. According to a 2018 research conducted by the Brookings Institution in Washington, US (a non-profit public policy organisation which does research on ideas for solving social problems), surveying 2,066 adult internet users, 61% said they were not inclined to ride in a self-driving car, while 21% said they were. They were then introduced with data which indicated that 40,000 Americans die in highway

181 MSc Design Thesis: Jana Voykova

accidents each year, and it is estimated that 90 percent of highway fatalities involve human error, distraction, or intoxication. Asked if they thought self-driving cars would reduce those numbers, 23% said yes and 55% said no. What do you think about those results? How would you respond to these questions yourself? Why? 17. According to WHO’s Global report on road safety for 2018, the road accident rate is 3 times higher in low-income countries than in high-income ones. The report quotes bad infrastructure, driver distraction, slow or no adoption of global best practices for road safety, accountability uncertainty, as main challenges remaining to prevent accidents. What do you think about these results? Is autonomous transport going to resolve the above-mentioned issues? 18. In 2019, the global law firm Perkins Coie LLP and the Association for Unmanned Vehicle Systems International (AUVSI, a non-profit organisation devoted to advancing the unmanned systems and robotics), issued a report based on a survey with 260+ leaders in the automotive and technology industry, state and federal regulators in the US. They quoted that the biggest obstacle before the successful launch and implementation of autonomous vehicles in society, is general concern with liability and safety. How should proponents of this technology tackle this scepticism? 19. In 2017, scholars from the Massachusetts Institute of Technology in Cambridge surveyed 2.3 million people around the globe as part of a research project called “The Moral Machine”. They asked people from 233 countries to answer 13 hypothetical crash scenarios with autonomous vehicles, where the vehicle has to make a decision about whom to sacrifice. Most people said they would rather save groups of people than individuals, young than old, humans than animals. There were differences in the answers between countries, in egalitarian countries with strong government presence, respondents would sacrifice illegally crossing people to those abiding by the law, in poorer countries, there wasn’t a preference to kill older citizens to younger, in countries with significant economic disparity, the preference was to kill a lower- status person. There are generally two camps when introducing the ethical dilemma into the autonomous transportation question: one group of researchers and people say those scenarios and decisions are unrealistic as they will occur fairly rarely, others support that we should consider ethical dilemmas when developing the technology. What do you think about this, given the fact that there is no single “right” decision on those topics and moral compass looks different in the different countries? How should we go about those aspects (ethical dilemmas) of the autonomous vehicles’ conversation?

SURVEY GUIDE: Social impact of autonomous vehicles Consent Form

182