Development (2019) 62:66–80 https://doi.org/10.1057/s41301-019-00208-1

DIALOGUE SECTION

Governance and Assessment of Future Spaces: A Discussion of Some Issues Raised by the Possibilities of Human– Mergers

Andelka M. Phillips1,2 · I. S. Mian3

Published online: 9 October 2019 © Society for International Development 2019

Abstract This article explores potential privacy, security, and ethical issues raised by that allow for human–machine mergers. The focus is on research, development and products at the intersection of robotics, artifcial intelligence, , and smart computing. We suggest that there is a need for a more holistic approach to the assessment of such and its . We argue that in order to determine how the law will need to respond to this particular future space, it is necessary to understand the full impacts of human–machine mergers on societies and our planet—to go beyond these three aforementioned issues. We aim to encourage further discussion and research on this as well as the broader organism-machine merger question, including on our ­FLE5SH (F = fnancial, L = legal, ­E5 = economic, ethical, equity, environmental, and eco- system, S = socio-political, H = historical) framework for the governance and assessment of these and other future spaces.

Keywords Human–machine merger · Technology assessment · FLE5SH · G.O.A.T.S · Precautionary principle · Artifcial intelligence

Introduction altered our world and while there will be some gains from many of these advances, most technologies pose both risks Today, it seems we stand at the beginning of an age of ubiq- and benefts and are not in themselves neutral. Since there uitous computing and attempts to merge the physical, digital, will be both winners and losers, there is a need for a broader and biological realms.1 This is also a time increasingly of assessment of the impact of new technologies on society as technological convergence (Kearns 1998: 975; O’Brolcháin a whole, the environment, and the planet. et al. 2016; Perakslis et al. 2016), with an ever-larger array Developments in Artifcial Intelligence (AI) and com- of objects having Internet connectivity. All of this poses sig- puting are often viewed as transformative technologies. nifcant risks for individual and group privacy and security However, given the potential that future developments in (Weber 2010; European Commission 2013; Global Privacy these related felds have to alter our natural and built envi- Enforcement Network 2016; UK Information Commis- ronment-, impacting not only humans, plants, and animals sioner’s Ofce 2016), but it also raises further issues for but also entire ecosystems, there should be a wider debate environmental, human, and animal health, as well as the about not only regulation and assessment of technology, prospect of unemployment for many as jobs are increasingly but also the type of world we want to live in. Furthermore, automated (Frey et al. 2016; Solon 2016; Williams 2017). although these technologies are often presented as trans- Developments in computing technology have drastically formative, developments in these related felds often have limitations (including misinterpreting data and failing to distinguish between things that humans would be able to * Andelka M. Phillips correctly identify (Jordan 2018; Vincent 2018; Broussard [email protected] 2018: chapter 1)) and current advances in AI are best viewed as giving rise to narrow AI (Jordan 2018; Bostrom 2014: 1 Te Piringa ‑ Faculty of Law, University of Waikato, Hamilton, New Zealand 1 An earlier version of this article was presented as a discussion paper at the Data For Policy 2017: by ? Con- 2 HeLEX Centre, University of Oxford, Oxford, UK ference held in London, 6–7 September, 2017, Phillips and Mian 3 Department of Computer Science, University College (2017); this article builds also upon our related work Phillips et al. London, London, UK (2015) and Phillips (2016).

Vol:.(1234567890) Governance Challenges with Human-Machine Mergers 67

14–16). A September 2019 Royal Society report argues fait accompli and further, if humans are to be re-engineered ‘Linking human brains to computers using the power of arti- in this way then the matter must be subject to extensive pub- fcial intelligence could enable people to merge the decision- lic debate, scrutiny, and regulatory oversight (for example, making capacity and emotional intelligence of humans with recent discussions of such issues in the biosciences can be the big data processing power of computers, creating a new found in Newmann and Stevens 2019a, b; HEGAAs http:// and collaborative form of intelligence.’ (Royal Society Steer- web.evolb​io.mpg.de/HEGAA​s/). Broadening our purview, ing Group on Neural Interface Technologies 2019: 15). This the same arguments apply if we generalize to organism- proposed merger of neural interfaces (NI) and AI ‘could machine mergers where the term ‘organism’ encompasses open the way to game-changing applications ... However, individual or groups of microbes and/or macrobes—entities the prospect also raises a number of ethical issues concern- such as bacteria, fungi, viruses, plants and animals (includ- ing our autonomy, privacy and perception of ‘normality’.’ ing humans)—and ‘merger’ refers to the two components (Royal Society Steering Group on Neural Interface Tech- ‘working together’ so that boundaries between them become nologies 2019: 49). blurred. Since coordination and control of such hybrid enti- There is signifcant interest and investment in technolo- ties will require tighter coupling of their activities, we antici- gies that increase connections between humans and com- pate and expect closer interactions between and alignment of puters. Key here have been developments in: AI (Simonite the molecular (Suda and Nakano 2018; Aky- 2017; Peet and Wilde 2017; Patterson 2017; ACM 2018); ildiz et al. 2019) and organism-machine future spaces. That (ML); wearable technology (such as FitBit is, the rise of research and development into technologies and Garmin); Virtual Reality (such as Oculus Rift, HTC inspired by chemical communication within and between the Vive, Samsung Gear VR, and Neurable)2; implants (such (a)biotic worlds and applications attempting to manipulate as Northwestern University’s tiny antennas (Dormehl 2017; human behaviour (Kupferschmidt 2019; Chemical commu- Nan et al. 2017) and Elon Musk’s Neuralink venture (Con- nication in humans 2019; Schmidt et al. 2019) much in the stine 2017a; Lopatto 2019; Statt 2017)); brain to computer same way as, for example, chemical ecology-inspired bio- interfaces (such as Neurable and Neurovigil’s iBrain3), and active molecules are used for pest management (Beck and exoskeletons and bionic limbs (such as Berkeley’s Lower Vannette 2016). Extremity Exoskeleton (BLEEX) and Human Universal As new and emerging technologies combine to facilitate Load Carrier (HULC)4). Many of these projects could allow the potential merger of a wide range of corporeal bodies, for humans to be augmented, enhanced, and altered. For there will also be diferences in the degree to which the example, implants could enable extra senses. Meanwhile, intercommunicating components are impacted. For instance, bionic limbs and brain-to-computer interfaces could alter organisms may do all or part of the computing for capacities and cababilities, which would in turn permit some (organism computation), may work together with assistance form of motor and/or thought control (further examples can from computers (computer-supported cooperative work), be found in Royal Society Steering Group on Neural Inter- may live lives intermingled with computer systems (social face Technologies (2019)). computing and computer-mediated communication), and If successful these aptitudes and facilities could change may form organism-robot hybrids (cybernetic organisms). the very nature of what it means to be human. Some conjec- Organism-machine mergers are examples of perhaps one of ture that humans in their current form will be replaced by the most important future spaces: systems with physical-bio- a posthuman or transhumanist future (Barfeld 2015: 1–20) logical-digital interfaces at the microscopic-, mesoscopic-, and that human–machine merger is inevitable (Barfield and/or macroscopic-scale. Genetic material (digital sequence 2015: 1–2) However, we suggest that this scenario is not a information pairings such as the genomes of agricultural crops and livestock (Hammond 2017)) are molecular-level exemplars. 2 Neurable http://www.neura​ble.com. Accessed 25 July 2019. Written from a broadly law perspective, the overall goal 3 Neurable, http://www.neura​ble.com accessed 25 August 2017; herein is to initiate a discourse between policy makers, law- Neurable. Neurable Funded to Power Brain-Controlled Virtual Real- makers, the general public(s), and industry on how to (1) ity. Press Release, December 2016, http://www.neura​ble.com/news/ think5 about governance and assessment in future spaces)6, neura​ble-funde​d-power​-brain​-contr​olled​-virtu​al-reali​ty accessed 25 August 2017; Metz (2017); Neurovigil, http://neuro​vigil​.com/index​ (2) develop appropriate legal and regulatory frameworks .php/techn​ology​/ibrai​n-devic​e. Accessed 25 July 2019; Suzuki et al. 2007. 5 4 Berkeley Robotics & Human Engineering Laboratory BLEEX, See note 1. This article builds upon and extends work presented http://bleex​.me.berke​ley.edu/resea​rch/exosk​eleto​n/bleex​/. Accessed frst in 2015 and subsequently in 2016 (Phillips et al. 2015,; Phillips 25 July 2019; Berkeley Robotics & Human Engineering Laboratory 2016). Human Universal Load Carrier (HULC), http://bleex​.me.berke​ley. 6 See for example Newmann and Stevens 2019a, b; HEGAAs http:// edu/resea​rch/exosk​eleto​n/hulc/. Accessed 25 July 2019. web.evolb​io.mpg.de/HEGAA​s/. 68 A. M. Phillips, I. S. Mian for existing technologies, and (3) facilitate free access to protection of an individual’s privacy? Will there be privacy meaningful information on advances in felds and impacts settings for an individual’s brain? How can we ensure that of technologies to local, regional, national and international an individual has control over their body and mind and is stakeholders. This is because whilst our primary focus is free from manipulation of their thoughts and bodies by third developments and ideas that could enable human–machine parties?7 What happens if malware could afect the human mergers, we believe it is also necessary to draw attention to brain? How do we ensure security of the human brain and advances in felds such as molecular communication, nano- body? How would it afect communication between the brain technology, genome sequencing, CRISPR, and gene drives. and the gut (Martin et al. 2018; Hu et al. 2019)? Normally, For instance, these technologies could allow a wider range before we allow drugs and medical devices to be marketed, of sensors to be featured on clothing or a person’s skin and they are subject to oversight and pre-market review. How could enable various entities to be implanted into humans, do we ensure that any implant is safe for human, plant, and animals, and plants. This could in turn permit (genetic) animal use before it is made widely available? modifcation of all or parts of many diferent life forms, and Further ethical and legal issues include: if there are vari- use micoorganisms to manipulate the complex behaviours of ous forms of humans, some augmented and some not, who animals (Rohrscheib and Brownlie 2013). Examples include: will be entitled to the protection of human rights? Could spinach leaves that have been embedded with carbon nano- distinctions be made between an augmented human and a tubes to detect explosives (Trafton 2016); the implantation robot that did not have a genetic link to the human species? of self-destructing nanobots in mice (Gao et al. 2015; Sep- How do we implement consent in the context of brain to pala 2015); commercial genetic tests; and gene editing of computer interfaces or other technologies that enable the plants, insects such as mosquitos, and now human embryos human body to be connected to the Internet? How should (Young 2017; Ma et al. 2017; Sanders 2017). Here, issues society address the loss of gainful employment and increased such as (bio)security, (bio)safety, and (bio)privacy impacts economic inequality produced by robot- and/or computer- on ecosystems and consent need to be considered (Reeve guided automation? How do we ensure the protection of et al. 2018; HEGAAs http://web.evolbio.mpg.de/HEGAA​ s/​ ; individual autonomy in this context? For instance, medical African Centre for Biodiversity 2019; Borger 2019). How law often afords strong protection to the rights of patients to ensure that genetically altered organisms—whether or not to refuse treatment.8 How will this play out if a govern- merged with machines—are not released into the environ- ment wanted to require its citizens to have microchips, as is ment accidentally is a vital issue that needs further attention. already required for animals such as dogs and cats? There These felds are one prong of a trend towards merging are some existing examples of this: SJ, a Swedish train com- the physical (built and/or natural) world with the cyber pany has introduced implanted microchips for its passengers world. This can also be seen in developments in the Inter- as a form of biometric train ticket (Cofey 2017) whilst two net of Things (IoT) and in smart computing systems more companies, the Swedish startup Epicenter and the American broadly—for instance, the rise of smart buildings and the Three Square Market, have introduced microchips for their connection of critical infrastructure such as the electrical employees (Brooks 2017; Grimm 2017; Michael et al. 2017; grid to the Internet. Reducing energy consumption and News.com.au 2017; Sheppard 2017; Solon 2017; Associated improving efciency are desirable goals. However, mak- Press 2017). ing an entire country’s energy supply reliant on the Internet As more of the natural and built environment are con- introduces risks and vulnerabilities, such as a large-scale nected to the Machine, such issues are amplifed. If humans attack disabling the power supply of an entire nation. If become part of a telecommunication network such as an power plants, dams, and other infrastructure have not been Internet of Everything,9 where our thoughts can be read, maintained properly, connecting them to the Internet may monitored, and potentially manipulated, then it will be very not necessarily improve their reliability or security. Hence, difcult to turn back the clock. An illustrative example programmes aimed at maintaining the physical security of facilities and ensuring the cyber security of industrial control systems are critical and necessary investments (Mo et al. 7 For example, Pfeifer et al. (2015) and Frischmann and Selinger 2012; Hahn et al. 2013; Tuptuk and Hailes 2016). (2018: 30–32) discuss actuated navigation. To a large extent attempts to merge humans with machines 8 Phillips (2017: 285) citing Montgomery (Appellant) v Lanarkshire depend on a mechanistic perception of both humans and the Health Board (Respondent) (Scotland) [2015] 2 All ER 1031, [2015] human brain and of a view of intelligence as computation UKSC 11 and Campbell (2015). (O’Connell 2017: 55–6). Developments that link humans 9 Internet of Everything (IoE) Group, https​://ioe.eng.cam.ac.uk/. with machines by direct means such as implants or brain Accessed 25 July 2019—here IoE is defned as ‘a seamless intercon- nection and autonomous coordination of massive number of comput- to computer interfaces raise a number of issues for privacy, ing elements and sensors, inanimate and living entities, people, pro- security and ethics. These include: how can we ensure the cesses and data through the Internet infrastructure’. Governance Challenges with Human-Machine Mergers 69 of this point is Facebook’s announcement that it wants to sibility of building intelligent machines, more often develop a brain to computer interface (Constine 2017b; than not androids with human features (Committee on Strickland 2017). It has already emerged that Facebook Legal Afairs. 2016). does monitor what its users type and delete without post- These depictions may also be infuencing inventors and ing (Sørensen 2016). Imagine if this was not just a matter shaping what they expect to develop (perhaps both con- of typing words on a screen, but a direct link to someone’s sciously and unconsciously). They may also be employed thoughts. This would potentially reduce privacy in quite a in marketing to foster acceptance of (bio)technology. revolutionary way, compounding the challenges we already In this article we seek to draw attention to some of the face with targeting marketing and online behavioural adver- issues raised by developments in this feld and to encourage tising (Duhigg 2012; Lubin 2012; Papadopoulos et al. 2017; discussion of not only appropriate regulation, but also tech- Narayanan and Reisman 2017). A well-known example of nology assessment—a task for which, in previous work, the ways that businesses can obtain information about cus- we have proposed the FLE­ 5SH (F = fnancial, L = legal, tomers is that of Target, a company that was able to make ­E5 = economic, ethical, equity, environmental, and ecosys- predictions about whether a customer was likely to be preg- tem, S = socio-political, H = historical) framework (Phillips nant based on the purchase of 25 products and then engaged et al. 2015; Phillips and Mian 2017). in targeted market with coupons for baby products (Ellen- However, we also wish to highlight the more recent pro- berg 2014). Ensuring security of these types of technologies posal by the ETC Group of Global Overview Assessments is a signifcant challenge that should not be underestimated of Technological Systems (G.O.A.T.S). ETC presented the (Bonaci et al. 2014: 47; Li et al.2015: 663–666). Software ‘G.O.A.T.S approach to Science, Technology, and Innova- based systems are prone to many vulnerabilities and recent tion (STI)’ Governance at the UN STI Forum in May 2017 research has demonstrated that it was possible to implant (ETC Group 2017a, b: 1). The G.O.A.T.S provides for a malware into synthetic DNA (Ney et al. 2017; Tracy 2017; ‘bottom up ‘technology landscaping’ project involving Greenberg 2017; Timmer 2017). multi-actor assessment organised thematically around the As technological convergence increases, there is the 17 SDGs’ (Sustainable Development Goals) (ETC Group potential for Big Social Engineering, which raises further 2017a). As the ETC Group notes ‘Technology is established questions. These include: How can we ensure transparency as a key cross-cutting theme of the 2030 Agenda for Sustain- about the full functionality of particular technologies? How able Development which charts a path to the future for gov- can we ensure that people have access to information about ernments, and 13 of the 17 SDGs specify that technological technologies that may be used to infuence them without solutions will be necessary to achieve them.’ (ETC Group their consent or knowledge so that they can make informed 2017a) This approach can ofer a means for ‘policymakers, choices about whether to use particular technologies and civil society and others to better perceive and navigate the reject adoption if they want to? What kind of pre-market innovation landscape’ considering both ‘the potential prom- review should social engineering technologies be subject ises and pitfalls’ of technologies (ETC Group 2017b: 1–2). to? What rights will people have to their private thoughts? We support this approach and our aim with FLE­ 5SH is to Could there be a privacy setting for a person’s brain and facilitate a similarly broad multi-dimensional assessment of what will happen if this is overridden? How can we ensure technologies. that existing rights and freedoms are protected? What about security and control? How will ‘brain hacking’ allow peo- ple to be infuenced or conditioned to act in particular ways AI and Augmented Humans without conscious knowledge of this infuence? What are the consequences when applications encourage addiction? Developments in science, technology, engineering, math- Depictions of AI, cyborgs, and androids from science ematics and medicine (STEMM) promise a tomorrow fction also exert a signifcant infuence on how many view where ‘errors’ or ‘defciencies’ in an organism’s genetic innovations in these felds, (Calo et al. 2016: 1–22) as well and/or phenotypic makeup can be modulated, enhanced, as infuencing lawmakers (Walter 2016; Warwick 2016). corrected, redefned or eradicated. A post-human world A good example is the EU call for civil laws on robotics. could be populated by people who have additional senses, The text of the European Parliament Committee on Legal such as artifcial eyes equipped with video cameras and Afairs’ Draft Report begins in paragraph A: the ability to feel electromagnetic pulses, enhanced intel- whereas from Mary Shelley’s Frankenstein’s Monster ligence, and direct connections with computers through a to the classical myth of Pygmalion, through the story variety of mechanisms including Virtual Reality, Augmented of Prague’s Golem to the robot of Karel Čapek, who Reality, prosthetics, implants, and other forms of brain to coined the word, people have fantasised about the pos- computer interfaces. Such beings may be human on some 70 A. M. Phillips, I. S. Mian level and machine on another, but they will not be able to suggestion is that AI will improve itself and quickly surpass retain privacy (or security) of their own thoughts. In the human intelligence. Bostrom defnes super intelligence as 2018 American science-fction dark comedy flm ‘Sorry to ‘any intellect that greatly exceeds the cognitive performance Bother You’, workers are made stronger and more obedient of humans in virtually all domains of interest.’ (Bostrom by transforming them into ‘equisapiens’—a half-human, 2014: 22–23). His work is timely and of great value to this half-horse hybrid created when a human snorts a gene- discussion. The book concludes with an analogy of the modifying powder (Riley 2018). development of super intelligence with a child playing with Already, there are a number of products and services on a bomb (Bostrom 2014: 260–61), a very useful starting point the market that are part of the Quantifed Self movement. to highlight the importance of paying sufcient attention to These range from direct-to-consumer genetic tests through getting this right. wearable ftness monitors such as FitBit and Garmin to It is also important to understand that the development of FashTech which incorporates sensors into clothing, exam- a super intelligent AI is not at present a forgone conclusion, ples are heart rate monitoring bras such as the Mi Pulse although a number of experts do view it as likely. How- Smart Bra and the Vitali Everyday Smart Bra.10 Some of ever, if this does come to pass, it does not necessitate that these devices have already begun to be used in the court- all humans must be augmented and merged with machines. room (Chauriye 2016; Jackson et al. 2017). The results of These are separate issues that are both in need of further the Global Privacy Enforcement Network’s 2016 Privacy attention. There is growing attention and concern over the Sweep of IoT highlighted problems with companies’ com- safe development of AI technology. The letter calling for munication with consumers regarding privacy and security a ban on lethal autonomous weapons released at the Inter- practices, as well as the sending of unencrypted informa- national Joint Conference on Artifcial Intelligence (IJCAI tion by medical devices (Irish Data Protection Commis- 2017) is an example (Vincent 2017; Future of Life Institute sioner 2016; Privacy Commissioner (New Zealand) 2016; 2017a, b). Others include the ‘Partnership on AI’12 and ‘A UK Information Commissioner’s Ofce 2016). Meanwhile, Unifed Framework of Five Principles for AI in Society’ research by Citizen Lab and Open Efect as well as HPE (Floridi and Cowls 2019). Fortify (HPE 2015; HP 2015; Hilts et al. 2016; Norwegian We cannot predict what the interests of a super intelli- Consumer Council 2017) has demonstrated that a number gent AI will be and we support the calls for more discussion of such devices (including ftness bands and smart watches) and oversight of this area. Recent research from ’s are prone to security vulnerabilities and that it is possible DeepMind has shown that AI can behave both collabora- to create a false ftness record on some devices. This is a tively and in more aggressive ways (Burgess 2017; Leibo signifcant issue if such devices are to be relied upon as et al. 2017; International Foundation for Autonomous evidence in the courtroom. Furthermore, as more forms of Agents and Multiagent Systems 2017). Since AI may personal information are collected and linked, there is an behave unpredictably and before we get to the advent of increasing risk to informational privacy for individuals and a super intelligent AI, it is vital that we understand more their families (Drabiak 2017). about how less advanced AI operate and what their interests There is also growing concern and debate around technol- could be. There is a growing literature, particularly in the ogy design and specifcally the issue of technologies being context of autonomous (Bradshaw-Martin and Eas- designed to be addictive, as well as the impact of the use ton 2014; Bonnefon et al. 2015; Etzioni and Etzioni 2016), of screens on children and young people.11 Germany has about the need for coding in human values into AI systems. banned the sale of smartwatches to children and it is possi- Although this seems advisable, since humans do not always ble that other countries will also begin to restrict the sale of share all values (Mignolo 2010, 2013; Grosfoguel 2012) per- certain products to young people (Johnsen 2016; Wakefeld haps one option is a requirement for some form of balancing 2017; O’Brien 2018). and explanation, which could assist AI to make decisions In recent years there has been increasing interest in the contextually, allowing for consideration of a number of fac- idea of approaching technological singularity or as Nick tors. An example from science fction can demonstrate this Bostrom terms it, an intelligence explosion (Bostrom 2014: point. In Arthur C Clarke’s 2001: A Space Odyssey, Hal is 4; 62–77). The basic premise here is centred around creating taught to lie, cheat, and deceive humans. Hal’s abilities are human level machine intelligence. Once this is achieved the linked closely with the achievement of particular goals, in this case the completion of Hal’s mission (Clarke 2016). However, much of what he is designed to do is not balanced

10 Mi Pulse, https​://www.mi-pulse​.com. Accessed 1 March 2018; Vitali, https​://vital​iwear​.com. Accessed 1 March 2018. 11 For example, see Atler (2017); Harris (2016); Tristan Harris’ web- 12 Partnership on AI, https​://www.partn​ershi​ponai​.org/. Accessed 25 site http://www.trist​anhar​ris.com/essay​s/. Accessed 25 July 2019. July 2019. Governance Challenges with Human-Machine Mergers 71 out by explanation. The point here is that in order for AI 2019). An interesting initiative to combat this throwaway and humans to work together successfully, AI will need to culture is the Swedish Government’s introduction of tax understand human motivations and the reasons we do or do breaks for the repair of common consumer products, includ- not behave in certain ways. Such understanding could help to ing clothing, bicycles, and washing machines (Starritt 2016). avoid AI deciding to do something that could result directly Many of these technologies involve the collection, stor- or indirectly in human extinction. age, transmission, and sharing of a variety of forms of infor- However, our concern here is also to highlight the sig- mation, which can include personal information, and sensi- nificance of developments that allow for humans to be tive information, including health, and genetic information. revamped, so that they are cyber-physical and for instance, There is growing use of cross-device and cross-platform elements of the Internet of Bio-Nano Things (Akyildiz et al. tracking, which attempts to harvest more information from 2019). While there should be discussion and oversight of individuals based on their purchasing behaviour, as busi- AI, implants, and brain to computer interfaces, other prod- nesses seek to identify whether viewing a particular adver- ucts also need attention. Since AI systems that have under- tisement results in the purchase of their products or services standing of human motivations and emotions might be use- (Chen et al. 2016; Federal Trade Commission 2017; Brook- ful in developments that merge humans and machines, the man et al. 2017). Precautionary Principle could assist or be invoked here. It There are now a growing variety of impact assessments also seems advisable to look at existing governance mecha- that are either encouraged or required by law. These include: nisms that have regulated medical devices and pharmaceu- privacy impact assessments; impact assess- tical drugs. While such systems are imperfect, they could ment; environmental impact assessments; and ethical trade be helpful in thinking further about governance of implants impact assessments. One example is that of data protection and brain to computer interfaces. Generally, it would seem impact assessments, which are required in article 35 of Euro- wise to ensure the safety of such products before implanting pean Union’s General Data Protection Regulation. These are them into people. to be carried out: Ideally, we do not want the occurrence of super intelli- Where a type of processing in particular using new gent AI to be made by a lone individual—be it a lay person technologies, and taking into account the nature, or researcher—in their basement or (computer) laboratory. scope, context and purposes of the processing, is likely Likewise, while there is a DIY biohacking movement already to result in a high risk to the rights and freedoms of (Barfeld 2015: 135–176; Bradley-Munn and Michael 2016; natural persons, the controller shall, prior to the pro- Mallonee 2017) and it is true that some individuals want to cessing, carry out an assessment of the impact of the alter their bodies in new ways, this is also something that envisaged processing operations on the protection of does need more oversight. Furthermore, the addition of new personal data.13 senses, diferent forms of implants, and brain to computer interfaces is not something that should be forced on people If we think about this in the context of technologies, such without their consent. as implants or brain to computer interfaces, it is likely that such technologies would be caught by this requirement. Our recently proposed FLE­ 5SH framework provides a Technology Assessment: Time for a Holistic new approach to help organize, interpret and assess past, Approach? extant, emerging and new research and development in STEMM (Phillips and Mian 2017; Phillips et al. 2015). A variety of technologies, such as smartphones, laptops, The nine lenses in this framework provide a more holistic tablets, wearables, as well as a burgeoning range of other approach to technology assessment and regulation. We are devices which form the IoT are now accessible and used by including such a broad range of lenses because we believe a signifcant portion of the world’s population. For example that many if not all technologies need to be assessed from Facebook now exceeds 2 billion monthly active users, (Con- as wide a perspective as possible. stine 2017c) Apple has sold more than 1.2 billion iPhones, To some extent the ­FLE5SH framework can be seen as (Morris 2017) and the number of mobile phone users will allowing the formation of a social , whereby all likely exceed 5 billion in 2019 (Statistica 2017). However, while technological solutions are often promoted as a means to solve many of the planet’s problems, much of this high 13 Regulation (EU) 2016/679 of the European Parliament and of technology consumer culture involves products that are not the Council of 27 April 2016 on the protection of natural persons made to last, but to be replaced on a regular basis, which is with regard to the processing of personal data and on the free move- ment of such data, and repealing Directive 95/46/EC (General Data depleting resources and also places burdens on resource, Protection Regulation), http://eur-lex.europ​a.eu/legal​-conte​nt/en/ particularly energy, consumption (Vince 2012; Mian et al. TXT/?uri=CELEX​%3A320​16R06​79. Accessed 1 March 2018. 72 A. M. Phillips, I. S. Mian stakeholders are required to engage in a review of this wider reputation. Fines for violating laws may be regarded as a cost spectrum of the possible impacts of technologies. Where of doing business. risks are seen as likely, imminent or serious then this may In relation to discussion of technology assessment, we (or probably should) trigger application of the Precaution- suggest utilising the Precautionary Principle. This principle ary Principle. has been invoked in the context of environmental policy, There is growing interest in central banks maintaining as well as in the context of public health. It is an important fnancial stability,14 together with interest in ethical invest- principle in International Environmental Law and is set out ment in sectors such as pension funds. Consequently, look- in the Rio Declaration on Environment and Development ing at digital technology, such as distributed ledger technol- (1992). Principle 15 of the Rio Declaration: ogy (for example, and smart ), In order to protect the environment, the precautionary in the round can help give a more balanced picture of the approach shall be widely applied by States according respective benefts, risks, and challenges raised by a spe- to their capabilities. Where there are threats of serious cifc technology (Wolbring 2009; ETC Group 2011, 2014, or irreversible damage, lack of full scientifc certainty 2015; Daño et al. 2013) such as Bitcoin or Ethereum (Reijers shall not be used as a reason for postponing cost-efec- et al. 2016; Zimmer 2017). In order to have a more holistic tive measures to prevent environmental degradation. assessment of technology, we advocate for a broad dialogue (Rio Declaration on Environment and Development amongst all stakeholders, including the public, and espe- 1992) cially groups that have historically been marginalized, such as Indigenous Peoples. It is also set out in article 191 of the Treaty on the Function- Taking a more holistic approach also allows for consid- ing of the European Union (Treaty on the Functioning of the eration of the relationship between technology and Nature European Union 2007). and its impact on Nature. Here we are thinking about not A useful depiction of when the Precautionary Principle only humans, but also organisms and the rights of Nature. ought to be relied upon stems from the Consensus Statement The granting of forms of legal personhood and human rights on the Precautionary Principle developed by the Wingspread for the protection of rivers in New Zealand and Ecuador Conference on the Precautionary Principle held in 1998 pro- are illuminating examples.15 Our proposed approach aims to vides that: assess the interactions amongst and between components of When an activity raises threats of harm to human all Earth systems: the lithosphere, atmosphere, hydrosphere health or the environment, precautionary measures and biosphere. The ­FLE5SH framework provides a common should be taken even if some cause and efect rela- toolbox that diverse stakeholders—researchers, policymak- tionships are not fully established scientifcally. In this ers, regional and national social movements, civil society context the proponent of an activity, rather than the organisations, and others—can use to evaluate technologies public, should bear the burden of proof. (Science and and if warranted, to choose a diferent future. Environmental Health Network 1998; Kriebel et al. At present, many products and services are coming to 2001: 871) market without pre-market review and without comprehen- sive impact assessments. Regulators have generally held back and there is a general tendency to let the market decide The Consensus Statement further suggests that: and promote industry self-regulation. The law may have a The process of applying the Precautionary Principle history of struggling to keep up with technological progress, must be open, informed and democratic and must but we should not accept this as a permanent state of afairs include potentially affected parties. It must also that stops discussion of appropriate regulation and account- involve an examination of the full range of alterna- ability. Unforeseen harms can occur if there is no incen- tives, including no action. (Science and Environmental tive for a company to behave responsibly other than loss of Health Network 1998) Once the Principle is triggered in relation to a particular technology, when more scientifc information becomes avail- 14 See for example Chang et al. (2019). able that would enable for assessment, the situation should 16 15 See for example the recent New Zealand Te Awa Tupua (Whan- then be reviewed. ganui River Claims Settlement) Act 2017, http://www.legis​latio​ n.govt.nz/act/publi​c/2017/0007/lates​t/DLM68​30851​.html. Accessed 1 March 2018; Community Environmental Legal Defense Fund (2015), 16 Som et al. (2009: 493); Communication from the Commission on Tanasescu (2017), O’Donnell and Talbot-James (2017), Biggs (2017), the precautionary principle/* COM/2000/0001 fnal */Precaution- Global Alliance for the Rights of Nature, http://theri​ghtso​fnatu​re.org. ary Principle. EUR-Lex, http://eur-lex.europ​a.eu/legal​-conte​nt/EN/ Accessed 1 March 2018. TXT/?uri=celex​:52000​DC000​1. Accessed 1 March 2018. Governance Challenges with Human-Machine Mergers 73

While the Precautionary Principle has often been invoked recommendations later in 2019. One of the authors of the in the context of environmental protection, as Som et al. present article, Andelka Phillips, is using the example of (2009) suggest, it can also be applied to social subjects and the DTC industry’s use wrap contracts as their dominant in thinking about potential frameworks for an information means of governance to illustrate the challenges disruptive society that is sustainable (Som et al. 2009; Danaher 2016). technologies pose for societies and for regulation. She has We suggest the need for invoking this Principle in the con- raised signifcant questions (Bates 2018) about whether the text of consideration of whether to adopt these new technol- services are: ft for their claimed purposes, the genetic data ogies. Although smart infrastructure has been promoted as and other personal information collected are being stored facilitating the development of more sustainable, cost efec- securely, sufcient protection for privacy is provided, com- tive, and efcient cities, connecting things such as energy, panies are sufciently transparent in their claims about ben- water and monetary supply chains to the Internet renders efts and limitations of their services, and consumers actually them vulnerable to physical and cyber attacks (Taylor 2015). understand the contracts they enter into when purchasing these tests. Beyond technical and legal solutions to such problems lie other concerns, notably the issue of biological Why is a Historical Lens Necessary? and genetic determinism. Dubbed ‘21st Century genetically informed social sci- Recently, IEEE released a report on ethical implementa- ence’, sociogenomics aims to understand the roots of com- tion of autonomous and intelligent systems (A/IS) A Vision plex behavior (Braudt 2018). For instance, genoeconomics for Prioritizing Human Well-being with Autonomous and posits that economic outcomes and preferences are about as Intelligent Systems (IEEE 2019). Thus, whilst STEMM stu- heritable as many medical conditions and personality traits dents entering a digital society require a frm foundation (Comfort 2018a). It suggests that fnancial behaviour can in discipline-related matters, they ought to be conversant be traced to a person’s DNA so ‘genetics could someday be also in non-technical issues. Given this need for students to used to build not just personalized medicine, but personal- become more multilingual but in light of an already highly ized policy that takes into account the genotypes that infu- constrained curriculum, how can the thorny transition from ence whether you and I are receptive to certain methods of fuency in STEMM to expressivity in SHTEAMM be made instruction, or punishment, or therapy’ (Ward 2018). Sociog- (S: Science, H: Humanities, T: Technology, E: Engineer- enomics has been characterized also as opening a new door ing, A: Arts, M: Mathematics, M: Medicine)? Consider to eugenics, new ways ‘genetic data could bolster scientifc the following exemplar. Direct-to-consumer genetic testing racism and encourage discrimination’ (Comfort 2018b). The (DTC, aka personal genomics) is a way a person can access Victorian scientist Francis Galton coined the term eugenics information about their genome from their home (National (eu = good or true + genus = birth, race or stock) to describe Human Genome Research Institute https​://www.genom​ the betterment of the overall quality of the gene pool (Das e.gov/dna-day/15-for-15/direc​t-to-consu​mer-genom​ic-testi​ 2015, 2017). In his Anthropometric Laboratory—established ng). Uses of AI/ML in this space range from calling of in 1883 (Boulter 2017) at the International Health Exhibition genetic variants from high-throughput DNA sequencing data in South Kensington, London—he measured, recorded and (for example, DeepVariant (Google Cloud 2019)) through evaluated the mental abilities and physical characteristics secure storage and sharing of genomic data (Mittos et al. of ∼ 10,000 people over a year (Das 2015). Galton’s histori- 2019) (for example, diferential privacy (Page et al. 2018)) cal connections to and the eugenics movement he initiated to developing apps for consumers to ‘interact and experience have forced University College London (UCL) to confront DNA-powered insights’ about, for example, health, ancestry, its past (Bartlett 2018). December 2018 saw the launch of a genetic relatedness, athletic ability, child talent, and infdel- Commission of Inquiry into the History of Eugenics at UCL ity (Phillips 2017, 2019). The DTC market is predicted to (UCL 2018). One key issue is the university’s role in teach- exceed $2.5 billion by 2024 (MarketWatch 2019). ing and researching eugenics in the past, present and future In January 2019, the UK’s Health Secretary Matt Han- (Osei-Mensah 2019). cock announced plans to ofer healthy people the option The United Kingdom Research and Innovation (UKRI)17 to have their whole genome sequenced by the NHS for a expects all researchers and their research to fee with these ‘genomics volunteers’—if they share data— commit to an approach that seeks continuously to ‘Antici- receiving a personalized health report (Semsarian 2019). pate, Refect, Engage and Act’ (AREA). This approach will: In March 2019, the House of Commons Science and Tech- nology Committee launched an inquiry into commercial genomic testing to establish what safeguards need to be put in place to protect those who get tested (Science and 17 UK Research and Innovation, https​://www.ukri.org. Accessed 25 Technology 2019). The Committee should be releasing July 2019. 74 A. M. Phillips, I. S. Mian

Anticipate—describing and analyzing the impacts, world that previously were experienced or existed only in intended or otherwise, (for example economic, social, qualitative, non-numeric form). environmental) that might arise. This does not seek to predict but rather to support an exploration of possible impacts and implications that may otherwise remain Conclusion uncovered and little discussed. Refect—refecting on the purposes of, motivations for It is hoped that this article will stimulate refection about and potential implications of the research, and the asso- the following matters: the need to engage in a more public, ciated uncertainties, areas of ignorance, assumptions, democratic, and open discussion of technologies and their framings, questions, dilemmas and social transformations potential impact on society, the environment, and the planet; these may bring. the need for greater oversight of technologies that pose sig- Engage—opening up such visions, impacts and question- nifcant risks to human and/or environmental health; the ing to broader deliberation, dialogue, engagement and need to ensure that technologies that allow for the alteration debate in an inclusive way. of the genetic makeup of biological organisms are subject to Act—using these processes to infuence the direction and oversight, especially regarding their safety; and the need for trajectory of the research and innovation process itself the development of appropriate laws and governance mecha- (EPSRC 2019). nisms that will protect the public, the environment, and the planet as a whole. The UCL Inquiry and the AREA framework are relevant It should be noted that we have developed bodies of law to AI/ML and their applications in human health and agri- such as consumer protection and product liability law for culture (Marr 2018). A shift from genetics to genomics—the sound reasons. Permitting commercialisation of technolo- study of organisms in terms of their full DNA sequences—at gies without any regulation other than industry-self regula- the turn of the 21 century is said to have given rise to a tion is unlikely to lead to a safer, fairer world. new form of eugenics, eugenomics (Aultman 2006). A move The issues raised both by developments in AI which could from ‘personalized medicine’ to ‘precision health’ and ‘well- lead to a super intelligent AI and other felds that could lead ness genomics’ during this period has raised the question to the merging of humans with machines raise issues that of whether a century from now, the latter two ideas will be need to be considered from a range of perspectives. If the viewed as eugenics is today (Jeungst et al. 2018). Might a future is Humanity 2.0 (or higher) then this should be a similar fate await ‘public health genomics’ and ‘precision choice that humans make, just as if super intelligent AI is to public health’, programmes such as whole genome sequenc- develop, we do need to ensure that its values are in line with ing on every newborn within a population (Molster 2018)? those of humanity and the planet. However, there is a pluriv- The HUMAN Project18 (Human Understanding Through ersal and not just a universal notion of what constitutes value Measurement and Analytics)—introduced in 2015, aims (Mignolo 2013). Perhaps, a more holistic approach to assess- to measure, aggregate and analyse the biology, behaviour, ing technology could also serve to guide policy contextually, environmental conditions and events of ∼ 10,000 New Yor- as a substitute for humanity’s conscience, and thereby shape kers over 20 years (Azmak et al. 2015). This Project uses technology in a consistent and more balanced way. medical records, biological samples, surveys, questionaires, digital device data, third party data and other modalities in Acknowledgements Thanks to our colleagues for their support and order to create ‘synoptic and granular views of how human fruitful discussion on these topics. health and behavior coevolve over the life cycle and why they evolve diferently for diferent people’ (Azmak et al. References 2015). Thus, it is important to understand Big Science and Big Data studies of humanity in Victorian and mod- ACM. 2018. Fathers of the Deep Learning Revolution Receive ACM ern times—anthropometry (UCL Culture https​://www.ucl. A.M. Turing Award. Bengio, Hinton and LeCun Ushered in ac.uk/culture/galto​ n-colle​ ction​ /galto​ n-and-anthr​ opome​ trics​ ​ Major Breakthroughs in Artifcial Intelligence. Association for ) yesterday, today and tomorrow—and more generally, the Computing Machinery, 27 March 2019, https​://www.acm.org/ past, present and future of datafcation (the quantifcation of media​-cente​r/2019/march​/turin​g-award​-2018. African Centre for Biodiversity. 2019. Civil Society Denounces the objects, actions, processes and other aspects of life and the Release of GM mosquitoes in Burkina Faso. Press Release, 2 July, https://acbio​ .org.za/en/civil​ -socie​ ty-denou​ nces-relea​ se-gm-​ mosqu​itoes​-burki​na-faso. Accessed 25 July 2019. Akyildiz, I.F., M. Pierobon and S. Balasubramaniam. 2019. Moving forward with molecular communication: from theory to human 18 The Human Project. https​://www.thehu​manpr​oject​.org. Accessed health applications. Proc IEEE 107:858–865. https​://ieeex​plore​ 23 July 2019. .ieee.org/docum​ent/87103​66. Governance Challenges with Human-Machine Mergers 75

Associated Press. 2017. Companies Start Implanting Microchips into Social Science. Sociology Compass 12(10): e12626. https​://doi. Workers’ Bodies. LA Times, 3 April, http://www.latim​es.com/ org/10.1111/soc4.12626​. busin​ess/techn​ology​/la-f-tn-micro​chip-emplo​yees-20170​403- Brookman, Justin, Phoebe Rouge, Aaron Alva, and Christina Yeung. story​.html. Accessed 2 March 2018. 2017. Cross-Device Tracking: Measurement and Disclosures. Atler, Adam. 2017. Irresistible: The Rise of Addictive Technology and Proceedings on Privacy Enhancing Technologies 2017(2): the Business of Keeping Us Hooked. Penguin Press. http://adama​ 133–148. https​://petsy​mposi​um.org/2017/paper​s/issue​2/paper​ ltera​uthor​.com/irres​istib​le. Accessed 12 September 2019. 29-2017-2-sourc​e.pdf. Aultman, Julie M. 2006. Eugenomics: Eugenics and Ethics in Brooks, James. 2017. A Swedish Start-Up has Started Implanting the 21st Century. Genom Soc Policy 2: 2–28. https​://doi. Microchips Into its Employees. CNBC, 3 April, https​://www. org/10.1186/1746-5354-2-2-28. cnbc.com/2017/04/03/start​-up-epice​nter-impla​nts-emplo​yees- Azmak, Okan, Hannah Bayer, Andrew Caplin, Miyoung Chun, Paul with-micro​chips​.html. Accessed 2 March 2018. Glimcher, Steven Koonin and Aristides Patrinos. 2015. Using Broussard, Meredith. 2018. Artifcial Unintelligence: How Computers Big Data to Understand the Human Condition: The Kavli Misunderstand the World. Cambridge, MA: MIT Press. https​:// HUMAN Project. Big Data. 2015, 3:3, https​://doi.org/10.1089/ mitpr​ess.mit.edu/books​/artif​ cial​-unint​ellig​ence. Accessed 12 big.2015.0012. September 2019. Barfeld, Woodrow. 2015. Cyber-Humans: Our Future with Machines. Burgess, Matt. 2017. DeepMind’s AI has Learnt to Become ‘Highly Springer. https​://www.sprin​ger.com/gp/book/97833​19250​489. Aggressive’ When it Feels like it’s Going to Lose. Wired, 9 Feb- Accessed 12 September 2019. ruary, http://www.wired.co.uk/artic​ le/arti​ f​ cial-intel​ ligen​ ce-socia​ ​ Bartlett, Georgina. 2018. Investigation: Moving Forward From Gal- l-impac​t-deepm​ind. Accessed 13 September 2019. ton’s UCL. PI Magazine, 22 January 2018, https​://pimed​iaonl​ Calo, Ryan, A. Michael Froomkin and Ian Kerr. 2016. Robot Law. MA: ine.co.uk/news/moving-forwa​ rd-from-galto​ ns-ucl/​ . Accessed 23 Edward Elgar Publishing. https​://www.e-elgar​.com/shop/robot​ July 2019. -law. Accessed 12 September 2019. Bates, Mary. 2018. Direct-To-Consumer Genetic Testing. IEEE Pulse, Campbell, Mark. 2015. Montgomery v Lanarkshire Health Board. 14 December 2018, https://pulse​ .embs.org/novem​ ber-2018/direc​ ​ Common Law World Review 44(3): 222–228. https​://doi. t-to-consu​mer-genet​ic-testi​ng/. Accessed 23 July 2019. org/10.1177/14737​79515​59211​8. Beck, John J. and Rachel L. Vannette. 2016. Harnessing Insect– Chang, Marina, Huang, C.-H. and Mian, I.S. 2019. A data science and Microbe Chemical To Control Insect Pests of historical global political ecology perspective on the fnancial Agricultural Systems. Journal of Agricultural and Food Chemis- system, agriculture and climate: from the trans-Atlantic slave try 65:23–28. https​://doi.org/10.1021/acs.jafc.6b042​98. trade to agroecology. Current version of the discussion paper pre- Biggs, Shannon. 2017. When Rivers Hold Legal Rights. Earth Island sented at Data for Policy 2017: ? 6-7 Journal, 17 April, http://www.earth​islan​d.org/journ​al/index​ September 2017. https​://zenod​o.org/recor​d/88450​3. Accessed .php/elist​/eList​Read/when_river​s_hold_legal​_right​s/. Accessed 16 September. 1 March 2018. Chauriye, Nicole. 2016. Wearable Devices as Admissible Evidence: Bonaci, Tamara, Ryan Calo and Howard Jay Chizeck. 2014. App Technology is Killing Our Opportunity to Lie. Catholic Univer- Stores for the Brain: Privacy & Security in Brain–Computer sity Journal of Law and Technology 24(2) art. 9: 494-528. https​ Interfaces. In Proceedings of the IEEE 2014 International Sym- ://schol​arshi​p.law.edu/jlt/vol24​/iss2/9/. Accessed 12 September posium on Ethics in Engineering, Science, and Technology, IEEE 2019. Press. https://doi.org/10.1109/ETHIC​ S.2014.68934​ 15​ . Accessed Chemical communication in humans. 2019. Royal Society Meeting 1-2 12 September 2019. April 2019. Kavli Royal Society Centre, Chicheley Hall, Newport Bonnefon, Jean-François, Azim Sharif and Iyad Rahwan, 2015. The Pagnell, Buckinghamshire. https://royal​ socie​ ty.org/scien​ ce-event​ ​ social dilemma of autonomous vehicles, https​://arxiv​.org/ s-and-lectu​res/2019/04/chemi​cal-commu​nicat​ion/. abs/1510.03346​. Accessed 12 September 2019. Chen, Kai, Xueqiang Wang, Yi Chen, Peng Wang, Yeonjoon Lee, Borger, Julian. 2019. House Orders Pentagon to Review if It Exposed XiaoFeng Wang, Bin Ma, Aohui Wang, Yingjun Zhang and Americans to Weaponised Tick. The Guardian, 16 July, https​ Wei Zou. 2016. Following Devil’s Footprints: Cross-Platform ://www.thegu​ardia​n.com/us-news/2019/jul/16/penta​gon-revie​ Analysis of Potentially Harmful Libraries on Android and iOS. w-weapo​nised​-ticks​-lyme-disea​se. Accessed 25 July 2019. In Security and Privacy (SP), 2016 IEEE Symposium: 357-76. Bostrom, Nick. 2014. Superintelligence: Paths, Dangers, Strategies. https​://doi.org/10.1109/SP.2016.29. Oxford: Oxford University Press. https​://globa​l.oup.com/acade​ Clarke, Arthur C. 2016. 2001: A Space Odyssey. 1st edn. 1968, reprint, mic/produ​ct/super​intel​ligen​ce-97801​98739​838. Accessed 13 Penguin Books. https​://www.pengu​inran​domho​use.com/books​ September 2019. /53885​1/2001-a-space​-odyss​ey-by-arthu​r-cclar​ke/97801​43111​ Boulter, Michael. 2017. Bloomsbury Scientists Science and Art in the 573/. Wake of Darwin. London: UCL Press, xi. https://www.ucl.ac.uk/​ Cofey, Helen. 2017. The future is here—A Swedish Rail Company ucl-press/brows​ e-books​ /bloom​ sbury​ -scien​ tists​ .​ Accessed 13 Sep- is Trialing Letting Passengers Use Biometric Chips as Tickets. tember 2019. The Independent, 16 June, http://www.indep​enden​t.co.uk/trave​ Bradley-Munn, Sharon R. and Katina Michael. 2016. Whose Body l/news-and-advic​e/sj-rail-train-ticke​ ​ts-hand-impla​nt-micro​chip- Is It?: The body as Physical Capital in a Techno-Society. IEEE biome​tric-swede​n-a7793​641.html. Accessed 2 March 2018. Consumer Electronics Magazine 5(3): 107–114. http://www. Comfort, Nathaniel. 2018a. Nature Still Battles Nurture in the Haunting katin​amich​ael.com/resea​rch/2016/8/10/the-body-as-physi​cal- World of Social Genomics. Nature 2018(553): 278–280. https​ capit​al-in-a-techn​o-socie​ty. Accessed 12 September 2019. ://www.natur​e.com/artic​les/d4158​6-018-00578​-5. Accessed 12 Bradshaw-Martin, Heather and Catherine Easton. 2014. Autonomous September 2019. or ‘Driverless’ Cars and Disability: A Legal and Ethical Analy- Comfort, Nathaniel. 2018b. Sociogenomics is Opening a New Door sis. European Journal of Current Legal Issues 20(3). http://webjc​ to Eugenics. MIT Technology Review, 23 October 2018, https​:// li.org/artic​le/view/344/471. Accessed 1 March 2018. www.techn​ology​revie​w.com/s/61227​5/socio​genom​ics-is-openi​ Braudt, David B. 2018. Sociogenomics in the 21st Century: An Intro- ng-a-new-door-to-eugen​ics/. Accessed 23 July 2018. duction to the History and Potential of Genetically Informed Committee on Legal Afairs. 2016. Draft Report, 31 May, http:// www.europ​arl.europ​a.eu/sides​/getDo​c.do?type=COMPA​ 76 A. M. Phillips, I. S. Mian

RL&mode=XML&langu​age=EN&refer​ence=PE582​.443. oup.org/conte​nt/un-moves​-towar​ds-techn​ology​-early​-liste​ning- Accessed 2 March 2018. syste​m. Accessed 1 March 2018. Community Environmental Legal Defense Fund. 2015. Rights of ETC Group. 2017a. The Wisdom of G.O.A.T.S. (Global Overview Nature: Overview. 4 August, updated 19 May, 2017, https://celdf​ ​ Assessment of Technological Systems). Draft Proposal for STI .org/right​s/right​s-of-natur​e/. Accessed 1 March 2018. Forum 2, New York, May, http://www.etcgr​oup.org/sites​/www. Constine, Josh. 2017a. Elon Musk’s Brain Interface Startup Neu- etcgroup.org/fles​ /fles​ /etc_goats​ _us_may20​ 17.pdf​ . Accessed 23 ralink Files $27 M Fundraise. TechCrunch, 25 August, https://​ July 2019. techc​runch​.com/2017/08/25/elon-musks​-brain​-inter​face-start​ ETC Group. 2017b. UN Sustainable Development Knowledge Plat- up-neura​link-fles​-27m-fundr​aise/. Accessed 25 August 2017. form. https​://susta​inabl​edeve​lopme​nt.un.org/tfm. Accessed 2 Constine, Josh. 2017b. Facebook is Building Brain-Computer Inter- March 2018. faces for Typing and Skin-Hearing. TechCrunch, 19 April, Etzioni, Amitai and Oren Etzioni. 2016. Designing AI Systems that https://techc​ runch​ .com/2017/04/19/faceb​ ook-brain​ -inter​ face/​ . Obey our Laws and Values. Communications of the ACM Accessed 2 March 2018. 59(9): 29–31. https​://paper​s.ssrn.com/sol3/paper​s.cfm?abstr​ Constine, Josh. 2017c. Facebook Now has 2 Billion Monthly Users act_id=28301​46. Accessed 12 September 2019. … and Responsibility. TechCrunch, 27 June, https://techc​ runch​ ​ European Commission. 2013. Report on the Public Consultation on .com/2017/06/27/facebook-2-billi​ on-users​ /​ . Accessed 1 March IoT Governance, https​://ec.europ​a.eu/digit​al-singl​e-marke​t/en/ 2018. news/conclusion​ s-inter​ net-thing​ s-publi​ c-consu​ ltati​ on​ . Accessed Danaher, John. 2016. New Technologies as Social Experiments: An 2 March 2018. Ethical Framework. Philosophical Disquisitions, 15 March, Federal Trade Commission. 2017. Cross-Device Tracking. Staf Report, http://philosophi​ caldi​ squis​ ition​ s.blogs​ pot.co.uk/2016/03/new-​ January, https​://www.ftc.gov/syste​m/fles​/docum​ents/repor​ts/ techn​ologi​es-as-socia​l-exper​iment​s.html. Accessed 1 March cross​-devic​e-track​ing-feder​al-trade​-commi​ssion​-staff​-repor​ 2018. t-janua​ry-2017/ftc_cross​-devic​e_track​ing_repor​t_1-23-17.pdf. Daño, Neth, Kathy Jo Wetter and Silvia Ribeiro. 2013. Addressing Accessed 1 March 2018. the ‘Technology Divides’: Critical Issues in Technology and Floridi, Luciano and Josh Cowls. 2019. A Unifed Framework of Five SDGs. Briefng Paper: Science, Technology and Innovation Principles for AI in Society. Harvard Data Science Review, 23 (STI) 6th Session of the Open Working Group on Sustainable June, Issue 1: 1-13. https://doi.org/10.1162/99608​ f92.8cd55​ 0d1​ . Development Goals: New York, 9-23 December, https​://susta​ Frey, Carl Benedikt and Michael Osborne et al. 2016. Technology at inabl​edeve​lopme​nt.un.org/conte​nt/docum​ents/4673d​ano.pdf. Work v2.0: The Future Is Not What It Used to Be. Oxford Martin Accessed 1 March 2018. School and CITI GPS Reports, January, http://www.oxfor​dmart​ Das, Subhadra. 2015. The Unbelievable Truth About Sir Francis in.ox.ac.uk/publi​catio​ns/view/2092. Accessed 2 March 2018. Galton. UCL Museums and Collections Blog, 5 November Frischmann, Brett, and Evan Selinger. 2018. Re-Engineering Human- 2015, https​://blogs​.ucl.ac.uk/museu​ms/2015/11/05/the-unbel​ ity, 30–32. Cambridge: Cambridge University Press. https​:// ievable-truth​ -about​ -sir-franc​ is-galto​ n/​ . Accessed 23 July 2019. www.cambridge.org/acade​ mic/subje​ cts/law/law-and-econo​ mics/​ Das, Subhadra. 2017. Racism, Eugenics and the Domestication re-engin​eerin​g-human​ity?forma​t=HB&isbn=97811​07147​096. of Humans. UCL Museums and Collections Blog, 25 Octo- Future of Life Institute. 2017a. Autonomous Weapons: An Open Letter ber 2017, https​://blogs​.ucl.ac.uk/museu​ms/2017/10/25/racis​ From AI & Robotics Researchers, https​://futur​eofi​fe.org/open- m-eugen​ics-and-the-domes​ticat​ion-of-human​s/. Accessed 23 lette​r-auton​omous​-weapo​ns/. Accessed 1 March 2018. July 2019. Future of Life Institute. 2017b. Killer Robots: World’s Top AI and Dormehl, Luke. 2017. Engineers Just Created A Tiny Antenna, Which Robotics Companies Urge United Nations to Ban Lethal Autono- Could Be Used For Brain Implants. Digital Trends. Digital mous Weapons. Future of Life Institute, 20 August, https​://futur​ Trends, 25 August, https​://www.digit​altre​nds.com/cool-tech/ eofli​fe.org/2017/08/20/kille​r-robot​s-world​s-top-ai-robot​ics- tiny-anten​na-brain​-impla​nt/. Accessed 25 August 2017. compa​nies-urge-unite​d-natio​ns-ban-letha​l-auton​omous​-weapo​ Drabiak, Katherine. 2017. Caveat Emptor: How the Intersection of Big ns/. Accessed 1 March 2018. Data and Consumer Genomics Exponentially Increases Informa- Gao, Wei, Renfeng Dong, Soracha Thamphiwatana, Jinxing Li, Wei- tional Privacy Risks. Health Matrix 27(1): 143–525. https://schol​ ​ wei Gao, Liangfang Zhang and Joseph Wang. 2015. Artifcial arlycommon​ s.law.case.edu/healt​ hmatr​ ix/vol27​ /iss1/8/​ . Accessed Micromotors in the Mouse’s Stomach: A Step Toward in Vivo 12 September 2019. Use of Synthetic Motors. ACS Nano 9(1): 117-123. https​://doi. Duhigg, Charles. 2012. How Companies Learn Your Secrets. org/10.1021/nn507​097k. New York Times Magazine, 16 February, https​://www.nytim​ Global Privacy Enforcement Network. 2016. GPEN Privacy Sweep, es.com/2012/02/19/magaz​ine/shopp​ing-habit​s.html. Accessed : Participating Authorities’ Press Releases. 12 September 2019. https​://www.priva​cyenf​orcem​ent.net/node/717. Accessed 2 Ellenberg, Jordan. 2014. What’s Even Creepier Than Target Guessing March 2018. That You’re Pregnant? Slate, 19 June, https​://slate​.com/human​ Google Cloud. Running DeepVariant. https://cloud​ .googl​ e.com/genom​ ​ -interest/2014/06/big-data-whats​ -even-creep​ ier-than-targe​ t-guess​ ​ ics/docs/tutor​ials/deepv​arian​t. Accessed 23 July 2019. ing-that-youre​-pregn​ant.html. Accessed 25 July 2019. Greenberg, Andy. 2017. Biohackers Encoded Malware In A Strand Of EPSRC. 2019. Anticipate, Refect, Engage and Act (AREA), https​:// DNA. Wired, 10 August, https​://www.wired​.com/story​/malwa​ epsrc.ukri.org/resea​ rch/frame​ work/area/​ . Accessed 25 July 2019. re-dna-hack. Accessed 2 March 2018. ETC Group. 2011. Why Technology Assessment? ETC Group Briefng Grimm, Nick. 2017. Swedish Employees Agree to Free Microchip Paper, New York, March, http://www.etcgr​oup.org/sites​/www. Implants Designed for Ofce Work. ABC News, 3 April, http:// etcgroup.org/fles​ /Why%20tec​ hnolo​ gy%20ass​ essme​ nt201​ 1.pdf​ . www.abc.net.au/news/2017-04-03/swedish-emplo​ yees-agree​ -to-​ Accessed 1 March 2018. micro​chip-impla​nts/84100​18. Accessed 2 March 2018. ETC Group. 2014. A Note for Discussion: UN Technology Assessment. Grosfoguel, Ramón. 2012. Decolonizing Western Uni-versalisms: 29 May, http://www.un-ngls.org/IMG/pdf/Techn​ology​_Asses​ Decolonial Pluri-Versalism from Aimé Césaire to the Zapatistas. sment​_Overv​iew21​May20​14.pdf. Accessed 1 March 2018. Transmodernity: Journal of Peripheral Cultural Production of ETC Group. 2015. UN Moves Towards a Technology Early Listening the Luso-Hispanic World 1(3): 88-104, http://escho​larsh​ip.org/ System. ETC Group, News Release, 16 July, http://www.etcgr​ uc/item/01w71​63v. Accessed 1 March 2018). Governance Challenges with Human-Machine Mergers 77

Hahn, Adam, Aditya Ashok, Siddharth Sridhar, and Manimaran Govin- -perso​naliz​ed-medic​ine-preci​sion-healt​h-and-welln​ess-genom​ darasu. 2013. Cyber-Physical Security Testbeds: Architecture, ics-matte​r/2018-09. Accessed 23 July 2019. Application, and Evaluation for Smart Grid. IEEE Transac- Kearns, Thomas B. 1998. Technology and the Right to Privacy: The tions on Smart Grid 4(2): 847–855. https​://doi.org/10.1109/ Convergence of Surveillance and Information Privacy Concerns. TSG.2012.22269​19. William & Mary Bill of Rights Journal 7(3), art. 10,. http://schol​ Hammond, Edward. 2017. Sequence Data And Beneft Sharing: Divs- arship.law.wm.edu/wmbor​ j/vol7/iss3/10​ . Accessed 12 September eek’s Pitfalls Show Need For Appropriate Policy Penang, Malay- 2019. sia: Third World Network, https://www.twn.my/title​ 2/serie​ s/bkr/​ Kriebel, David, Joel Tickner, Paul Epstein, John Lemons, Richard bkr00​5.htm. Accessed 1 March 2018. Levins, Edward L. Loechler and Michael Stoto. 2001. The Harris, Tristan. 2016. How Technology Hijacks People’s Minds — from Precautionary Principle in Environmental Science. Environ- a Magician and Google’s Design Ethicist. Thrive Global, 19 May, mental health perspectives 109(9): 871–876, 871, citing C. https​://journ​al.thriv​eglob​al.com/how-techn​ology​-hijac​ks-peopl​ Rafensperger and J. A. Tickner, (eds) Protecting public health es-minds​-from-a-magic​ian-and-googl​e-s-desig​n-ethic​ist-56d62​ and the environment: implementing the precautionary princi- ef5ed​f3. Accessed 1 March 2018. ple, Island Press, 1999: 8. https​://doi.org/10.1289/ehp.01109​ HEGAAs. Horizontal Environmental Genetic Alteration Agents. http:// 871. web.evolb​io.mpg.de/HEGAA​s/. Accessed 25 July 2019. Kupferschmidt, Kai. 2019. Can atmospheric chemists rescue the Hilts, Andrew, Christopher Parsons and Jefrey Knockel. 2016. Every stalled quest for a human pheromone? Science Apr 11, 2019. Step you Fake: A Comparative Analysis of Fitness Tracker Pri- https​://doi.org/10.1126/scien​ce.aax66​30. vacy and Security. Open Efect Report 76(24): 31-33, https​:// Leibo, Joel Z., Vinicius Zambaldi, Marc Lanctot, Janusz Marecki opene​fect​.ca/repor​ts/Every​_Step_You_Fake.pdf. Accessed 1 and Thore Graepel. 2017. Multi-agent Reinforcement Learn- March 2018. ing in Sequential Social Dilemmas. In Proceedings of the 16th HP. 2015. HP Study Reveals Smartwatches Vulnerable to Attack. HP Conference on Autonomous Agents and MultiAgent Systems, News Advisory, 22 July, http://www8.hp.com/us/en/hp-news/ pp. 464–473. https​://dl.acm.org/citat​ion.cfm?id=30911​94. press​-relea​se.html?id=20373​86. Accessed 1 March 2018. Accessed 12 September 2019. HPE. 2015. Fortify and the Internet of Things. Internet of Things Secu- Li, QianQian, Ding Ding and Mauro Conti. 2015. Brain–Computer rity Study: Smartwatches, http://go.saas.hpe.com/fod/internet-of-​ Interface Applications: Security and Privacy Challenges. Com- thing​s. Accessed 1 March 2018. munications and Network Security (CNS). In IEEE Conference Hu, Shaohua, Yiru Fang, Chee H Ng and J. John Mann. 2019. Editorial: on Network Security, https://doi.org/10.1109/CNS.2015.73468​ ​ Involvement of Neuro-Immune Mechanism and Brain–Gut Axis 84. Accessed 25 July 2019. in Pathophysiology of Mood Disorders. Frontiers in Psychiatry. Lopatto, Elizabeth. 2019. Elon Musk unveils Neuralink’s https​://doi.org/10.3389/fpsyt​.2019.00403​. Plans for Brain-Reading ‘Threads’ and a Robot to Insert Hull, Dana. 2017. Elon Musk’s Neuralink Gets $27 Million to Build Them. The Verge, 16 July 2019, https​://www.theve​rge. Brain Computers. Bloomberg Technology, 25 August, https​:// com/2019/7/16/20697​123/elon-musk-neura​link-brain​-readi​ www.bloom​berg.com/news/artic​les/2017-08-25/elon-musk-s- ng-threa​d-robot​. Accessed 23 July 2019. neura​link-gets-27-milli​on-to-build​-brain​-compu​ters. Accessed Lubin, Gus. 2012. The Incredible Story Of How Target Exposed A 1 March 2018. Teen Girl’s Pregnancy. Business Insider, 16 February, http:// IEEE. 2019. Ethically Aligned Design. First Edition, 2019, https​:// www.busin​essin​sider​.com/the-incre​dible​-story​-of-how-targe​ stand​ards.ieee.org/conte​nt/dam/ieee-stand​ards/stand​ards/web/ t-expos​ed-a-teen-girls​-pregn​ancy-2012-2. Accessed 13 Sep- docum​ents/other​/ead1e​.pdf. Accessed 23 July 2019. tember 2019. International Foundation for Autonomous Agents and Multiagent Sys- Ma, Hong, Nuria Marti-Gutierrez, Sang-Wook Park, Wu Jun, Yeonmi tems. 2017. https​://stora​ge.googl​eapis​.com/deepm​ind-media​/ Lee, Keiichiro Suzuki, Amy Koski, et al. 2017. Correction paper​s/multi​-agent​-rl-in-ssd.pdf. Accessed 1 March 2018. of a Pathogenic Gene Mutation in Human Embryos. Nature Irish Data Protection Commissioner. 2016. Findings of International 548(7668): 413–419. https​://doi.org/10.1038/natur​e2330​5. Privacy Sweep 2016 published. 22 September, https​://www. Mallonee, Laura. 2017. The DIY Cyborgs Hacking Their Bodies datap​rotec​tion.ie/docs/23-9-2016-Inter​natio​nal-Priva​cy-Sweep​ For Fun. Wired, 8 June, https​://www.wired​.com/story​/hanne​ -2016/i/1597.htm. Accessed 1 March 2018. s-wiede​mann-grind​ers/. Accessed 1 March 2018. Jackson, Brian A., Duren Banks, Dulani Woods and Justin C. Dawson. MarketWatch. 2019. DTC Genetic Testing Market will surpass 2017. Future-Proofng Justice: Building a Research Agenda to $2.5 Billion by 2024. Press Release, 22 February 2019, https​ Address the Efects of Technological Change on the Protection ://www.marke​twatc​h.com/press​-relea​se/dtc-genet​ic-testi​ng- of Constitutional Rights. Santa Monica, CA: RAND Corpora- market-will-surpa​ ss-25-billi​ on-by-2024-2019-02-22​ . Accessed tion, https://www.rand.org/pubs/resea​ rch_repor​ ts/RR174​ 8.html​ . 23 July 2019. Accessed 1 March 2018. Marr, Bernard. 2018. The Wonderful Ways Artifcial Intelligence Is Johnsen, Andreas. 2016. Investigation of Privacy and Security Transforming Genomics and Gene Editing. Forbes, 16 Novem- Issues with Smart toys. Norwegian Consumer Council Report, ber 2018, https://www.forbe​ s.com/sites​ /berna​ rdmar​ r/2018/11/16/​ Version 1, https​://fil.forbr​ukerr​adet.no/wp-conte​nt/uploa​ the-amazing-ways-arti​ f​ cial-intel​ ligen​ ce-is-trans​ formi​ ng-genom​ ​ ds/2016/12/2016-11-technical-analy​ sis-of-the-dolls​ -bouve​ t.pdf​ . ics-and-gene-editi​ng/#cafc0​d042c​11. Accessed 23 July 2019. Accessed 1 March 2018. Martin, Chris R., Vadim Osadchiy, Amir Kalani, and Emeran A. Maye. Jordan, Michael I. 2018. Artifcial Intelligence—The Revolution Hasn’t 2018. The Brain-Gut-Microbiome Axis. Cellular and Molecu- Happened Yet. Medium, 18 April 2018, https​://mediu​m.com/@ lar Gastroenterology and Hepatology 6(2): 133–148. https://doi.​ mijordan3/arti​ f​ cial-intel​ ligen​ ce-the-revol​ ution​ -hasnt​ -happe​ ned-​ org/10.3389/fpsyt​.2019.00403​. yet-5e1d5​812e1​e7. Accessed 23 July 2019. Metz, Rachel. 2017. Controlling VR with Your Mind. MIT Tech Jeungst, Eric T, and Michelle L McGowan. 2018. Why Does the Shift Review, March, http://www.neura​ble.com/news/contr​ollin​g-vr- from “Personalized Medicine” to “Precision Health” and “Well- your-mind. Accessed 12 September 2019. ness Genomics” Matter? AMA Journal of Ethics, September Mian, I.S., D. Twiselton and D. Timm. 2019. What is the Resource 2018, https://journ​ alofe​ thics​ .ama-assn.org/artic​ le/why-does-shift​ ​ Footprint of a Computer Science Department? Place, People, and Pedagogy. Current version of the discussion paper presented at 78 A. M. Phillips, I. S. Mian

Data for Policy 2017: Government by Algorithm? 6-7 Septem- Newmann, Stuart and Tina Stevens. 2019b. Biotech Juggernaut: Hope, ber 2017. https​://doi.org/10.5281/zenodo.88449​ ​2. Accessed 16 Hype, and Hidden Agendas of Entrepreneurial BioScience. https​ September. ://www.routl​edge.com/Biote​ch-Jugge​rnaut​-Hope-Hype-and- Michael, Katina, M.G. Michael Anas Aloudat, and Christine Perak- Hidden-Agend​ as-of-Entre​ prene​ urial​ -BioSc​ ience​ /Steve​ ns-Newma​ ​ slis. 2017. You Want to Do What with RFID?: Perceptions of n/p/book/97811​38043​237. Accessed 16 September 2019. Radio-Frequency Identifcation Implants for Employee Identif- Norwegian Consumer Council. 2017. #WatchOut Analysis of cation in the Workplace. IEEE Consumer Electronics Magazine Smartwatches for Children. Norwegian Consumer Council 6(3): 111–117. https​://ieeex​plore​.ieee.org/docum​ent/79488​58. Report, October, https://fl.forbr​ ukerr​ adet.no/wp-conte​ nt/uploa​ ​ Accessed 12 September 2019. ds/2017/10/watch​out-rappo​rt-octob​er-2017.pdf. Accessed 1 Mignolo, Walter. 2010. The communal and the decolonial. Turbu- March 2018. lence, http://www.turbu​lence​.org.uk/index​.html@p=391.html. O’Brien, Carl. 2018. Call for Smartphone Use to be Banned for Accessed 1 March 2018. Primary Students. The Irish Times, 30 January, https​://www. Mignolo, Walter. 2013. On Pluriversality. 20 October, http://walte​ irishtimes​ .com/news/educa​ tion/call-for-smart​ phone​ -use-to-be-​ rmign​olo.com/on-pluri​versa​lity/. Accessed 1 March 2018. banne​d-for-prima​ry-stude​nts-1.33728​48. Accessed 1 March Mittos, Alexandros, Bradley Malin, and Emiliano De Cristofaro. 2019. 2018. Systematizing Genome Privacy Research: A Privacy-Enhancing O’Brolcháin, Fiachra, Tim Jacquemard, David Monaghan, Noel Technologies Perspective. In Proceedings on Privacy Enhancing O’Connor, Peter Novitzky, and Bert Gordijn. 2016. The Con- Technologies. https​://doi.org/10.2478/popet​s-2019-0006. vergence of Virtual Reality and Social Networks: Threats to Mo, Yilan, Tifany Hyun-Jin Kim, Kenneth Brancik, Dona Dickinson, Privacy and Autonomy. Science and Engineering Ethics 22(1): Heejo Lee, Adrian Perrig, and Bruno Sinopoli. 2012. Cyber– 1–29, https​://doi.org/10.1007/s1194​8-014-9621-1. Accessed 12 Physical Security of a Smart Grid Infrastructure. Proceedings September 2019. of the IEEE 100(1): 195–209. https​://doi.org/10.1109/JPROC​ O’Connell, Mark. 2017. To Be A Machine. London: Granta Publica- .2011.21614​28. tions. http://grant​abook​s.com/to-be-a-machi​ne-2. Accessed 12 Molster, Caron M, Faye L. Bowman, Gemma A. Bilkey, Angela S. Cho, September 2019. Belinda L. Burns, Kristen J. Nowak and Hugh J. S. Dawkins. O’Donnell, Erin and Julia Talbot-James. 2017. Three Rivers are Now 2018. The Evolution of Public Health Genomics: Exploring Its Legally People—But That’s Just the Start of Looking After Past, Present, and Future. Frontiers in Public Health, 4 Septem- Them. The Conversation, 23 March, https://theco​ nvers​ ation​ .com/​ ber 2018, https​://doi.org/10.3389/fpubh​.2018.00247​. Accessed three​-river​s-are-now-legal​ly-peopl​e-but-thats​-just-the-start​-of- 23 July 2019. looki​ng-after​-them-74983​. Accessed 1 March 2018. Morris, Ian. 2017. Apple Has Sold 1.2 Billion iPhones Worth $738 Bil- Osei-Mensah, Jennifer. 2019. “A History That Has Never Gone Away”: lion In 10 Years. Forbes, 29 June, https://www.forbe​ s.com/sites​ /​ UCL Eugenics Inquiry Holds Divisive First Meeting. PI Maga- ianmorris/2017/06/29/apple​ -has-sold-1-2-billi​ on-iphon​ es-worth​ ​ zine, 7 March 2019, https​://pimed​iaonl​ine.co.uk/news/a-histo​ -738-billi​on-in-10-years​/. Accessed 1 March 2018. ry-that-has-never​-gone-away-ucl-eugen​ics-inqui​ry-holds​-divis​ Nan, Tianxiang, Hwaider Lin, Yuan Gao, Alexei Matyushov, Yu. ive-frst​-meeti​ng/. Accessed 23 July 2019. Guoliang, Huaihao Chen, Neville Sun, Shengjun Wei, Zhiguang Page, Hector, Charlie Cabot, and Kobbi Nissim. 2018. Diferential Pri- Wang, Menghui Li, Xinjun Wang, Amine Belkessam, Rongdi vacy: An Introduction for Statistical Agencies. Government Sta- Guo, Brian Chen, James Zhou, Yu. Zhenyun Qian, Matteo tistical Service. December 2018, https​://gss.civil​servi​ce.gov.uk/ Rinaldi Hui, Michael E. McConney, Brandon M. Howe, Hu wp-conte​nt/uploa​ds/2018/12/12-12-18_FINAL​_Privi​tar_Kobbi​ Zhongqiang, John G. Jones, Gail J. Brown, and Nian Xiang Sun. _Nissi​m_artic​le.pdf. Accessed 23 July 2019. 2017. Acoustically Actuated Ultra-Compact NEMS Magneto- Papadopoulos, Elias P., Michalis Diamantaris, Panagiotis Papadopou- electric Antennas. Nature Communications 8(1): 296. https​:// los, Thanasis Petsas, Sotiris Ioannidis and Evangelos P. Mar- doi.org/10.1038/s4146​7-017-00343​-8. katos. 2017. The Long-Standing Privacy Debate: Mobile Web- Narayanan, Arvind, and Dillon Reisman. 2017. The Princeton Web sites Vs Mobile Apps. In Proceedings of the 26th International Transparency and Accountability Project, 45–67. Transparent Conference on World Wide Web. International World Wide Web Data Mining for Big and Small Data: Springer International Pub- Conferences Steering Committee, https://doi.org/10.1145/30389​ ​ lishing. https​://doi.org/10.1007/978-3-319-54024​-5_3. 12.30526​91. National Human Genome Research Institute. 15 for 15: Direct-to-Con- Patterson, Anna 2017. Introducing Gradient Ventures. Google Blog, sumer Genomic Testing, https​://www.genom​e.gov/dna-day/15- 11 July, https​://www.blog.googl​e/topic​s/machi​ne-learn​ing/intro​ for-15/direc​t-to-consu​mer-genom​ic-testi​ng. Accessed 23 July ducin​g-gradi​ent-ventu​res/. Accessed 2 March 2018. 2019. Peet, Alastair and Tom Wilde. 2017. Artifcial Intelligence: The Invest- Ney, Peter, Karl Koscher, Lee Organick, Luis Ceze and Tadayoshi ment Of 2017 And Beyond. Financier Worldwide, February, Kohno. 2017. Computer Security, Privacy, and DNA Sequenc- https​://www.finan​cierw​orldw​ide.com/artif​icial​-intel​ligen​ce- ing: Compromising Computers with Synthesized DNA, Privacy the-inves​tment​-of-2017-and-beyon​d/. Accessed 2 March 2018. Leaks, and More. In 26th USENIX Security Symposium. 25 Octo- Perakslis, Christine, Katina Michael and M. G. Michael. 2016. Smart ber, https​://www.useni​x.org/syste​m/fles​/confe​rence​/useni​xsecu​ Environments & The Convergence of the Veillances: Privacy rity1​7/sec17​-ney.pdf. Accessed 2 March 2018. Violations to Consider. MBA Faculty Conference Papers & News.com.au. 2017. Swedish Company Epicenter Implants Microchips Journal Articles. Paper 92, http://schol​arsar​chive​.jwu.edu/ into Employees. News.com.au., 4 April, http://www.news.com. mba_fac/92. Accessed 2 March 2018. au/technology​ /scien​ ce/human​ -body/swedi​ sh-compa​ ny-epice​ nter-​ Pfeifer, Max, Tim Dunte, Stefan Schneegass, Florian Alt and Michael impla​nts-micro​chips​-into-emplo​yees/news-story​/5c487​00ebb​ Rohs. 2015. Cruise Control for Pedestrians: Controlling Walking 54262​ae389​db085​593ab​12. Accessed 2 March 2018. Direction using Electrical Muscle Stimulation. Proceedings of Newmann, Stuart and Tina Stevens. 2019a. A ban, Not a Moratorium, the 33rd Annual ACM Conference on Human Factors in Comput- on Human Embryo Modifcation. The Berkeley Daily Planet. 28 ing Systems. https​://doi.org/10.1145/27021​23.27021​90. June 2019, https://www.berke​ leyda​ ilypl​ anet.com/issue​ /2019-06-​ Phillips, Andelka M. 2016. Wake Up and Smell the Cofee! A FLE 28/artic​le/47714​. Accessed 25 July 2019. 5SH approach to New and Emerging Technologies … Beyond ‘Responsible Research and Innovation. University of Edinburgh’s Governance Challenges with Human-Machine Mergers 79

IP/IT/Media Law Discussion Group seminar, Edinburgh Law org/wings​pread​-confe​rence​-on-the-preca​ution​ary-princ​iple/. Faculty. https://www.iash.ed.ac.uk/event/andelka-phillips-wake- Accessed 1 March 2018. and-smell-cofee-fe5sh-approach-new-and-emerging-technolo- Seppala, Timothy. J. 2015. Scientists successfully Implant Self- gies-%E2%80%A6-beyond. Accessed 1 March 2018. Destructing Nanobots into Live Mice. Engadget, 23 January, Phillips, Andelka M. 2017. Reading the Fine Print When Buying Your https​://www.engad​get.com/2015/01/23/nanob​ots-in-mice-do- Genetic Self Online: Direct-to-Consumer Genetic Testing Terms the-twist​/. Accessed 2 March 2018. and Conditions. New Genetics and Society 36(3): 273–295. https​ Sheppard, David. 2017. Microchipping Workers: What are the Moral, ://doi.org/10.1080/14636​778.2017.13524​68. Practical and Legal Implications? Personnel Today, 22 August, Phillips, Andelka M. 2019. Buying Your Self on the Internet: Wrap http://www.perso​nnelt​oday.com/hr/micro​chipp​ing-worke​rs- Contracts and Personal Genomics, 2019. Edinburgh: Edinburgh moral​-pract​ical-legal​-impli​catio​ns/. Accessed 2 March 2018. University Press. https​://edinb​urghu​niver​sityp​ress.com/book- Science and Technology Committee (Commons). 2019. Commercial buyin​g-your-self-onthe​-inter​net.html. genomics Inquiry Launched. 4 March 2019, https​://www.parli​ Phillips, Andelka M. and I.S. Mian. 2017. Governance and Assessment ament​.uk/busin​ess/commi​ttees​/commi​ttees​-a-z/commo​ns-selec​ of Future Spaces: A Discussion of Some Issues Raised by the t/scien​ce-and-techn​ology​-commi​ttee/news-parli​ament​-2017/ Possibilities of Human-Machine Mergers. In Data For Policy comme​rcial​-genom​ics-inqui​ry-launc​h-17-19/. Accessed 23 July 2017: Government by Algorithm? London, 6–7 September, https​ 2019. ://doi.org/10.5281/zenod​o.89610​9. Simonite, Tom. 2017. Google Unveils An AI Investment Fund. It’s Phillips, Andelka M., I.S. Mian and J. Charbonneau. 2015. Mol- Betting On An App Store For . Wired, 23 June, https​ ecule say ‘hello’ to molecule: Technological Innovation under ://www.wired​.com/story​/googl​e-ai-ventu​re-fund/. Accessed 2 the , GikII Conference Berlin, http://www.gikii​ March 2018. .org/?p=280. Accessed 25 July 2019. Solon, Olivia. 2016. Self-Driving Trucks: What’s the Future for Amer- Phillips, Andelka M., I.S. Mian and J. Charbonneau. 2016. ‘Living in ica’s 3.5 Million Truckers? The Guardian, 17 June, https://www.​ a Panopticon City: the Biological-Geographic-Economic-Social- thegu​ardia​n.com/techn​ology​/2016/jun/17/self-drivi​ng-truck​ Behavioural-Physical Complex—People and Places Under s-impac​t-on-drive​rs-jobs-us. Accessed 1 March 2018. Dynamic Surveillance’. In Surveillance: Power, Performance & Solon, Olivia. 2017. World’s lamest cyborg? My Microchip Isn’t Cool Trust (7th Biannual Surveillance and Society Conference, Bar- Now—But it Could be the Future. The Guardian, 2 August, https​ celona, April. http://www.ssn20​16.net/blog/wp-conte​nt/uploa​ ://www.thegu​ardia​n.com/techn​ology​/2017/aug/02/micro​chip- ds/Bookl​et_Abstr​acts_2016.pdf. Accessed 16 September 2019. conta​ctles​s-payme​nt-three​-squar​e-marke​t-bioha​x. Accessed 2 Privacy Commissioner (New Zealand). 2016. International study Finds March 2018. Privacy Shortfalls In Internet of Things Devices. 28 September, Som, Claudia, Lorenz M. Hilty, and Andreas R. Kohler. 2009. The https​://www.priva​cy.org.nz/news-and-publi​catio​ns/state​ments​ Precautionary Principle as a Framework for a Sustainable Infor- -media​-relea​ses/inter​natio​nal-study​-fnds​-priva​cy-short​falls​-in- mation Society. Journal of Business Ethics 85(3): 493–505. https​ inter​net-of-thing​s-devic​es/. Accessed 1 March 2018. ://doi.org/10.1007/s1055​1-009-0214-x. Reeve, R.G., S. Voeneky, D. Caetano-Anollés, F. Beck, and C. Boëte. Starritt, Alexander. 2016. Sweden is Paying People to Fix Their 2018. Agricultural Research, or a New Bioweapon System? Sci- Belongings Instead of Throwing Them Away. World Eco- ence 362(6410): 35–37. https://doi.org/10.1126/scien​ ce.aat76​ 64​ . nomic Forum, 27 October 27, https​://www.wefor​um.org/agend​ Reijers, Wessel, Fiachr O’Brolcháin and Paul Haynes. 2016. Govern- a/2016/10/swede​n-is-tackl​ing-its-throw​away-cultu​re-with-tax- ance in Technologies & Social Contract Theories. break​s-on-repai​rs-will-it-work/. Accessed 1 March 2018. Ledger, 1: 134–151, https​://www.ledge​rjour​nal.org/ojs/index​ Statistica. 2017. Number of Smartphone Users Worldwide from 2014 .php/ledge​r/artic​le/view/62. Accessed 25 July 2019. to 2020 (in Billions). https://www.stati​ sta.com/stati​ stics​ ​/33069​ Riley, Boots. 2018. Sorry To Bother You, http://www.sorrytobot​ heryo​ ​ 5/numbe​r-of-smart​phone​-users​-world​wide/. Accessed 1 March u.movie​. Accessed 25 July 2019. 2018. Rio Declaration on Environment and Development. 1992. Principle Statt, Nick. 2017. Elon Musk launches Neuralink, a Venture to Merge 15, http://www.unesco.org/educa​ tion/pdf/RIO_E.PDF​ . Accessed the Human Brain with AI. The Verge, 27 March, https​://www. 1 March 2018. theve​rge.com/2017/3/27/15077​864/elon-musk-neura​link-brain​ Rohrscheib, Chelsie E. and Jeremy C. Brownlie. 2013. Microorgan- -compu​ter-inter​face-ai-cybor​gs. isms that Manipulate Complex Animal Behaviours by Afecting Strickland, Eliza. 2017. Facebook Announces ‘Typing-by-Brain’ the Host’s Nervous System. Springer Science Reviews. 1: 1–2, Project. IEEE Spectrum, 20 April, http://spect​rum.ieee.org/the- 133–40, https​://doi.org/10.1007/s4036​2-013-0013-8. Accessed human​-os/biome​dical​/bioni​cs/faceb​ook-annou​nces-typin​g-by- 25 July 2019. brain​-proje​ct. Accessed 2 March 2018. Royal Society Steering Group on Neural Interface Technologies. Suzuki, Tohru, Norio Fujimaki and Kazuhisa Ichikawa. 2007. iBrain: A 2019. iHuman perspective: Neural interfaces. https​://royal​socie​ Simulation and Visualization Tool for Activation of Brain Areas ty.org/topic​s-polic​y/proje​cts/ihuma​n-persp​ectiv​e/. Accessed 16 on a Realistic 3D Brain Image. BMC Neuroscience 8(S2), https​ September. ://www.ncbi.nlm.nih.gov/pmc/articles/PMC44​ 36429​ /​ . Accessed Semsarian, Christopher. 2019. Genome Sequencing for Sale on the 1 March 2018. NHS. British Medical Journal 364:l789, https​://www.bmj.com/ Sørensen, Evelyne J.B. 2016. The Post that Wasn’t: Facebook Moni- conte​nt/364/bmj.l789. Accessed 23 July 2019. tors Everything Users Type and Not Publish. Computer Law Sanders, Robert. 2017. Defense Department Pours $65 Million into & Security Review 32(1): 146–151. https​://doi.org/10.1016/j. Making CRISPR Safer. Berkeley News, 19 July, http://news. clsr.2015.12.007. berke​ley.edu/2017/07/19/defen​se-depar​tment​-pours​-65-milli​ Suda, T. and T. Nakano. 2018. Molecular Communication: A Personal on-into-makin​g-crisp​r-safer​/. Accessed 2 March 2018. Perspective. IEEE Trans Nanobioscience 17:424-432. https://doi.​ Schmidt R., D. Ulanova, L.Y. Wick, H.B. Bode and P. Garbeva. 2019. org/10.1109/TNB.2018.28519​51. Microbe-driven chemical ecology: past, present and future. ISME Tanasescu, Mihnea. 2017. Rivers Get Human Rights: They Can Sue to J. 9 July 2019. https​://doi.org/10.1038/s4139​6-019-0469-x. Protect Themselves. Scientifc American, 19 June, https​://www. Science and Environmental Health Network. 1998. The Wingspread scientifc​ ameri​ can.com/artic​ le/river​ s-get-human​ -right​ s-they-can-​ Consensus Statement on the Precautionary Principle. http://sehn. sue-to-prote​ct-thems​elves​/. Accessed 1 March 2018. 80 A. M. Phillips, I. S. Mian

Taylor, Harriet. 2015. Biggest Cybersecurity Threats in 2016. CNBC, Vincent, James. 2018. Google ‘Fixed’ Its Racist Algorithm by Remov- 28 December, http://www.cnbc.com/2015/12/28/bigge​st-cyber​ ing Gorillas from its Image-Labeling Tech. The Verge, 12 Janu- secur​ity-threa​ts-in-2016.html. Accessed 1 March 2018. ary 2018, https://www.theve​ rge.com/2018/1/12/16882​ 408/googl​ ​ Templeton, Brad. 2019. Tesla Autopilot Repeats Fatal Crash; Do They e-racis​t-goril​las-photo​-recog​nitio​n-algor​ithm-ai. Accessed 23 Learn From Past Mistakes? Forbes, 21 May 2019, https​://www. July 2019. forbes.com/sites​ /bradt​ emple​ ton/2019/05/21/tesla​ -autop​ ilot-repea​ ​ Wakefeld, Jane. 2017. Germany bans children’s smartwatches. BBC, ts-fatal​-crash​-do-they-learn​-from-past-mista​kes. Accessed 23 17 November, http://www.bbc.com/news/technology​ -42030​ 109​ . July 2019. Accessed 1 March 2018. Timmer, John. 2017. Researchers Encode Malware in DNA, Compro- Walter, Damien. 2016. When AI Rules the World: What SF Novels mise DNA Sequencing Software. Ars Technica, 12 August, https​ Tell us About our Future Overlords. The Guardian, 18 March, ://arste​chnic​a.com/scien​ce/2017/08/resea​rcher​s-encod​e-malwa​ https​://www.thegu​ardia​n.com/books​/books​blog/2016/mar/18/ re-in-dna-compr​omise​-dna-seque​ncing​-softw​are/. Accessed 2 ai-sf-novel​s-artif​ cial​-intel​ligen​ce-scien​ce-fcti​on-gibso​n-neuro​ March 2018). mance​r. Accessed 2 March 2018. Tracy, Phillip. 2017. Infected DNA Successfully Hacks Computer in Ward, Jacob. The ‘Geno-Economists’ Say DNA Can Predict Our Terrifying Experiment. The Daily Dot, 10 August, https​://www. Chances of Success. The New York Times Magazine, 16 Novem- daily​dot.com/debug​/dna-hack-compu​ter/. Accessed 2 March ber 2018. https​://www.nytim​es.com/inter​activ​e/2018/11/16/ 2018. magaz​ine/tech-desig​n-econo​mics-genes​.html. Accessed 23 July Trafton, Anne. 2016. Nanobionic Spinach Plants Can Detect Explo- 2019. sives. MIT News, 30 October, http://news.mit.edu/2016/nanob​ Warwick, Kevin. 2016. The Future of Artifcial Intelligence and Cyber- ionic-spina​ ch-plant​ s-detec​ t-explo​ sives​ -1031​ . Accessed 1 March netics. MIT Technology, 10 November, https​://www.techn​ology​ 2018. revie​w.com/s/60283​0/the-futur​e-of-artif​ cial​-intel​ligen​ce-and- Treaty on the Functioning of the European Union. 2007. Ofcial cyber​netic​s/. Accessed 2 March 2018). Journal C 326, 26/10/2012 P. 0001—0390, http://eur-lex.europ​ Weber, Rolf H. 2010. Internet of Things—New Security and Privacy a.eu/legal​-conte​nt/EN/TXT/?uri=celex​%3A120​12E%2FTXT​. Challenges. Computer Law & Security Review 26: 23–30. https​ Accessed 1 March 2018. ://doi.org/10.1016/j.clsr.2009.11.008. Tuptuk, Nilufer and Stephen Hailes. 2016. The Cyber attack on Williams, Lucy. 2017. Driverless lorries could mean 600,000 Lost Jobs. ’s Power Grid is a Warning of What’s to Come. The Con- It’s Time We Took a Universal Basic Income Seriously. Evolve versation, 13 January, http://theconvers​ ation​ .com/the-cyber​ attac​ ​ Politics, 28 August 2017, http://evolv​epoli​tics.com/drive​rless​ k-on-ukraines-power​ -grid-is-a-warni​ ng-of-whats​ -to-come-52832​ ​ -lorri​es-could​-mean-600000-lost-jobs-its-time-we-took-a-unive​ ​ . Accessed 2 March 2018. rsal-basic​-incom​e-serio​usly/. Accessed 2 March 2018. UCL. 2018. Inquiry Launches into History of Eugenics at UCL. 5 Wolbring, Gregor. 2009. Innovation for Whom? Innovation for What? December 2018. https​://www.ucl.ac.uk/news/2018/dec/inqui​ry- The Impact of Ableism. 2020 Science Guest Blog, 14 December, launc​hes-histo​ry-eugen​ics-ucl. Accessed 23 July 2019. http://2020scienc​ e.org/2009/12/14/wolbr​ ing/​ . Accessed 1 March UCL Culture. Galton and Anthropometrics. UCL Culture, Galton Col- 2018. lection. https​://www.ucl.ac.uk/cultu​re/galto​n-colle​ction​/galto​ Young, Stephen L. 2017. Unintended Consequences of 21st Century n-and-anthr​opome​trics​. Accessed 23 July 2019. Technology for Agricultural Pest Management. EMBO Reports UK Information Commissioner’s Ofce. 2106. Privacy Regulators 18(9): 1478. https​://doi.org/10.15252​/embr.20174​4660. Study Finds Internet of Things Shortfalls. 22 September, https​ Zimmer, Zac. 2017. Bitcoin and Potosí Silver: Historical Perspectives ://ico.org.uk/about​-the-ico/news-and-event​s/news-and-blogs​ on . Technology and Culture 58: 307–334. https​ /2016/09/privacy-regul​ ators​ -study​ -fnds​ -inter​ net-of-thing​ s-short​ ​ ://doi.org/10.1353/tech.2017.0038. falls​/. Accessed 2 March 2018. Vince, Gaia. 2012. The High Cost of Our Throwaway Culture. BBC, 29 Publisher’s Note Springer Nature remains neutral with regard to November, http://www.bbc.com/future/story​ /20121​ 129-the-cost-​ jurisdictional claims in published maps and institutional afliations. of-our-throw​away-cultu​re. Accessed 1 March 2018. Vincent, James. 2017. Elon Musk and AI Leaders Call for a Ban on Killer Robots. The Verge, 21 August, https​://www.theve​rge. com/2017/8/21/16177​828/kille​r-robot​s-ban-elon-musk-un-petit​ ion. Accessed 1 March 2018.