Annotated Table of Contents AI Education: Deep Neural Network Learning Resources Welcome to AI Matters, Volume 3(3) Todd W

Home , Peter Norvig, SIGAI, Turing Award, Yolanda Gil

AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 AI Matters

Annotated Table of Contents AI Education: Deep Neural Network Learning Resources Welcome to AI Matters, Volume 3(3) Todd W. Neller Eric Eaton & Amy McGovern Full article: http://doi.acm.org/10.1145/3137574.3137580 Full article: http://doi.acm.org/10.1145/3137574.3137575 Interested in deep learning? Want to learn more or teach it in your classes? Look no further! AI Events Michael Rovatsos Celebrating the Past, Present, and Full article: http://doi.acm.org/10.1145/3137574.3137576 Future of Computing Timothy E. Lee & Justin Svegliato Listing of upcoming SIGAI-related events. Full article: http://doi.acm.org/10.1145/3137574.3137581

ACM SIGAI Activity Report Two SIGAI student scholars cover “The 50 Years of the ACM Turing Award Celebration.” Sven Koenig, Sanmay Das, Rose- mary Paradis, Eric Eaton, Yolanda Gil, Katherine Guo, Bojun Huang, Albert ACM SIGAI CHINA: A New Incubator Jiang, Benjamin Kuipers, Nicholas Mat- for AI in China tei, Amy McGovern, Larry Medsker, Le Dong, Man Yuan, Ming-Liang Xu & Ji Todd Neller, Plamen Petrov, Michael Wan Rovatsos & David Stork Full article: http://doi.acm.org/10.1145/3137574.3137582 Full article: http://doi.acm.org/10.1145/3137574.3137577 Read about the new China chapter of ACM SIGAI. This report summarizes the annual activity of ACM SIGAI from July 2016 to June 2017. On the Importance of Monitoring and Directing Progress in AI AI Proﬁles: An Interview with Maja Lukas Prediger Mataric´ Full article: http://doi.acm.org/10.1145/3137574.3137583 Amy McGovern & Eric Eaton ACM SIGAI Student Essay Contest Winner Full article: http://doi.acm.org/10.1145/3137574.3137578

An interview with Maja Mataric´ from USC. Truth in the ‘Killer Robots’ Angle? Matthew Rahtz AI Buzzwords Explained: Multi-Agent Full article: http://doi.acm.org/10.1145/3137574.3137584 Path Finding Hang Ma & Sven Koenig ACM SIGAI Student Essay Contest Winner Full article: http://doi.acm.org/10.1145/3137574.3137579 How Do We Ensure That We Re- If your multi-agent system needs to ﬁnd paths, this main In Control of Our Autonomous article will help get you started. Weapons? Ilse Verdiesen

1 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Full article: http://doi.acm.org/10.1145/3137574.3137585 Contents Legend ACM SIGAI Student Essay Contest Winner

The Ethics of Automated Behavioral Book Announcement Microtargeting Ph.D. Dissertation Brieﬁng Dennis G. Wilson Full article: http://doi.acm.org/10.1145/3137574.3139451 AI Education ACM SIGAI Student Essay Contest Winner Event Report

Hot Topics Links Humor SIGAI website: http://sigai.acm.org/ Newsletter: http://sigai.acm.org/aimatters/ AI Impact Blog: http://sigai.acm.org/ai-matters/ Twitter: http://twitter.com/acm sigai/ AI News

Edition DOI: 10.1145/3137574 Opinion

Paper Precis´ Join SIGAI Spotlight Students $11, others $25 For details, see http://sigai.acm.org/ Video or Image Beneﬁts: regular, student Details at http://sigai.acm.org/aimatters Also consider joining ACM. Our mailing list is open to all.

Submit to AI Matters!

We’re accepting articles and announcements now for future issues. Details on the submission process are available at http://sigai.acm.org/aimatters.

AI Matters Editorial Board

Eric Eaton, Editor-in-Chief, U. Pennsylvania Amy McGovern, Editor-in-Chief, U. Oklahoma Sanmay Das, Washington Univ. in Saint Louis Alexei Efros, Univ. of CA Berkeley Susan L. Epstein, The City Univ. of NY Yolanda Gil, ISI/Univ. of Southern California Doug Lange, U.S. Navy Kiri Wagstaff, JPL/Caltech Xiaojin (Jerry) Zhu, Univ. of WI Madison Contact us: [email protected]

2 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Welcome to AI Matters, Volume 3, Issue 3 Eric Eaton, Co-Editor (University of Pennsylvania; [email protected]) Amy McGovern, Co-Editor (University of Oklahoma; [email protected]) DOI: 10.1145/3137574.3137575

Welcome to the third issue in our third year of Thanks for reading! Don’t forget to send your AI Matters! In the spring, ACM SIGAI spon- ideas and future submissions to AI Matters! sored a student essay contest on the “Re- sponsible Use of AI Technologies.” This issue features the first set of winning essays from Eric Eaton is a Co-Editor the contest, with the second set of winning es- of AI Matters. He is a fac- says to appear in the next issue. ulty member at the Uni- In addition to having their essay appear in AI versity of Pennsylvania in Matters, the contest winners received either the Department of Com- monetary prizes or one-on-one Skype ses- puter and Information Sci- sions with leading AI researchers. The stu- ence, and in the Gen- dents reported exceptional experiences: eral Robotics, Automa- tion, Sensing, and Per- I just had an absolutely phenomenal con- ception (GRASP) lab. His versation with Eric Horvitz. Really so research is in machine spectacular. Thank you so so so much for learning and AI, with applications to robotics, enabling this to happen. .... This was such sustainability, and medicine. a privilege and wonderful experience. Amy McGovern is a Co- It was great talking to [Peter Norvig] and Editor of AI Matters. She we discussed a broad range of topics from is an Associate Profes- Autonomous Weapons, changes in the sor of computer science job market caused by automation to AI at the University of Okla- techniques to help people programming. homa and an adjunct as- It was very inspiring to talk to him! sociate professor of me- teorology. She directs Speaking of leading researchers, this issue’s the Interaction, Discovery, AI Interviews column highlights Maja Mataric,´ Exploration and Adapta- the Vice Dean for Research and the Direc- tion (IDEA) lab. Her re- tor of the Robotics and Autonomous Systems search focuses on machine learning and Center at the Univ. of Southern California. data mining with applications to high-impact In AI news, this issue includes the 2016-2017 weather. ACM SIGAI Activity Report, which summarizes the annual activities of the SIGAI, re- flections on The 50 Years of the ACM Tur- ing Award Celebration by two student SIGAI scholars, and a report on the new China chapter of SIGAI. The (buzz)word of this issue is multi-agent path finding. “What’s that?” you say. Well, you can read all about it in the AI Buzzwords Ex- Submit to AI Matters! plained column. Or, if deep learning is more We’re accepting articles and announce- your thing, then check out the AI Education ments for future issues. Details on column on resources for teaching and learn- the submission process are available at ing about deep neural networks. http://sigai.acm.org/aimatters. Copyright c 2017 by the author(s).

3 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

AI Events Michael Rovatsos (University of Edinburgh; [email protected]) DOI: 10.1145/3137574.3137576

This section features information about up- niques, and tools for automating the analysis, coming events relevant to the readers of AI design, implementation, testing, and mainte- Matters, including those supported by SIGAI. nance of large software systems. ASE 2017 We would love to hear from you if you are are features high quality contributions describing organizing an event and would be interested significant, original, and unpublished results. in cooperating with SIGAI, or if you have announcements relevant to SIGAI. For more information about conference support visit 9th International Joint Conference on sigai.acm.org/activities/requesting sponsor- Computational Intelligence ship.html. (IJCCI 2017) Funchal, Portugal, November 1-3, 2017 International Conference on www.ijcci.org Computational Approaches to The purpose of the International Joint Confer- Diversity in Interaction and Meaning ence on Computational Intelligence (IJCCI) is to bring together researchers, engineers and San Servolo/Venice, Italy, October 7-9, 2017 practitioners interested on the field of Compu- www.essence-network.com/essence-final- tational Intelligence both from theoretical and conference application perspectives. Four simultaneous Diversity-awareness, understood as the ca- tracks will be held covering different aspects pability of an intelligent system to take the of Computational Intelligence, including evolu- heterogeneity of human or artificial agents it tionary computation, fuzzy computation, neu- is interacting with into account when making ral computation and cognitive and hybrid sys- decisions, has been researched by many tems. The connection of these areas in all communities across several disciplines in the their wide range of approaches and applica- past, including semantic technologies, mul- tions forms the International Joint Conference tiagent systems, knowledge representation on Computational Intelligence. and reasoning, and NLP. This conference will provide a forum for leading experts from the above (and other) areas to discuss key CRA Summit on Technology and Jobs research issues surrounding diversity-aware Washington DC, USA, December 12, 2017 AI. Participation is by invitation only, and cra.org/events/summit-technology-jobs/ financial support is available – please con- The goal of the summit is to put the issue of tact [email protected] if you are technology and jobs on the national agenda in interested in attending. an informed and deliberate manner. The summit will bring together leading technologists, 32nd International Conference on economists, and policy experts who will offer their views on where technology is headed Automated Software Engineering and what its impact may be, and on policy is- Urbana-Champaign, USA, October 30 to sues raised by these projections and possible November 3, 2017 policy responses. The summit is hosted by the ase2017.org Computing Research Association, as part of The IEEE/ACM Automated Software Engi- its mission to engage the computing research neering (ASE) Conference series is the pre- community to provide trusted, non-partisan in- mier research forum for automated software put to policy thinkers and makers. engineering. Each year, it brings together researchers and practitioners from academia and industry to discuss foundations, tech- Copyright c 2017 by the author(s).

4 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

10th International Conference on tion systems. Six simultaneous tracks will be Agents and Artificial Intelligence held, covering different aspects of Enterprise (ICAART 2018) Information Systems Applications, including Enterprise Database Technology, Systems In- Funchal, Portugal, January 16-18, 2018 tegration, Artificial Intelligence, Decision Sup- www.icaart.org port Systems, Information Systems Analysis The purpose of the International Conference and Specification, Internet Computing, Elec- on Agents and Artificial Intelligence is to bring tronic Commerce, Human Factors and Enter- together researchers, engineers and practi- prise Architecture. tioners interested in the theory and applica- Submission deadline: October 18, 2017. tions in the areas of Agents and Artificial In- telligence. Two simultaneous related tracks 31st International Conference on will be held, covering both applications and current research work. One track focuses Industrial, Engineering & Other on Agents, Multi-Agent Systems and Soft- Applications of Applied Intelligent ware Platforms, Distributed Problem Solving Systems (IEA/AIE-2018) and Distributed AI in general. The other Montreal, Canada, June 25-28, 2018 track focuses mainly on Artificial Intelligence, ieaaie2018.encs.concordia.ca Knowledge Representation, Planning, Learn- IEA/AIE 2018 continues the tradition of em- ing, Scheduling, Perception Reactive AI Sys- phasizing applications of applied intelligent tems, and Evolutionary Computing and other systems to solve real-life problems in all ar- topics related to Intelligent Systems and Com- eas including engineering, science, industry, putational Intelligence. automation & robotics, business & finance, medicine and biomedicine, bioinformatics, cy- 23rd International Conference on berspace, and human-machine interactions. Intelligent User Interfaces (IUI 2018) IEA/AIE-2018 will include oral presentations, Tokyo, Japan, March 7-11, 2018 invited speakers, and special sessions. iui.acm.org Submission deadline: November 27, 2017 ACM IUI 2018 is the 23rd annual meeting of the intelligent interfaces community that will be 17th International Conference on held in Tokyo, on March 7th-11th, 2018. IUI is Autonomous Agents and Multiagent where the HCI and AI communities meet, and Systems (AAMAS 2018) the focus of the conference is to improve the Stockholm, Sweden, July 10-15, 2018 interaction between humans and machines, celweb.vuse.vanderbilt.edu/aamas18 by leveraging both traditional HCI approaches AAMAS is the largest and most influential and solutions that involve state-of-the art AI conference in the area of agents and multi- techniques like machine learning, natural lan- agent systems. The aim of the conference guage processing, data mining, knowledge is to bring together researchers and practi- representation and reasoning. Along with 25 tioners in all areas of agent technology and topics in AI and HCI, this year IUI especially to provide a single, high-profile, internation- encourages submissions on explainable intel- ally renowned forum for research in the the- ligent user interfaces. ory and practice of autonomous agents and Submission deadline: October 8, 2017 multiagent systems. AAMAS is the flag- 20th International Conference on ship conference of the non-profit Interna- tional Foundation for Autonomous Agents and Enterprise Information Systems Multiagent Systems (IFAAMAS). This edition (ICEIS 2018) will be a part of the Federated AI Meeting Funchal, Portugal, March 21-24, 2018 (FAIM) where AAMAS is co-located with sev- www.iceis.org eral AI conferences including the International The purpose of the 20th International Con- Joint Conference on Artificial Intelligence (IJ- ference on Enterprise Information Systems CAI)/European Conference on Artificial Intel- (ICEIS) is to bring together researchers, en- ligence (ECAI) and the International Confer- gineers and practitioners interested in the ad- ence on Machine Learning (ICML). vances and business applications of informa- Submission deadline: November 14, 2017.

5 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Michael Rovatsos is the Conference Coordination Ofﬁcer for ACM SIGAI, and a faculty member of the School of Informatics at the University of Edin- burgh, UK. His research in in multiagent systems, social computation, and human-friendly AI. Con- tact him at [email protected].

6 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

ACM SIGAI Activity Report Sven Koenig (elected; ACM SIGAI Chair) Sanmay Das (elected; ACM SIGAI Vice-Chair) Rosemary Paradis (elected; ACM SIGAI Secretary/Treasurer) Eric Eaton (appointed; ACM SIGAI Newsletter Editor-in-Chief) Yolanda Gil (appointed; ACM SIGAI Past Chair) Katherine Guo (appointed; ACM SIGAI Membership and Outreach Officer) Bojun Huang (appointed; ACM SIGAI Information Officer) Albert Jiang (appointed; ACM SIGAI Education Officer) Benjamin Kuipers (appointed; ACM SIGAI Ethics Officer) Nicholas Mattei (appointed; ACM SIGAI Ethics Officer) Amy McGovern (appointed; ACM SIGAI Newsletter Editor-in-Chief) Larry Medsker (appointed; ACM SIGAI Public Policy Officer) Todd Neller (appointed; ACM SIGAI Education Activities Officer) Plamen Petrov (appointed; ACM SIGAI Industry Liaison Officer) Michael Rovatsos (appointed; ACM SIGAI Conference Coordination Officer) David Stork (appointed; ACM SIGAI Award Officer) DOI: 10.1145/3137574.3137577

Abstract better. In order to do so, ACM SIGAI has created several new appointed officer positions, We are happy to present the annual activity namely an industry liaison officer, an award of- report of ACM SIGAI, covering the period from ficer and two ethics officers. July 2016 to June 2017. In the course of the last year, ACM SIGAI has been responsive to specific events and cir- The scope of ACM SIGAI consists of the cumstances as well as continued to support study of intelligence and its realization in and expand a range of regular activities. computer systems (see also its website at sigai.acm.org). This includes areas such as Responsive Initiatives in the Last Year

autonomous agents, cognitive modeling, ACM SIGAI actively supported the founding computer vision, constraint programming, hu- of a new ACM SIGAI chapter in China with man language technologies, intelligent user help from the membership and outreach offi- interfaces, knowledge discovery, knowledge cer. ACM SIGAI China held its first event, the representation and reasoning, machine learn- ACM SIGAI China Symposium on New Chal- ing, planning and search, problem solving and lenges and Opportunities in the Post-Turing AI robotics. Era, as part of the ACM China Turing 50th Cel- ebration Conference on May 12-14, 2017 in Members of ACM SIGAI come from academia, Shanghai. industry and government agencies worldwide. ACM SIGAI is proud of the fact that many AI ACM SIGAI held the ACM SIGAI Student Es- researchers in the past year received ACM say Contest on the Responsible Use of AI honors, including becoming ACM fellows as Technologies (run by one of the ethics of- well as receiving other awards. ficers), where students could win five cash ACM SIGAI is committed to increase its ac- prizes of US$500 or skype conversations with tivities in order to support its members even five very senior AI researchers from academia or industry if their essay provided good an- Copyright c 2017 by the author(s). swers to the following two questions:

7 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

• What do you see as the 1-2 most pressing ethi- expression and communication for scientists. cal, social or regulatory issues with respect to AI All individuals are entitled to participate technologies? in any ACM activity.’ SIGAI is working on • What position or steps can governments, in- policies to support inclusive participation dustries or organizations (including ACM SIGAI) in our AI-related activities. We encourage take to address these issues or shape the dis- event organizers to share their efforts and cussions on them? experiences with us through our AI Matters newsletter at [email protected] The winning essays will be published in the and blog postings at sigai.acm.org/aimatters/blog/ ACM SIGAI newsletter. .” ACM SIGAI extended its coordination and col- ACM SIGAI also actively supported the Jour- laboration with a variety of groups, both inside nal of Human-Robot Interaction (JHRI) in its and outside of ACM: desire to become an ACM journal and be included in the ACM Digital Library. JHRI will be- • ACM SIGAI started to participate in the ACM come the ACM Transactions on Human-Robot US Public Policy Council (USACM). USACM ad- Interaction in January 2018. dresses US public policy issues related to computing and information technology and regularly educates and informs US Congress, the US Ad- Continuing Activities ministration and the US courts about significant developments in the computing field and how Organizing events: those developments affect public policy. The ACM SIGAI organized the annual ACM SIGAI public policy officer, for example, facilitated talks between the leaderships of USACM and the Career Network Conference (CNC), overseen American Association for the Advancement of by the vice chair. CNC showcases the work Science (AAAS) on areas of potential collabo- of early career researchers (including stu- ration. dents) to their potential mentors and employ- • ACM SIGAI started to participate in the IEEE ers. Each early career researcher is men- Global Initiative for Ethical Considerations in AI tored by a senior AI researcher, with small and Autonomous Systems. The purpose of this group mentoring sessions as well as individ- initiative is to ensure that every technologist is ual advice. CNC 2016 was held at Northeast- educated, trained and empowered to prioritize ern University in Boston, Massachusetts, on ethical considerations in the design and devel- October 19-20, 2016 (and ACM SIGAI grate- opment of autonomous and intelligent systems. fully acknowledges the support and hospital- • ACM SIGAI also provided a response to the pub- ity of Northeastern University). 36 early ca- lic request for information from the US Office of reer researchers presented their work via talks Science and Technology Policy (OSTP) in 2016 and posters, and two panels (on Career Op- on Preparing for the Future of AI, thus support- tions and Getting Started, featuring senior AI ing the US government in making decisions con- researchers and practitioners from academia, cerning AI technologies and their applications. industry and the public sector) completed the program. In response to developments regarding international travel policies, ACM SIGAI released ACM SIGAI has an agreement with the As- the following public policy statement in 2017 sociation for the Advancement of AI (AAAI) via an effort of the public policy officer: to jointly organize the annual joint job fair at the AAAI conference, where attendees can “The ACM SIGAI executive committee find out about job and internship opportunities shares the view of its parent organization from representatives from industry, universi- that ’the open exchange of ideas and the ties and other organizations. This event was freedom of thought and expression are held at AAAI 2017 as planned. central to the aims and goals of ACM. ACM supports the statute of International Council for Science in that the free and responsible Supporting international conferences and practice of science is fundamental to scientific other events: advancement and human and environmental wellbeing. Such practice, in all its aspects, ACM SIGAI processed requests for co- requires freedom of movement, association, sponsored and in-cooperation status from 27

8 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 conferences. The conference coordination of- • 16th International Conference on Autonomous ficer improved the support provided to confer- Agents and Multiagent Systems (AAMAS 2017) ence organizers by contacting them person- • 16th International Conference on Artificial Intel- ally immediately after approval, inviting them ligence and Law (ICAIL 2017) to publicize their conference via the ACM • International Conference on Industrial, Engi- SIGAI newsletter and mailing lists, and follow- neering & Other Applications of Applied Intelli- ing up with a request for a conference report gent Systems (IEA/AIE 2017) after the conference, in order to publish it in • 14th International Conference on Informatics the ACM SIGAI newsletter and blog. in Control, Automation and Robotics (ICINCO ACM SIGAI has an agreement with AAAI to 2017) co-sponsor, jointly with AAAI, the annual joint • 11th ACM Conference on Recommender Sys- doctoral consortium at the AAAI conference. tems (RecSys 2017) The doctoral consortium provides an oppor- • International Conference on the Foundations of tunity for Ph.D. students to discuss their re- Digital Games (FDG 2017) search interests and career objectives with the • International Joint Conference on Rules and other participants and a group of established Reasoning (RuleML+RR 2017) AI researchers that act as their mentors. • 4th International Workshop on Sensor-based ACM SIGAI also co-sponsored the following Activity Recognition and Interaction (iWOAR conferences (future events are in italics): 2017) • Data Institute San Francisco Conference (DSCO • ACM/IEEE International Conference on Human- 2017) Robot Interaction (HRI 2017) • 9th International Joint Conference on Knowl- • 22nd International Conference on Intelligent edge Discovery, Knowledge Engineering and User Interfaces (IUI 2017) Knowledge Management (IC3K 2017) • ACM/IEEE International Conference on Human- • 9th International Joint Conference on Computa- Robot Interaction (HRI 2018) tional Intelligence (IJCCI 2017) • ACM/IEEE International Conference on Auto- • 10th International Conference on Agents and Ar- mated Software Engineering (ASE 2017) tificial Intelligence (ICAART 2017) • ACM/IEEE International Conference on Auto- • 7th International Conference on Pattern Recog- mated Software Engineering (ASE 2018) nition Applications and Methods (ICPRAM 2018) • International Conference on Web Intelligence • 11th International Joint Conference on Biomed- (WI 2017) ical Engineering Systems and Technologies (BIOSTEC 2018) • International Conference on Web Intelligence (WI 2018) • 31st International Conference on Industrial, En- gineering & Other Applications of Applied Intelli- • 23rd International Conference on Intelligent gent Systems (IEA/AIE 2018) User Interfaces (IUI 2018) • 12th ACM Conference on Recommender Sys- • 24th International Conference on Intelligent tems (RecSys 2018) User Interfaces (IUI 2019) • 31th Annual ACM Symposium on User Interface Software and Technology (UIST 2018) In addition, ACM SIGAI granted in- cooperation status to the following con- • 20th International Conference on Enterprise In- ferences: formation Systems (ICEIS 2018)

• Swarm/Human Blended Intelligence Workshop ACM SIGAI has an agreement with the Inter- (SHBI 2016) national Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS) to spon- • 6th International Conference on Pattern Recog- sor the ACM SIGAI Autonomous Agents Re- nition Applications and Methods (ICPRAM 2017) search Award. The ACM SIGAI Autonomous • International Conference on Agents and Artiﬁ- Agents Research Award is an annual award cial Intelligence (ICAART 2017) for excellence in research in the area of au- • International Knowledge System Conference tonomous agents. The recipient is invited to (KMIKS 2017) give a talk at the International Conference on

9 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Autonomous Agents and Multiagent Systems • AI Interviews (with interesting people from (AAMAS). The 2017 ACM SIGAI Autonomous academia, industry and government), Agents Research Award was presented at AA- • AI Amusements (including AI humor, puzzles MAS 2017 to David Parkes from Harvard Uni- and games), versity for his work on a variety of topics in • AI Education (led by one of the education activi- multi-agent systems and economics. ties officers), • AI Policy Issues (led by the public policy officer), Increasing the visibility of AI research: • AI Buzzwords (which explains new AI concepts or terms), ACM SIGAI actively supports the Research Highlight Track of the Communications of the • AI Events (which includes conference an- ACM (CACM) by nominating publications of nouncements and reports), recent, significant and exciting AI research re- • AI Dissertation Abstracts and sults that are of general interest to the com- • News from AI Groups and Organizations. puter science research community to the Re- search Highlight Track. This way, ACM SIGAI To promote readership, the editors-in- helps to make important AI research results chief have moved to an open-access visible to many computer scientists. model (where AI Matters is openly available on the ACM SIGAI website) and instituted a new AI Matters blog (at Supporting student members: sigai.acm.org/aimatters/blog/) to ACM SIGAI believes that funding students is a feature timely posts and promote discussion good way to ensure vitality in the AI commu- among community members. For example, nity and thus a good investment in the future. the public policy officer posts new informa- Consequently, it awarded a number of schol- tion every two weeks in the blog to survey arships to students to attend conferences co- and report on current AI policy issues and sponsored by it as well as the 50 Years of the raise awareness about the activities of other ACM Turing Award Celebration. The amounts organizations that share interests with ACM of the scholarships vary but are generally in SIGAI. Behind the scenes, the editors-in-chief the range of US$1,000 to US$10,000 per con- have also revamped the editorial process and ference, depending on the conference size. created tools to streamline the assembly of ACM SIGAI changed the period of time be- each issue. fore students who join ACM SIGAI can apply ACM SIGAI organized or co-organized ACM for financial benefits. There is now a 3-month webinars to inform ACM members about AI membership requirement before students can topics, such as the Town Hall on AI, Ma- apply for fellowships and travel support. chine Learning, and More in 2016 (run by the secretary/treasurer) and the Panel and Communicating with and supporting Town Hall on Big Thoughts and Big Ques- members: tions about Ethics in AI (run by one of the ethics officers). The webinars were ACM SIGAI publishes four issues of its streamed live but the videos are archived at newsletter AI Matters per year. AI Matters fea- learning.acm.org/webinar/. tures articles of general interest to the AI community, from research overview articles to dis- Additional ACM SIGAI membership benefits sertation abstracts. The editors-in-chief insti- include reduced registration fees at many of its tuted a number of reforms over the last year. co-sponsored and in-cooperation conferences For example, they started a number of recur- and access to the proceedings of many of ring columns in an effort to focus on the needs these conferences in the ACM Digital Library. and interests of individual populations of the membership and promote content creation for Planning for the Future each issue. These columns are led by individual column editors, who are responsible for ACM SIGAI is working on increasing the com- soliciting content or writing the column each munication with its current membership (for quarter. These columns have included example, via social media and membership

10 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 surveys) and intensifying its activities that reach students (for example, via the student essay contest) and industry professionals (for example, via the webinars). Additional activities in this directions are currently being planned. ACM SIGAI also intends to continue expanding the number of co-sponsored and in-cooperation conferences and its efforts to influence public policy and further the discussion on the responsible use of AI technologies. Finally, it intends to reach out to more AI groups worldwide that could benefit from ACM support, such as providing financial support, making the proceedings widely accessible in the ACM Digital Library and providing speakers via the ACM Distinguished Speakers program.

To help further the ACM SIGAI’s activities, you should consider becoming a SIGAI member!

For details, see http://sigai.acm.org/. The SIGAI mailing list is open to all.

11 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

AI Proﬁles: An Interview with Maja Mataric´ Amy McGovern (University of Oklahoma; [email protected]) Eric Eaton (University of Pennsylvania; [email protected]) DOI: 10.1145/3137574.3137578

Abstract Getting to Know Maja Mataric´ How did you become interested in This column is the fourth in our series proﬁling robotics and AI? senior AI researchers. This month we interview Maja Mataric.´ When I moved to the US in my teens, my un- cle wisely advised me that “computers are the future” and that I should study computer sci- Introduction ence. But I was always interested in human behavior. So AI was the natural combination Our fourth proﬁle for the interview series is of the two, but I really wanted to see behav- Maja Mataric,´ Vice Dean for Research and the ior in the real world, and that is what robotics Director of the Robotics and Autonomous Sys- is about. Now that is especially interesting as tems Center at the University of Southern Cal- we can study the interaction between people ifornia. and robots, my area of research focus.

Do you have any suggestions for people interested in doing outreach to K-12 students or the general public? Getting involved with K-12 students in incredi- bly rewarding! I do a huge amount of K-12 outreach, including students, teachers, and families. I find the best way to do so is by including my PhD students and undergraduates, who are naturally more relatable to the K-12 students: I always have them say what “grade” Figure 1: Maja Mataric´ they are in and how much more fun “school” is once they get to do research. The other key parts to outreach include letting the audience Biography do more than observe: the audience should get involved, touch, and ask questions. And Maja Mataric´ is professor and Chan Soon- finally, the audience should get to take some- Shiong chair in Computer Science Depart- thing home, such as concrete links to more in- ment, Neuroscience Program, and the De- formation and accessible and affordable activ- partment of Pediatrics at the University of ities so the outreach experience is not just a Southern California, founding director of one-off. Above all, I think it’s critical to convey the USC Robotics and Autonomous Sys- that STEM is changing on almost a daily ba- tems Center (RASC), co-director of the USC sis, that everyone can do it, and that whoever Robotics Research Lab and Vice Dean for gets into it can shape its future and with it, the Research in the USC Viterbi School of Engi- future of society. neering. She received her PhD in Computer Science and Artificial Intelligence from MIT in How do you think robotics or AI 1994, MS in Computer Science from MIT in researchers in academia should best 1990, and BS in Computer Science from the connect to industry? University of Kansas in 1987. Recently connections to industry have be- Copyright c 2017 by the author(s). come especially pressing in robotics, which

12 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 has gone, during my career so far, from being needs, from those with autism to those with a small area of specialization to being a mas- Alzheimer’s, to reach their potential. Seeing sive and booming area of employment oppor- that ﬁeld become established and grow from tunity and huge technology leaps. This means the enthusiasm of wonderful students and undergraduate and graduate students need to young researchers is an unparalleled source be trained in latest and most relevant skills of professional satisfaction. and methods, and all students need to be inspired and empowered to pursue skills and ca- reers in these areas, not just those who self- What do you wish you had known as a select as their most obvious path; we have to Ph.D. student or early researcher? proactively work on diversity and inclusion as Nobody, no matter how senior or famous, these are clearly articulated needs by indus- knows how things are going to work out and try. There are great models of companies that how much another person can achieve. So have strong outreach to researchers, such as when receiving advice, believe encourage- Microsoft and Google to name two, both hold- ment and utterly ignore discouragement. I am ing annual faculty research summits and hav- fortunate to be very stubborn by nature, but ing grant opportunities for faculty to connect it was still a hard lesson and I see too many with their research and business units. As in young people taking advice too seriously; it’s all contexts, it is best to develop personal rela- good to get advice but take it with a grain of tionships with contacts at relevant companies, salt: keep pushing for what you enjoy and as they tend to lead to most meaningful col- believe in, even if it makes some waves and laborations. raises some eyebrows.

What was your most difficult professional decision and why? What would you have chosen as your career if you hadn’t gone into robotics? It’s hard to pick one, but here are, briefly, three that are interesting: 1) I had to actively choose I think about that when I talk to K-12 students; whether to speak up against unfair treatment I try to tell them that it is fine to have a mean- when I was still pre-tenure and in a very under- dering path. I finally understand that what re- repreresented group, or to stay silent and not ally fascinates me is people and what makes make waves. I spoke up and never regret- us tick. I could have studied that from various ted being true to myself. 2) I had to choose perspectives, including medicine, psychology, whether to take part of my time away from neuroscience, anthropology, economics, his- research to get involved and stay involved in tory... but since I was advised (by my un- academic administration. I chose to do so, but cle, see above) to go into computer science, also chose to never let it take more than the I found a way to connect those paths. It’s al- official half time, and never stomp on my re- most arbitrary but it turned out to be lucky, as search. 3) I had to choose whether to leave I love what I do. academia for a startup or industry. These days, that is an increasingly complex choice, What is a “typical” day like for you? but as long as academia allows us to explore and experiment, it will remain the best choice. I have no typical day, they are all crazy in enjoyable ways. I prefer to spend my time What professional achievement are you in face-to-face interactions with people, and most proud of? there are so many to collaborate with, from PhD students and undergraduate students, The successes of my students and of my re- to research colleagues, to dean’s office col- search field. Seeing my PhD students re- leagues, to neighbors on my floor and around ceive presidential awards while having bal- my lab, to K-12 students we host. It’s all about anced lives with families and still responding people. And sure, there is a lot of on-line work, to my emails just makes me beam with pride. too, too much of it given how much less satis- Pioneering a field, socially assistive robotics, fying it is compared to human-human interac- that focuses on helping users with special tions, but we have to read, review, evaluate,

13 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 recommend, rank, approve, certify, link, pur- chase, pay, etc.

What is the most interesting project you are currently involved with? Since I got involved with socially assistive robotics, I truly love all my research projects: we are working with children with autism, with reducing pain in hospital patients, and addressing anxiety, loneliness and isolation in the elderly. I share with my students the cu- riosity to try new things and enjoy the opportunity to do so collaborative and often in a very interdisciplinary way, so there is never a short- age of new things to discover, learn, and over- come, and, hopefully, to do some good.

How do you balance being involved in so many different aspects of the robotics and AI communities? With daily difﬁcult choices: it’s an hourly struggle to focus on what is most important, set the rest aside, and then get back to enough of it but not all of it and, above all, to know what is in what category. I ﬁnd that my family provides an anchoring balance that helps greatly with prioritizing.

What is your favorite CS or AI-related movie or book and why? Wall*E: it’s a wonderfully human (vulnerable, caring, empathetic, idealistic) portrayal of a robot, one that has all the best of our quali- ties and none of the worst. After that, Robot and Frank and Big Hero 6.

Help us determine who should be in the AI Mat- ters spotlight!

If you have suggestions for who we should pro- ﬁle next, please feel free to contact us via email at [email protected].

14 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

AI Buzzwords Explained: Multi-Agent Path Finding (MAPF) Hang Ma (University of Southern California; [email protected]) Sven Koenig (University of Southern California; [email protected]) DOI: 10.1145/3137574.3137579

Kiva Systems was founded in 2003 to develop to minimize the makespan, that is, the num- robot technology that automates the fetching ber of time steps until all robots are at their of goods in order-fulfillment centers. It was goal cells. During each time step, each robot acquired by Amazon in 2012 and changed its can move from its current cell to its current name to Amazon Robotics in 2014. Ama- cell (that is, wait in its current cell) or to an zon order-fulfillment centers have inventory unblocked neighboring cell in one of the four stations on the perimeter of the warehouse main compass directions. Robots are not al- and storage locations in its center, see Fig- lowed to collide. Two robots collide if and only ure1. Each storage location can store one if, during the same time step, they both move inventory pod. Each inventory pod holds one to the same cell or both move to the current or more kinds of goods. A large number of cell of the other robot. Figure2 shows an ex- warehouse robots operate autonomously in ample, where the red and blue robots have to the warehouse. Each warehouse robot is able move to the red and blue goal cells, respec- to pick up, carry and put down one inventory tively. pod at a time. The warehouse robots move inventory pods from their storage locations There are also versions of the multi-agent to the inventory stations where the needed path-finding problem with different optimiza- goods are removed from the inventory pods tion objectives than makespan (such as the (to be boxed and eventually shipped to cus- sum of the time steps of each robot until it tomers) and then back to the same or different is at its goal cell) or slightly different collision empty storage locations (Wurman, D’Andrea, or movement rules. For example, solving the & Mountz, 2008).1 eight-puzzle (a toy with eight square tiles in a three by three frame, see Figure3) is a version These order-fulfillment centers raise a num- of the multi-agent path-finding problem where ber of interesting optimization problems, such the tiles are the robots. as which paths the robots should take and at which storage locations inventory pods should Researchers in theoretical computer science, be stored. Path planning, for example, is tricky artificial intelligence and robotics have stud- since most warehouse space is used for stor- ied multi-agent path finding under slightly dif- age locations, resulting in narrow corridors ferent names. They have developed fast where robots that carry inventory pods cannot (polynomial-time) algorithms that find solu- pass each other. Warehouse robots operate tions for different classes of multi-agent path- all day long but a simplified one-shot version finding instances (for example, those with at of the path-planning problem is the multi-agent least two unblocked cells not occupied by path-finding (MAPF) problem, which can be robots) although not necessarily with good described as follows: On math paper, some makespans. They have also characterized cells are blocked. The blocked cells and the the complexity of finding optimal (or bounded- current cells of n robots are known. A dif- suboptimal) solutions and developed algo- ferent unblocked cell is assigned to each of rithms that find them. A bounded-suboptimal the n robots as its goal cell. The problem is solution is one whose makespan is at most a to move the robots from their current cells to given percentage larger than optimal. their goal cells in discrete time steps and let Interestingly, it is slow (NP-hard) to find opti- them wait there. The optimization objective is mal solutions (Yu & LaValle, 2013c; Ma, Tovey, Copyright c 2017 by the author(s). Sharon, Kumar, & Koenig, 2016), although 1See the following YouTube video: a slight modification of the multi-agent path- https://www.youtube.com/watch?v= finding problem can be solved in polynomial 6KRjuuEVEZs time with flow algorithms, namely where n un-

15 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Figure 1: Warehouse robot (left), inventory pods (center), and the layout of a small simulated warehouse (right). The left and center photos are courtesy of Amazon Robotics.

1 2 3 4

A Robot Goal Cell

B time step 0 time step 1 time step 2 time step 3

Figure 2: A multi-agent path-ﬁnding instance with two robots.

not, then ... 4 1 2 8 7 • there are multi-agent path-finding algo- 6 3 5 rithms that group all colliding robots together and find a solution for the group with minimal makespan (by ignoring the other Figure 3: The eight-puzzle. robots), and then repeat the process. The hope is to find a solution before all robots have been grouped together into one big blocked cells are given as goal cells but it is group (Standley, 2010; Standley & Korf, up to the algorithm to assign a different goal 2011). cell to each one of the n robots (Yu & LaValle, 2013a). Researchers have also studied ver- • there are other multi-agent path-finding al- sions of the multi-agent path-finding problem gorithms that pick a collision between two where goal cells require robots with certain ca- robots (for example, robots A and B both pabilities (Ma & Koenig, 2016) or robots can move to cell x at time step t) and then exchange their payloads (Ma et al., 2016). consider recursively two cases, namely one where robot A is not allowed to move to cell In principle, one can model the original multi- x at time step t and one where robot B is not agent path-finding problem as a shortest-path allowed to move to cell x at time step t. The problem on a graph whose vertices corre- hope is to find a solution before all possi- spond to tuples of cells, namely one for each ble constraints have been imposed (Sharon, robot, as shown in Figure4 (where the red Stern, Felner, & Sturtevant, 2015). path shows the optimal solution), but the number of vertices can be exponential in the num- These state-of-the-art multi-agent path-finding ber of robots and the shortest path thus can- algorithms are currently not quite able to not be found quickly. Instead, researchers find bounded-suboptimal solutions for 100 have suggested to plan a shortest path for robots in small warehouses in real-time. The each robot independently (by ignoring the tighter the space, the longer the runtime. other robots), which can be done fast. If all Researchers have also suggested a vari- robots can follow their paths without colliding, ety of other multi-agent path-finding tech- then an optimal solution has been found. If niques (Silver, 2005; Sturtevant & Buro, 2006;

16 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Start Vertex =B1 =B2

=B1 =A2 =B2 =A2 =B1 =B3 =B2 =B3

=B3 =A2 =B3 =B2 . . .

Goal Vertex =B4 =A2 =B4 =B2 =B4 =B3

Figure 4: Partial graph for the multi-agent path-ﬁnding instance from Figure2.

Ryan, 2008; Wang & Botea, 2008, 2011; Luna A. (2014). Suboptimal variants of the & Bekris, 2011; Sharon, Stern, Goldenberg, conflict-based search algorithm for the & Felner, 2013; de Wilde, ter Mors, & Wit- multi-agent pathfinding problem. In An- teveen, 2013; Barer, Sharon, Stern, & Fel- nual symposium on combinatorial search ner, 2014; Goldenberg et al., 2014; Wagner (pp. 19–27). & Choset, 2015; Boyarski et al., 2015; Ma & Boyarski, E., Felner, A., Stern, R., Sharon, Koenig, 2016; Cohen et al., 2016), including G., Tolpin, D., Betzalel, O., & Shimony, some that transform the problem into a differ- S. (2015). ICBS: Improved conflict- ent problem for which good solvers exist, such based search algorithm for multi-agent as satisfiability (Surynek, 2015), integer linear pathfinding. In International joint confer- programming (Yu & LaValle, 2013b) and an- ence on artificial intelligence (pp. 740– swer set programming (Erdem, Kisa, Oztok, 746). & Schueller, 2013). Researchers have also Cirillo, M., Pecora, F., Andreasson, H., Uras, studied how to execute the resulting solutions T., & Koenig, S. (2014). Integrated mo- on actual robots (Cirillo, Pecora, Andreasson, tion planning and coordination for indus- Uras, & Koenig, 2014; Hoenig et al., 2016). trial vehicles. In International conference Two workshops have recently been held on on automated planning and scheduling the topic, namely the AAAI 2012 Workshop on (pp. 463–471). Multi-Agent Pathfinding2 and the IJCAI 2016 Cohen, L., Uras, T., Kumar, T. K. S., Xu, H., Workshop on Multi-Agent Path Finding3. Re- Ayanian, N., & Koenig, S. (2016). Im- cent dissertations include (Wang, 2012; Wag- proved solvers for bounded-suboptimal ner, 2015; Sharon, 2016). multi-agent path finding. In International joint conference on artificial intelligence (pp. 3067–3074). References de Wilde, B., ter Mors, A., & Witteveen, C. Barer, M., Sharon, G., Stern, R., & Felner, (2013). Push and rotate: Cooperative multi-agent path planning. In Interna- 2See the following URL: http://movingai.com/mapf tional conference on autonomous agents 3See the following URL: and multi-agent systems (pp. 87–94). http://www.andrew.cmu.edu/user/ Erdem, E., Kisa, D., Oztok, U., & Schueller, gswagner/workshop/ijcai 2016 P. (2013). A general formal frame- multirobot path finding.html work for pathfinding problems with mul-

17 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

tiple agents. In AAAI conference on arti- rithms for cooperative pathfinding prob- ficial intelligence (pp. 290–296). lems. In International joint conference on Goldenberg, M., Felner, A., Stern, R., Sharon, artificial intelligence (pp. 668–673). G., Sturtevant, N., Holte, R., & Schaef- Sturtevant, N., & Buro, M. (2006). Improv- fer, J. (2014). Enhanced partial expan- ing collaborative pathfinding using map sion A*. Journal of Artificial Intelligence abstraction. In Artificial intelligence and Research, 50, 141–187. interactive digital entertainment (pp. 80– Hoenig, W., Kumar, S., Cohen, L., Ma, H., Xu, 85). H., Ayanian, N., & Koenig, S. (2016). Surynek, P. (2015). Reduced time- Multi-agent path finding with kinematic expansion graphs and goal decompo- constraints. In International conference sition for solving cooperative path find- on automated planning and scheduling ing sub-optimally. In International joint (pp. 477–485). conference on artificial intelligence (pp. Luna, R., & Bekris, K. (2011). Push and swap: 1916–1922). Fast cooperative path-finding with com- Wagner, G. (2015). Subdimensional ex- pleteness guarantees. In International pansion: A framework for computa- joint conference on artificial intelligence tionally tractable multirobot path plan- (pp. 294–300). ning (Unpublished doctoral dissertation). Ma, H., & Koenig, S. (2016). Optimal target Carnegie Mellon University. assignment and path finding for teams of Wagner, G., & Choset, H. (2015). Subdimen- agents. In International conference on sional expansion for multirobot path plan- autonomous agents and multiagent sys- ning. Artificial Intelligence, 219, 1–24. tems (pp. 1144–1152). Wang, K. (2012). Scalable cooperative Ma, H., Tovey, C., Sharon, G., Kumar, multi-agent pathfinding with tractability T. K. S., & Koenig, S. (2016). Multi- and completeness guarantees (Unpub- agent path finding with payload transfers lished doctoral dissertation). Australian and the package-exchange robot-routing National University. problem. In AAAI conference on artificial Wang, K., & Botea, A. (2008). Fast and intelligence (pp. 3166–3173). memory-efficient multi-agent pathfind- Ryan, M. (2008). Exploiting subgraph struc- ing. In International conference on au- ture in multi-robot path planning. Jour- tomated planning and scheduling (pp. nal of Artificial Intelligence Research, 31, 380–387). 497–542. Wang, K., & Botea, A. (2011). MAPP: a scal- Sharon, G. (2016). Novel search tech- able multi-agent path planning algorithm niques for path finding in complex en- with tractability and completeness guar- vironment (Unpublished doctoral disser- antees. Journal of Artificial Intelligence tation). Ben-Gurion University of the Research, 42, 55–90. Negev. Wurman, P., D’Andrea, R., & Mountz, M. Sharon, G., Stern, R., Felner, A., & Sturtevant, (2008). Coordinating hundreds of co- N. (2015). Conflict-based search for op- operative, autonomous vehicles in ware- timal multi-agent pathfinding. Artificial In- houses. AI Magazine, 29(1), 9–20. telligence, 219, 40–66. Yu, J., & LaValle, S. (2013a). Multi-agent Sharon, G., Stern, R., Goldenberg, M., & Fel- path planning and network flow. In ner, A. (2013). The increasing cost tree E. Frazzoli, T. Lozano-Perez, N. Roy, search for optimal multi-agent pathfind- & D. Rus (Eds.), Algorithmic founda- ing. Artificial Intelligence, 195, 470-495. tions of robotics x, springer tracts in ad- Silver, D. (2005). Cooperative pathfinding. In vanced robotics (Vol. 86, pp. 157–173). Artificial intelligence and interactive digi- Springer. tal entertainment (pp. 117–122). Yu, J., & LaValle, S. (2013b). Planning optimal Standley, T. (2010). Finding optimal solutions paths for multiple robots on graphs. In to cooperative pathfinding problems. In Ieee international conference on robotics AAAI conference on artificial intelligence and automation (pp. 3612–3617). (pp. 173–178). Yu, J., & LaValle, S. (2013c). Structure and Standley, T., & Korf, R. (2011). Complete algo- intractability of optimal multi-robot path

18 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

planning on graphs. In AAAI conference on artiﬁcial intelligence (pp. 1444–1449).

Hang Ma is a Ph.D. student in computer science at the University of Southern California. He is the recipient of a USC Annenberg Graduate Fellowship. His research interests include artiﬁcial intelligence, machine learning and robotics. Additional information about Hang can be found on his webpages: http://www-scf.usc.edu/˜hangma. Contact him at [email protected]. Sven Koenig is a professor in computer science at the University of South- ern California. Most of his research centers around techniques for decision making (planning and learning) that enable single situated agents (such as robots) and teams of agents to act intelligently in their environments and exhibit goal-directed behavior in real-time. Additional information about Sven can be found on his webpages: http://idm-lab.org. Contact him at [email protected].

19 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

AI Education: Deep Neural Network Learning Resources Todd W. Neller (Gettysburg College; [email protected]) DOI: 10.1145/3137574.3137580

Introduction Web Resources

In this column, we focus on resources for One of the best news feeds for following Deep learning and teaching deep neural network Learning research developments and learning learning. Many exciting advances have been resources is Waikit Lau and Arthur Chan’s Ar- made in this area of late, and so many re- tificial Intelligence and Deep Learning (AIDL) sources have become available online that the Facebook group. At the time of this writing, flood of relevant concepts and techniques can it has over 30,000 members and features an be overwhelming. Here, we hope to provide active and steady flow of research results, tu- a sampling of high-quality resources to guide torials, announcements, and Q&A discussion the newcomer into this booming field. relevant to deep learning. Recommendations much like these can be found in questions 2-4 of the AIDL FAQ. Other good sites for suggested starting points Textbooks and Papers for learning about DL is A Guide to Deep Learning by YerevaNN Labs, Piotr Migdał’s Deep Learning (Goodfellow, Bengio, & Learning Deep Learning with Keras, a16z Courville, 2016) is a popular recent textbook team’s reference links, Stanford’s CS 231n that seeks to briefly introduce background Convolutional Networks course website, and, mathematical topics (e.g. linear algebra, of course, various Wikipedia pages concern- probability and information theory, numerical ing artificial neural networks. computation) as well as machine learning ba- sics. The second part of Deep Learning treats core material of deep learning practice (e.g. MOOCs deep feedforward networks, regularization, convolutional networks, recurrent networks, In April 2017, David Venturi collected an im- etc.), whereas the third part covers topics of pressive list of Deep Learning online courses modern research interest in deep learning. along with ratings data. In August 2016, Arthur Chan listed his top 5 lists. Concurring This textbook is available in HTML form on the with these bloggers, we found Geoffrey Hin- authors’ Deep Learning Book website and it is ton’s Neural Networks for Machine Learning not difficult to find other e-book formats online course lectures to be a good high-level intro- that have been built from these HTML pages. duction to the field. However, this course is However, this text alone is not the easiest in- not oriented towards the beginner. troduction to the field. We would recommend Andrew Ng’s Machine Learning MOOC, An In- We recommend taking Andrew Ng’s Machine troduction to Statistical Learning with Applica- Learning MOOC for background coverage tions in R (James, Witten, Hastie, & Tibshirani, and then supplementing Hinton’s MOOC with 2014), and other resources listed in this AI Ed- applied tutorial exercises found elsewhere.1 ucation Matters column in Volume 3, Number Kaggle is a data science competition web- 2 as starting points for background material site that features tutorials, datasets, and chal- relevant to all machine learning. lenges that offer practical experiential learning opportunities. Beyond Hinton’s general intro- Also recommended as a gentler introduction is duction, Arthur Chan also recommends Hugo Michael Nielson’s Neural Networks and Deep Larochelle’s graduate-level online Neural Net- Learning online book. work course.

1At time of writing, Andrew Ng has announced Copyright c 2017 by the author(s). a new Coursera specialization in deep learning.

20 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Software ter with our wiki and add them to our col- lection at http://cs.gettysburg.edu/ There are several popular software frame- ai-matters/index.php/Resources. works that facilitate rapid prototyping of deep learning systems. Most popular is Google’s TensorFlow. An even higher-level layer that References has become popular is Keras, which can run as a layer on top of TensorFlow, Microsoft Goodfellow, I., Bengio, Y., & Courville, A. Cognitive Toolkit (CNTK), or Theano. To grasp (2016). Deep learning. MIT Press. how high-level Keras is, consider these small (http://www.deeplearningbook MNIST digit recognition training examples fea- .org) turing multi-class logistic regression, single- James, G., Witten, D., Hastie, T., & Tibshi- hidden-layer neural network training, and con- rani, R. (2014). An introduction to sta- volutional network training implemented in un- tistical learning: With applications in r. der ten lines each. Springer Publishing Company, Incorpo- rated. (http://www-bcf.usc.edu/ Other popular software for deep learning in- ˜gareth/ISL/) cludes Torch (and PyTorch), Caffe, MXNet, and DeepLearning4J. While Python appears to be the most popular language for deep learning development, support exists for other Todd W. Neller is a Pro- programming languages. fessor of Computer Sci- ence at Gettysburg Col- lege. A game enthu- Hardware siast, Neller researches Researchers seem to obtain hardware with game AI techniques and GPUs that support fast deep learning in three their uses in undergradu- main ways. One expensive route is to buy ate education. machines marketed directly for deep learning that typically have high-end GPU speci- ﬁcations and come preinstalled with popular deep learning software. However, there are numerous DIY tutorials for buying the necessary parts and putting together an inexpen- sive deep learning machine. These options are at the extremes of the high-cost/low-effort and low-cost/high-effort spectrum. We would recommend a middle-ground approach for time-strapped faculty with tight budgets: Buy a high-end gaming machine (e.g. Dell’s Alienware desktops) with good GPUs and install the necessary software as needed. Tim Dettmers has shared results of a recent GPU comparison study, and Ved’s d4datascience blog entry describes the process of installing CUDA libraries and Tensor- Flow in detail.

Your Favorite Resources? These are but a few good starting points for learning about deep neural network learning. If there are other resources you would recommend, we invite you to regis-

21 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Celebrating the Past, Present, and Future of Computing Timothy E. Lee (Carnegie Mellon University; [email protected]) Justin Svegliato (University of Massachusetts Amherst; [email protected]) DOI: 10.1145/3137574.3137581

Abstract

Timothy Lee and Justin Svegliato, two Student SIGAI Scholars, cover The 50 Years of the ACM Turing Award Celebration, which con- vened in San Francisco last June. The semi- centennial celebration addressed the past, present, and future advancements of computing, ranging from deep learning and ethics to augmented reality and quantum computing.

As Student Scholars sponsored by SIGAI, we are grateful for the opportunity to be a part of The 50 Years of the ACM Turing Award Celebration. For two days in June, hundreds Figure 1: SIGAI at The 50 Years of the ACM Tur- of professors, researchers, and students from ing Award Celebration. Pictured from left to right: across the globe gathered together in San Timothy E. Lee, Yolanda Gil, and Justin Svegliato. Francisco to celebrate the legacy of the Tur- The bronze bust of Alan Turing unveiled during the ing Award (often referred to as the Nobel Prize conference is also shown here. of computing) and the incredible advances in computing over the last 50 years. The semi- centennial celebration also honored this year’s The first panel Advances in Deep Neural Net- Turing Award recipient, Tim Berners-Lee, for works was particularly relevant to the SIGAI inventing the World Wide Web and related net- community. Moderated by Judea Pearl, the working technologies such as the Semantic panel featured Michael Jordan (UC Berke- Web. ley), Fei-Fei Li (Stanford), Stuart Russell After opening remarks from a Turing laure- (UC Berkeley), Ilya Sutskever (OpenAI), and ate each day, we heard from several panels Raquel Urtasun (Toronto). As a popular area that spanned the field of computing, ranging not only in AI but also in computing in gen- from deep learning and ethics to augmented eral, deep learning has emerged as a powerful reality and quantum computing. Every panel approach for enabling machine intelligence. featured a distinguished moderator and sev- Sutskever explained that neural networks are eral panelists that included Turing laureates, essentially tunable circuits that learn high- prominent researchers, and rising stars in the dimensional mappings from data. Deep neu- field. Interleaved with the panels were sev- ral networks have emerged from the conflu- eral short films. These films featured the life ence of several factors: the recent advances in and work of the father of computer science, hardware (the “oxygen” of neural networks ac- Alan Turing, and highlighted the Turing lau- cording to Sutskever), the availability of mas- reates’ contributions, including those made to sive datasets via the Internet, and the accel- the field of artificial intelligence by the AI Tur- erated progress of data science. ing Award laureates. We were honored by the attendance of several AI Turing Award recipi- Despite their promising results, a common ents: Judea Pearl (2011 Turing laureate), Ed theme emerged from the panel. Although ef- Feigenbaum (1994 Turing laureate), and Raj fective in particular domains, deep learning in Reddy (1994 Turing laureate). its current form cannot be the fundamental abstraction of machine intelligence sought after Copyright c 2017 by the author(s). by researchers. There are many questions

22 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 about its long-term viability as the bedrock opening day of the celebration also featured of machine intelligence. As Jordan argued, four other panels with many prominent re- today’s neural networks are deep architec- searchers from industry and academia, along turally, but not semantically. Pearl questioned with a talk by 2008 Turing laureate Barbara whether these networks could reason about Liskov that explored the history of comput- causality, a central theme in his foundational ing. First, in Restoring Personal Privacy with- work on Bayesian networks. out Compromising National Security, Whitfield Diffie, 2015 Turing laureate, along with sev- Outlining the weaknesses of today’s deep eral leaders in security, cryptography, and net- neural networks segued into a discussion on working discussed how governments could the types of intelligent behavior that humans obtain useful information using backdoors and exhibit but these networks currently lack, such other intentional vulnerabilities to aid crimi- as semantic understanding, contextual rea- nal investigations without jeopardizing the pri- soning, abstraction, and reasoning under un- vacy of society. Following a short film on certainty, all of which are easily handled by hu- Alan Turing’s life, we then turned to Vint Cerf, mans despite little training data. Russell drew 2004 Turing laureate, and several other distin- a fitting analogy between Allen Newell, Cliff guished researchers in Preserving Our Past Shaw, and Herbert Simon’s General Problem for the Future. They considered the problem Solver and the need for exponential comput- of how to store data for centuries to come and ing with deep learning and the need for ex- whether corporations or governments should ponential data. In Russell’s opinion, hoping fund such an endeavor. to achieve “tabula rasa” machine intelligence with only deep learning may be infeasible in Later that day, Moore’s Law Is Really Dead: some—or all—domains due to the data de- What’s Next? headlined the 1992 Turing lau- mands, and so we must continue to search reate Butler Lampson. The panel explored for better techniques. Li offered a similar the ways in which the field can continue the anecdote from her work with ImageNet. With trend of exponential technological growth de- enough data, deep neural networks are by far spite that Moores Law has continued to slow the state of the art in object recognition, but down. During the panel, a common theme they perform poorly and cannot reason effec- emerged: researchers will eventually leverage tively without massive datasets. In the case special-purpose hardware and quantum com- of robotics, Urtasun noted that being unable puting to push the boundaries of computing to model uncertainty well in deep learning is a forward. considerable drawback in her current work on self-driving vehicles where algorithm robust- At the end of the day, we heard from Raj ness is critical. Reddy in Challenges in Ethics and Comput- ing. Given the increasing relevance of AI and Still, even with these shortcomings, deep machine learning, Reddy believes that ethi- learning performs quite impressively in nar- cal questions in computing have become more row problems, such as computer vision, image important than ever. Noel Sharkey added captioning, and object segmentation. In some several important questions in light of re- cases, such as AlphaGo, it enables decision- cent progress in self-driving cars and machine making capabilities that are superior to human learning. How can self-driving cars make de- intelligence. Several of the panelists agreed cisions that were once reserved for humans in that deep learning has matured enough to be life and death situations? And how do we en- used in industry, but the search for machine in- sure that data-driven algorithms escape bias telligence must continue. Ultimately, the panel against minorities in the justice system? could be best summarized by Li’s comments: we are entering the “end of the beginning” for On the final day of the conference, there were AI. Deep neural networks may be one of our two panels on some of the most rapidly grow- best existing tools for enabling the developing fields in computing following a talk from ment of intelligent agents, but even greater Donald Knuth, 1974 Turing laureate. Quan- breakthroughs are yet to come. tum Computing: Far Away? Around the Cor- ner? Or Maybe Both at the Same Time? that In addition to the deep learning panel, the featured the 2000 Turing laureate Andrew Yao

23 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 investigated the current state of quantum computing and how it might drive software devel- Timothy E. Lee is a M.S. opment in the next 50 years. Like the deep student in Robotics at learning panel, John Martinis cautioned that Carnegie Mellon Univer- quantum computing is only a powerful tool in sity. As a member of certain combinatorial problems but useless in the Robust Adaptive Sys- others. However, in areas like AI, machine tems Lab under Profes- learning, and cryptography, it has the poten- sor Nathan Michael, Tim- tial to revolutionize the field. othy’s research focuses The celebration culminated with a panel on an on improving mobile robot area of computing that has recently seen rapid intelligence to enable the progress: Augmented Reality: From Gaming automation of challenging to Cognitive Aids and Beyond. Fred Books, tasks in real-world set- 1999 Turing laureate, and Ivan Sutherland, tings. He is currently in- 1988 Turing laureate, reminisced about the vestigating robust, vision-based navigation of early work of augmented reality while they a submersible robot to automate the precision were aspiring researchers. Peter Lee dis- inspection of underwater infrastructure. cussed the impact of augmented reality on Justin Svegliato is a the gaming industry. Other panelists consid- second year Ph.D. stu- ered Google Glass, Pokemon Go, and Oculus dent in Computer Science Rift and explored the inevitable future of aug- at the University of Mas- mented reality in the home and at the work- sachusetts Amherst. In place. the Resource-Bounded For all attendees across the spectrum of com- Reasoning Lab under puting, the advancements of the last 50 years Professor Shlomo Zilber- honored during the conference will undoubt- stein, Justin’s research edly shape our own contributions for the next focuses on bounded ratio- 50 years to come. And, for the SIGAI com- nality, real-time decision munity, the experts in our field gave insight making, and autonomous into the ongoing search for machine intelli- agent architectures. He is currently develop- gence and how deep learning might play a ing metareasoning techniques that monitor role. Given the recent groundbreaking ad- and control algorithms that trade decision vances in AI, it was only fitting that the cele- quality with computation time. bration of computing’s greatest achievements was in honor of who many call the grandfather of AI, Alan Turing.

Acknowledgements We gratefully acknowledge the support of SIGAI for the opportunity to attend The 50 Years of the ACM Turing Award Celebration. We also thank Yolanda Gil for her help with this article.

24 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

ACM SIGAI CHINA:A New Incubator for AI in China Le Dong (University of Electronic Science and Technology of China; [email protected]) Man Yuan (University of Edinburgh; [email protected]) Ming-Liang Xu (Zhengzhou University; [email protected]) Ji Wan (Zhengzhou University; [email protected]) DOI: 10.1145/3137574.3137582

ACM SIGAI CHINA ACM TURC 2017 - SIGAI CHINA SYMPOSIUM The China Chapter of ACM SIGAI commit- tees have been preparing ACM SIGAI China The ACM Turing 50th Celebration Conference for a year. In March 2016, Le Dong and the - China (ACM TURC 2017), was held from ACM China council decided to establish a new May 12-14, 2017 in Shanghai, China with its chapter–ACM SIGAI China. key theme on “Trustworthy Network Big Data”. The conference served as a highly selective In May 2016, the issue was further discussed and premier international forum on computer in ACM China Council Meeting in Beijing. science research. The Chair of ACM, Vicki L. Hanson, CEO, Robert B. Schnabel, and COO, Pat Ryan gave their approval to the foundation of ACM SIGAI China. The Chair of ACM China Council, Yunhao Liu from Tsinghua University and the Chair of ACM SIGAI, Sven Koening from Uni- versity of California also gave their special support to the foundation of this new chapter in China. The issue was also reported to ACM and ACM China for permission. In Octo- ber 2016, the organization structure of ACM Figure 1: ACM TURC 2017 SIGAI China was discussed in ACM China Council Meeting which was held in Taiyuan, Alan Mathison Turing is the father of computer China. science, artificial intelligence, the founder of computer logic. He put forward the important In March 2017, an official email from ACM in- concept of “Turing machine” and the “Turing formed that ACM SIGAI China chapter was test”. In honor of this distinguished scientist, successfully registered and ACM SIGAI A.M. Turing Award is established according to China was officially founded. In this new his name in 1966 by the Association for Com- chapter, the Executive Chair & General puting Machinery (ACM). It devotes to reward Secretary is Le Dong from University of Elec- individuals who have made outstanding con- tronic Science and Technology of China. tributions to the computer industry. The Turing In this year, ACM SIGAI China symposium Award is recognized as the highest distinction aims to provide a world0s premier forum of in computer science and the Nobel Prize of renowned researchers to share their insightful computing. opinions and discuss cutting-edge research On the occasion of the 50th anniversary on the artificial intelligence. The symposium of the establishment of A.M. Turing Award, features in types of sessions including dis- ACM TURC 2017 was hosted by Shang- tinguished talks and panel discussion. This hai Jiao Tong University, the Third Research summit forum expects to promote the devel- Institute of the Ministry of Public Security, opment of artificial intelligence from the aca- and Shanghai Municipal People’s Govern- demic, technical to industry and applica- ment, co-organized by China Computer Fed- tions. eration, Tsinghua University, Peking Univer- sity, Huazhong University of Science and Copyright c 2017 by the author(s). Technology, and Tongji University.

25 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

In addition to the main sessions, the conference included workshops, panels, demon- strations, and exhibits. Twelve distinguished speakers including A.M. Turing Award Recipient were invited to discuss the frontier issues in today’s society as well as the interdisciplinary development trends together. ACM SIGAI is the Association for Computing Machinery’s Special Interest Group on Artifi- cial Intelligence. The Chair of ACM SIGAI is Sven Koening from University of California who is also AAAI Fellow and distinguished speaker of ACM. Figure 2: Le Dong & Kai-Fu Lee @ TURC 2017 This time, ACM SIGAI China invited many distinguished speakers to share their ideas with our members in the symposium. They Turing Award talked about Exciting ideas in were Yangsheng Xu (Chinese University of computer science. The ACM/IEEE Fellow, Hong Kong), Sven Koenig (University of Gao Wen also brought up his ideas about Southern California), Shipeng Li (CTO of Co- Evolution of the Artificial Visual System. gobuy Group and IngDan), Shiyi Chen (Fu- In the banquet, Haifeng Wang, Vice President dan University), Bill Huang (CloudMinds Inc.), of Baidu discussed AI Makes the Internet Gansha Wu (Uisee Technology Inc.), Gang Smarter. The Chairman and CEO of Sino- Pan (Zhejiang University), Shuicheng YAN vation Ventures, President of Sinovation Ven- (Chief Scientist of Qihoo/360, Director of 360 tures Artificial Intelligence Institute and Sci- AI Institute), Heng Tao SHEN (University of ence, Kai-Fu Lee also shared his ideas. Lee Electronic Science and Technology of China), reviewed the history of computer and empha- Chunyuan Liao (Hiscene), Kai Yu (Horizon sized the application of quantum computer. Robotics), Liang Lin (SenseTime Group Lim- He pointed out that today, people learn from ited and Sun Yat-Sen University), Enhong mass data. In the future, people are able to Chen (University of Science and Technology learn from experience and have the ability of China), Junping Du(Beijing University of of transfer learning. Posts and Telecommunications), Zenglin Xu (University of Electronic Science and Tech- From the discussion, we can know that with nology of China), Fei Wu (Zhejiang Univer- the progress of science, AI will be used sity), Rui Hou (Chinese Academy of Sci- in cross-disciplinary programs and natural ences), Zhongxing XUE (KingPoint), Jinglei language understanding which will eventually ZHAO (ReadSense), Xuyao Hao, and Pi- be platformed and become the most effective anpian He(TF Securities). Yidan XU (Founder tool which can be applied in deep learning, & CEO of Topplus), Linsen BAI (Founder of transfer learning, reinforcement learning, sta- Alpha Brick), Wenzhi LIU (Engineering Direc- tistical learning, and so on. AI will eventu- tor of SenseTime),and Yao Chen (Marketing ally step into and have a huge impact on the Manager of Horizon Robotics) also delivered physical world and also make remarkable ac- their speech in the Technical Review part. Our complishments. But we have to be aware of sponsors and enterprises also joined in the the limitations of artificial intelligence. It lacks symposium panel discussion: From AI to Ap- self-awareness, aesthetic, feelings or love. At plications, what it takes to REALLY get the same time, it demands mass data, sin- there? gle domain and top scientists. People must be aware of these obstacles and realize that In conference panel, Vinton Cerf, Alexander a new opportunity beckons in China. Human Wolf, Wen Gao, Kai-Fu Lee, Xiangyan Li and select AI as nature selects the fittest.There Andrew discussed Big Data or Brain Pow- are several approaches for scientists: start ered Artificial Intelligence: Turing or Quan- their own business, empower technology, find tum? John Hopcroft, Recipient of the ACM a partner and publish papers.

26 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Figure 4: Le Dong, The Executive Chair&General Secretary of ACM SIGAI CHINA Was Awarded by Vicki L.Hanson in The ACM Turing 50th Celebra- tion Conference - China

Figure 3: Alexander Wolf Left His Words in the Au- The Future of AI tograph Book @ TURC 2017 In May 4th, 2014, President Xi Jinping once pointed out that China is going to initiate world-class universities. In 2016, Report on the Work of the Gov- In the banquet, scientists, entrepreneurs and ernment highlighted a new higher educa- students exchanged their thoughts and ideas tion policy, aiming to promote the construc- freely and left their precious signatures and tion of “Double First-Rate”(developing First- some words in our autograph book. class Universities and Academic Programs) in China. The development of cutting-edge tech- The feature of Turing Conference is AI. In nology relies heavily on higher learning insti- ACM SIGAI China Committee, Executive tutions. Chair & General Secretary is Le Dong from In 2017, President Xi attended the Opening University of Electronic Science and Technol- Ceremony of Belt and Road Forum for Inter- ogy of China. Vice General Secretary are national Cooperation(BRF) and he pointed out Mingliang Xu from Zhengzhou University and that in order to pursue innovation-driven de- Yanli Ji from University of Electronic Science velopment, China should strengthen the coop- and Technology of China. Vice chair are Kun eration on the digital economy, artificial intel- Zhou from Zhejiang University, Kai Yu from ligence, nanotechnology, and quantum com- Horizon Robotics, Gansha Wu from Uisee puting, and advance the development of big Technology Inc., and Liang Lin from Sense- data, cloud computing and smart cities so as time Tech. Man Yuan and Ji Wan are also to turn them into a digital silk road of the 21st ACM SIGAI China secretary. In ACM TURC century. He also said that we should spur the 2017, other organizers, XueLong Li, Hanli full integration of science and technology into Wang, and Xiang Bai also made their contri- industries and finance, improve the environ- butions to this event. In particular, 2017 Out- ment for innovation and pool resources for in- standing Contribution Award and Excellent novation. Artificial Intelligence orients the de- ACM China Lecturer was presented to Le velopment of future technology. Dong by ACM Turing 50TH Celebration Con- ference - China for her extraordinary contribu- In July, 2017, the State Council notified the tions to ACM TURC 2017 by Vicki Hanson, plan of the development for artificial intel- the Chair of ACM. ligence in the following decades in China.

27 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

In such context, ACM SIGAI China is founded Find your opportunities at with three major missions: Our website: First, ACM SIGAI China provides an incuba- http://china.acm.org/TURC/2017/SIGAI.html tor for this emerging industry. It connects AI and education, also with other ﬁelds. Aca- WeChat Ofﬁcial Account QR Code: demic faculty, student, enterprises are very welcomed to join ACM SIGAI China to promote the interaction and impact of disciplinar- ies in AI.

Second, we encourage original AI innovation and international cooperation within communities via various activities and events, through different branches in ACM and SIGAI, especially in China.

Third, we strongly advocate the variability of SIGAI CHINA and especially welcome students and females to become members of Le Dong received her ACM SIGAI China. PhD degree in Elec- tronic Engineering and We shall strengthen the context from senior Computer Science from researchers to junior students, as well as link Queen Mary, University of partners among different regional areas. The London in 2009. She is a new era of artiﬁcial intelligence needs every full professor in University one of you! Learn and exchange the sparks of of Electronic Science thoughts here! and Technology of China. She is the coordinator of More information about conference, recent several National Natural events, registration, business cooperation and Science Foundation Projects such as NSFC further details can be found on our website Surface Project, NSFC Youth Project, NSFC and WeChat Ofﬁcial Account. Your participa- Important Research Project and so on. She tion is warmly welcomed! has now published more than 40 papers in international journals and conferences including several top journals and high level International Conference, such as TIP, ACM MM, TMM, PR, TCSVT, ICPR. She has served as a reviewer for several top journals and conferences. Now she is the Executive Chair & General Secretary of ACM SIGAI China , General Secretary of Vision And Learning Seminar (VALSE), General Chair of ACM TURC 2017 SIGAI CHINA Symposium, General Secretary of National Program of Global Experts, Youth Talents Committee, and the Executive Secretary of next-generation of Sichuan Provincial Engineering Laboratory. She got the Best Talk Award in the 10th ACM International Workshop on IOT and Cloud Computing, Excellent ACM CHINA Lecturer and ACM 2017 Outstanding Contri- Figure 5: AI Helps Innovation bution Award in ACM Turing 50th Celebration Conference - China.

28 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Man Yuan is currently Ji Wan is currently a a postgraduate student postgraduate student studying Teaching En- majoring in Computer glish to Speakers of Other science and technology Languages in Moray in Zhengzhou University. House School of Edu- He received his Bachelor cation in the College of of Engineering degree Humanities and Social in Computer science Science at the University and technology from of Edinburgh, Scotland, Zhengzhou University, United Kingdom. She China in 2015. He is the received her Bachelor secretary of ACM SIGAI of Arts degree in English from Shanghai China. His research interests include mobile International Studies University, and Bachelor and spatial data management, location-based of Science in Management Accounting as her services , and smart city computing. minor specialty from Shanghai University of International Business and Economics, China in 2017. She has served as secretary of ACM SIGAI China, and translator in Huaqiao Foun- dation. Her research interests include English for speciﬁc purposes, teaching English to engineering and science students, language testing, and second language acquisition.

Ming-Liang Xu is a full professor in the School of Information Engineer- ing of Zhengzhou Uni- versity, China, and currently is the director of CI- ISR (Center for Interdis- ciplinary Information Sci- ence Research) and the Vice General Secretary of ACM SIGAI China. His research interests include computer graphics and artiﬁcial intelligence. He has now published more than 40 papers in international journals and conferences including ACM TOG, IEEE TPAMI, IEEE TIP, IEEE TCSVT. Xu got his Ph.D. degree in computer science and technology from the State Key Lab of CAD&CG at Zhejiang University.

29 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

On the Importance of Monitoring and Directing Progress in AI Lukas Prediger (RWTH Aachen University; [email protected]) DOI: 10.1145/3137574.3137583

Abstract AI will consequently empower researchers further and likely help us solve problems that we This essay argues that the speed with which struggle with today. More capable narrow AI AI development proceeds will be an impor- can also reduce labor costs in manufacturing tant factor for a beneficial adoption and thus or services, yielding higher profit margins for should be closely monitored and controlled, companies and reduced consumer prices. if necessary. It also discusses privacy and manipulation, inequality of access and (value As a result, these AI applications receive learning) superintelligence as major issues for tremendous interest in current research and all development trajectories. can be expected to be further refined in the near future since they promise a very direct and obvious value to the companies applying Introduction them. Recent years have seen steady progress in Other than AlphaGo and Libratus, these ap- the development and application of artificial plications of AI already have a direct impact intelligence, mainly in the form of machine on society. For example, our notion of privacy learning artifacts. The most prominent pub- is based upon the intuition that having more lic achievements were DeepMind’s AlphaGo information about a person gives a greater mastering the Go board game last year ability to predict and manipulate this person’s (Hassabis, 2016) and, even more recently, behavior in subtle or even open ways and Libratus, a program developed at Carnegie thus access to this information should be re- Mellon University, winning a match of Poker stricted. Deriving information from seemingly against professional players for the first time innocouos data thus naturally raises concerns in history (Condliffe, 2017). about the privacy of the data subjects. As companies gain more and more data on their While these events mark important points in customers, they can more accurately predict AI development, they represent only a small preferences and behaviors, not only to offer part of the ”state of the art”. The more im- more specifically tailored products and ser- portant, but not as spectacular, development vices but also to influence customer decisions is the ever increasing number of narrow AI in ways that these might not even recognize. programs, especially the powerful combination of data mining and machine learning ap- One example for how such a manipulation plications that help in recognizing patterns in might take place is the recent discussion on all kinds of huge data bases, thus e.g. allow- the extent to which content selection proce- ing companies to predict customer behavior. dures in social networks might have skewed This greater ability to make predictions based the public debate during the recent election in on data is a major advantage and suggests the United States. that AI can yield massive benefits to whoever controls it as well as society as a whole. As an Acknowledging this, it is clear that like any example, current research in almost all fields other technology AI opens up possibilities for of science - especially natural and engineering benefits and harm alike and it is the respon- sciences - relies on the ability to process data sibility of those who develop and employ it to and was thus empowered and accelerated by take measures such that the benefits outweigh computing machines. A greater ability to pro- the possible harm. In parallel to the techni- cess data and (automatically) derive knowl- cal development, recent years have also seen edge in the form of more accurate models and a rising awareness about the ethical implica- predictions as enabled by more sophisticated tions of AI, including (but not limited to) the possibility of massive job loss, privacy degra- Copyright c 2017 by the author(s). dation, autonomous weapons, increases so-

30 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 cial inequity and more (cf. (Steinhardt, 2015; are replaced by automation, pushing the work- Brundage, 2015; Open Philantrophy Project, force into sectors that cannot yet be auto- 2015)). There is also concern about pos- mated. Not only does this change the job sible superintelligent agents and a resulting landscape but also the importance of certain existential threats to humanity (cf. (Bostrom, skills and, by extent, the prestige of a person 2014)). The consensus is that AI will have that holds them. Often, the values prevalent a large - maybe unprecedented - transform- in a society change as behaviors enabled by ing and probably irreversible impact on hu- new technology, while first usually regarded as manity. However, there is much uncertainty strange and being rejected by the larger part over how exactly this transformation will take of the population, become normal. However, shape, mostly because there is much uncer- as mentioned above, these processes usually tainty about the future progress in AI regard- take several years at least. ing both the levels of capability that can be achieved and how long it will take to reach One of the reasons for this is that, since soci- each level. This translates into some un- eties are complex systems, the consequences certainty about the urgency and extent to of a certain new technology or paradigm can- which each of these concerns needs to be ad- not be anticipated to a sufficiently accurate de- dressed. gree. Its implementation thus requires a monitoring during the process and adapting related I will argue in the first section that establish- regulatory measures in a reactive and itera- ing a proper framework of AI development will tive fashion as the impact becomes gradually be essential to streamline the discussion of more evident. This opens a window in which measures to keep AI beneficial. However, some aspects of new technologies might be there are also certain issues, namely its im- unregulated and are open to exploitation. pact on the economy and labor market as well as privacy and manipulation hazards, that Given these observations, the faster the dis- will almost certainly occur in the near future ruptions caused by more advanced AI appli- and need to be addressed immediately, as I cations proceed and, consequently, the less will point out in the following sections, con- time societies and regulatory bodies have to tinued by a short argument about the long- observe and adapt, the greater the probability term prospect of superintelligence. Note in ad- and magnitude of societal instability or disor- vance that most the arguments I will state are der is likely to be. Taking the labor market as not groundbreakingly new. I merely aim to em- an example, if jobs are automated in a grad- phasize their importance and add some minor ual fashion, there is more time to retrain and points. educate those workers that lose their previous jobs. Further, assuming a gradual transition, there are more jobs still available for each Monitoring and Predicting AI wave of replaced workers and longer time win- Development dows for new jobs to emerge in the newly shaped economy. If, in contrast, waves of au- Motivation tomation follow each other with very brief inter- Much of the impact that AI will have depends mittent time periods, a large part of the work- on the speed of the development and applica- force might be suddenly unemployed without tion of new AI capabilities. Societies change having time to adapt during which the still em- constantly due to new circumstances, tech- ployed support and smoothen the transition nologies or ideas but it usually takes several (cf. (Steinhardt, 2015)). Furthermore, while years, if not generations, for new paradigms to new job opportunities might open, they would become accepted in mainstream opinion. This probably also be swiftly automated without a is especially true if they do not benefit every- sufficiently large part of the population having one equally. a chance to acquire the required skills to pursue them. Disruptive technologies break up the prevalent composition of society and force it to While these examples only focus on the la- adapt to new circumstances. Often this is bor market, the same arguments can be con- due to a shift in employment because jobs strued for other aspects of society in similar

31 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 ways. crete years when a certain new capability (e.g. whole brain emulation) is achieved, mostly Recent trends seem to suggest that the time from the ongoing exponential growth of com- for adoption of new technology into soci- putational power that he expects, combined ety is shortening (cf. (McGrath, 2013)), sug- with an estimation of the required computa- gesting some kind of resilience of society to tional power to achieve these feats. He also getting outpaced by technological advance- points out that even if his estimates are off by ment. However, the exact dynamics of this some orders of magnitude, this would only de- and whether it will hold up with more dis- lay these technologies for a small number of ruptive changes than simple convenience de- years due to the exponential nature of advanc- vices such as smartphones or whether there ing technology. is some limit to the adoption speed (as suggested above) remain unclear. Research Critics pointed out that the exponential growth aimed in that direction, especially on varia- paradigm is not a reasonable assumption tions between different subgroups of society, since past development of AI capability does e.g. groups of age or ethnicity, might also re- not follow the increase in computational power veal relevant measures to ensure a more sta- in a linear fashion (cf. Myhrvold’s contribution ble and beneficial transition. to the Edge conversation (Brockman, 2014)). Recent findings however suggest that hard- Modeling Progress ware development has a large contribution to current AI performance (Brundage, 2016) Following from the above, it seems paramount and that AI development seems to be, in fact, to have as accurate knowledge as possible accelerating (Stone et al., 2016). It should about the speed with which AI development nevertheless be pointed out that some of the will probably progress and when to expect milestones that Kurzweil predicts have already which changes in capability of these systems. been missed, indicating that his estimations Having this knowledge will allow us to antici- are, at least, overly optimistic. There was also pate the most disruptive changes in advance criticism that there is no reason why events and smoothen their impact through prepara- should play out in the order that he estab- tory measures or, if necessary, delaying de- lishes. velopment or implementation just enough to allow for a more gradual transition. However, there is some merit in the approach We are still in the dark about which exact qual- of construing one possible scenario and eval- ities or properties make someone (or some- uating the impact is has, as pointed out by thing) ”intelligent” and thus the final complex- Goertzel(2007). Given the large uncertainty ity of AI cannot be properly estimated. So far, we are facing when predicting AI progress, it we cannot even reliably establish how many might be helpful to explore several scenarios scientific breakthroughs are approximately re- in detail and refine them over time, as the tra- quired or what the the ultimate final result of jectory of development we are really on be- AI research will be. Without even knowing the comes more clear. required steps, it is certainly impossible to pre- As an example, a more recent effort by Stan- dict when they will occur. Establishing a pre- ford University’s AI100 project attempts to cise model for future progress thus seems a to forecast how AI systems might be imple- be a hopeless endeavor. mented in a typical American city in 2030, However, there might be ways to get a reason- providing insight into medium-term develop- ably accurate understanding of the degree of ment and pointing out possible concerns that capability by observing past trends and pro- should be addressed, as well as research di- jecting them into the future. A prominent ex- rections to do so (Stone et al., 2016). Further- ample of this is Kurzweil’s book (Kurzweil, more, the concrete issue of economic impact 2005) in which he observes past trends in has already received significant interest and overall increasing complexity of biological and a wealth of literature exists (e.g. (Brynjolfsson then technical systems and uses them to for- & McAfee, 2014)). However, studies that try mulate a scenario of how future progress to estimate a more general impact of AI on might play out. From this he derives con- society and additionally explore different sce-

32 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 narios and directions of possible development It seems uncertain whether or not qualitative might yield significant additional and required superintelligence, i.e. AI that ’thinks’ in ways insight. superior to humans, is possible. However, any human-level AI could benefit from advance- Finally, any prediction into the future requires ments in hardware technology which will with us to have a reasonable understanding of the high probability enable it to gradually speed up current state of development. Unfortunately, the involved calculations and thus evolve into attempts to accurately assess the past de- a ’speed superintelligence’, i.e. an intelligence velopment are few and suffer from a vague- of similar capability as the human brain, but ness surrounding the term of AI. Brundage operating vastly faster (cf. (Bostrom, 2014)). reviews some recent attempts in (Brundage, 2016) and concludes that more research is re- To me, the above appears to be a probable quired and proposes relevant directions that development and is kept very broad intention- should be explored. ally. I will not try to give any precise dates or milestones here. As pointed out above, An Outline of Assumed Progress this would need more rigorous thinking and research. However, the above outline will suffice Following the above proposal of constructing for me to argue about some issues that are likely scenarios for future progress, I want to currently already present and might get mag- briefly establish what I believe will be a proba- nified by the development I foresee. ble trajectory before addressing the issues resulting from it in the following section. Primary Concerns for Human Society First, there seems to be no compelling argument as to why human-level intelligence Privacy, Manipulation and Control should be theoretically unreachable in an AI Implications implementation. There are several ways to As briefly mentioned before, the ability to ex- achieve this as pointed out by Bostrom(2014), tract knowledge about a person’s beliefs, opin- e.g. full brain emulation or mathematical- ions and behavior not only allows to offer bet- functional simulation of the brain or even a ter services to that person but also makes completely artificial solution. All of these either him/her vulnerable to manipulation and ex- require or will reveal insights into how the hu- ploitation. This is, in a sense, already happen- man brain works. Human brain research and ing and, so far, the protection of that privacy development of general AI are thus deeply in- is lacking behind the ever new frontiers that tertwined endeavors and progress in both will might compromise it. advance at a similar rate. An infamous example of this was reported by Notwithstanding the current trend of narrow Duhigg(2012). Allegedly, the discount store AI research, I do further believe that the retailer Target used data it collected or bought incentives for implementing human-level AI about its consumers to predict the pregnancy are strong enough that it will be created at of women. Given that the habits of new par- some point in the future (cf. (Brundage, 2015; ents tend to break up, the intention was to Kurzweil, 2005)). Already there are compa- exploit this knowledge to acquire new per- nies supplied with large amounts of resources, manent customers by sending special adver- such as Alphabet’s DeepMind, working on this tisement and thus stimulating newly forming very task with impressive results such as the buying habits to incorporate shopping at Tar- previously mentioned AlphaGo ((Silver et al., get stores. The article further mentions that 2016)) as well as a system that learns to play customers did not initially respond well to the classic Atari 2600 Games without any previ- targeted advertisements, feeling unduly spied ous knowledge (Mnih et al., 2015). There upon, to which target adapted by concealing seems to be a general trend to expand the the specialized pregnancy-related advertise- narrow domains of AI systems as they grow ments within unrelated product ads. more capable for their tailored tasks (e.g., autonomous cars have come a long way from It should be pointed out that this article is not simply driving in a desert to being able to nav- without criticism. Critics argue that the pre- igate roads in traffic). diction algorithm was probably not as exact as

33 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 portrayed in the article, the misprediction rate use. Overall, this leads to a devaluation and was likely to be high and that ”Target mixes hence degradation of the notion of privacy in up its offers not because it would be weird itself. to send an all-baby coupon-book to a woman Left unchecked, this potential shift of value who was pregnant but because the company might continue to open up access to personal knows that many of those coupon books will data until no person presides over his/her own be sent to women who arent pregnant after personal data alone anymore, leaving that all.” (Harford, 2014). data, or at least parts of it, in the public do- While this might be true, the underlying mo- main. This must not necessarily happen and tivation of (subconsciously) influencing cus- admittedly seems unlikely now, but the current tomers based on the knowledge extracted trend hints in that direction. With it might come from data does not change even if the predic- a greater favoring of interconnectedness and tion is less accurate than described. Progress sharing between people instead of the indi- in AI will likely increase the reliability of these vidualism currently prevalent in Western soci- predictions and, in accordance to the inter- eties. twindness of AI and human brain research, of- Such a development must not be a bad thing fer new insights into how a person can be ma- but should not happen without reflection. Peo- nipulated and directed. Left for exploitation, ple must be made aware of the data they pro- this is a scary prospect. vide and should be educated more thoroughly Of course, all of marketing and even most on the potential consequences. This should, interactions between persons are manipula- however, not aim to evoke irrational fears tive to some extent and everyone counteracts causing rejection of technology but merely such attempts on a daily basis. However, we provide the necessary baseline for reflection usually do not have as large a difference in and critical usage. If users know which infor- available data about the other party and as the mation is required for a system to work, they possibilities to take influence grow, we need are not only able to spot where unnecessary to explore the extent of manipulation that we data is harvested but might also be able to deem acceptable. increase the quality and thus usefulness of the required data they provide (cf. the Con- To prevent these concerns from becoming tent Awareness privacy property described real threats, governments and other regula- by Deng, Wuyts, Scandariato, Preneel, and tory bodies should follow closely on the de- Joosen(2011)). velopments and regulate the extent of the implementation of manipulating behavior. All the An interesting topic of research to support or above issues also obviously apply to (state) defeat the claim made above is whether peo- surveillance agencies, leaving governments in ple in Asian societies, where the collective and a conflict of interest that I cannot see how to group is often valued higher than in Western resolve for now. ones, display a greater willingness to share (private) data. A related issue is the willingness with which many software users surrender their private data in exchange for services. This is, to an Equality of Access extent, in itself an exploitation of the human Another concern of importance is the equal risk-reward-system: The benefits offered by a access to AI systems and the required data service are most often obvious and immediate to drive them. AI will empower whoever con- while the associated risks of giving up private trols it by granting him/her/it superior informa- data are abstract. Negative consequences tion processing capability and automation of might only occur after a long time, if at all, and tasks, thus vastly increasing the overall capa- might seem unrelated to the act of hading over bilities of a single entity (person or institution). the data. An emphasizing factor might be that This bears the danger of opening a significant users are dealing with systems instead of per- wealth-gap between those who have access sons, which makes the surrendering of private to that technology and those who have not. data feel much more impersonal. It seems as if it is stored in some system for private future Access will most likely initially align itself with

34 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 the already existing rich-poor divide as the Superintelligence more wealthy can afford adopting new technology earlier. They might thus be able to As pointed out before, I believe it is highly establish a dominance in that field before the probable that superintelligent AI will be cre- rest of the population is able to adopt, thus fur- ated at some point. There are serious con- ther improving their advantages following cur- cerns that this, if not handled correctly, might rent trends (cf. (OECD, 2008; OECD, 2015)). pose a serious threat to continued existence Furthermore, at least in the early stages of AI of human life (as we know and value it). development, greater knowledge of how the It is, of course, unreasonable to assume that technology works will amplify its usefulness, an AI will arbitrarily decide to turn on its mas- putting higher educated persons at an advan- ters. We design these systems with the pur- tage as well. Greater wealth typically also im- pose to serve (some of) our goals encoded plies better education, suggesting a continued in them, otherwise there would be no incen- dominance of the currently wealthy. These tive to create them. However, as pointed out are, of course, no new observations but might by Omohundro(2014) and Bostrom(2014) be amplified by emerging AIs of sufficient ca- and formalized by Benson-Tilsen and Soares pability. We should ensure that every individ- (2016), there are certain instrumental goals ual is sufficiently computer and data literate to such as self-preservation (including the pro- be able to prevail in a society where AI based tection of its current goal) and acquiring a data processing is prevalent. maximal amount of resources that any AI agent striving to maximize any objective is ex- tremely likely to converge to. Given that a suf- However, not only access to the hardware and ficiently capable superintelligent agent is likely algorithms that constitute an AI will be cru- to outcompete ordinary humans and possibly cial, but also widespread availability of the rel- human societies as a whole in any conceiv- evant data. Without accurate and sufficiently able way, trying to control it by means of dom- large amounts of data, the usefulness of any inance is not a viable option. Hence, if a con- AI will be severely restrained, putting again cern for human values is not sufficiently em- those who have access to that data at a signif- bedded in the agents overall goal, these in- icant advantage. Currently, the data collected strumental goals will be a serious threat. by the large Internet companies (e.g. Face- book, Google, etc.) is and will likely continue There is opposition to these concerns which to be the foundation of their business models, mostly seems to take the point that it is ei- making open access to them unlikely. Poli- ther impossible or there is no incentive to cre- cies that allow ordinary persons access to that ate human-level and, consequently, superin- data bases as part of a business plan (i.e. for telligent AI, that such a system would pose no a fee) can address this issue while covering threat, or that it is so far of in the future as to be the cost that these (and other) companies or irrelevant now (some of these arguments can maybe potential governmental agencies have be found in the Edge conversation (Brockman, in collecting and maintaining the data. Provid- 2014), stated in (Open Philantrophy Project, ing compensation for those who provide rele- 2015) or in (Bryson, Kime, & Zurich¨ , 2011)). vant data should be discussed (cf. (Brockman, However, given the arguments laid out so far, 2014)). I do not find these counterclaims convincing. It is thus important to solve this so-termed control problem before the advent of advanced The speed with which technology becomes AI. With the current high uncertainty in AI available will again be a crucial factor in en- progress which makes it unforeseeable when suring stability. Efforts at monitoring and, if this might occur and the additional uncertainty necessary, establishing equality in access can about the difficulty of solving the control prob- more safely take place during a slow transi- lem, research on this should not be post- tion, making small adjustments along the way, poned. than in a rapid progress that requires larger and more complex interventions with more un- Fortunately, recent years have seen an in- certain outcomes at a larger scale. creasing interest towards this and started to

35 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 explore possible solutions. A currently often tions to our conflicting values and misaligned proposed method is the self-learning of hu- behavior as displayed by humans first - before man values from observing behavior and cul- trying to create some uncontrollable entity that tural evidence by the agent instead of trying might incorporate them. to encode them by hand (cf. (Bostrom, 2014)). This last argument also further strengthens However, there are certain objections to this the necessity of equal access mentioned in that should be carefully investigated. the previous section. If only a reasonable bal- Caliskan-Islam, Bryson and Narayanan ance of power between its constituents main- (2016) observe that the semantics of natural tains the stability of a society and access to AI language already encode bias and harmful has the ability to massively empower individu- prejudices and that systems learning associ- als, ensuring equality of access is of tremen- ations from language corpora pick up these dous importance. biases. This suggests that an agent that is supposed to learn human values from the Conclusion evidence it sees will not only pick up those that we would label as good but also all the In this essay I tried to lay out some of the as- underlying biases we have towards each pects of continued development and integra- other that might harmfully skew the ultimate tion of AI into human societies that I found values it comes to extract. While embedded in most concerning and supply some arguments society these prejudices are already harmful to the discussion. Since AI’s impact will be but we can examine them and try to find enormous and is hard to anticipate exactly, it remedies. In a superintelligent agent they might be that my prioritization is wrong and could be beyond correction and vastly more other issues turn out to be more pressing. harmful. Military AI applications come to mind, for example. An ongoing discussion and future re- Further, as pointed out by Isaksen, Togelius, search of the possible and the monitoring of Lantz, and Nealen(2016), humans engage in the real progress of AI development will yield games of dominance with animals (i.e., beings insight into this over time. of lesser intelligence) that sometimes involve even the death of the animal. This might sug- The discussed issues have an immediate im- gest a tendency towards the exertion of dom- portance and will likely remain relevant over inance, including violence, if no significant re- the entire time frame of transitioning into an sistance is to expect, that a value-learning AI AI empowered society (and possibly there- might also pick up. Hobbes (1651) suggested after), so working on solutions to them will ad- that societies provide a stable and peaceful dress long-term and short-term concerns in environment because of the limited ways in like manner. which single individuals can exert their powers I want to emphasize that, while it has seri- over the many - mostly due to the fact that any ous concerns attached to it, AI also has an single entity will be easily overpowered by a enormous beneficial potential in helping us conglomerate of many entities if all have sim- solving outstanding problems and create suf- ilar capability. If this were true, a superintel- ficient wealth for everyone to profit from, po- ligent AI that embodies these human tenden- tentially ending poverty on a global scale. By cies and is unconstrainable in this sense will no means should these developments be sup- be dangerous. pressed out of misplaced fear. However, care should be taken that it is these beneficial con- The ultimate point made here is that human sequences that come to pass, which should behavior often does not align with the no- not be taken for granted if we are lacking the ble values we claim to hold. Ultimately, be- will or care to shape the development. tween different individuals, groups and cul- tures, we appear to not even agree on the Most of these measures are more in the realm ultimate values of humanity (although some of policy making, either for governments or the baseline has been established). To ensure research community, than in technical AI re- that value-learning will load an agent with truly search and thus involve an open and honest benevolent goals we might need to find solu- discussion outside of a purely academic set-

36 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 ting. The overall public must have knowledge shop on AI, ethics and society at of and agency in the involved decisions. Given AAAI-2016. Phoenix, AZ. Retrieved the global scale of AI impact, discussion and from http://www.aaai.org/ocs/ policy decisions must be globally coordinated. index.php/WS/AAAIW16/paper/ In potential impact, vagueness of threat (due view/12662/12348 to the long time scales) and difficulty to ad- Brynjolfsson, E., & McAfee, A. (2014). The dress, we are facing issues on a scale similar second machine age: Work, progress, to climate change. That we have been unable and prosperity in a time of brilliant tech- to address the latter in a permanent way so far nologies. WW Norton & Company. might well be a disheartening omen. Bryson, J. J., Kime, P. P., & Zurich,¨ C. (2011). Just an artifact: Why machines However, a recent surge in discussion of are perceived as moral agents. In IJ- ethics and AI as well as AI impact displays CAI proceedings - International joint con- that, at least in parts of the academic commu- ference on artificial intelligence (Vol. 22, nity, the issues have been realized and action p. 1641). is taking place. This can be seen in part by the many great works referred to in this text that Condliffe, J. (2017, January 31). An AI provide amazing insights as well as the estab- poker bot has whipped the pros. MIT lishment of several institutes concerned with Technology Review. Retrieved from https://www.technologyreview these topics or codes of ethics for AI research .com/s/603544/an-ai-poker-bot as well as research proposals. Given that, be- -has-whipped-the-pros/ tween some overly optimistic and pessimistic views, it seems that we are on an overall sta- Deng, M., Wuyts, K., Scandariato, R., Pre- ble trajectory so far. If we do not stray from it neel, B., & Joosen, W. (2011). A privacy and watch our steps closely, the future, in so threat analysis framework: Supporting far as it is affected by AI, might turn out fine. the elicitation and fulfillment of privacy requirements. Requirements Engineer- ing, 16(1), 3–32. References Duhigg, C. (2012, February 16). How companies learn your secrets. The New York Benson-Tilsen, T., & Soares, N. (2016). Times. Formalizing convergent instrumental Goertzel, B. (2007). Human-Level artificial goals. In 2nd International Work- general intelligence and the possibility of shop on AI, ethics and society at a technological singularity: A reaction to AAAI-2016. Phoenix, AZ. Retrieved Ray Kurzweil’s The Singularity Is Near, from http://www.aaai.org/ocs/ and McDermott’s critique of Kurzweil. Ar- index.php/WS/AAAIW16/paper/ tificial Intelligence, 171(18), 1161–1173. view/12634/12347 Harford, T. (2014, March 28). Big data: Are Bostrom, N. (2014). Superintelligence: Paths, we making a big mistake? Financial dangers, strategies. OUP Oxford. Times. Retrieved from https:// Brockman, J. (2014, November 14). The www.ft.com/content/21a6e7d8 myth of AI - A conversation with Jaron -b479-11e3-a09a-00144feabdc0 Lanier. Edge. Retrieved from https:// Hassabis, D. (2016, March 16). What www.edge.org/conversation/ we learned in Seoul with AlphaGo. jaron lanier-the-myth-of-ai Google’s ’The Keyword’ Blog. Re- Brundage, M. (2015). Economic possibilities trieved from https://blog.google/ for our children: Artificial intelligence topics/machine-learning/ and the future of work, education, and what-we-learned-in-seoul-with leisure. In 1st International Workshop -alphago/ on AI and ethics at AAAI-2015. Re- Hobbes, T. (1651). Leviathan. London, trieved from http://aaai.org/ocs/ Michael Oaskeshott edition. index.php/WS/AAAIW15/paper/ Isaksen, A., Togelius, J., Lantz, F., & Nealen, view/10155 A. (2016). Playing games across the Brundage, M. (2016). Modeling progress superintelligence divide. In 2nd Interna- in AI. In 2nd International Work- tional Workshop on AI, ethics and society

37 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

at AAAI-2016. Phoenix, AZ. Retrieved Go with deep neural networks and tree from http://www.aaai.org/ocs/ search. Nature, 529(7587), 484–489. index.php/WS/AAAIW16/paper/ Steinhardt, J. (2015, June 24). Long-Term view/12645/12350 and short-term challenges to ensuring Islam, A. C., Bryson, J. J., & Narayanan, the safety of AI systems. Personal A. (2016). Semantics derived au- Blog ’Academically Interesting’. Re- tomatically from language corpora trieved from https://jsteinhardt necessarily contain human biases. .wordpress.com/2015/06/24/ CoRR, abs/1608.07187. Retrieved long-term-and-short-term from http://arxiv.org/abs/ -challenges-to-ensuring-the 1608.07187 -safety-of-ai-systems/ Kurzweil, R. (2005). The singularity is near: Stone, P., Brooks, R., Brynjolfsson, E., When humans transcend biology. Pen- Calo, R., Etzioni, O., Hager, G., . . . guin Books. Teller, A. (2016, September). Artifi- McGrath, R. (2013, November 25). The pace cial intelligence and life in 2030 (One of technology adoption is speeding up. Hundred Year Study on Artificial Intelli- Harvard Business Review. Retrieved gence: Report of the 2015-2016 Study from https://hbr.org/2013/ Panel). Stanford University, Stanford, 11/the-pace-of-technology CA. Retrieved from https://ai100 -adoption-is-speeding-up .stanford.edu/2016-report Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., . . . others (2015). Human-level control through deep reinforcement learning. Lukas Prediger is a Nature, 518(7540), 529–533. Master’s degree student OECD. (2008). Growing unequal?: In- in Computer Science at come distribution and poverty in RWTH Aachen University, OECD countries. OECD Publishing. Germany, with a recent Retrieved from http://www.oecd stay at Aalto University, -ilibrary.org/social-issues Finland, during which -migration-health/growing this essay came to be. -unequal 9789264044197-en doi: His main study interests 10.1787/9789264044197-en are artificial intelligence, OECD. (2015). In it together: machine learning, big Why less inequality benefits all. data and IT security and OECD Publishing. Retrieved from all their entailed consequences for society, http://www.oecd-ilibrary.org/ individual freedom and privacy. employment/in-it-together -why-less-inequality-benefits -all 9789264235120-en doi: 10.1787/9789264235120-en Omohundro, S. (2014). Autonomous technology and the greater human good. Jour- nal of Experimental & Theoretical Artifi- cial Intelligence, 26(3), 303–315. Open Philantrophy Project. (2015, Au- gust). Potential risks from advanced artificial intelligence. Retrieved from http://www.openphilanthropy .org/research/cause-reports/ ai-risk Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., . . . others (2016). Mastering the game of

38 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Truth in the ‘Killer Robots’ Angle? Matthew Rahtz (University of Zurich/ETH¨ Zurich;¨ [email protected]) DOI: 10.1145/3137574.3137584

I had no idea, getting interested in AI two Unemployment years ago, that being involved in the field would involve such a persistent sense of un- I was quite prepared to spend this essay argu- ease. I started out unequivocally excited, per- ing that unemployment resulting from use of AI haps a little naive; but over the years, the con- was by far the most pressing issue right now. cerned voices of economists, philosophers, The more I’ve read, though, the more I’ve got and the mass media have gradually seeped the impression that the change is going to be into me, leaving me with an ill-defined feel- slower than it might appear. ing of hesitation about what we’re heading to- At the start of my reading, the threat of unem- wards. ployment seemed clear. Automation in various Trying to understand the situation a bit bet- forms has been encroaching on the job market ter over the past months has been somewhat for centuries. With the recent step change in overwhelming. AI touches so many different our machine learning capabilities, it seemed a areas that it’s hard to hold everything in mind very reasonable concern that we may not be at once. While there are points of concern far away from a step change in automation, in many of these areas, there are three ar- and hence unemployment, too. eas in particular which have stood out to me: unemployment, the long-term risks of AI, and Take autonomous vehicle technology. Com- lethal autonomous weapons systems (other- panies like Google and Tesla seem to be get- wise known as ‘killer robots’). ting pretty good at it. For most of us, self- driving cars are going to be an unambigu- Reading into these areas in more depth has ously good thing; promises of safer roads left me surprised. AI risk research is some- and more leisure time abound. The ques- thing that’s interested me for a while, but the tion is: what happens when that technol- pressing issue right now doesn’t seem to be ogy makes, say, self-driving trucks possible? the immediate need for research - rather, the There are 1.6 million long-haul truck drivers in connotations that are becoming associated the US (McArdle, 2015). It’s the most popu- with that research. With the second of these lar job in 29 states (Solon, 2016). It’s one of areas, autonomous weapons, I began looking the last jobs left offering middle-class pay with- into it only for the sake of completeness, but out a college degree (Kitroeff, 2016). When found myself completely unaware of the grav- autonomous vehicle technology reaches the ity of the situation. The biggest surprise, how- trucking industry and makes all these drivers ever, has been a change of opinion about the redundant – what then? danger of unemployment. I realise I am no longer nearly so concerned about the immi- Despite the apparently obvious problem, I was nent threat of automation. surprised to find that some people don’t seem to be all that worried. The common arguments Given all that we’ve been hearing on the topic I’ve heard for this position, though, haven’t of AI-related unemployment recently, this last convinced me. statement clearly requires some explanation. So that the scene is set properly for the former “Jobs will be lost, yes; but the AI revolution issues, let us in fact begin our discussion with will create new jobs at the same time,” some this last point. say. But I see no guarantee that the jobs created will match the skills of the people being made redundant. This is already a problem: the global talent gap. The issue is not that there aren’t enough jobs. The issue is that the unfilled jobs require skills that the unemployed Copyright c 2017 by the author(s). population don’t have.

39 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Others say, “We’ve seen step changes in em- matter). And it’s this dynamic that makes me ployment before; what about the industrial rev- think that change is going to be much slower olution? We survived that alright.” But this than we might expect. doesn’t comfort me, because the world is such a different place now. The internet, for exam- Even Google’s efforts seem to be in line with ple, enables advancements in technology to this principle. Their self-driving buggy with- spread much quicker than ever before; and out even a steering wheel is pretty cool, but this is especially significant when the rele- there’s a catch: it can only safely go 25 vant technology is software. In general, it mph (Farivar, 2015). Also, it doesn’t work in seems like a bad idea to try and make pre- snow (McArdle, 2015). Turning back to self- dictions based on only superficially-similar his- driving trucks, if it’s taken Google this long to toric precedents. get to achieve autonomy in even these limited conditions, I’d guess it’s going to be a very The difficulty of making predictions in a highly long time before complete autonomy can be unpredictable world is clearly one of the ma- achieved for a multi-tonne truck travelling at jor factors limiting the quality of these discus- high speed down a freeway in rain, sleet, fog sions. Whatever ideas one may have, it’s hard and, indeed, often snow. to avoid concluding with anything but the in- It’s unsurprising, therefore, that current ef- evitable cop-out of, “But then again, technol- forts at truck automation, such as those from ogy changes so rapidly, who knows what may Daimler (Davies, 2015) and recent startup happen?” Otto (Lee, 2016), are instead targeting semi- This got me to thinking: what are the autonomous solutions. In good conditions, the technology-invariant factors here? What dy- truck will drive itself. In bad conditions, a hu- namics will stay relevant regardless of what man driver in the cab can take over. In good new kinds of technology we come up with? conditions, you still get the benefits of machine Of these factors, there’s been one in partic- control, like the ability to drive throughout the ular that’s struck me as significant: the Pareto night. But you’re spared the difficulty of push- principle. And there’s no better example of ing all the way to the “99.9999%” that’s re- this principle than in the development of self- quired for complete automation. driving cars. I suspect this is a pattern we’ll see through- My impression with autonomous vehicle tech- out many industries. Sure, new technologies nology is that we’re a lot further away from are going to pop up. And we’ll see those tech- complete human replacement than one might nologies progress to the level of useful semi- think at first glance. Sure, we’ve seen the ex- automation pretty quickly. But it’s going to take citing demos of self-driving technology. But much longer for the technology to mature to while these demos are impressive, it’s worth the point where complete automation is possi- bearing in mind that even back in 2015, ble. a single talented hacker could get similar This is not to say that complete automation demo-level functionality working in about a won’t happen eventually. But because of the month (Vance, 2015). The difficulty is appar- Pareto principle, I think the change is going to ently not in getting something basically work- be gradual. We’re going to get advance warning; the difficulty is in getting it to work reliably, ing of what’s happening. It seems unlikely that in a wide range of conditions. As Tesla point it’s going to be a step change. out in their response to said hacker’s efforts: “This is the true problem of autonomy: getting I also don’t mean to suggest that eventual a machine learning system to be 99% correct wide-scale automation isn’t something worth is relatively easy, but getting it to be 99.9999% thinking about and preparing for. Indeed, I’m correct, which is where it ultimately needs to glad to see the issue receiving as much atten- be, is vastly more difficult.” (Tesla, 2015) tion as it is. I only mean to say that it may be a better use of our energies right now to focus This dynamic is, essentially, what the Pareto on other areas which are more pressing and, principle states: that the first 80% of the re- perhaps, more neglected. sults tends to be achieved with the first 20% of the efforts (though the exact proportions don’t This brings us to the first of what I believe

40 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 our current concerns really are: lethal au- parts (Arkin, 2015). tonomous weapons systems. The most forceful argument I’ve come across, however, concerns the likely consequences of Lethal Autonomous Weapons Systems a LAWS arms race. The idea is: once one nation starts deploying sophisticated LAWS, One of the tropes brought up often in the me- other countries will feel the need to step up dia over the past few years has been the im- their own efforts to develop and deploy LAWS age of ‘killer robots’. For a long time, the of their own, leading to a positive feedback hyperbole that invariably accompanied such loop (Future of Life Institute, 2015b). That media lead me not to take the issue seri- race is going to lead to even more sophisti- ously. Even in retrospect, I’m not surprised cated forms of LAWS being developed, and at my ignorance. Those kinds of discussions at ever lower prices. With proliferation hap- never touched on what seems to be the real pening all around the world, at some point issue: the consequences of an autonomous it seems inevitable that some units will fall weapons arms race. (or be sold) into the wrong hands. Consider “the availability on the black market of mass First, let’s clarify what we’re talking about. quantities of low-cost, anti-personnel micro- Lethal autonomous weapons systems robots that can be deployed by one person (LAWS), as they’re known more dryly, re- to anonymously kill thousands or millions of fer to weapons which can make the decision people who meet the user’s targeting crite- to kill entirely on their own, without explicit ria” (Russell, Tegmark, & Walsh, 2015), and go-ahead from a human. For example, the you get the picture. drones currently in use don’t fall into this category, because the decision to kill must The proposal, therefore, is to ban LAWS be- be made by a remote human operator. What fore this arms race can get started. we’re talking about is, say, a drone that can Indeed, this is the direction that the gears of find and kill a target while being completely international government - the UN Conven- disconnected from a pilot. tion on Certain Conventional Weapons (CCW) LAWS have already existed for a while – think – are moving in. The problem is that they land mines. More recently, though, advances may not be moving fast enough. Only at in AI are starting to enable more sophisticated the end of 2016, after three years of dis- forms of LAWS. For example, automated sen- cussion, has the CCW agreed to establish a try guns have already been developed and de- Group of Governmental Experts under whom ployed along the border between North and the creation of new international law can be South Korea (Rabiroff, 2010). This trend looks discussed (Wareham, 2017a). Whether this set to continue: with the advantages of LAWS group will move quickly enough to prevent the (e.g. immunity to communications jamming; start of the race is still uncertain (Wareham, potentially more precise and accurate target- 2017a). It’s not even clear whether this discus- ing; fewer soldiers’ lives on the line), many na- sion will really lead to a complete ban, or only tions are now investing heavily in their further regulation limiting LAWS’ use (Goose & Ware- development (Goose & Wareham, 2017). The ham, 2017). Given that deployment of sophis- question we now face is: should we allow this ticated LAWS may be only years away (Future trend to continue? of Life Institute, 2015b), we’re at a decisive moment. There are arguments both ways. On the one hand, avoiding danger to soldiers’ lives, LAWS Reading about LAWS, I’ve been forced to ad- lower the threshold of entry to conflict. And mit that the ‘killer robot’ angle really does have because of the difficulty in distinguishing com- a grain of truth in it. There is, however, a sec- batants from non-combatants, LAWS could ond angle of the scare I’ve gradually become lead to an increase in the number of civilian convinced it’s worth taking seriously: the long- casualties (Goose & Wareham, 2017). On the term risks of AI. But it’s not the risks them- other hand, if we can program them with hu- selves that I think are the most pressing is- manitarian law, perhaps LAWS could be more sue right now. The bigger issue at the mo- ethical than their stressed-out human counter- ment is the culture that’s becoming associated

41 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 with such concerns – and the limiting effect given that it may be difficult to alter our trajec- that culture might have on the field’s growth. tory once popularity of potentially unsafe al- This brings us to our second pressing issue: gorithms has passed some critical level, it’s opinion of AI risk research. worth us getting started as early as possible on making sure that we get the good path. Opinion of AI Risk Research The most pressing issue right now, however, isn’t the immediate need for research. Yes, Though the dangers of ‘killer robots’ have the proportion of the AI community that’s ded- been talked about for decades, research into icated to risk research is less than what it ide- the long-term risks of AI only seems to have ally would be. But given the long-term na- started being taken seriously with the publica- ture of the problem, what’s important is not the tion in 2014 of Nick Bostrom’s book ‘Superin- starting level, but the rate of growth the field telligence’ (Bostrom, 2016). Bostrom argues will experience over the coming decades. And that the real threat will come not from robots, this is where I get worried. but from artificial general intelligence (AGI): AI which is superhumanly capable across a wide The publication of Superintelligence around range of different tasks, rather than just the the middle of 2014 succeeded in bringing narrow domains that current AI can deal with. awareness of the issue to a broader audi- Consider AI with superhuman cognitive abili- ence. Some of that audience were in a posi- ties (without the rest of our ancestral baggage, tion to rebroadcast to a broader audience still: like emotions) that can be put to work on arbi- over the subsequent months, we saw public trary problems, and you get the idea. statements of concern from the likes of Elon Musk in October 2014 (Gibbs, 2014), Stephen Such technology is, of course, not around Hawking in December 2014 (Cellan-Jones, the corner. Reflecting, though, that over 2014), and Bill Gates in January 2015 (Mack, the course of my father’s life, we went 2015). Though beneficial in publicising the is- from complete ignorance of DNA to being sue, taken out of the context of the broader able to precisely engineer super-muscular discussion, these statements seem to have dogs (Regalado, 2015), and from complete had some undesirable consequences. lack of digital technology to small devices we can fit in our pockets with radio access to the One of the consequences has come about sum of all human knowledge, it seems within through the coincident media hype about re- the realms of possibility that AGI may happen cent advances in machine learning. This within our lifetimes. AGI may be a way away, seems to have given some the impression that but not so far as to be completely intangible to the concern about the risks of AGI is based on us. an assumption of imminence. A report in Jan- uary to the US Department of Defense by the Despite being such a long way away, Su- JASON advisory group, for example, states perintelligence concludes, AI risk research is that “the claimed ‘existential threats’ posed by nonetheless something we need to start work- AI seem at best uninformed. . . in the midst of ing on now. Why? Because the advent of an AI revolution, there are no present signs of AGI is likely to be one of the most momen- any corresponding revolution in AGI” (JASON, tous events in the history of mankind. After 2017). But this is not the basis for the concern AGI, we’re likely to be forced onto one of two at all. In fact, it’s a measure of just how se- paths. There’s the ‘bad’ path, where, for ex- riously those involved believe the future dan- ample, AGI allows one organisation or state gers to be that even though AGI is likely to be to assume control; or where an errant AGI a very long way away, it’s still worth preparing set up with an faulty objective gradually con- for now. It would be unfortunate if misunder- sumes all the world’s resources in order to standing on this point were to lead to misallo- achieve its goal. But there’s also the ‘good’ cation of resources. path, where AGI allows us to solve a whole slew of problems that have thus far proved There is, however, a second, more seri- beyond the reach of our small, metabolically- ous consequence. These statements, com- limited brains. Given the all-or-nothing nature bined with the above-mentioned Terminator of the outcome, Superintelligence argues, and articles, seem to have created a media at-

42 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 mosphere where questions about the dan- What can we do? gers of AI have become appealingly provoca- tive (Bostrom, 2016). This provocation has, in In summary: turn, seem to have stirred up an (understand- • It seems unlikely that unemployment able) feeling of defensiveness among some in through automation will occur as a step the AI community. Mustafa Suleyman, one change. Assuming that real-world appli- of the co-founders of Google DeepMind, for cation of AI continues to follow the Pareto example, was quoted at a conference 2015 principle, we’re more likely to see that telling the audience that “Any talk of a su- change happening gradually. Furthermore, perintelligent machine vacuuming up all the we’re more likely to see human-machine knowledge in the world and then going about hybrid jobs than complete replacement of making its own decisions are absurd. There humans. Given these two factors, and given are engineers in this room who know how that the issue of unemployment is already difficult it is to get any input into these sys- receiving a lot of attention, our further tems.” (Arthur, 2015) efforts might be better spent on other issues Indirectly, these statements and the media re- both more pressing and more neglected. action to them seem to have created a perception of AI risk research as being something Within this category, I see two particularly im- of a silly thing to work on. And it’s this per- portant issues: ception of silliness that makes me concerned • about the field’s growth. Deployment of LAWS based on sophisticated AI could lead to an arms race. An One problem is that it’s going to make it harder arms race will lead to technology prolifer- to attract more people to the area. Given that ation. Proliferation will make it easier for the field is already talent-limited (Whittlestone, groups with malicious intent to get their 2017), it would be a mistake to stunt its growth hands on the technology and use it to, for even further. example, oppress a populace. • AI risk research is in danger of becoming The bigger issue, though, is that this percep- seen as a silly topic. This is concerning tion could lead to the broader AI community partly because the connotation will make it becoming actively hostile towards those in- hard to attract extra minds to an already volved in risk research. If such hostility arises, talent-limited field. The bigger concern, then no matter how many people are working however, is that without collaboration be- on risk research, they may be prevented from tween the risk community and the rest of the having any impact. They may, for example, be AI community, the impact of risk research unable to persuade those pursuing real-world may be limited. implementation to investigate safer alterna- tives to existing algorithms (such as the inclu- So what can we actually do about these is- sion of reward uncertainty into reward learn- sues? ing algorithms (Alexander, 2017)). Without a sense of everyone being on the same side, the Despite being perhaps the most urgent prob- venture seems doomed from the start. lem, LAWS may be the simplest to actually deal with. A lot of progress has been made It’s hard to say just how much investment in towards a full ban. All that remains is to make risk research is going to be needed to ensure sure that the real issues are not drowned out a ‘safe’ future AGI-wise. It may be enough to by irrelevant hyperbole; to maintain sufficient have only a small portion of the AI community attention on the situation to ensure that the re- as a whole dedicated to the issue. But judging maining steps towards a full ban take place. from our current trajectory, there’s no guarantee that we’ll get that balance right by just let- One way that organisations can assist in this is ting things happen. That balance looks to be by lending their weight to the push for a ban. something we’ll need to work on deliberately. Specifically, they can endorse the Campaign to Stop Killer Robots – a group of NGOs that This brings us to our final section: what can has been instrumental in influencing the UN. we actually starting doing? Public statements of support, such as those

43 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 from Clearpath Robotics in 2014 (Hennessey, apparent success of the Future of Life Insti- 2014), give the campaign clout with which to tute’s ‘Beneficial AI’ conference held at the be- maintain their influence (Wareham, 2017b). ginning of the year in Asilomar. Part of the success of the conference was a step towards Organisations may also be able to help by a greater sense of unity in the field, drawing holding events to get the issue known about on the range of expertise represented at the more widely, encouraging more people to get conference to form the Asilomar AI principles involved. The positions of the individual mem- – a set of 29 principles agreed by the partic- bers of the Group of Government Experts ipants of the conference as important to up- will be partially a function of, for example, hold, touching on aspects from AI safety to the number of people writing to national rep- ensuring that the benefits of AI will be shared resentatives, and media coverage resulting throughout society. However, the event was from events publicising the issue. Organisa- also apparently successful in starting to break tions such as the ACM may promote such ac- down the cultural barriers surrounding the is- tion directly through member communications; sue. One participant noted that the confer- those at academic institutions might organise ence was in part “a coming-out party for AI local talks to get more people aware of what safety research. One of the best received the risks really are. talks was about ‘breaking the taboo’ on the The question of how to get AI risk research to subject, and mentioned a postdoc who had be taken seriously is a more difficult one. The pursued his interest in it secretly lest his pro- various tropes have become so firmly estab- fessor find out, only to learn later that his pro- lished in our collective consciousness that it’s fessor was also researching it secretly, lest ev- going to be hard to directly affect public per- eryone else find out.” (Alexander, 2017) Creat- ception. However, I think there is at least hope ing more opportunities for this kind of conver- for change within the limited scope of the aca- sation – whether in the form of conferences, demic community. an evening of talks, or simply group discus- One low-hanging fruit may be for more organi- sions – can only be a good thing. sations to offer awards and grants for work re- Having got a better sense of the bigger picture lated to AI risk, as the Future of Life Institute is throughout the course of writing this essay, I already doing (Future of Life Institute, 2015a). find myself feeling optimistic. Though it’s clear As well as enabling research, grants may help there are dangers ahead of us – those cov- to signal to the community that risk research ered here, along with many others – they’re is something credible to be working on. not just being swept under the rug. People are Another source of easy gains may be to en- taking notice of them. courage universities to offer courses on the broader context of AI – an area that seems Of all that I’ve read about, I think it’s the Asilo- to be conspicuously lacking in the curriculum mar conference that has given me the most currently. In addition to informing students hope. The fact that people from so many dif- of the real arguments for risk research, such ferent parts of the field – machine learning courses could address other problems that (Yann LeCun; Yoshua Bengio), risk research have been pointed out in the computer sci- (Nick Bostrom; Eliezer Yudkowsky), funding ence curriculum, such as the need for ethics (Sam Altman; Elon Musk), and so on – were education and awareness of the dangers of willing to come together to talk about where dataset bias (National Science and Technol- things are going gives me a sense that cultur- ogy Council, 2016). Attracting computer sci- ally, we’re on the right track. ence students towards such courses is, of Whatever issues we may face over the com- course, going to be a challenge; but I see a lot ing decades, at the broader level, it’s nurturing of possibility for creative solutions here, such and maintaining this culture that strikes me as as courses based on readings in science fic- the most important thing going forward. It’s tion (Burton, Goldsmith, & Mattei, 2015). only through this attitude that we’re going to Perhaps the most effective remedy to the is- be able to continue making course corrections sue would simply be getting people together where necessary – and that long-term, there- to talk about it. Consider, for example, the fore, we really will be able to reach the kind of

44 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 future that we all hope AI will take us to. Future of Life Institute. (2015a). First AI Grant Recipients. Retrieved 2017-02-25, from https://futureoflife.org/ References first-ai-grant-recipients Future of Life Institute. (2015b, 07). Alexander, S. (2017, 02). Notes from the Open Letter on Autonomous Asilomar Conference on Beneficial AI. Weapons. Retrieved 2017-02-07, from Retrieved 2017-02-25, from https:// https://futureoflife.org/open slatestarcodex.com/2017/02/ -letter-autonomous-weapons 06/notes-from-the-asilomar Gibbs, S. (2014, 10). Elon Musk: artificial -conference-on-beneficial-ai intelligence is our biggest existential Arkin, R. (2015, 08). Warfighting Robots threat. Retrieved 2017-02-15, from Could Reduce Civilian Casualties, https://www.theguardian.com/ So Calling for a Ban Now Is Pre- technology/2014/oct/27/elon mature. Retrieved 2017-02-07, from -musk-artificial-intelligence http://spectrum.ieee.org/ -ai-biggest-existential automaton/robotics/artificial -threat -intelligence/autonomous Goose, S., & Wareham, M. (2017, 01). -robotic-weapons-could-reduce The Growing International Movement -civilian-casualties Against Killer Robots. Retrieved 2017- Arthur, C. (2015, 06). DeepMind: ‘Artificial 02-07, from http://hir.harvard intelligence is a tool that humans can .edu/growing-international control and direct’. Retrieved 2017-02- -movement-killer-robots/ 07, from https://www.theguardian Hennessey, M. (2014, 08). Clearpath .com/technology/2015/jun/ Robotics Takes Stance Against ‘Killer 09/deepmind-artificial Robots’. Retrieved 2017-02-22, from -intelligence-tool-humans http://www.clearpathrobotics -control .com/press release/clearpath Bostrom, N. (2016). Superintelligence: Paths, -takes-stance-against-killer Dangers, Strategies. -robots/ Burton, E., Goldsmith, J., & Mattei, N. (2015). JASON. (2017, 01). Perspectives on Re- Teaching AI Ethics Using Science Fic- search in Artificial Intelligence and tion. In 1st International Workshop on Artificial General Intelligence Relevant AI, Ethics and Society, Workshops at the to DoD. Retrieved 2017-02-28, from Twenty-Ninth AAAI Conference on Artifi- https://fas.org/irp/agency/ cial Intelligence. dod/jason/ai-dod.pdf Cellan-Jones, R. (2014, 12). Stephen Hawk- Kitroeff, N. (2016, 09). Robots could replace ing warns artificial intelligence could 1.7 million American truckers in the end mankind. Retrieved 2017-02-15, next decade. Retrieved 2017-02-04, from http://www.bbc.com/news/ from http://www.latimes.com/ technology-30290540 projects/la-fi-automated Davies, A. (2015, 05). The World’s First -trucks-labor-20160924 Self-Driving Semi-Truck Hits the Lee, T. B. (2016, 10). Self-driving trucks are Road. Retrieved 2017-02-04, from here, but they won’t put truck drivers out https://www.wired.com/2015/ of work - yet. Retrieved 2017-01-31, from 05/worlds-first-self-driving http://www.vox.com/new-money/ -semi-truck-hits-road/ 2016/10/25/13404974/otto-self Farivar, C. (2015, 11). Cops pull over -driving-trucks Google car for doing 24mph in a Mack, E. (2015, 01). Bill Gates Says 35mph zone. Retrieved 2017-02-26, You Should Worry About Artificial from https://arstechnica.com/ Intelligence. Retrieved 2017-02-15, tech-policy/2015/11/google from https://www.forbes.com/ -self-driving-car-pulled-over sites/ericmack/2015/01/ -for-not-going-fast-enough 28/bill-gates-also-worries

45 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

-artificial-intelligence-is-a Vance, A. (2015, 12). The First Person to -threat Hack the iPhone Built a Self-Driving McArdle, M. (2015, 05). When Will Self- Car. Retrieved 2017-02-20, from Driving Trucks Destroy America? https://www.bloomberg.com/ Retrieved 2017-01-31, from http:// features/2015-george-hotz origin-www.bloombergview.com/ -self-driving-car/ articles/2015-05-27/when-will Wareham, M. (2017a, 01). Ban- -self-driving-trucks-destroy ning Killer Robots in 2017. Re- -america- trieved 2017-02-08, from https:// National Science and Technology Coun- www.hrw.org/news/2017/01/15/ cil. (2016, 10). Preparing for the banning-killer-robots-2017 Future of Artificial Intelligence. Re- Wareham, M. (2017b, 08). Personal commu- trieved 2017-01-13, from https:// nication. obamawhitehouse.archives Whittlestone, J. (2017). Research .gov/sites/default/files/ into risks from artificial intelli- whitehouse files/microsites/ gence. Retrieved 2017-02-23, ostp/NSTC/preparing for the from https://80000hours.org/ future of ai.pdf career-reviews/artificial Rabiroff, J. (2010, 07). Machine -intelligence-risk-research gun-toting robots deployed on DMZ. Retrieved 2017-02-23, from http://www.stripes.com/news/ pacific/korea/machine-gun Matthew Rahtz is a stu- -toting-robots-deployed-on dent on the MSc in Neu- -dmz-1.110809 ral Systems and Compu- Regalado, A. (2015, 10). First Gene- tation joint between the Edited Dogs Reported in China. Re- University of Zurich¨ and trieved 2017-02-23, from https:// ETH Zurich.¨ His research www.technologyreview.com/s/ interests are in the long- 542616/first-gene-edited-dogs term risks of artificial in- -reported-in-china/ telligence, with a particu- Russell, S., Tegmark, M., & Walsh, T. lar focus on safe reinforcement learning. (2015, 08). Why We Really Should Ban Autonomous Weapons: A Re- sponse. Retrieved 2017-02-08, from http://spectrum.ieee .org/automaton/robotics/ artificial-intelligence/ why-we-really-should-ban -autonomous-weapons Solon, O. (2016, 06). Self-driving trucks: what’s the future for America’s 3.5 million truckers? Retrieved 2017-01-31, from https://www.theguardian.com/ technology/2016/jun/17/ self-driving-trucks-impact -on-drivers-jobs-us Tesla. (2015, 12). Correction to article: “The First Person to Hack the iPhone Built a Self-Driving Car”. Retrieved 2017-02- 20, from https://www.tesla.com/ support/correction-article -first-person-hack-iphone -built-self-driving-car

46 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

How Do We Ensure That We Remain In Control of Our Autonomous Weapons? Ilse Verdiesen (Delft University of Technology; [email protected]) DOI: 10.1145/3137574.3137585

‘... our AI systems must do what we want Galactica, are planning to conquer the world. them to do.‘’ Many AI applications are already being used today. Smart meters, search engines, per- This quote is mentioned in the Open Letter: sonal assistance on mobile phones, autopi- Research Priorities for Robust and Beneficial lots and self-driving cars are examples of this. Artificial Intelligence (AI) (Future of Life In- One of the applications of AI is that in Au- stitute, 2016) signed by over 8.600 people tonomous Weapons. Research found that Au- including Elon Musk and Stephan Hawking. tonomous Weapons are increasingly deployed This open letter received a lot of media at- on the battlefield (Roff, 2016). It is already re- tention with news headlines as: ‘Musk, Woz- ported that China has autonomous cars which niak and Hawking urge ban on warfare AI and carry an armed robot (Lin & Singer, 2014), autonomous weapons’ (Gibbs, 2015) and it Russia claims it is working on autonomous fused the debate on this topic. Although this tanks (W. Stewart, 2015), the US christened type of ‘War of the Worlds’ news coverage their first ‘self-driving’ war-ship in May 2016 might seem exaggerated at first glance, the (P. Stewart, 2016) and the Russian arms underlying question on how we ensure that manufacturer Kalashnikov recently disclosed our Autonomous Weapons remain under our that they developed a fully automated combat control, is in my opinion one of the most press- module that uses neural networks (RT, 2017). ing issues for AI technology at this moment in time. Autonomous systems can have many benefits for the military, for example when the au- To remain in control of our Autonomous topilot of the F-16 prevents a crash (US Air- Weapons and AI in general, meaning that its force, 2016) or the use of robots by the Ex- actions are intentional and according to our plosive Ordnance Disposal (EOD) to disman- plans (Cushman, 2015), we should design tle bombs (Carpenter, 2016). The US Airforce it in a responsible manner and to do so I expects the deployment of robots with fully believe we must find a way incorporate our autonomous capabilities between the years moral and ethical values into their design. The 2025 and 2047 (Royakkers & Orbons, 2015). ART principle, an acronym for Accountabil- There are many more applications which can ity, Responsibility and Transparency can sup- be beneficial for the Defence organization. port a responsible design of AI. The Value- Goods can be supplied with self-driving trucks Sensitive Design (VSD) approach can be used and small UAVs can be programmed with to cover the ART principle. In this essay, I swarm behaviour to support intelligence gath- show how Autonomous Weapons can be de- ering (CBS, 2017). Yet, the nature of Au- signed responsibly by applying the VSD ap- tonomous Weapons might also lead to uncon- proach which is an iterative process that con- trollable activities and societal unrest. The siders human values throughout the design Stop Killer Robots campaign of 61 NGOs di- process of technology (Davis & Nathan, 2015; rected by Human Rights Watch (Campaign Friedman & Kahn Jr, 2003). Stop Killer Robots, 2015) is voicing concerns, but also the United Nations are involved in Introduction the discussion and state that ‘Autonomous weapons systems that require no meaningful Artificial Intelligence is not just a futuristic human control should be prohibited, and re- science-fiction scenario in which the ‘Ulti- motely controlled force should only ever be mate Computer‘ takes over the Enterprise or used with the greatest caution’ (General As- human-like robots, like the Cylons in Battlestar sembly United Nations, 2016). Copyright c 2017 by the author(s). In the remainder of this essay, I will define AI

47 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 and Autonomous Weapons in a short intro- insight into societal concerns about this kind duction, followed by an explanation the Value- of technology. Principles to describe Respon- Sensitive Design approach. I will use the three sible AI are Accountability, Responsibility and different phases of this approach to investi- Transparency (ART) which are depicted in the gate the conceptual, empirical and technical outer layer of figure1. Accountability refers to aspects of a design of Autonomous Weapons the justification of the actions taken by the AI, in which human values are the central compo- Responsibility allows for the capability to take nent. blame for these actions and Transparency is concerned with describing and reproducing the decisions the AI makes and adepts to its Defining Artificial Intelligence environment (Dignum, 2016). Artificial Intelligence is described by Neapolitan and Jiang(2012, p. 8) as ‘an intelligent entity that reasons in a changing, complex environment’, but this definition also applies to natural intelligence. Russell, Norvig, and Intelligence(1995) provide an overview of many definitions combining views on systems that think and act like humans and systems that think and act rational, but they do not present a clear definition of their own. For now, I adhere to the description Bryson, Kime, and Zrich(2011) provide. They state that a machine (or system) shows intelligent behaviour if it can select an action based Figure 1: Concepts of Responsible AI (based on on an observation in its environment. The (Dignum, 2016)) intervention of the autopilot that prevented the crash of the F-16 is an example of this ‘action selection’ (US Airforce, 2016). The autopilot Defining Autonomous Weapons assessed its environment, in this case the rapid loss of altitude and the fact that the pilot Royakkers and Orbons(2015) describe sev- did not act on warning signals, and took an eral types of Autonomous Weapons and make action to improve the situation; it pulled up a distinction between (1) Non-Lethal Weapons to a safe altitude. In scientific literature, AI is which are weapons ‘without causing (inno- described as more than an Intelligent System cent) casualties or serious and permanent alone. It is characterized by the concepts harm to people.’ (Royakkers & Orbons, 2015, of Adaptability, Interactivity and Autonomy p. 617), such as an Active Denial System (Floridi & Sanders, 2004) as depicted in the which uses a beam of electromagnetic en- inner layer of figure1(Dignum, 2016). Adapt- ergy to keep people at a certain distance ability means that the system can change from an object or troops, and (2) Military based on its interaction and can learn from its Robots which they define ‘as reusable un- experience. Machine learning techniques are manned systems for military purposes with an example of this. Interactivity occurs when any level of autonomy.’ (Royakkers & Orbons, the system and its environment act upon each 2015, p. 625). Altmann, Asaro, Sharkey, and other and Autonomy means that the system Sparrow(2013) closely follow the definition itself can change its state. These character- of autonomous robots stated above, but add istics may lead to undesirable behaviour or ‘that once launched [they] will select and en- uncontrollable activities of AI as scenarios of gage targets without further human interven- many science-fiction movies have shown us. tion.’ (Altmann et al., 2013, p. 73). The de- Although these scenarios are often not realis- ployment of Autonomous Weapons on the bat- tic, a growing body of researchers is focusing tlefield without direct human oversight is not on responsible design of AI, for example on only a military revolution according to Kaag the social dilemmas of Autonomous Vehicles and Kaufman(2009), but can also be consid- (Bonnefon, Shariff, & Rahwan, 2016), to get ered as a moral one. As large scale deploy-

48 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

ment of AI on the battleﬁeld seems unavoid- able (Rosenberg, 2016), the discussion about ethical and moral responsibility is imperative. I found that substantive empirical research on values related to Autonomous Weapons is lacking and it is unclear which moral values people, for example politicians, engineers, military and the general public, would want to be incorporated into the design of Autonomous Weapons. The Value-Sensitive Design could be used as a proven design approach to ﬁgure out which values are relevant for a responsible design of Autonomous Weapons (Friedman & Kahn Jr, 2003; van der Hoven & Mander- sHuits, 2009).

Value-Sensitive Design approach Figure 2: Example of Value-Sensitive Design approach The Value Sensitive Design is a three-partite approach (figure2) that allows for considering human values throughout the design pro- Conceptual investigation cess of technology. It is an iterative process for the conceptual, empirical and technologi- In the conceptual investigation phase I will cal investigation of human values implicated look at the direct and indirect stakeholders by the design (Davis & Nathan, 2015; Fried- of who will use and will be effected by Au- man & Kahn Jr, 2003). It consists of three tonomous Weapons. I will also investigate phases: universal human values and the values that specifically relate to Autonomous Weapons. 1.A conceptual investigation that splits in two parts: a) Identifying the direct stakeholders, Stakeholders Many stakeholder groups are those who will use the technology, and the involved in the case of Autonomous Weapons indirect stakeholders, those whose lives are and each of these groups could be further influenced by the technology, and b) Identi- subdivided, but for the scope of this essay I fying and defining the values that the use of will use a high level of analysis which already the technology implicates. results in a fair number of direct and indirect stakeholders. The direct stakeholders that will 2. The empirical investigation looks into the use Autonomous Weapons are the Military, understanding and experience of the stake- for example the Air Force in case of drones, holders in a context relating to the technol- the Navy who uses unmanned ships and sub- ogy and implicated values will be examined. marines, and the Army that can use robots or 3. In the technical investigation, the specific automated missile systems. Also, at a more features of the technology are analysed political level the Department of Defense and (Davis & Nathan, 2015). the government are involved as these stakeholders decide on funding research and de- The VSD should not been seen as a separate ploying military personnel in armed conflicts. design method, but it can be used to augment Indirect stakeholders, whose lives are influ- an already used and established design pro- enced by Autonomous Weapons are the res- cess such as the waterfall or spiral model. The idents living in conflict areas who might be VSD can be used as a roadmap for engineers affected by the use of these weapons, the and students to incorporate ethical considera- general public whose support for the troops tions into the design (Cummings, 2006). I will abroad is imperative, the engineers who de- use the three phases of the VSD approach as sign and develop the technology, but also civil a method to show the elicitation of values for a society organizations (Gunawardena, 2016), responsible design of Autonomous Weapons. such as the 61 NGOs directed by Human

49 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Rights Watch (Campaign Stop Killer Robots, Values relating to Autonomous Weapons 2015) and the United Nations (General As- The recent advances in AI technology led to sembly United Nations, 2016) that are con- increase in the ethical debate on Autonomous cerned about these type of weapons. Weapons and scholars are getting more and more involved in these discussions. Most Values In this section, first universal hu- studies on weapons do not explicitly mention man values in general are defined and sec- values, but some do discuss some ethical is- ondly values found in literature related to Au- sues that relate to values. Cummings(2006), tonomous Weapons are described. in her case study of the Tactical Tomahawk missile, looks at the universal values posed Definition of values Values have been by Friedman and Kahn Jr(2003) and states studied quite intensively over the past twenty- that next to accountability and informed con- five years and many definitions have been sent, the value of human welfare is a fun- drafted. For example, Schwartz(1994, p. 21) damental core value for engineers when de- describes values as: ‘desirable transsitua- veloping weapons as it relates to the health, tional goals, varying in importance, that serve safety and welfare of the public. She also as guiding principles in the life of a person or mentions that the legal principles of propor- other social entity.’. This is quite a specific tionality and discrimination are the most im- description compared to Friedman, Kahn Jr, portant to consider in the context of just con- Borning, and Huldtgren(2013, p. 57) who duct of war and weapon design. Proportion- state that values refer to: ‘what a person or ality refers to the fact that an attack is only group of people consider important in life.’. justified when the damage is not considered The existing definitions have been summa- to be excessive. Discrimination means that rized by Cheng and Fleischmann(2010, p. 2) a distinction between combatants and non- in their meta-inventory of values in that: ‘val- combatants is possible (Hurka, 2005). Asaro ues serve as guiding principles of what people (2012) also refers to the principles of propor- consider important in life’. Although a quite tionality and discrimination and states that Au- simple description, I think it captures the def- tonomous Weapons open-up a moral space in inition of a value best so I will adhere to this which new norms are needed. Although he definition for now. Many lists of values exist, does not explicitly mention values in his argu- but I will stay close to the values that Friedman ment, he does refer to the value of human life and Kahn Jr(2003) describe in their proposal and the need for humans to be involved in the of the Value-Sensitive Design method: Hu- decision of taking a human life. Other studies man welfare, Ownership and property, Pri- primarily describe ethical issues, such as pre- vacy, Freedom from bias, Universal usability, venting harm, upholding human dignity, secu- Trust, Autonomy, Informed consent, Account- rity, the value of human life and accountabil- ability, Courtesy, Identity, Calmness and En- ity (Horowitz, 2016; UNDIR, 2015; Walsh & vironmental Sustainability. Values can be dif- Schulzke, 2015; Williams, Scharre, & Mayer, ferentiated from attitudes, needs, norms and 2015). behaviour in that they are a belief, lead to behaviour that guides people and are ordered Empirical investigation in a hierarchy that shows the importance of the value over other values (Schwartz, In this phase, I will examine the values of 1994). Values are used by people to jus- direct and indirect stakeholders in a con- tify their behaviours and define which type of text relating to the technology to under- behaviours are socially acceptable (Schwartz, stand how they will experience the deploy- 2012). They are distinct from facts in that val- ment of Autonomous Weapons. One method ues do not only describe an empirical state- of empirically investigating how stakeholders ment of the external world, but also adhere to experience the deployment of Autonomous the interests of humans in a cultural context Weapons is by means of testing a scenario in (Friedman et al., 2013). Values can be used a randomized controlled experiment (Oehlert, to motivate and explain individual decision- 2010). I will sketch one scenario and analyse making and for investigating human and social the values that can be inferred from it. How- dynamics (Cheng & Fleischmann, 2010). ever, I need to remark that I will not conduct an

50 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 actual experiment and that for valid results a The analysis shows that different stakeholders more extensive empirical study is needed than will have different values regarding the actions the brief analysis I provide in this essay. of an Autonomous Weapon. The values that can be derived from this particular scenario Scenario: Humanitarian mission A mili- are security, accountability, responsibility and tary convoy is on its way to deliver food pack- human life. Of all of these values, the univer- ages to a refugee camp in Turkey near the sal value of accountability relates to the justifi- Syrian border. The convoy is supported in cation of an action, it is most mentioned in re- the air by an Autonomous drone that car- search and it fits the ART principle described ries weapons and that scans the surround- in the introduction, therefore I will use it in the ing for enemy threats. When the convoy is technical investigation phase to show how Au- at 3-mile distance of the refugee camp, the tonomous Weapons can be designed in a re- Autonomous drone detects a vehicle behind sponsible manner upholding this value. a mountain range on the Syrian side of the border that approaches the convey at high Technical investigation speed and will reach the convoy in less than one minute. The Autonomous drones im- In the technical investigation phase the spe- agery detection system spots four people in cific features of the Autonomous Weapons the car who carry large weapons shaped ob- technology are analysed and requirements for jects. Based on a positive identification of the the design can be specified. Translating val- driver of the vehicle, who is a known mem- ues into design requirements can be done ber of an insurgency group, and intelligence by means of a value hierarchy (Van de Poel, information uploaded to the drone prior to its 2013). This hierarchical structure of values, mission the drone decides to attack the vehi- norms and design requirements makes the cle when it is still at a considerable distance value judgements, that are required for the of the convoy which results in the death of all translation, explicit, transparent and debat- four passengers. able. The explicitly of values allows for critical reflection in debates and pinpoint the value judgements that are disagreed on. In this sec- Analysis In the analysis of the incident, the tion I will use this method to create a value stakeholders would probably interpret the sce- hierarchy for Autonomous Weapons for the nario in numerous ways resulting in a differ- value of accountability. ent emphasis of inferred values. For example, as direct stakeholders, military personnel The top level of a value hierarchy consists of (especially those in the convoy) will probably the value, as depicted in figure3, the middle see the actions of the drone as protecting their level contains the norms, which can be capa- security. Politicians, as another direct stake- bilities, properties or attributes of Autonomous holder, will also take the value of responsibil- Weapons, and the lower level are the design ity into account. Indirect stakeholders, such requirements that can be identified based on as residents of the area who might be related the norms. The relation between the levels to the passengers in the car will look at val- is not deductive and can be constructed top- ues as accountability and human life. Non- down, by means of specification, or bottom-up governmental organisations (NGOs) who are by seeking for the motivation and justification working in the camp might relate to both the of the lower level requirements. The bottom- value of security for the refugees and respon- up conceptualisation of values is a philosophi- sibility for the delivery of the food packages, cal activity which does not require specific do- but would also call for accountability of the ac- main knowledge and the top-down specification taken by the drone, especially if local res- tion of values requires context or domain spe- idents claim that the passengers had no in- cific knowledge that adds content to the de- tention of attacking the convoy and were just sign (Van de Poel, 2013). This might prove to driving by. The NGOs might call for further in- be quite difficult as insight is needed in the investigation of the incident by a third party in tended use and intended context of the value which the principles of proportionality and dis- which is not always clear from the start of a de- crimination are looked at to determine if the sign project. Also, as artefacts are often used attack was justified. in an unintended way or context, new values

51 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

are being realized or a lack of values is discov- tribunal of the United Nations. ered (van Wynsberghe & Robbins, 2014). An example of this are drones that were initially designed for military purposes, but are now also used by civilians for filming events and even as background lights during the 2017 Su- per bowl half-time show. The value of safety is interpreted differently for military users that use drones in desolated regions compared to 300 drones flying in formation over a popu- lated area. The different context and usage of a drone will lead to a different interpretation of the safety value and could lead to more strict norms for flight safety which in turn could be Figure 3: Value hierarchy for Autonomous further specified in alternate design require- Weapons (based on Van de Poel(2013, p. 264)) ments for rotors and software for proximity alerts to name two examples. Van de Poel (2013, p. 262) defines specification as: ‘as the Conclusion translation of a general value into one or more In this essay, I have argued that the most specific design requirements’ and states that pressing issue for AI technology of this time is this can be done in two steps: that we remain in control of our Autonomous Weapons which means that its actions are ac- 1. Translating a general value into one or more cording to our intentions and plans. For this, general norms; we must ensure that our human values, such 2. Translating these general norms into more as accountability, are incorporated into the de- specific design requirements. sign so that we can investigate if its actions are justified based on the legal principles of In the case of Autonomous Weapons, I trans- proportionality and discrimination. If it turns lated the value of accountability into norms for out that its action is not justified, the design ‘transparency of decision-making’ and ‘insight or the algorithm of the Autonomous Weapon into the algorithm’ that will allow users to get needs to be adjusted to prevent this action of an understanding of the decision choices the happening in the future. Autonomous Weapon makes so that its ac- The Value-Sensitive Design approach is a pro- tions can be traced and justified. The norms cess that can be used to augment the exist- for transparency lead to specific design re- ing design process of Autonomous Weapons quirements. In this case, a feature to visu- for the elicitation of human values. I showed alise the decision-tree, but also to present the that the elicitation of human values in the de- decision variables the Autonomous Weapons sign process will lead to a different design of used, for example trade-offs in collateral dam- AI technology. In the case of Autonomous age percentages of different attack scenarios Weapons, the value of accountability would to provide in-sight into the proportionality of lead to a design in which a screen as user an attack. The Autonomous Weapon should interface is added. Also, the weapon needs also be able to present the sensor information, to be designed with features to download the such as imagery of the site, in order to show information and visualisation of the decision- that it discriminated between combatants and making process, for example by means of non-combatants. To get insight into the algo- a decision-tree. Without explicitly consider- rithm, an Autonomous Weapon should be de- ing the value of accountability, these features signed with features that it normally will not are overlooked in current design processes contain. For example, a screen as user in- and not incorporated into an Autonomous terface that shows the algorithm in a human Weapon. readable form and the functionality to download the changes made by the algorithm as Therefore I argue, that if we want to remain in part of its machine learning abilities that can control of our Autonomous Weapons, we will be studied by an independent party like a war have to start designing this AI technology in a

52 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 responsible way using the ART principle and the American Society for Information Sci- the elicitation of human values by means of ence and Technology, 47(1), 1-10. the Value-Sensitive Design process. I would Cummings, M. L. (2006). Integrating ethics like to call on governments, industries and or- in design through the value-sensitive de- ganisation, including the ACM SIGAI, to ap- sign approach [Journal Article]. Science ply a Value-Sensitive Design approach early and Engineering Ethics, 12(4), 701-715. in the design of Autonomous Weapons to cap- Cushman, F. (2015). Deconstructing intent ture human values in the design process and to reconstruct morality [Journal Article]. make sure that this AI technology does what Current Opinion in Psychology, 6, 97- we want it to do. 103. Davis, J., & Nathan, L. P. (2015). Value sensitive design: applications, adaptations, References and critiques [Journal Article]. Handbook Altmann, J., Asaro, P., Sharkey, N., & Spar- of Ethics, Values, and Technological De- row, R. (2013). Armed military robots: sign: Sources, Theory, Values and Appli- editorial [Journal Article]. Ethics and In- cation Domains, 11-40. formation Technology, 15(2), 73. Dignum, V. (2016). [Web Page]. Re- Asaro, P. (2012). On banning autonomous trieved from https://rai2016.tbm weapon systems: human rights, automa- .tudelft.nl/contents/ tion, and the dehumanization of lethal Floridi, L., & Sanders, J. W. (2004). On the decision-making [Journal Article]. In- morality of artiﬁcial agents [Journal Arti- ternational Review of the Red Cross, cle]. Minds and machines, 14(3), 349- 94(886), 687-709. 379. Bonnefon, J.-F., Shariff, A., & Rahwan, Friedman, B., & Kahn Jr, P. H. (2003). Human I. (2016). The social dilemma of values, ethics, and design [Journal Ar- autonomous vehicles [Journal Arti- ticle]. The human-computer interaction cle]. Science, 352(6293), 1573-1576. handbook, 1177-1201. Retrieved from http://science Friedman, B., Kahn Jr, P. H., Borning, A., & .sciencemag.org/content/sci/ Huldtgren, A. (2013). Value sensitive 352/6293/1573.full.pdf design and information systems [Book Bryson, J. J., Kime, P. P., & Zrich, C. Section]. In Early engagement and new (2011). Just an artifact: why machines technologies: Opening up the laboratory are perceived as moral agents [Confer- (p. 55-95). Springer. ence Proceedings]. In Ijcai proceedings- Future of Life Institute. (2016). Ai international joint conference on artiﬁcial open letter [Pamphlet]. Retrieved intelligence (Vol. 22, p. 1641). from http://futureoflife.org/ai Campaign Stop Killer Robots. (2015). -open-letter (Vol. 2017) (Web Page No. 15-07- General Assembly United Nations. (2016, 2017). Retrieved from https://www 4 February 2016). [Government Docu- .stopkillerrobots.org/ ment]. Carpenter, J. (2016). Culture and human- Gibbs, S. (2015). [Web Page]. Retrieved from robot interaction in militarized spaces: A https://www.theguardian.com/ war story [Book]. Taylor & Francis. Re- technology/2015/jul/27/ trieved from https://books.google musk-wozniak-hawking-ban-ai .nl/books?id=q8ijCwAAQBAJ -autonomous-weapons CBS. (2017, 08-01-2017). [Audio- Gunawardena, U. (2016). Legality of lethal visual Material]. Retrieved from autonomous weapons aka killer robots http://www.cbsnews.com/ [Web page]. Retrieved from https:// news/60-minutes-autonomous ssrn.com/abstract=2892447 -drones-set-to-revolutionize Horowitz, M. C. (2016). The ethics & morality -military-technology/ of robotic warfare: Assessing the debate Cheng, A., & Fleischmann, K. R. (2010). De- over autonomous weapons [Journal Arti- veloping a metainventory of human val- cle]. Daedalus, 145(4), 25-36. ues [Journal Article]. Proceedings of Hurka, T. (2005). Proportionality in the moral-

53 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

ity of war [Journal Article]. Philosophy & aspects in the structure and contents of Public Affairs, 33(1), 34-66. human values? [Journal Article]. Journal Kaag, J., & Kaufman, W. (2009). Mili- of social issues, 50(4), 19-45. tary frameworks: Technological know- Schwartz, S. H. (2012). An overview of the how and the legitimization of warfare schwartz theory of basic values [Journal [Journal Article]. Cambridge Review of Article]. Online readings in Psychology International Affairs, 22(4), 585-606. and Culture, 2(1), 11. Lin, J., & Singer, P. (2014). China’s new Stewart, P. (2016). [Web Page]. Retrieved military robots pack more robots inside from http://www.reuters.com/ (starcraft style) [Web page]. Retrieved article/us-usa-military-robot from http://www.popsci.com/ -ship-idUSKCN0X42I4 blog-network/eastern-arsenal/ Stewart, W. (2015). [Web Page]. Re- chinas-new-military-robots trieved from http://www.dailymail -pack-more-robots-inside .co.uk/news/article-3271094/ -starcraft-style Russia-turned-T-90-tank-robot Neapolitan, R. E., & Jiang, X. (2012). Contem- -plans-hire-gamers-fight porary artificial intelligence [Book]. CRC -future-wars.html Press. UNDIR. (2015). [Government Document]. Oehlert, G. W. (2010). A first course in design Retrieved from http://www.unidir and analysis of experiments [Book]. Re- .org/files/publications/pdfs/ trieved from https://conservancy considering-ethics-and-social .umn.edu/bitstream/handle/ -values-en-624.pdf 11299/168002/A%20First% US Airforce. (2016, 15 sep. 2016). [Au- 20Course%20in%20Design%20and% diovisual Material]. Retrieved from 20Analysis%20of%20Experiments https://www.youtube.com/ OehlertG 2010.pdf?sequence= watch?v=RQBFJkMnBcA 1&isAllowed=y Van de Poel, I. (2013). Translating values Roff, H. M. (2016). [Web Page]. Retrieved into design requirements [Book Section]. http://foreignpolicy.com/ from In Philosophy and engineering: Reflec- 2016/09/28/weapons-autonomy tions on practice, principles and process -is-rocketing/ (p. 253-266). Springer. Rosenberg, M. (2016). [Newspa- van der Hoven, J., & MandersHuits, N. (2009). per Article]. Retrieved from Valuesensitive design [Book]. Wiley On- http://www.nytimes.com/2016/ line Library. 10/26/us/pentagon-artificial -intelligence-terminator.html van Wynsberghe, A., & Robbins, S. (2014). ? r=0 Ethicist as designer: a pragmatic approach to ethics in the lab [Journal Ar- Royakkers, L., & Orbons, S. (2015). Design ticle]. Science and engineering ethics, for values in the armed forces: Nonlethal 20(4), 947-961. weapons weapons and military military robots robot [Journal Article]. Handbook Walsh, J. I., & Schulzke, M. (2015). The of Ethics, Values, and Technological De- ethics of drone strikes: Does reducing sign: Sources, Theory, Values and Appli- the cost of conflict encourage war? (Re- cation Domains, 613-638. port). DTIC Document. RT. (2017, 5-07-2017). (Web Page Williams, A. P., Scharre, P. D., & Mayer, C. No. 15-07-2017). Retrieved from (2015). Developing autonomous sys- https://www.rt.com/news/ tems in an ethical manner [Book Sec- 395375-kalashnikov-automated tion]. In Autonomous systems: Issues -neural-network-gun/ for defence policymakers. NATO Al- Russell, S., Norvig, P., & Intelligence, A. lied Command Transformation (Capabil- (1995). A modern approach [Journal Ar- ity Engineering and Innovation). ticle]. Artificial Intelligence. Prentice-Hall, Egnlewood Cliffs, 25. Schwartz, S. H. (1994). Are there universal

54 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Major Ilse Verdiesen has worked in Royal Nether- lands Armed Forces since 1995 and has been deployed to Bosnia and Afghanistan. She has a Master degree in Infor- mation Architecture from Delft University of Tech- nology and graduated in the ﬁeld of Responsible Artiﬁcial Intelligence under supervision of Dr. Virginia Dignum. She conducted her graduation project on the ethics of Autonomous Weapons at the Scalable Co- operation Lab of Dr. Iyad Rahwan at MIT Me- dia Lab.

55 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

The Ethics of Automated Behavioral Microtargeting Dennis G Wilson (IRIT, University of Toulouse; [email protected]) DOI: 10.1145/3137574.3139451

One day AM woke up and knew who he times scripted responses to financial cues, was, and he linked himself, and he began sometimes real AI decisions about stocks feeding all the killing data, until everyone made in fractions of seconds, has earned was dead, except for the five of us, and the ire of governments worldwide for creating AM brought us down here. volatile markets with flash crashes and decep- I was the only one still sane and whole. tive upswings. This threat is in the process of Really! AM had not tampered with my mitigation, the world now more wary of algo- mind. Not at all. rithmic decisions. I Have No Mouth and I Must Scream Elli- However, as politics have recently turned the son(1967) world upside, with a major force for globaliza- AI is undeniably powerful in its modern form. tion losing a key member, and a US President It has surpassed human performance in board who lost the popular vote by almost 3 million, games, trivia game shows, and even recogniz- it seems pertinent to investigate the role of AI ing other humans’ handwriting. Soon, it will be in politics. Issues abound in this field as they evident how much better at driving it is, and do in others, although separating the symp- then maybe at all forms of navigation. In its toms of AI from those of malevolent actors and wake, we are left to ponder the ethical and so- competing political factions is a daunting task. cial implications of the tools we have created. AI, by its definition, enables us. It is a tool. For a technology that began development, ar- The dystopian concerns of a hate-filled ma- guably, over 60 years ago, we are woefully chine manipulating and torturing humans are unprepared when it comes to ethical, social, nowhere near our reality. However, as a pow- and regulatory understanding of AI, and less erful and novel tool, the ways in which it en- so concerning precedent. ables us must be considered. By examining How should the work a robot does, especially the use of AI in political campaigning, it is ev- in the case of direct human job replacement, ident that AI can realize undesired potential. be taxed? How should AI contribute fiscally to Specifically, AI can be used to manipulate and society? Are they complicit in a social con- suppress human ideas. It can facilitate the for- tract, and are there basic rights that extend mation of ideological barriers that serve to di- to non-human intelligences? Who should be vide people. It can enable the concerted ef- held accountable for the actions of an AI, such forts of few to disrupt the marketplace of ideas. as the operator of a self-driving car? How These are the most pressing issues related to can AI’s power be equally distributed across AI technologies, and we must identify and ad- society, to ensure that all benefit and that dress them fully. it isn’t used to disadvantage select groups? These and more are now capturing the atten- The Personal Web tion of great thinkers from prestigious universities (Stone et al., 2016) to the White House Personalized content recommendation has (Executive Office of the President & Technol- long been a hallmark of AI success. The Net- ogy Council, 2016). flix Prize, a competition for predicting user ratings of films, was started in 2006 as an effort However, when tasked with finding the most to increase the quality of film recommenda- pressing issues related to AI, we must look tions. Based solely on user ratings of previ- to the present and to the impact AI has al- ously watched films, the competitors devised ready made. The self driving car fatality count algorithms to accurately predict what rating stands at one, which is unfortunate but far a user would give a new film, a metric then from pressing. High frequency trading, some- usable by Netflix to determine recommenda- Copyright c 2017 by the author(s). tion priority of this new film to the user. Re-

56 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 searchers from IBM, AT&T Labs, visionary erhouses,” they note, “will be able to micro- professor Geoffrey Hinton’s lab, and many oth- analyze and micro-serve content to increas- ers competed in this prestigious event, im- ingly specialized segments of the population proving upon and showcasing the power of down to the individual.” (Stone et al., 2016) what was then mostly called machine learn- These media powerhouses will be able to con- ing. trol, on a large scale and yet with a high level of specificity, the exposure to different prod- More than a decade later, personalized rec- ucts, media, and ideas. ommendation is an increasingly normal part of the web. Advertisers have long since under- The control of idea exposure is not the only stood the benefits of personalizing their mes- ethical issue exacerbated by the increasingly sage and targeting individuals based on intel- capable personality analysis performed regu- ligent personality analysis. Google and Face- larly online. The same data that determines book have been leaders in this market, with a user’s product interest can reveal private advertising revenues in the billions. The ad- details and identifying factors. As early as vertisement software platforms from Google, 2011, there was research showing AI capable AdSense and AdWords, accounted for 89% of of determining the political alignment of indi- the company’s revenue in 2014. Both com- viduals based on their Twitter data. (Conover, panies wield intelligent personality metrics to Goncalves, Ratkiewicz, Flammini, & Menczer, build their advertising renown. Both are now 2011). Even the seemingly innocuous Netflix heavily investing in AI. Prize was dogged for years after its termina- tion by lawsuits claiming that the users’ data Google’s AdSense uses the term matched had violated their privacy, with researchers content to describe showing advertisements able to identify a number of users from the to specifically profiled individuals. By their Netflix Prize datasets by cross-referencing claims, matched content recommendations in- user data from the Internet Movie Database. crease the number of pages viewed by 9% In 2013, research demonstrated that it was and the time spent on site by 10%. (Google, possible to recover a large amount of personal 2017a) Similarly, Facebook has Custom Au- information from Facebook Likes. In 88% of diences that advertisers can create for their the tested cases, an AI model correctly dis- campaign based on selected demographics. criminated between homosexual and hetero- Interestingly, Facebook also allows advertis- sexual men, between African Americans and ers to select target users, such as existing fol- Caucasian Americans in 95% of cases, and lowers of the product’s Page or previous vis- between Democrat and Republican in 85% itors to their site, to build a Lookalike Audi- of cases. Age, intelligence, gender, and the ence. In their words, “A Lookalike Audience personality metrics openness to experience, is a way to reach new people who are likely to conscientiousness, extraversion, agreeable- be interested in your business because they’re ness, and neuroticism (OCEAN) were also es- similar to people who already are.” (Facebook, timated from Like data with a high degree of 2017b) Similarity is commonly used in AI prob- certainty. (Kosinski, Stillwell, & Graepel, 2013) lem formulation as it simplifies multiple prob- This model is now available online for public lems, whether a user will be interested in spe- use.1 cific products, to a single one. While the ethical issue of privacy invasion in As AI capabilities increase, the ability of these this analysis is apparent, so is the marketing platforms to deliver very specifically personal- potential. More recent work from the same re- ized content increases. The capabilities of AI searcher uses the information gleaned from in media were discussed in a report from Stan- Likes to draw users to different posts, as ad- ford’s One Hundred Year Study on AI (AI100). vertisers would. The early findings show that The positives of entertainment that is more marketing using individual personality analy- interactive, personalized, and engaging were sis, deemed personality targeting or behav- considered, as was the potential for media ioral microtargeting, can attract up to 63% conglomerates to act as Big Brothers, control- ling the ideas and online experiences to which 1https://applymagicsauce.com/ specific individuals are exposed. “Media pow- demo.html

57 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 more clicks, a clear profit for advertisers. is the use of this technology that raises concerns, not its potential influence on outcome. In the development of AI, it is often neces- The CEO of Cambridge Analytica, Alexander sary to define a numeric goal for optimiza- Nix, argues in favor of behavioral microtarget- tion. In the aforementioned Netflix prize, it ing: was the accuracy of user movie rating predictions. Often in advertising, it is a proxy of Your behavior is driven by your personal- the interest generated by the ad, using met- ity and actually the more you can under- rics such as clicks through to an advertiser’s stand about peoples personality as psy- website. YouTube measures the amount of chological drivers, the more you can actu- time a user watches a video, and how much ally start to really tap in to why and how that contributes to a session of watching mul- they make their decisions, Nix explained tiple videos, to determine a video’s popular- to Bloombergs Sasha Issenburg. We call ity sorting. Specific videos are suggested to this behavioral microtargeting and this is users based on their ability to capture the at- really our secret sauce, if you like. This is tention of the user, elongating their session what were bringing to America. (Anderson and increasing their exposure to more adver- & Horvath, 2017) tisements. Yvonne Hofstetter, a lawyer, AI expert, and the Cambridge Analytica informed the campaigns Managing Director of Teramark Technologies of individuals who matched specific psycho- GmbH, writes logical profiles for canvassing. It analyzed communities to determine talking points and Even Google Search is a control strat- campaign strategies for visiting candidates. egy. When typing a keyword, a user re- “We can use hundreds or thousands of indi- veals his intentions. The Google search vidual data points on our target audiences to engine, in turn, will not just present a list understand exactly which messages are going with best hits, but a link list that embod- to appeal to which audiences,” Nix claimed in ies the highest (financial) value rather for a lecture on the Cruz campaign. (Nix, 2016) the company than for the user. Doing it that way, i.e. listing corporate offerings at Developing political strategies based on citi- the very top of the search results, Google zen information such as demographics is nei- controls the users next clicks. (Helbing, ther novel nor ethically questionable. How- Frey, Gigerenzer, & Hafen, 2017) ever, the use of AI has enabled profiling to a degree that violates citizen privacy. It is The intent of the actor performing behavioral founded on a basis of data that some would microtargeting comes through in this numeric argue belongs first to the citizens and only to goal, for which an AI can be optimized. While political campaigns with explicit consent. Most the ethical and regulatory issues surround- importantly, though, when this form of analysis ing the use of this technology for commer- is used to deliver personalized political con- cial purposes, such as advertising, are poten- tent, the diversity of opinions citizens are ex- tially troubling, the danger of this technology posed to becomes artificially limited. is apparent when other goals are considered. Facebook is not only a vehicle for advertise- In 2016, the data firm Cambridge Analytica ment. Of the 67% of American adults who use used behavioral microtargeting in two major Facebook, two thirds of them, being 44% of US political campaigns, that of Senator Ted the adult population, cite it as a part of their Cruz in the Republican primary, and of Don- news sources. Facebook is not the only social ald Trump. These campaigns, and how they media site that functions as a news source for used data-driven AI to profile and persuade Americans, but it is by far the largest in terms the public, are useful as a case study on the of reach. The majority of users, 64%, who get current issues surrounding behavioral micro- news from a social networking site rely solely targeting. The efficacy of microtargeting in the on that site for their news. Most commonly, mentioned political campaigns is not the focus that solitary news source is Facebook. (Got- of this work; while Senator Cruz lost the pri- tfried & Shearer, 2016) mary despite aid from AI, and many factors contributed to President Trump’s election, it Mark Zuckerberg is aware of the implications

58 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 of this reliance on Facebook. In a recent let- Automated Interaction ter to the community of Facebook, he detailed a plan to report and remove terrorist propa- In a lecture describing the company’s ap- ganda from the site. The plan involves us- proach during the Cruz campaign, Nix used ing AI to flag suspect content for administra- the example of a private beach owner show- tive review, a system that currently generates ing an intentionally misleading sign warning one-third of all reports to Facebook’s content of shark sightings as an example of behav- review team. (Zuckerberg, 2016) ioral communication, the new technique that trumps older techniques of informational com- However, the site is already widely used to munication, like a sign that states that the push political agendas. Facebook played a beach is private property. (Nix, 2016) While pivotal role in fundraising for the Trump cam- Cambridge Analytica itself did not appear to paign and was a main focus of their advertise- support any intentional misleading during the ment. User feedback from political ads, such campaign, it became a focal issue in a cam- as clicks or shares, informed the usage of over paign based on behavioral communication. forty thousand different ad variants the campaign used. Many have argued that this bar- Large-scale manipulation of public opinion rage of tightly focused advertisement lead to and understanding is a growing ethical issue the creation of virtual echo chambers, spaces related to AI. While much of the existing threat where a limited set of ideas were constantly is due simply to automation, bots that have reinforced. (Anderson & Horvath, 2017) no independent intelligence, the potential for damage is already visible. AI is poised to re- The social issues at hand were captured well place existing bots and worsen this issue if al- in an article of Scientific American: lowed. To illustrate this potential, further examples of political manipulation are shown. In order for manipulation to stay unno- Standing less popular nationally than Face- ticed, it takes a so-called resonance ef- book, Twitter is used by 16% of US adults. fect suggestions that are sufficiently cus- Of those, 56% use the site as a news source. tomized to each individual. In this way, lo- Twitter is an attractive platform for automated cal trends are gradually reinforced by rep- users, or bots, as there are accessible appli- etition, leading all the way to the ”filter cation programming interfaces (APIs) in multi- bubble” or ”echo chamber effect”: in the ple programming languages, and programs for end, all you might get is your own opin- tasks such as tweet repetition and automatic ions reflected back at you. This causes liking. social polarization, resulting in the formation of separate groups that no longer un- By simply selecting random popular words derstand each other and find themselves and parroting other users’ tweets, one re- increasingly at conflict with one another. searcher’s Twitter bot was able to reach in- In this way, personalized information can fluence scores close to celebrities and higher unintentionally destroy social cohesion. than many human users. (Messias, Schmidt, (Helbing et al., 2017) Oliveira, & Benevenuto, 2013) This bot was intended to deceive human users in to be- While geographic boundaries or social class lieving it was also human, and it appears to have in the past limited the landscape of ideas have succeeded. The difficulty of separating available to individuals, it is surprising that this a bot, even a simple scripted one, from a hu- issue has resurfaced in the Age of Informa- man user on Twitter is so difficult that mod- tion. As this divide was enabled by novel tech- ern AI has been utilized to perform the task. nologies, among them the AI used in behav- BotOrNot2 uses random decision forests, an ioral microtargeting, it is fitting that we evalu- AI classification technique, to determine if a ate the appropriate use of these technologies Twitter user is a bot or not. (Davis, Varol, Fer- and propose methods to maximize their soci- rara, Flammini, & Menczer, 2016) etal benefit. This will be discussed further in With the difficulty of discerning humans from section 3. First, however, we will highlight the bots on the platform, and the ease with which technologies used to create these echo chambers and to push specific messages. 2https://botometer.iuni.iu.edu/

59 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 bots can be created and updated, the stage social network with other humans. Humans is set for technical users to exert influence use social information to modify their behav- far beyond what their single human account ior and make decisions, and when that social could have. Bots have been shown to partic- information is easily manipulated, human de- ipate and potentially manipulate Venezuelan cision can also be manipulated. (Bhanji & Del- politics on Twitter, with nearly 10 percent of gado, 2014) all politician retweets coming from bot-related platforms. The most active bots in this study Networks can be created with a high density were those used by Venezuelas radical oppo- of bots, or to connect individuals who have sition. (Forelle, Howard, Monroy-Hernandez, similar personality traits seen by a campaign & Savage, 2015) as exploitable. Users already tend to aggre- gate around common interests in a phenom- Political bots were also highly active during ena known as homophily, but this can be en- the 2016 US election, perhaps unprecedent- hanced with automated users that link previ- edly so. Highly automated pro-Trump activ- ously unknown users together via follows and ity increased until the final results, outnum- retweets. Echo chambers can be created bering pro-Clinton bot activity 5:1. (Kollanyi, with a mix of bots and human users, unknow- Howard, & Woolley, 2016) One group, using ingly selected together. Beyond limiting their BotOrNot, found that roughly 400,000 bots en- exposure to ideas, this type of organization gaged in political discussion about the Pres- has been show to facilitate rumor spreading idential election, responsible for roughly 3.8 (Aiello et al., 2012). Polarization is another million tweets, about one-fifth of the entire factor in misinformation spreading (Anagnos- conversation. (Bessi & Ferrara, 2016) topoulos et al., 2014), meaning a campaign The AI100 report details one of the ethical is- with knowledge of polarized individuals, based sues of this trend: on behavioral analysis, could facilitate rumor spreading by linking these individuals with au- AI technologies are already being used tomated accounts that reinforce desired ru- by political actors in gerrymandering and mors. targeted robocalls designed to suppress This is not a new phenomena. The technol- votes, and on social media platforms in ogy behind these bots is far from sophisti- the form of bots. They can enable co- cated, and more technical AI has been used to ordinated protest as well as the ability study it. Truthy, an earlier project of the same to predict protests, and promote greater team that created BotOrNot, used SVM and transparency in politics by more accu- AdaBoost to determine how factual a trend- rately pinpointing who said what, when. ing idea was. (Ratkiewicz et al., n.d.) In the Thus, administrative and regulatory laws course of this study, they noted the alarming regarding AI can be designed to promote ease with which false information could be en- greater democratic participation or, if ill- couraged to spread widely on Twiter. conceived, to reduce it. (Stone et al., 2016) Even while presenting honest content from human users, the combination of automa- However, there is a specific danger in the com- tion with behavioral microtargeting has trou- bination of behavioral microtargeting and the bling consequences, and it is not restricted use of bots: users can be targeted by other to Twitter. The platform’s automation acces- seemingly human users for coercion and idea sibility facilitates it, but these tactics are pos- suppression. In a psychology study, anony- sible on other platforms as well. While Face- mous users were more likely to make riskier book strictly verifies the identities of its users, gambles if they knew other users had chosen posts can automated to maximally convey to do so, even if the other users were anony- their message. The phrasing and presentation mous strangers. (Chung, Christopoulos, King- of a post, regardless of its content, has been Casas, Ball, & Chiu, 2015) The reward mech- shown to affect its potential for spreading. (Al- anism of targeted users can be manipulated habash et al., 2013) by artificially inflating retweets or likes of their posts, which will then inform their future be- Facebook’s automated rules allow advertis- havior, and artificially raise their standing in a ing campaigns to create rules that modify

60 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 their advertisement based on assigned condi- making. tions. (Facebook, 2017a) While this can be For this reason, there is currently a human as simple as stopping an ad if it isn’t per- barrier between the two technologies. Person- forming well, Cambridge Analytica appears to alities are analyzed using AI, and then a hu- have done much more complex automated man actor uses the information to decide and advertisement administration. Based on the design automated strategies on social media. ads selected by users, content was added Cambridge Analytica is an exception to this, to their feed in posts personalized for them, as their posts seem to be selected from a determined by their behavior profile. Auto- large pool based on input from the analyti- matically selecting from the thousands of ad cal techonology, but this selection is also rudi- variants available, these rules targeted spe- mentary compared to state of the art genera- cific individuals and seem to have created the tive AI. same echo chambers as described in Twitter. (Grassegger & Krogerus, 2017) Even without When this gap between the technologies is bots, these tightly networked groups are still closed by AI, and fully autonomous processes restricted from exposure to a diversity of opin- go from personality profiling to specialized ions and are susceptible to the spread of false content delivery and generation, we must information. (Anagnostopoulos et al., 2014) have well established guidelines for the ethics of such systems. Twitter bots and Facebook ad manipulation are not using state of the art AI and natural language processing, for the most part. Propositions Some bots aren’t even fully artificial. Users To address these concerns, proposed direc- like Daniel Sobieski have automated programs tions for the government, industry, and public that tweet more than 1,000 times a day us- organizations and academia are examined in ing schedulers that work through a queue of the next three section. These issues can not their previously written tweets. (As a conser- be resolved by any one sector alone. Rather, vative Twitter user sleeps, his account is hard there must be a coordinated effort of those at work, 2017) While scripts are far from AI, that work with AI in all three sectors. While their use informs a discussion on human ma- there are many other strides that could be chine interaction that is vital as AI capabili- taken to address the issues related to AI, ties increase. Microsoft’s disastrous attempt the initiatives proposed below are those best at a teenage Twitter chatbot, Tay, must be suited to combat the pressing issues raised in given credit for creating seemingly human re- this article. sponses, albeit tainted by the preferences of users that hijacked the experiment. As bots on social media gain increased social capa- Government bility, and as artificially generated content fur- The current drive of AI is data. The compa- ther resembles human generated content, our nies that own the most data have been mak- interactions on social media must be well in- ing the greatest strides in AI, and this data is formed. largely generated by their users. The Euro- pean Union has been a powerful force in coun- The ethical issue at hand is therefore the large tering corporate data ownership by declaring scale manipulation of human ideas, opinions, citizen’s rights over their data. Governments and agency using AI. The same technologies must continue to enforce and expand this type have created anew the social issue of ideo- of law. The right of a person to all of the data logically isolated communities, now manufac- associated with their identity, and the agency tured artificially to reduce opinion diversity and of each person to control that data, must be facilitate misinformation. The first technology respected. behind these issues is the powerful personality analysis now possible due to greater data Second to that is the funding of AI initiatives. availability and more accurate AI. The sec- While there is a surplus of funding for the de- ond is automation on social media platforms, velopment of AI, it mostly fits the individual which, for now, is mostly rudimentary script- desires of the company using that AI. Qual- ing and does not resemble intelligent decision ity AI research that doesn’t appear to have

61 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017 a corporate application should be supported sites, should be cautious about applying in- by the government. Furthermore, research house AI to combat what they see as nega- into the study of AI itself and how it affects tive trends in their content or user interaction. society must be done without corporate inﬂu- The numerical optimization used in AI should ence. The Obama Administration was very be considered carefully, as overemphasis on supportive of increasing AI research funding. a particular metric or disregard of another (Executive Ofﬁce of the President & Technol- can have drastic consequences once that op- ogy Council, 2016) timized AI is put to use. Zuckerberg’s community initiative, while seemingly well-intended, Lastly, the government must apply strict ad- comes off as nave in its understanding of both vertisement laws to new forms of marketing as AI and democracy, as AI must enforce local AI continues to change marketing. By requir- Community Standards by potentially limiting ing that advertisements are clearly marked as content while simultaneously fostering open such, the issue of unaware manipulation be- democratic policies. (Zuckerberg, 2016) comes much less concerning.

Industry Industries can not be expected to sacrifice po- Organizations and Academia tential profits by not utilizing the powerful user analysis enabled by AI. Nor can they be ex- Organizations and academia have the great- pected to invest their resources in endeavors est potential to shape the discourse around AI. that do not benefit them in return. However, in A first major step in that is recognizing, as the the event of government reforms of data pol- AI100 initiative has, that what we call AI is a icy, it would be in the interest of companies moving target, and is often one placed just out to develop tools that allow users to person- of reach. (Stone et al., 2016) Various forms ally perform the type of analysis being done of AI have existed and been in use for half a with their data currently. For example, on century, but there is a great hesitation to call Google News, specific news items are sug- something that would have been considered gested based on personality. Users can dis- AI 10 years ago the same now. Furthermore, able these articles and they can modify their technologies that were or are outside the tech- interests in different pre-selected categories, nical term’s strict definition should be consid- but they can only influence the personality ered when discussing the impact of AI, such metrics Google has built around them by indi- as the Twitter automation scripts discussed in cating their interest for or against new articles. this article. (Google, 2017b) This does not afford the user understanding nor control of their data. Academics can also fill in the research gaps Furthermore, industries that allow automated that industry will not, and organizations can users to interact with human users need to support this effort. The BotOrNot and Truthy make dedicated efforts to allow their human applications are examples of useful tools out- users to distinguish between actions of bots side the corporate interest of the platform they and of humans. While tools such as BotOrNot interact with. By making these tools indepen- are good research efforts, they should not dently and available to the public, society is be necessary. An example of good policy able to better understand the tool it is wield- is found in Slack, a messaging app that al- ing. lows for bots and software service integration. Lastly, but importantly, organizations and Bot users are clearly marked, even though academia must remain independent and unbi- they come from a variety of automation tools. ased in their evaluation of industrial and gov- The distinction is important; human reaction ernmental use of AI. The dangers of good AI in video games has been markedly different in the wrong hands have already been demon- when players are aware that the opponent is strated, and they can come from a number a bot as opposed to a human. (Smith & Del- of sources. Independent organizations must gado, 2015) support academic research into the fair and Lastly, media sites, including social network appropriate use of AI in all sectors.

62 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Conclusion the-rise-of-the-weaponized-ai -propaganda-machine. Power is in tearing human minds to pieces As a conservative twitter user sleeps, his and putting them together again in new account is hard at work. (2017). shapes of your own choosing. https://www.washingtonpost 1984 Orwell(1949) .com/business/economy/ as-a-conservative-twitter This is not the dystopia Ellison nor Orwell -user-sleeps-his-account imagined. Efforts are being made to check -is-hard-at-work/2017/02/ the use of private data. Social media is re- 05/18d5a532-df31-11e6-918c evaluating its newfound place as the news -99ede3c8cafa story.html?utm provider to many. People are learning to think term=.0938a67a1ce5. critically about what they see online before Bessi, A., & Ferrara, E. (2016). Social bots believing it. Some in the private sector, like distort the 2016 u.s. presidential election Google, have offered their research capabili- online discussion. First Monday, 21(11). ties in the form of open source code and pub- http://journals.uic.edu/ojs/ lications. All of these are marked progress index.php/fm/article/view/ against the concerns of AI manipulating and 7090. suppressing human ideas, slicing up the mar- Bhanji, J. P.,& Delgado, M. R. (2014). The so- ketplace of ideas into small despots. cial brain and reward: Social information processing in the human striatum. Wiley Still, there is a ways to go and basic attitudes Interdisciplinary Reviews: Cognitive Sci- must change. The tech mantra of “Move fast ence, 5(1), 61–73. and break things” must give way to cautious, Chung, D., Christopoulos, G. I., King-Casas, considered approaches when AI is concerned. B., Ball, S. B., & Chiu, P. H. (2015). Whether the fault of technology, and specif- Social signals of safety and risk con- ically AI, or not, things have broken enough fer utility and have asymmetric effects already. on observers’ choices. Nature neuroscience(6), 912–6. Conover, M. D., Goncalves, B., Ratkiewicz, References J., Flammini, A., & Menczer, F. (2011, Aiello, L. M., Barrat, A., Schifanella, R., Cat- oct). Predicting the Political Alignment tuto, C., Markines, B., & Menczer, F. of Twitter Users. In 2011 ieee third (2012). Friendship prediction and ho- int’l conference on privacy, security, risk mophily in social media. ACM Transac- and trust and 2011 ieee third int’l con- tions on the Web, 6(2), 1–33. ference on social computing (pp. 192– Alhabash, S., McAlister, A. R., Hagerstrom, 199). IEEE. http://ieeexplore A., Quilliam, E. T., Rifon, N. J., & .ieee.org/document/6113114/. Richards, J. I. (2013). Between Likes Davis, C. A., Varol, O., Ferrara, E., Flammini, and Shares: Effects of Emotional Appeal A., & Menczer, F. (2016). BotOrNot: A and Virality on the Persuasiveness of An- System to Evaluate Social Bots. arXiv, ticyberbullying Messages on Facebook. : 1602.009, 2. http://arxiv.org/ Cyberpsychology, Behavior, and Social abs/1602.00975. Networking(3), 130201060931000. Ellison, H. (1967). I have no mouth and i must Anagnostopoulos, A., Bessi, A., Caldarelli, G., scream. Galaxy Publishing Corp. Del Vicario, M., Petroni, F., Scala, A., Executive Ofﬁce of the President, N. S., . . . Quattrociocchi, W. (2014). Viral & Technology Council, C. o. T. Misinformation: The Role of Homophily (2016). Preparing for the future and Polarization. arXiv:1411.2893, 1– of artiﬁcial intelligence. https:// 12. http://arxiv.org/abs/1411 obamawhitehouse.archives .2893. .gov/sites/default/files/ Anderson, B., & Horvath, B. (2017). The rise whitehouse files/microsites/ of the weaponized ai propaganda ma- ostp/NSTC/preparing for the chine. https://scout.ai/story/ future of ai.pdf.

63 AI MATTERS, VOLUME 3, ISSUE 3 SUMMER 2017

Facebook. (2017a). Auto- Orwell, G. (1949). 1984. Secker and War- mated rules. https://www burg. .facebook.com/business/help/ Ratkiewicz, J., Conover, M., Meiss, M., 247173332297374/. Gonçalves, B., Patil, S., Flammini, A., & Facebook. (2017b). Lookalike audiences. Menczer, F. (n.d.). Detecting and Track- https://www.facebook.com/ ing the Spread of Astroturf Memes in business/help/231114077092092. Microblog Streams. Proceedings of the Forelle, M. C., Howard, P. N., Monroy- 20th International Conference Compan- Hernandez, A., & Savage, S. (2015). Po- ion on World Wide Web, 249–252. litical Bots and the Manipulation of Public Smith, D. V., & Delgado, M. R. (2015). So- Opinion in Venezuela. SSRN Electronic cial nudges: utility conferred from others. Journal, 1–8. Nature Neuroscience(6), 791–792. Google. (2017a). Matched content. Stone, P., Brooks, R., Brynjolfsson, E., Calo, https://support.google.com/ R., Etzioni, O., Hager, G., . . . others adsense/answer/6111336?hl= (2016). Artificial intelligence and life in en&ref topic=6111161. 2030. One Hundred Year Study on Ar- Google. (2017b). Personalize your news tificial Intelligence: Report of the 2015- settings. https://support.google 2016 Study Panel. .com/news/answer/1146405?hl= Zuckerberg, M. (2016). Building global en&ref topic=2428815. community. https://www.facebook Gottfried, J., & Shearer, E. (2016). News .com/notes/mark-zuckerberg/ use across social media platforms 2016. building-global-community/ http://www.journalism.org/ 10103508221158471/. 2016/05/26/news-use-across . -social-media-platforms-2016/. Grassegger, H., & Krogerus, M. (2017). The data that turned the world upside down. https://motherboard.vice.com/ Dennis G Wilson is a en us/article/how-our-likes PhD candidate at the -helped-trump-win. Institut de Recherche en Helbing, D., Frey, B. S., Gigerenzer, G., Informatique de Toulouse & Hafen, E. (2017). Will democ- studying artificial neural racy survive big data and arti- models, after studying ficial intelligence? https:// evolutionary computation www.scientificamerican.com/ and gene regulatory networks at the Anyscale article/will-democracy Learning For All group at CSAIL, MIT. Their -survive-big-data-and professional and personal information can be -artificial-intelligence/. found at https://d9w.xyz/ Kollanyi, B., Howard, P. N., & Woolley, S. C. (2016). Bots and Automation over Twitter during the First U.S Presidential Debate. Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the Na- tional Academy of Sciences of the United States of America, 110(15), 5802–5. Messias, J., Schmidt, L., Oliveira, R., & Ben- evenuto, F. (2013). You followed my bot! transforming robots into influential users in twitter. First Monday, 18(7). Nix, A. (2016). The power of big data and psy- chographics. https://www.youtube .com/watch?v=n8Dd5aVXLCc.

64 RADIO AI - CALL FOR PARTICIPATION 2018

RADIO AI: An Important Invitation to AI Educators and Professionals to Talk About Your Work, Ideas and Thoughts On AI

Call for Participation: This invitation is extended across all the subfields of AI. The purpose of the RADIO AI project is to help educate the public and other professions about AI, with a crowd sourced collective of podcasts by people who work on AI. Submit podcasts in .mp3 or .wav by email to [email protected] See http://www.radioai.net for examples.

Some of the people talking the loudest about AI right now don’t actually work on AI. We hope to hear from many individuals who wish to counter fear of AI by educating the public with lighthearted informative podcasts. People who are passionate about their work in AI tend to see the good it can do to help society, the environment, healthcare, and all aspects of life. We hope you will share this vision with others. Our goal is to bring together the collective vision through short podcasts and create easy to understand informative lectures by the people who work in AI - academia, business, finance, healthcare, inventors, and programmers. The lectures can be as short as 3 minutes or as long as 10. Some of you might wish to do a series of podcasts.

Deadlines:

Intent to submit: Nov. 1, 2017

Notifications Due: Dec. 1, 2017

Draft podcasts due: Dec. 12, 2017

Final podcasts due: Jan. 12, 2017

Audience: Please gage the audience as either teenage or in another field of study that is non-technical.

Directions: Example podcasts are located on www.radioai.net All you need is some peace and quiet, a microphone and your inspired thinking.

Topics:

History of AI, Software agents, Robots, Machine learning, Fuzzy AI, Overview of AI, AI and Society, Human-Robot Interaction and Applications - Healthcare, Legal, Transportation, Energy, Environment, etc. All topics related to AI are welcome. Other - if you're working in this field and are aware of the social changes that will or already have been taking place, we welcome your insights for the public.

Travel/hotel: There is no travel or hotel reservation required.

Contact: Dr. C. Mason [email protected]