Forecasting Transformative AI by Richard Ross Gruetzemacher A dissertation submitted to the Graduate Faculty of Auburn University in partial fulfillment of the requirements for the Degree of Doctor of Philosophy Auburn, Alabama August 8, 2020 Forecasting, Artificial Intelligence, Transformative AI Copyright 2020 by Richard Ross Gruetzemacher Approved by David Paradice, Chair, Harbert Eminent Scholar, Department of Systems and Technology Dianne Hall, Torchmark Professor, Department of Systems and Technology Kang Bok Lee, EBSCO Associate Professor, Department of Systems and Technology Kelly Rainer, George Phillips Privett Professor, Department of Systems and Technology “The whole purpose of science is to find meaningful simplicity in the midst of disorderly complexity.” Herbert Simon (1991) For my parents. Thank you for all of your support over the years. ii Abstract Technological forecasting is challenging, and the forecasting of AI progress is even more challenging. Forecasting transformative AI – AI that has great potential for societal transformation – which presents even more difficult challenges. This study addresses the research question of “How can we best forecast transformative AI?” To this end it explores a wide variety of techniques that are used for technological forecasting, demonstrates their viability in the context of transformative AI and evaluates their value for use in future efforts to forecast transformative AI. The study focuses on scenario analysis techniques, a variety of judgmental forecasting techniques and simple statistical forecasting techniques. These include a survey, interviews, bibliometric analysis and the Delphi technique. The literature review identifies a new subclass of scenario planning techniques called scenario mapping techniques. These techniques are well- suited for forecasting transformative AI because of their incorporation of large numbers of scenarios (i.e., future technologies) in directed graphs. Two novel techniques that meet the criteria of scenario mapping techniques are developed and demonstrated. The two methods, along with a variety of other forecasting techniques, are components of a proposed holistic forecasting framework for transformative AI. More generally, the holistic forecasting framework suggests that a combination of judgmental, statistical and scenario analysis techniques are necessary for forecasting complex future technologies such as transformative AI. This study is the first of which we are aware of to demonstrate the use of a holistic forecasting framework in any context. However, another framework(s) that satisfies the proposed criteria is identified and discussed. The results of the study include forecasts generated from two of the techniques demonstrated along with significant insights for future work on forecasting transformative AI. A iii research agenda for forecasting AI progress was also created in demonstrating and evaluating the Delphi technique for the purpose of using expert elicitation to generate questions of interest for use as forecasting targets in prediction markets or forecasting tournaments. The results from the survey also generate insights into limitations of existing methods used for future of work research. The study produces numerous novel contributions including theoretical elements from the definition of transformative AI, the holistic forecasting framework, the two novel methods and the concept of the subclass of scenario mapping methods. Beyond this, a large variety of insights are identified and reported as a result of the demonstration of the various techniques. Moreover, elicitation of some of the world’s leading experts on specific subdisciplines of AI yields some significant insights regarding future progress in AI. iv Acknowledgments I would like to thank Ashish Gupta for choosing to work with me, helping me to realize my potential for academic research and for helping me to put myself in a position to work on the world’s most important problems. I would also like to thank David Paradice for letting me pursue my interests and for supporting me in a very non-traditional course of study. I would like to thank Shahar Avin, as well, for giving me an opportunity to collaborate on a much larger project that enabled me to further enhance the research conducted for this dissertation by strengthening my network and making available incredible opportunities that would have otherwise been out of reach. I also thank Kang Bok Lee, Dianne Hall and Kelly Rainer for agreeing to serve on my committee and for supporting this unique research topic. This section would be woefully incomplete if I failed to acknowledge the contributions of Abi Arabshahi, who, upon completion of my master’s study on computational fluid dynamics encouraged me earnestly to pursue my passion (as he has continued to do since leaving, also), even against his own judgement, rather than continuing to work with him for my doctoral studies. Without the guidance and support of these academics at different stages of my 9 years of postgraduate study, this dissertation, and the body of research that has resulted from it, would not have been possible. Lastly, I would like to thank undergraduate instructors who initially helped me to realize my potential for post graduate studies: James Hiestand, Lucas Van der Merwe, Cecelia Wigal, Ron Bailey, Matt Matthews, Ron Goulet and Ed McMahon. Regarding this study, I first would also like to thank all of the organizers of AI conferences where I collected data for consenting to me attending and soliciting participants for my study. This was a requirement for IRB approval (protocol #17-484), and I was only unable to get an affirmative response for one event. I also owe great thanks to all of the survey participants, interviewees, v workshop participants, Delphi participants and other experts who I spoke to over the course of this study. Without their willingness to engage with a PhD student from outside their own discipline for no publicity or recognition the completion of this expansive study would not have been possible. Some of the work herein comes from prior publications, and some of this work was conducted with collaborators. Despite this, the major ideas presented here are my own. However, framing of these ideas and execution decisions for some of the elements of this study involved contributions from others. Particularly, in Chapter 2 Jess Whittlestone’s contributions greatly enhanced the end result and I owe her great thanks. Also, in Chapter 7, and the other sections related to the Delphi’s execution and analysis, I would like to thank my team, including Flo Dorner, Niko Bernaola-Alvarez and Charlie Giattino. Without their contributions the result would have been of lesser quality and would likely have been replaced with a poorly executed and low value Delphi demonstration in a less impactful context. I would also like to thank Danila Medvedev for helping to organize the scenario network mapping workshop in Moscow, and for his other numerous contributions in adapting the workshopping technique to be implemented using the NeyroKod concept mapping software package (also for granting me access to NeyroKod). Finally, I have a list of fellow researchers in the AI research community, the AI strategy research community and the Effective Altruism community who (collectively) advised me on all different aspects of this study (order does not denote level of contribution): Jordi Bieger, Miles Brundage, Ryan Carey, Zoe Cremer, Eric Drexler, Adam Gleave, Ozzie Gooen, Danny Hernandez, Jose Hernandez-Orallo, Luke Muehlhauser, Alan Porter, Mark Ring and Jan Youtie. Because of the breadth of this study I have surely left out some parties who are due acknowledgement here, and if this is indeed the case, I apologize. vi Table of Contents Abstract ........................................................................................................................................ iii Acknowledgments......................................................................................................................... v List of Tables ............................................................................................................................. xiv List of Figures ............................................................................................................................. xv List of Abbreviations ............................................................................................................... xviii Chapter 1: INTRODUCTION ...................................................................................................... 1 1.1 Artificial Intelligence .................................................................................................. 2 1.2 Artificial General Intelligence .................................................................................... 7 1.3 Transformative AI ...................................................................................................... 9 1.4 The Present Study ....................................................................................................... 9 1.5 Summary ................................................................................................................... 12 Chapter 2: DEFINING TRANSFORMATIVE AI .................................................................... 13 2.1 Background ............................................................................................................... 13 2.1.1 Notions