Artificial Intelligence As a Positive and Negative Factor in Global Risk
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Infinite Ethics
INFINITE ETHICS Nick Bostrom Faculty of Philosophy Oxford University [Published in Analysis and Metaphysics, Vol. 10 (2011): pp. 9-59] [This is the final version. Earlier versions: 2003, 2005, 2008, 2009] www.nickbostrom.com ABSTRACT Aggregative consequentialism and several other popular moral theories are threatened with paralysis: when coupled with some plausible assumptions, they seem to imply that it is always ethically indifferent what you do. Modern cosmology teaches that the world might well contain an infinite number of happy and sad people and other candidate value-bearing locations. Aggregative ethics implies that such a world contains an infinite amount of positive value and an infinite amount of negative value. You can affect only a finite amount of good or bad. In standard cardinal arithmetic, an infinite quantity is unchanged by the addition or subtraction of any finite quantity. So it appears you cannot change the value of the world. Modifications of aggregationism aimed at resolving the paralysis are only partially effective and cause severe side effects, including problems of “fanaticism”, “distortion”, and erosion of the intuitions that originally motivated the theory. Is the infinitarian challenge fatal? 1. The challenge 1.1. The threat of infinitarian paralysis When we gaze at the starry sky at night and try to think of humanity from a “cosmic point of view”, we feel small. Human history, with all its earnest strivings, triumphs, and tragedies can remind us of a colony of ants, laboring frantically to rearrange the needles of their little ephemeral stack. We brush such late-night rumination aside in our daily life and analytic 1 philosophy. -
Apocalypse Now? Initial Lessons from the Covid-19 Pandemic for the Governance of Existential and Global Catastrophic Risks
journal of international humanitarian legal studies 11 (2020) 295-310 brill.com/ihls Apocalypse Now? Initial Lessons from the Covid-19 Pandemic for the Governance of Existential and Global Catastrophic Risks Hin-Yan Liu, Kristian Lauta and Matthijs Maas Faculty of Law, University of Copenhagen, Copenhagen, Denmark [email protected]; [email protected]; [email protected] Abstract This paper explores the ongoing Covid-19 pandemic through the framework of exis- tential risks – a class of extreme risks that threaten the entire future of humanity. In doing so, we tease out three lessons: (1) possible reasons underlying the limits and shortfalls of international law, international institutions and other actors which Covid-19 has revealed, and what they reveal about the resilience or fragility of institu- tional frameworks in the face of existential risks; (2) using Covid-19 to test and refine our prior ‘Boring Apocalypses’ model for understanding the interplay of hazards, vul- nerabilities and exposures in facilitating a particular disaster, or magnifying its effects; and (3) to extrapolate some possible futures for existential risk scholarship and governance. Keywords Covid-19 – pandemics – existential risks – global catastrophic risks – boring apocalypses 1 Introduction: Our First ‘Brush’ with Existential Risk? All too suddenly, yesterday’s ‘impossibilities’ have turned into today’s ‘condi- tions’. The impossible has already happened, and quickly. The impact of the Covid-19 pandemic, both directly and as manifested through the far-reaching global societal responses to it, signal a jarring departure away from even the © koninklijke brill nv, leiden, 2020 | doi:10.1163/18781527-01102004Downloaded from Brill.com09/27/2021 12:13:00AM via free access <UN> 296 Liu, Lauta and Maas recent past, and suggest that our futures will be profoundly different in its af- termath. -
Comments to Michael Jackson's Keynote on Determining Energy
Comments to Michael Jackson’s Keynote on Determining Energy Futures using Artificial Intelligence Prof Sirkka Heinonen Finland Futures Research Centre (FFRC) University of Turku ENERGIZING FUTURES 13–14 June 2018 Tampere, Finland AI and Energy • Concrete tools: How Shaping Tomorrow answers the question How can AI help? • Goal of foresight crystal clear: Making better decisions today Huge Challenge – The Challenge The world will need to cut energy-related carbon dioxide emissions by 60 percent by 2050 -even as the population grows by more than two billion people Bold Solution on the Horizon The Renewable Energy Transition Companies as pioneers on energy themes, demand, supply, consumption • Google will reach 100% RE for its global operations this year • GE using AI to boost different forms of energy production and use tech-driven data reports to anticipate performance and maintenance needs around the world BUT …also the role of governments, cities and citizens …NGOs, media… new actors should be emphasised AI + Energy + Peer-to-Peer Society • The transformation of the energy system aligns with the principles of a Peer-to-Peer Society. • Peer-to-peer practices are based on the active participation and self-organisation of citizens. Citizens share knowledge, skills, co-create, and form new peer groups. • Citizens will use their capabilities also to develop energy-related products and services Rethinking Concepts Buildings as Power stations – global (economic) opportunity to construct buildings that generate, store and release solar energy -
Less Wrong Sequences Pdf
Less wrong sequences pdf Continue % Print Ready Lesswrong Sequences Print Friendly Versions of Lesswrong Sequence, Enjoy! The basic sequences of Mysterious Answers to Mysterious Questions How to See through many disguises of answers or beliefs or statements that do not respond or say or mean nothing. The first (and probably most important) main sequence on Less Wrong. the epub pdf-version of the reductionism discount the second core sequence is less wrong. How to make the reality apart... and live in this universe where we have always lived without feeling frustrated that complex things are made of simpler things. Includes zombies and Joy in just real subsequences epub (en) pdf-version marking quantum physics is not a mysterious introduction to quantum mechanics, designed to be available to those who can grok algebra and complex numbers. Cleaning up the old confusion about SM is used to introduce basic issues into rationality (such as the technical version of Occam's Razor), epistemology, dredonism, naturalism, and philosophy of science. Do not dispense reading, although the exact reasons for the retreat are difficult to explain before reading. epub pdf-version of the markup Fun Theory is a specific theory of transhuman values. How much pleasure there is in the universe; We will someday run out of fun; We have fun yet; we could have more fun. Part of the complexity of the value thesis. It is also part of a fully general response to religious theododicy. epub pdf-version marking Minor sequences smaller collection of messages. Usually parts of the main sequences that depend on some, but not all points are entered. -
Global Catastrophic Risks
Global Catastrophic Risks Edited by Nick Bostrom Milan M. Cirkovic OXPORD UNIVERSITY PRESS Contents Acknowledgements ν Foreword vii Martin J. Rees 1 Introduction 1 Nick Bostrom and Milan M. Cirkoviô 1.1 Why? 1 1.2 Taxonomy and organization 2 1.3 Part I: Background 7 1.4 Part II: Risks from nature 13 1.5 Part III: Risks from unintended consequences 15 1.6 Part IV: Risks from hostile acts 20 1.7 Conclusions and future directions 27 Part I Background 31 2 Long-term astrophysics! processes 33 Fred C. Adams 2.1 Introduction: physical eschatology 33 2.2 Fate of the Earth 34 2.3 Isolation of the local group 36 2.4 Collision with Andromeda 36 2.5 The end of stellar evolution 38 2.6 The era of degenerate remnants 39 2.7 The era of black holes 41 2.8 The Dark Era and beyond 41 2.9 life and information processing 43 2.10 Conclusion 44 Suggestions for further reading 45 References 45 3 Evolution theory and the future of humanity 48 Christopher Wills· 3.1 Introduction 48 3.2 The causes of evolutionary change 49 xiv Contents 3.3 Environmental changes and evolutionary changes 50 3.3.1 Extreme evolutionary changes 51 3.3.2 Ongoing evolutionary changes 53 3.3.3 Changes in the cultural environment 56 3.4 Ongoing human evolution 61 3.4.1 Behavioural evolution 61 3.4.2 The future of genetic engineering 63 3.4.3 The evolution of other species, including those on which we depend 64 3.5 Future evolutionary directions 65 3.5.1 Drastic and rapid climate change without changes in human behaviour 66 3.5.2 Drastic but slower environmental change accompanied by changes in human behaviour 66 3.5.3 Colonization of new environments by our species 67 Suggestions for further reading 68 References 69 4 Millennial tendencies in responses to apocalyptic threats 73 James J. -
Beneficial AI 2017
Beneficial AI 2017 Participants & Attendees 1 Anthony Aguirre is a Professor of Physics at the University of California, Santa Cruz. He has worked on a wide variety of topics in theoretical cosmology and fundamental physics, including inflation, black holes, quantum theory, and information theory. He also has strong interest in science outreach, and has appeared in numerous science documentaries. He is a co-founder of the Future of Life Institute, the Foundational Questions Institute, and Metaculus (http://www.metaculus.com/). Sam Altman is president of Y Combinator and was the cofounder of Loopt, a location-based social networking app. He also co-founded OpenAI with Elon Musk. Sam has invested in over 1,000 companies. Dario Amodei is the co-author of the recent paper Concrete Problems in AI Safety, which outlines a pragmatic and empirical approach to making AI systems safe. Dario is currently a research scientist at OpenAI, and prior to that worked at Google and Baidu. Dario also helped to lead the project that developed Deep Speech 2, which was named one of 10 “Breakthrough Technologies of 2016” by MIT Technology Review. Dario holds a PhD in physics from Princeton University, where he was awarded the Hertz Foundation doctoral thesis prize. Amara Angelica is Research Director for Ray Kurzweil, responsible for books, charts, and special projects. Amara’s background is in aerospace engineering, in electronic warfare, electronic intelligence, human factors, and computer systems analysis areas. A co-founder and initial Academic Model/Curriculum Lead for Singularity University, she was formerly on the board of directors of the National Space Society, is a member of the Space Development Steering Committee, and is a professional member of the Institute of Electrical and Electronics Engineers (IEEE). -
F.3. the NEW POLITICS of ARTIFICIAL INTELLIGENCE [Preliminary Notes]
F.3. THE NEW POLITICS OF ARTIFICIAL INTELLIGENCE [preliminary notes] MAIN MEMO pp 3-14 I. Introduction II. The Infrastructure: 13 key AI organizations III. Timeline: 2005-present IV. Key Leadership V. Open Letters VI. Media Coverage VII. Interests and Strategies VIII. Books and Other Media IX. Public Opinion X. Funders and Funding of AI Advocacy XI. The AI Advocacy Movement and the Techno-Eugenics Movement XII. The Socio-Cultural-Psychological Dimension XIII. Push-Back on the Feasibility of AI+ Superintelligence XIV. Provisional Concluding Comments ATTACHMENTS pp 15-78 ADDENDA pp 79-85 APPENDICES [not included in this pdf] ENDNOTES pp 86-88 REFERENCES pp 89-92 Richard Hayes July 2018 DRAFT: NOT FOR CIRCULATION OR CITATION F.3-1 ATTACHMENTS A. Definitions, usage, brief history and comments. B. Capsule information on the 13 key AI organizations. C. Concerns raised by key sets of the 13 AI organizations. D. Current development of AI by the mainstream tech industry E. Op-Ed: Transcending Complacency on Superintelligent Machines - 19 Apr 2014. F. Agenda for the invitational “Beneficial AI” conference - San Juan, Puerto Rico, Jan 2-5, 2015. G. An Open Letter on Maximizing the Societal Benefits of AI – 11 Jan 2015. H. Partnership on Artificial Intelligence to Benefit People and Society (PAI) – roster of partners. I. Influential mainstream policy-oriented initiatives on AI: Stanford (2016); White House (2016); AI NOW (2017). J. Agenda for the “Beneficial AI 2017” conference, Asilomar, CA, Jan 2-8, 2017. K. Participants at the 2015 and 2017 AI strategy conferences in Puerto Rico and Asilomar. L. Notes on participants at the Asilomar “Beneficial AI 2017” meeting. -
Philos 121: January 24, 2019 Nick Bostrom and Eliezer Yudkowsky, “The Ethics of Artificial Intelligence” (Skip Pp
Philos 121: January 24, 2019 Nick Bostrom and Eliezer Yudkowsky, “The Ethics of Artificial Intelligence” (skip pp. 7–13) Jeremy Bentham, Introduction to the Principles of Morals and Legislation, Ch. I–II Suppose we tell AI: “Make paper clips.” Unfortunately, AI has become advanced enough to design ever more intelligent AI, which in turn designs ever more intelligent AI, and so on. AI then tries to turn the universe into a paper-clip making factory, diverting all resources from food production into paper-clip production. Getting hungry, we try to turn it off. AI interprets us and our attempts to turn it off as obstacles to making paper clips. So it eliminates the obstacles… So… what should we tell AI to do, if we have only one chance to get it right? This is very close to the basic question of moral philosophy: In the most basic and general terms, what should we, or any other intelligent agent, do? The road to utilitarianism: First premise (Hedonism): What is good for us as an end is pleasure, and what is bad for us as an end is pain. Good as a means vs. good as an end: • Good as a means: good because it helps to bring about something else that is good. • But this chain of means has to end somewhere. There has to be something that is: • Good as an end: good because of what it is in itself and not because of what it brings about. What is good for us as an end? The answer at the end of every chain seems to be: pleasure and the absence of pain. -
Are We Living at the Hinge of History?
Are We Living At The Hinge Of History? 0. Introduction In the final pages of On What Matters, Volume II (2011), Derek Parfit made the following comments: We live during the hinge of history. Given the scientific and technological discoveries of the last two centuries, the world has never changed as fast. We shall soon have even greater powers to transform, not only our surroundings, but ourselves and our successors. If we act wisely in the next few centuries, humanity will survive its most dangerous and decisive period. Our descendants could, if necessary, go elsewhere, spreading through this galaxy.1 These comments hark back to a statement he made twenty-seven years earlier in Reasons and Persons (1984): the part of our moral theory... that covers how we affect future generations... is the most important part of our moral theory, since the next few centuries will be the most important in human history.2 He also subsequently made the same claim in even stronger terms during a talk sponsored by Giving What We Can at the Oxford Union in June 2015: I think that we are living now at the most critical part of human history. The twentieth century I think was the best and worst of all centuries so far, but it now seems fairly likely that there are no intelligent beings anywhere else in the observable universe. Now, if that’s true, we may be living in the most critical part of the history of the universe… [The reason] why this may be the critical period in the history of the universe is if we are the only rational intelligent beings, it’s only we who might provide the origin of what would then become a galaxy-wide civilisation, which lasted for billions of years, and in which life was much better than it is for most human beings. -
1 Strategies for an Unfriendly Oracle AI with Reset Button Olle Häggström
1 Strategies for an unfriendly oracle AI with reset button Olle Häggström Chalmers University of Technology and the Institute for Future Studies http://orcid.org/0000-0001-6123-3770 Abstract. Developing a superintelligent AI might be very dangerous if it turns out to be unfriendly, in the sense of having goals and values that are not well-aligned with human values. One well-known idea for how to handle such danger is to keep the AI boxed in and unable to influence the world outside the box other than through a narrow and carefully controlled channel, until it has been deemed safe. Here we consider the special case, proposed by Toby Ord, of an oracle AI with reset button: an AI whose only available action is to answer yes/no questions from us and which is reset after every answer. Is there a way for the AI under such circumstances to smuggle out a dangerous message that might help it escape the box or otherwise influence the world for its unfriendly purposes? Some strategies are discussed, along with possible countermeasures by human safety administrators. In principle it may be doable for the AI, but whether it can be done in practice remains unclear, and depends on subtle issues concerning how the AI can conceal that it is giving us dishonest answers. 1. Introduction Once artificial intelligence (AI) development has been so successful as to produce a superintelligent machine – i.e., one that clearly outperforms humans across the whole range of capabilities that we associate with human intelligence – what will happen? From the perspective of humanity, it seems plausible both that things can go very well, and that they can go very 2 badly. -
Intelligence Explosion Microeconomics
MIRI MACHINE INTELLIGENCE RESEARCH INSTITUTE Intelligence Explosion Microeconomics Eliezer Yudkowsky Machine Intelligence Research Institute Abstract I. J. Good’s thesis of the “intelligence explosion” states that a sufficiently advanced ma- chine intelligence could build a smarter version of itself, which could in turn build an even smarter version, and that this process could continue to the point of vastly exceeding human intelligence. As Sandberg (2010) correctly notes, there have been several attempts to lay down return on investment formulas intended to represent sharp speedups in economic or technological growth, but very little attempt has been made to deal formally with Good’s intelligence explosion thesis as such. I identify the key issue as returns on cognitive reinvestment—the ability to invest more computing power, faster computers, or improved cognitive algorithms to yield cognitive labor which produces larger brains, faster brains, or better mind designs. There are many phenomena in the world which have been argued to be evidentially relevant to this question, from the observed course of hominid evolution, to Moore’s Law, to the competence over time of machine chess-playing systems, and many more. I go into some depth on some debates which then arise on how to interpret such evidence. I propose that the next step in analyzing positions on the intelligence explosion would be to for- malize return on investment curves, so that each stance can formally state which possible microfoundations they hold to be falsified by historical observations. More generally, Yudkowsky, Eliezer. 2013. Intelligence Explosion Microeconomics. Technical report 2013-1. Berkeley, CA: Machine Intelligence Research Institute. -
Ethical Artificial Intelligence
Ethical Artificial Intelligence Bill Hibbard Space Science and Engineering Center University of Wisconsin - Madison and Machine Intelligence Research Institute, Berkeley, CA 17 November 2015 First Edition Please send typo and error reports, and any other comments, to [email protected]. Copyright © Bill Hibbard 2014, 2015 Preface Recent research is giving us ways to define the behaviors of future artificial intelligence (AI) systems, before they are built, by mathematical equations. We can use these equations to describe various broad types of unintended and harmful AI behaviors, and to propose AI design techniques that avoid those behaviors. That is the primary subject of this book. Because AI will affect everyone's future, the book is written to be accessible to readers at different levels. Mathematical explanations are provided for those who want details, but it is also possible to skip over the math and follow the general arguments via text and illustrations. The introductory and final sections of the mathematical chapters (2 −4 and 6 −9) avoid mathematical notation. While this book discusses dangers posed by AI and proposes solutions, it is only a snapshot of ongoing research from my particular point of view and is not offered as the final answer. The problem of defining and creating ethical AI continues, and interest in it will grow enormously. While the effort to solve this problem is currently much smaller than is devoted to Internet security, ultimately there is no limit to the effort that ethical AI merits. Hollywood movies such as the Terminator and Matrix series depict violent war between AI and humans.