Policy Desiderata in the Development of Machine Superintelligence
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Effective Altruism William Macaskill and Theron Pummer
1 Effective Altruism William MacAskill and Theron Pummer Climate change is on course to cause millions of deaths and cost the world economy trillions of dollars. Nearly a billion people live in extreme poverty, millions of them dying each year of easily preventable diseases. Just a small fraction of the thousands of nuclear weapons on hair‐trigger alert could easily bring about global catastrophe. New technologies like synthetic biology and artificial intelligence bring unprece dented risks. Meanwhile, year after year billions and billions of factory‐farmed ani mals live and die in misery. Given the number of severe problems facing the world today, and the resources required to solve them, we may feel at a loss as to where to even begin. The good news is that we can improve things with the right use of money, time, talent, and effort. These resources can bring about a great deal of improvement, or very little, depending on how they are allocated. The effective altruism movement consists of a growing global community of peo ple who use reason and evidence to assess how to do as much good as possible, and who take action on this basis. Launched in 2011, the movement now has thousands of members, as well as influence over billions of dollars. The movement has substan tially increased awareness of the fact that some altruistic activities are much more cost‐effective than others, in the sense that they do much more good than others per unit of resource expended. According to the nonprofit organization GiveWell, it costs around $3,500 to prevent someone from dying of malaria by distributing bed nets. -
Robin Hanson Statement on Teaching
Robin Hanson Statement on Teaching Long ago I was one of the top teaching assistants at the University of Chicago physics department. I had to learn to translate this ability into teaching economics here. For example, I learned that students are far more willing to believe physicists about physics, than to believe economists about economics. To help overcome this skepticism, I run some classroom experiments in my microeconomics courses. These experiments give students a good initial hook for the concepts of marginal value and cost, and they show that standard theory predicts experimental outcomes reasonably well. One physics practice that translates well to economics is “back of the envelope” analysis. For example, I ask why poets are paid less than romance novelists, and teachers are paid less than garbage men (hint: note supply side effects). And I ask whether a subsidy or tax would best correct for law mower externalities (hint: neighbors can negotiate easily). While many undergraduates prefer teachers to give them the one true answer on policy questions, I try to avoid taking sides. In class discussions, I ask students to take and defend policy positions, and then I defend any undefended major positions, and mention any unmentioned major criticisms of these positions. The important thing is for students to understand the sorts of arguments that an economist would respect. I am big on normative analysis. I review how normative economics differs from normative analysis elsewhere, and the basic types of market failure. I cover counter- arguments that most teachers don’t cover, such as consistency checks (e.g., if this were the failure then we should also see that) and minimal interventions (e.g., this milder alternative should work just as well). -
SPACE and DEFENSE
SPACE and DEFENSE Volume Three Number One Summer 2009 Space Deterrence: The Delicate Balance of Risk by Ambassador Roger G. Harrison, Collins G. Shackelford and Deron R. Jackson with commentaries by Dean Cheng Pete Hays John Sheldon Mike Manor and Kurt Neuman Dwight Rauhala and Jonty Kasku-Jackson EISENHOWER CENTER FOR SPACE AND DEFENSE STUDIES Space and Defense Scholarly Journal of the United States Air Force Academy Eisenhower Center for Space and Defense Studies Editor-in-Chief: Ambassador Roger Harrison, [email protected] Director, Eisenhower Center for Space and Defense Studies Academic Editor: Dr. Eligar Sadeh, [email protected] Associate Academic Editors: Dr. Damon Coletta U.S. Air Force Academy, USA Dr. Michael Gleason U.S. Air Force Academy, USA Dr. Peter Hays National Security Space Office, USA Major Deron Jackson U.S. Air Force Academy, USA Dr. Collins Shackelford U.S. Air Force Academy, USA Colonel Michael Smith U.S. Air Force, USA Reviewers: Andrew Aldrin John Logsdon United Launch Alliance, USA George Washington University, USA James Armor Agnieszka Lukaszczyk ATK, USA Space Generation Advisory Council, Austria William Barry Molly Macauley NASA, France Resources for the Future, USA Frans von der Dunk Scott Pace University of Nebraska, USA George Washington University, USA Paul Eckart Xavier Pasco Boeing, USA Foundation for Strategic Research, France Andrew Erickson Wolfgang Rathbeger Naval War College, USA European Space Policy Institute, Austria Joanne Gabrynowicz Scott Trimboli University of Mississippi, USA University -
FROM LONGITUDE to ALTITUDE: INDUCEMENT PRIZE CONTESTS AS INSTRUMENTS of PUBLIC POLICY in SCIENCE and TECHNOLOGY Clayton Stallbau
FROM LONGITUDE TO ALTITUDE: INDUCEMENT PRIZE CONTESTS AS INSTRUMENTS OF PUBLIC POLICY IN SCIENCE AND TECHNOLOGY Clayton Stallbaumer* I. INTRODUCTION On a foggy October night in 1707, a fleet of Royal Navy warships ran aground on the Isles of Scilly, some twenty miles southwest of England.1 Believing themselves to be safely west of any navigational hazards, two thousand sailors discovered too late that they had fatally misjudged their position.2 Although the search for an accurate method to determine longitude had bedeviled Europe for several centuries, the “sudden loss of so many lives, so many ships, and so much honor all at once” elevated the discovery of a solution to a top national priority in England.3 In 1714, Parliament issued the Longitude Act,4 promising as much as £20,0005 for a “Practicable and Useful” method of determining longitude at sea; fifty-nine years later, clockmaker John Harrison claimed the prize.6 Nearly three centuries after the fleet’s tragic demise, another accident, also fatal but far removed from longitude’s terrestrial reach, struck a nation deeply. Just four days after its crew observed a moment’s silence in memory of the Challenger astronauts, the space shuttle Columbia disintegrated upon reentry, killing all on board.7 Although * J.D., University of Illinois College of Law, 2006; M.B.A., University of Illinois College of Business, 2006; B.A., Economics, B.A., History, Lake Forest College, 2001. 1. DAVA SOBEL, LONGITUDE: THE TRUE STORY OF A LONE GENIUS WHO SOLVED THE GREATEST SCIENTIFIC PROBLEM OF HIS TIME 12 (1995). -
Future of Research on Catastrophic and Existential Risk
ORE Open Research Exeter TITLE Working together to face humanity's greatest threats: Introduction to The Future of Research in Catastrophic and Existential Risk AUTHORS Currie, AM; Ó hÉigeartaigh, S JOURNAL Futures DEPOSITED IN ORE 06 February 2019 This version available at http://hdl.handle.net/10871/35764 COPYRIGHT AND REUSE Open Research Exeter makes this work available in accordance with publisher policies. A NOTE ON VERSIONS The version presented here may differ from the published version. If citing, you are advised to consult the published version for pagination, volume/issue and date of publication Working together to face humanity’s greatest threats: Introduction to The Future of Research on Catastrophic and Existential Risk. Adrian Currie & Seán Ó hÉigeartaigh Penultimate Version, forthcoming in Futures Acknowledgements We would like to thank the authors of the papers in the special issue, as well as the referees who provided such constructive and useful feedback. We are grateful to the team at the Centre for the Study of Existential Risk who organized the first Cambridge Conference on Catastrophic Risk where many of the papers collected here were originally presented, and whose multi-disciplinary expertise was invaluable for making this special issue a reality. We’d like to thank Emma Bates, Simon Beard and Haydn Belfield for feedback on drafts. Ted Fuller, Futures’ Editor-in-Chief also provided invaluable guidance throughout. The Conference, and a number of the publications in this issue, were made possible through the support of a grant from the Templeton World Charity Foundation (TWCF); the conference was also supported by a supplementary grant from the Future of Life Institute. -
31 Nuclear Weapons—Use, Probabilities and History
1 ROTARY DISTRICT 5440 PEACEBUILDER NEWSLETTER MARCH 2020 NUMBER 31 NUCLEAR WEAPONS—USE, PROBABILITIES AND HISTORY William M. Timpson, Robert Meroney, Lloyd Thomas, Sharyn Selman and Del Benson Fort Collins Rotary Club Lindsey Pointer, 2017 Rotary Global Grant Scholarship Recipient In these newsletters of the Rotary District Peacebuilders, we want to invite readers for contributions and ideas, suggestions and possibilities for our efforts to promote the foundational skills for promoting peace, i.e., nonviolent conflict resolution, improved communication and cooperation, successful negotiation and mediation as well as the critical and creative thinking that can help communities move through obstacles and difficulties. HOW THE MOST VIOLENT OF HUMAN ACTIVITY, A NCLEAR WWIII, HAS BEEN AVOIDED, SO FAR Robert Lawrence, Ph.D. is an Emeritus Professor of Political Science at Colorado State University who has written extensively on this issue of nuclear weapons Let’s play a mind game. Think back in time and pick an actual war, any war. Now imagine you are the leader who began the war. Would you have ordered the attack if you knew for certain that 30 minutes later you, your family, and your people would all be killed. Probably not. Your logic led Presidents Reagan and Gorbachev to jointly declare “a nuclear war can never be won and must never be fought.” This is the sine qua non of American and Russian nuclear deterrence policy. Always before wars made sense to some because they could be won, and there were many winners. How then has this no-win war situation---the first in history-- happened? Because the U.S. -
The Transhumanist Reader Is an Important, Provocative Compendium Critically Exploring the History, Philosophy, and Ethics of Transhumanism
TH “We are in the process of upgrading the human species, so we might as well do it E Classical and Contemporary with deliberation and foresight. A good first step is this book, which collects the smartest thinking available concerning the inevitable conflicts, challenges and opportunities arising as we re-invent ourselves. It’s a core text for anyone making TRA Essays on the Science, the future.” —Kevin Kelly, Senior Maverick for Wired Technology, and Philosophy “Transhumanism has moved from a fringe concern to a mainstream academic movement with real intellectual credibility. This is a great taster of some of the best N of the Human Future emerging work. In the last 10 years, transhumanism has spread not as a religion but as a creative rational endeavor.” SHU —Julian Savulescu, Uehiro Chair in Practical Ethics, University of Oxford “The Transhumanist Reader is an important, provocative compendium critically exploring the history, philosophy, and ethics of transhumanism. The contributors anticipate crucial biopolitical, ecological and planetary implications of a radically technologically enhanced population.” M —Edward Keller, Director, Center for Transformative Media, Parsons The New School for Design A “This important book contains essays by many of the top thinkers in the field of transhumanism. It’s a must-read for anyone interested in the future of humankind.” N —Sonia Arrison, Best-selling author of 100 Plus: How The Coming Age of Longevity Will Change Everything IS The rapid pace of emerging technologies is playing an increasingly important role in T overcoming fundamental human limitations. The Transhumanist Reader presents the first authoritative and comprehensive survey of the origins and current state of transhumanist Re thinking regarding technology’s impact on the future of humanity. -
Wise Leadership & AI 3
Wise Leadership and AI Leadership Chapter 3 | Behind the Scenes of the Machines What’s Ahead For Artificial General Intelligence? By Dr. Peter VERHEZEN With the AMROP EDITORIAL BOARD Putting the G in AI | 8 points True, generalized intelligence will be achieved when computers can do or learn anything that a human can. At the highest level, this will mean that computers aren’t just able to process the ‘what,’ but understand the ‘why’ behind data — context, and cause and effect relationships. Even someday chieving consciousness. All of this will demand ethical and emotional intelligence. 1 We underestimate ourselves The human brain is amazingly general compared to any digital device yet developed. It processes bottom-up and top-down information, whereas AI (still) only works bottom-up, based on what it ‘sees’, working on specific, narrowly defined tasks. So, unlike humans, AI is not yet situationally aware, nuanced, or multi-dimensional. 2 When can we expect AGI? Great minds do not think alike Some eminent thinkers (and tech entrepreneurs) see true AGI as only a decade or two away. Others see it as science fiction — AI will more likely serve to amplify human intelligence, just as mechanical machines have amplified physical strength. 3 AGI means moving from homo sapiens to homo deus Reaching AGI has been described by the futurist Ray Kurzweil as ‘singularity’. At this point, humans should progress to the ‘trans-human’ stage: cyber-humans (electronically enhanced) or neuro-augmented (bio-genetically enhanced). 4 The real risk with AGI is not malice, but unguided brilliance A super-intelligent machine will be fantastically good at meeting its goals. -
Global Catastrophic Risks 2016
Global Challenges Foundation Global Catastrophic Risks 2016 © Global Challenges Foundation/Global Priorities Project 2016 GLOBAL CATASTROPHIC RISKS 2016 THE GLOBAL CHALLENGES FOUNDATION works to raise awareness of the The views expressed in this report are those of the authors. Their Global Catastrophic Risks. Primarily focused on climate change, other en- statements are not necessarily endorsed by the affiliated organisations. vironmental degradation and politically motivated violence as well as how these threats are linked to poverty and rapid population growth. Against this Authors: background, the Foundation also works to both identify and stimulate the Owen Cotton-Barratt*† development of good proposals for a management model – a global gover- Sebastian Farquhar* nance – able to decrease – and at best eliminate – these risks. John Halstead* Stefan Schubert* THE GLOBAL PRIORITIES PROJECT helps decision-makers effectively prior- Andrew Snyder-Beattie† itise ways to do good. We achieve his both by advising decision-makers on programme evaluation methodology and by encouraging specific policies. We * = The Global Priorities Project are a collaboration between the Centre for Effective Altruism and the Future † = The Future of Humanity Institute, University of Oxford of Humanity Institute, part of the University of Oxford. Graphic design: Accomplice/Elinor Hägg Global Challenges Foundation in association with 4 Global Catastrophic Risks 2016 Global Catastrophic Risks 2016 5 Contents Definition: Global Foreword 8 Introduction 10 Catastrophic Risk Executive summary 12 1. An introduction to global catastrophic risks 20 – risk of events or 2. What are the most important global catastrophic risks? 28 Catastrophic climate change 30 processes that would Nuclear war 36 Natural pandemics 42 Exogenous risks 46 lead to the deaths of Emerging risks 52 Other risks and unknown risks 64 Our assessment of the risks 66 approximately a tenth of 3. -
GPI's Research Agenda
A RESEARCH AGENDA FOR THE GLOBAL PRIORITIES INSTITUTE Hilary Greaves, William MacAskill, Rossa O’Keeffe-O’Donovan and Philip Trammell February 2019 (minor changes July 2019) We acknowledge Pablo Stafforini, Aron Vallinder, James Aung, the Global Priorities Institute Advisory Board, and numerous colleagues at the Future of Humanity Institute, the Centre for Effective Altruism, and elsewhere for their invaluable assistance in composing this agenda. 1 Table of Contents Introduction 3 GPI’s vision and mission 3 GPI’s research agenda 4 1. The longtermism paradigm 6 1.1 Articulation and evaluation of longtermism 6 1.2 Sign of the value of the continued existence of humanity 8 1.3 Mitigating catastrophic risk 10 1.4 Other ways of leveraging the size of the future 12 1.5 Intergenerational governance 14 1.6 Economic indices for longtermists 16 1.7 Moral uncertainty for longtermists 18 1.8 Longtermist status of interventions that score highly on short-term metrics 19 2. General issues in global prioritisation 21 2.1 Decision-theoretic issues 21 2.2 Epistemological issues 23 2.3 Discounting 24 2.4 Diversification and hedging 28 2.5 Distributions of cost-effectiveness 30 2.6 Modelling altruism 32 2.7 Altruistic coordination 33 2.8 Individual vs institutional actors 35 Bibliography 38 Appendix A. Research areas for future engagement 46 A.1 Animal welfare 46 A.2 The scope of welfare maximisation 48 Appendix B. Closely related areas of existing academic research 51 B.1 Methodology of cost-benefit analysis and cost-effectiveness analysis 51 B.2 Multidimensional economic indices 51 B.3 Infinite ethics and intergenerational equity 53 B.4 Epistemology of disagreement 53 B.5 Demandingness 54 B.6 Forecasting 54 B.7 Population ethics 55 B.8 Risk aversion and ambiguity aversion 55 B.9 Moral uncertainty 57 1 B.10 Value of information 58 B.11 Harnessing and combining evidence 59 B.12 The psychology of altruistic decision-making 60 Appendix C. -
Downloads of Technical Information
Florida State University Libraries Electronic Theses, Treatises and Dissertations The Graduate School 2018 Nuclear Spaces: Simulations of Nuclear Warfare in Film, by the Numbers, and on the Atomic Battlefield Donald J. Kinney Follow this and additional works at the DigiNole: FSU's Digital Repository. For more information, please contact [email protected] FLORIDA STATE UNIVERSITY COLLEGE OF ARTS AND SCIENCES NUCLEAR SPACES: SIMULATIONS OF NUCLEAR WARFARE IN FILM, BY THE NUMBERS, AND ON THE ATOMIC BATTLEFIELD By DONALD J KINNEY A Dissertation submitted to the Department of History in partial fulfillment of the requirements for the degree of Doctor of Philosophy 2018 Donald J. Kinney defended this dissertation on October 15, 2018. The members of the supervisory committee were: Ronald E. Doel Professor Directing Dissertation Joseph R. Hellweg University Representative Jonathan A. Grant Committee Member Kristine C. Harper Committee Member Guenter Kurt Piehler Committee Member The Graduate School has verified and approved the above-named committee members, and certifies that the dissertation has been approved in accordance with university requirements. ii For Morgan, Nala, Sebastian, Eliza, John, James, and Annette, who all took their turns on watch as I worked. iii ACKNOWLEDGMENTS I would like to thank the members of my committee, Kris Harper, Jonathan Grant, Kurt Piehler, and Joseph Hellweg. I would especially like to thank Ron Doel, without whom none of this would have been possible. It has been a very long road since that afternoon in Powell's City of Books, but Ron made certain that I did not despair. Thank you. iv TABLE OF CONTENTS Abstract..............................................................................................................................................................vii 1. -
Letters to the Editor
Articles Letters to the Editor Research Priorities for is a product of human intelligence; we puter scientists, innovators, entrepre - cannot predict what we might achieve neurs, statisti cians, journalists, engi - Robust and Beneficial when this intelligence is magnified by neers, authors, professors, teachers, stu - Artificial Intelligence: the tools AI may provide, but the eradi - dents, CEOs, economists, developers, An Open Letter cation of disease and poverty are not philosophers, artists, futurists, physi - unfathomable. Because of the great cists, filmmakers, health-care profes - rtificial intelligence (AI) research potential of AI, it is important to sionals, research analysts, and members Ahas explored a variety of problems research how to reap its benefits while of many other fields. The earliest signa - and approaches since its inception, but avoiding potential pitfalls. tories follow, reproduced in order and as for the last 20 years or so has been The progress in AI research makes it they signed. For the complete list, see focused on the problems surrounding timely to focus research not only on tinyurl.com/ailetter. - ed. the construction of intelligent agents making AI more capable, but also on Stuart Russell, Berkeley, Professor of Com - — systems that perceive and act in maximizing the societal benefit of AI. puter Science, director of the Center for some environment. In this context, Such considerations motivated the “intelligence” is related to statistical Intelligent Systems, and coauthor of the AAAI 2008–09 Presidential Panel on standard textbook Artificial Intelligence: a and economic notions of rationality — Long-Term AI Futures and other proj - Modern Approach colloquially, the ability to make good ects on AI impacts, and constitute a sig - Tom Dietterich, Oregon State, President of decisions, plans, or inferences.