OUP CORRECTED PROOF – FINAL, 09/08/20, SPi Moral Uncertainty OUP CORRECTED PROOF – FINAL, 09/08/20, SPi OUP CORRECTED PROOF – FINAL, 09/08/20, SPi Moral Uncertainty WILLIAM MACASKILL, KRISTER BYKVIST, AND TOBY ORD 1 OUP CORRECTED PROOF – FINAL, 09/08/20, SPi 1 Great Clarendon Street, Oxford, OX2 6DP, United Kingdom Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries © William MacAskill, Krister Bykvist and Toby Ord 2020 The moral rights of the authors have been asserted First Edition published in 2020 Impression: 1 Some rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, for commercial purposes, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by licence or under terms agreed with the appropriate reprographics rights organization. This is an open access publication, available online and distributed under the terms of a Creative Commons Attribution – Non Commercial – No Derivatives 4.0 International licence (CC BY-NC-ND 4.0), a copy of which is available at http://creativecommons.org/licenses/by-nc-nd/4.0/. Enquiries concerning reproduction outside the scope of this licence should be sent to the Rights Department, Oxford University Press, at the address above Published in the United States of America by Oxford University Press 198 Madison Avenue, New York, NY 10016, United States of America British Library Cataloguing in Publication Data Data available Library of Congress Control Number: 2020932188 ISBN 978–0–19–872227–4 Printed and bound in Great Britain by Clays Ltd, Elcograf S.p.A. Links to third party websites are provided by Oxford in good faith and for information only. Oxford disclaims any responsibility for the materials contained in any third party website referenced in this work. OUP CORRECTED PROOF – FINAL, 09/08/20, SPi For Derek Parfit, a mentor and friend. OUP CORRECTED PROOF – FINAL, 09/08/20, SPi OUP CORRECTED PROOF – FINAL, 09/08/20, SPi Acknowledgements The authors would like to thank Ron Aboodi, Per Algander, Amanda Askell, Frank Arntzenius, Gustaf Arrhenius, Ralf Bader, Nick Bostrom, Richard Bradley, John Broome, Owen Cotton-Barratt, Erik Carlson, Sven Danielsson, Dan Deasy, Ben Eidelson, Anna Folland, Hilary Greaves, Johan Gustafsson, Anandi Hattiangadi, Iwao Hirose, Benjamin Kiesewetter, Jimmy Lenman, Jeff McMahan, Andreas Mogensen, Dan Moller, Graham Oddie, Jonas Olson, Jan Österberg, Mike Otsuka, Peter Pagin, Derek Parfit, Jessica Pepp, Filip Poignant, Wlodek Rabinowicz, Stefan Riedener, Michael Ridge, Olle Risberg, Simon Rosenqvist, Jacob Ross, Julian Savulescu, Andrew Sepielli, Peter Singer, Howard Sobel, Michael Smith, Bastian Stern, Folke Tersman, Torbjörn Tännsjö, Alex Voorhoeve, Ralph Wedgwood, Alex Worsnip, and Michael Zimmerman. For especially detailed comments, we would like to thank Christian Tarsney. We would also like to thank audiences at: the Rocky Mountain Ethics Congress, Boulder; the Stockholm Centre for Healthcare Ethics Workshop on Moral Uncertainty; the CRNAP workshops on moral epistemology at Princeton and Oxford; the Economics and Philosophy workshop at Princeton; the Centre de recherche en éthique de l’Université de Montréal; the Cumberland Lodge Weekend, organized by LSE; the British Society for Ethical Theory conference; and seminars at the Universities of Leeds, LSE, Oxford, and Princeton. Some of the work in this book has been published before. Chapters 1 (sections I–II), 2–6, and 8–9 are based in part on William MacAskill’s DPhil thesis (though many passages within that thesis cited unpublished work by Toby Ord). Chapter 1 (section III) draws, to some extent, on two papers by Krister Bykvist: ‘Evaluative Uncertainty and Consequentialist Environmental Ethics’ in: Leonard Kahn and Avram Hiller (eds), Consequentialism and Environmental Ethics, pp. 122–35, London: Routledge, doi: 10.4324/9780203379790, Copyright © 2014, and ‘How to Do Wrong Knowingly and Get Away with it’ in: Rysiek Sliwinski and Frans Svensson (eds.), Neither/Nor: Philosophical Papers Dedicated to Erik Carlson on the Occasion of His Fiftieth Birthday, Volume 58 in UPPSALA PHILOSOPHICAL STUDIES, pp. 31–47, Uppsala: Uppsala University, Copyright © 2011. OUP CORRECTED PROOF – FINAL, 09/08/20, SPi viii Acknowledgements Chapter 2 is significantly based on William MacAskill and Toby Ord. ‘Why Maximize Expected Choice-Worthiness’, Noûs (July 2019), Copyright © 2019 by Wiley Periodicals, Inc. The bulk of Chapter 3 is based on William MacAskill, ‘Normative Uncertainty as a Voting Problem’, Mind, Volume 125, Issue 500, pp. 967–1004, doi: 10.1093/mind/fzv169, Copyright © 2016, reprinted by permission of Oxford University Press: https://academic.oup.com/mind. Chapter 4 is significantly based on Owen Cotton-Barratt, William MacAskill, and Toby Ord, ‘Statistical Normalization Methods in Interpresonal and Intertheoretic Comparisons’, Journal of Philosophy (forthcoming), Copyright © 2019. Part of Chapter 6 is based on William MacAskill, ‘The Infectiousness of Nihilism’, Ethics, Volume 123, Issue 3, pp. 508–20, doi: 10.1086/669564, Copyright © 2013 by the University of Chicago Press. All rights reserved. Chapter 7 is based on two papers co-authored by Krister Bykvist and Jonas Olson: ‘Expressivism and Moral Certitude’, The Philosophical Quarterly, Volume 59, Issue 235, pp. 202–15, doi: 10.1111/j.1467-9213.2008.580.x, Copyright © 2008 The Editors of The Philosophical Quarterly, reprinted by permission of Oxford University Press: https://academic.oup.com/pq, and ‘Against the Being For Account of Normative Certitude’, Journal of Ethics and Social Philosophy, Volume 6, Issue 2, pp. 1–8, doi: 10.26556/jesp.v6i2.63, Copyright © 2012. OUP CORRECTED PROOF – FINAL, 09/08/20, SPi Contents Introduction 1 1. Why We Should Take Moral Uncertainty Seriously 11 2. Maximizing Expected Choiceworthiness 39 3. Ordinal Theories and the Social Choice Analogy 57 4. Interval-Scale Theories and Variance Voting 77 5. Intertheoretic Comparisons of Choice-Worthiness 112 6. Fanaticism and Incomparability 150 7. Metaethical Implications: Cognitivism versus Non-Cognitivism 160 8. Practical Ethics Given Moral Uncertainty 179 9. Moral Information 197 Conclusion 211 Bibliography 217 Index 225 OUP CORRECTED PROOF – FINAL, 09/08/20, SPi OUP CORRECTED PROOF – FINAL, 08/08/20, SPi Introduction We are often uncertain about what we ought, morally, to do. Suppose that Alice has £20 to spend. With that money, she could eat out at a pleasant restaurant. Alternatively, she could pay for four long-lasting insecticide- treated bed nets that would protect eight children against malaria for two years.1 Let’s suppose that Alice knows all the morally relevant empirical facts about what that £20 could do. Even so, it might be that she still doesn’t know whether she’s obligated to donate that money or whether it’s per mis- sible for her to pay for the meal out, because she just doesn’t know how strong her moral obligations to distant strangers are. If so, then even though Alice knows all the relevant empirical facts, she doesn’t know what she ought to do. Or suppose that the members of a government are making a decision about whether to tax carbon emissions. Let’s assume that they know all the relevant facts about what would happen as a result of the tax: it would make presently existing people worse off, since they would consume less oil and coal, and would therefore be less economically productive; but it would slow down climate change, thereby on balance increasing the welfare of people living in the future. But the members of the government don’t know how to weigh the interests of future people against the interests of presently existing people. So, again, the members of this government don’t ultimately know what they ought to do. These are instances ofmoral uncertainty: uncertainty that stems not from uncertainty about descriptive matters, but about moral or evaluative mat- ters. Moral uncertainty is commonplace: given the difficulty of ethics and the widespread disagreement about ethical issues, moral uncertainty is not the exception, but the norm. Moral uncertainty matters. If we don’t know how to weigh the interests of future generations against the current generation, then we don’t yet know 1 For the relevant estimates, see GiveWell, ‘Against Malaria Foundation’, November 2016, http://www.givewell.org/charities/against-malaria-foundation/November-2016-version. Moral Uncertainty. William MacAskill, Krister Bykvist, and Toby Ord, Oxford University Press (2020). © William MacAskill, Krister Bykvist and Toby Ord. DOI: 10.1093/oso/9780198722274.001.0001 OUP CORRECTED PROOF – FINAL, 08/08/20, SPi 2 Introduction how we ought to act in response to climate change. If we don’t know how to weigh the interests of distant strangers against compatriots, then we don’t yet know the extent of our duties to the global poor. We aren’t going to resolve these difficult moral questions any time soon. But we still need to act now. So we need to know how to act, despite our uncertainty. Given the prevalence and importance of moral uncertainty, one would expect ethicists to have devoted considerable research effort to the topic of how one ought to make decisions in the face of moral uncertainty. But this topic has been neglected. In modern times, only one book and fewer than twenty published articles deal with the topic at length.2 The book you are reading attempts to begin to address this gap. In this book, we address the questions of whether there are norms that are distinct from first-order moral norms that govern how one ought to act given one’s fundamental moral uncertainty and, if so, what those norms are. These questions raise many difficult theoretical issues, and we don’t pre- tend to have comprehensive solutions to all of them.