Global Challenges Foundation

Home , Anthropic Bias (book), Chelyabinsk meteor, Doomsday argument, Dysgenics, Economic collapse, Foresight Institute

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial Extreme Future Bad Global Global System Major Asteroid Global Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Challenges Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Ecological NanotechnologyIntelligence NuclearClimate WarChange Super-volcanoGlobal Governance PandemicSynthetic UnknownCollapse Impact Risks that threaten Catastrophe Biology Consequences humanArtificial civilisationExtreme Future Bad Global Global System Major Asteroid 12 Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial Extreme Future Bad Global Global System Major Asteroid IntelligenceEcological ClimateNanotechnology Change NuclearGlobal Governance War Super-volcanoPandemic CollapseSynthetic UnknownImpact Catastrophe Biology Consequences

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Ecological Nanotechnology Nuclear War Super-volcano SyntheticIntelligence ClimateUnknown Change Global Governance Pandemic Collapse Impact Catastrophe Biology Consequences

The case for Ecologicala new riskNanotechnology categoryNuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences Published February 2015 by Global Challenges Foundation. The opinions expressed in this report are those of the authors and do not necessarily reflect those of Global Challenges Foundation. Any inaccuracies in the report remain the responsibility of the authors. The material and the geographical designations in this report do not imply the expression of any opinion whatsoever on the part of Global Challenges Foundation concerning the legal status of any country, territory, or area, or concerning the delimitation of its frontiers or boundaries.

For comments and/or queries on this report, please contact the co-authors:

Dennis Pamlin Executive Project Manager Global Risks Global Challenges Foundation [email protected] globalchallenges.org or

Stuart Armstrong James Martin Research Fellow Future of Humanity Institute Oxford Martin School University of Oxford stuart.armstrong@philosophy.ox.ac.uk fhi.ox.ac.uk The main authors of this report are Dennis collaborative approach where many people important role. Patrick McSharry, Pamlin, Executive Project Manager, Global have provided invaluable contributions. Head of Smith School’s Catastrophe Challenges Foundation and Dr Stuart Risk Financing research area, provided Armstrong, James Martin Research Fellow, The authors would therefore like to thank invaluable input regarding complex Future of Humanity Institute, Oxford Martin a few people in particular. First and systems and ways that the economic School & Faculty of Philosophy, University foremost László Szombatfalvy, Chairman system can respond to infinite impacts. of Oxford. Dr Stuart Armstrong wrote of the Global Challenges Foundation, Alex Kirby also played a key part as the chapter covering the twelve global whose work is the basis for this report he did so much more than proofread challenges, under the direction of Dennis and whose guidance on all levels has the text; the report would hardly be Pamlin who served as project manager been invaluable. The rest of the board possible to read without his help. Various and himself wrote and edited the rest of of the Global Challenges Foundation additional edits and changes were made the report. Seth Baum, Executive Director have also contributed in many different by Peter Brietbart. of the Global Catastrophic Risk Institute ways, in particular, Johan Rockström has and affiliate researcher at the Center for provided important input regarding the Others that must be mentioned, including Research on Environmental Decisions, structure and methodology. Outside the those who participated in the workshop Columbia University, also played an foundation Prof Nick Bostrom, Professor on 14 January 2014, at the Future of important role as he helped develop & Director of the Future of Humanity Humanity Institute (FHI), University of the methodology chapter regarding the Institute, Oxford Martin School & Faculty Oxford and the workshop at the Munich selection of the global challenges with of Philosophy, University of Oxford, who RE office in London on 15 January 2014, potentially infinite impacts as well as initiated the possibility of working with and helped provide input regarding the providing helpful input throughout the the Future of Humanity Institute at the economic and finance aspects, include process. The report is the result of a University of Oxford, played a particularly (in alphabetical order):

Dr Nick Beckstead, Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Kennette Benedict, Executive Director and Publisher of the Bulletin of the Atomic Scientists Oliver Bettis, Pricing Actuary, Munich RE and Fellow of the Chartered Insurance Institute and the Institute & Faculty of Actuaries Dr Eric Drexler, Academic Visitor, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Madeleine Enarsson, Transformative Catalyst, 21st Century Frontiers Dr Seán Ó hÉigeartaigh, Senior Academic Manager, Future of Humanity Institute, University of Oxford and Executive Director, Centre for the Study of Existential Risk, University of Cambridge Martin Hellman, Professor Emeritus of Electrical Engineering, Stanford University Pan Jiahua, Director of the Institute for Urban and Environmental Studies, Chinese Academy of Social Sciences (CASS); Professor of economics at CASS; Vice-President Chinese Society for Ecological Economics; Member of the National Expert Panel on Climate Change and National Foreign Policy Advisory Committee, China Aled Jones, Director of the Global Sustainability Institute (GSI) at Anglia Ruskin University Nick Mabey, Chief Executive and Founding Director of E3G (Third Generation Environmentalism) Jennifer Morgan, Founder & Co-Convener, The Finance Lab Prof Vincent Müller, James Martin Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Robert de Neufville, Professional Associate, Global Catastrophic Risk Institute Prof Toby Ord, James Martin Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Jules Peck, Founding Partner, Jericho Chambers; Trustee, New Economics Foundation Dr Anders Sandberg, James Martin Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Nick Silver, Director of Callund Consulting and founder and director of the Climate Bonds Initiative (CBI) Andrew Simms, Author, Fellow at the New Economics Foundation and Chief Analyst at Global Witness Andrew Snyder-Beattie, Academic Project Manager, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford James Taplan, Principal Sustainability Advisor, Forum for the Future Raj Thamotheram, CEO, Preventable Surprises Nathan Wolfe, Director of Global Viral and the Lorry I. Lokey Visiting Professor in Human Biology at Stanford University Liang Yin, Investment Consultant at Towers Watson

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 1 Contents

Contents

Executive Summary 4

Preface 26

1. Twelve risks that threaten human civilisation 28

2. Risks with infinite impact: A new category of risks 30

2.1 Report structure ...... 37 2.2 Goals ...... 38 2.3 Global challenges and infinite impact ...... 40 2.3.1 Definition of infinite impact ...... 40 2.4 Methodology...... 49 2.4.1 A scientific review of key literature ...... 50 2.4.2 A review of organisations working on global challenges...... 52 2.4.3 Workshops ...... 55 2.5 The list of global risks...... 55 2.5.1 Risks not included ...... 56 2.6 Relationship between impact levels beyond the infinite threshold ...... 58

3. Twelve Global Challenges 60

3.1 Current risks / 3.1.1 Extreme Climate Change ...... 62 3.1.2 Nuclear War ...... 70 3.1.3 Ecological Catastrophe ...... 78 3.1.4 Global Pandemic ...... 84 3.1.5 Global System Collapse ...... 90 3.2 Exogenic risk / 3.2.1 Major Asteroid Impact ...... 96 3.2.2 Super-volcano...... 102 3.3 Emerging risk / 3.3.1 Synthetic Biology ...... 108 3.3.2 Nanotechnology ...... 114 3.3.3 Artificial Intelligence ...... 120 3.3.4 Unknown Consequences ...... 126 3.4 Global policy risk / 3.4.1 Future Bad Global Governance ...... 132

2 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Contents

4. Relations between global risks 138

4.1 General relations between global risks and their potential impacts ...... 139 4.2 Specific relations between global risks ...... 141

5. Probabilities and uncertainties – an initial overview 142

6. Underlying trends of key importance 170

6.1 Poverty ...... 171 6.2 Population growth ...... 173 6.3 Technological development...... 174 6.4 Demographic changes...... 175

7. Possible ways forward ...... 176

Endnotes 184

Appendix 1 – Global Challenges Bibliography 198 Appendix 2 – Workshops 206

Notes 208

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3 Executive Summary

Executive Summary

This is the executive summary of a report about a limited number of global risks that pose a threat to human civilisation, or even possibly to all human life.

4 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Executive Summary

With such a focus it may surprise some readers to find that the report’s essential aim is to inspire action and dialogue as well as an increased use of the methodologies used for risk assessment. The real focus is not on the almost unimaginable impacts of the risks the report outlines. Its fundamental purpose is to encourage global collaboration and to use this new category of risk as a driver for innovation.

The idea that we face a number of global challenges threatening the very basis of our civilisation at the beginning of the 21st century is well accepted in the scientific community, and is studied at a number of leading universities.I However, there is still no coordinated approach to address this group of challenges and turn them into opportunities.

I http://en.wikipedia.org/wiki/Global_catastrophic_risk

It is only 70 years ago that Robert Oppenheimer, who History: Edward Teller, one of the led the Manhattan Project to greatest physicists of his time, develop the nuclear bomb, the LA-602 with his back-of-the-envelope halted the project to see document calculations, produced results whether Teller’s calculations that differed drastically from were correct. The resulting all that had gone before. His document, LA-602: Ignition of calculations showed that the the Atmosphere with Nuclear explosion of a nuclear bomb – a Bombs, concluded that Teller creation of some of the brightest was wrong. But the sheer minds on the planet, including complexity drove the assessors Teller himself – could result in to end their study by writing a chain reaction so powerful that “further work on the that it would ignite the world’s subject [is] highly desirable”. atmosphere, thereby ending The LA-602 document can human life on Earth. be seen as the first global challenge report addressing a category of risks where the worst possible impact in all practical senses is infinite.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 5 Executive Summary

This report has, to the best of the authors’ knowledge, 12 Global created the first list of global risks with impacts that risks for all practical purposes can be called infinite. It is also the first structured overview of key events related to such challenges and has tried to provide initial rough quantifications for the probabilities of these impacts. In the next phase of the project, these placeholder estimates will be improved and refined by a variety of methods (expert elicitation, fault trees, simulations, etc.) appropriate to each specific risk.

The report conducts its exploration It is worth noting that complex within carefully defined bounds, systems are often stable only within resulting in a list of twelve risks with certain boundaries outside which potentially infinite outcomes. the system can collapse and rapidly change to a new stable state. Such a There were many challenges which collapse can trigger a process where might have been included on the change continues for a long time until list because of their ability to pose a new stable state is found. None of severe damage to humanity. They the risks in this report are likely to were excluded for one or more of result directly in an infinite impact, three reasons: and some cannot do so physically. All the risks however are big enough to 1. Limited impact – tsunamis, for reach a threshold where the social and example, and chemical pollution. ecological systems become so unstable that an infinite impact could ensue. 2. No effective countermeasures – the report focuses on promoting This is a report about two extremes, effective interventions and so not one. It is about how a better ignores challenges where nothing understanding of the magnitude of useful can be done to prevent or the challenges can help the world mitigate the impact, as with to address the risks it faces, and nearby gamma-ray bursts. can help to create a path towards more sustainable development. It 3. Included in other challenges – is a scientific assessment about many challenges are already the possibility of oblivion, certainly, covered by others, or are very but more than that it is a call for similar to them. Population growth, action based on the assumption that for one, is significant for climate humanity is able to rise to challenges change and ecosystem and turn them into opportunities. catastrophe, but without direct We are confronted with possibly large-scale impacts of its own. the greatest challenge ever and our response needs to match this through global collaboration in new and innovative ways.

6 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance PandemicExecutive Summary Collapse Impact

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact ArtificialEcologicalArtificial ExtremeNanotechnologyExtreme FutureNuclearFuture Bad BadWar GlobalSuper-volcanoGlobal GlobalSyntheticGlobal System System MajorUnknownMajor Asteroid Asteroid IntelligenceCatastropheIntelligence ClimateClimate Change Change GlobalGlobal Governance Governance PandemicPandemic CollapseBiologyCollapse ImpactConsequencesImpact

ArtificialArtificial ExtremeExtreme FutureFuture Bad Bad GlobalGlobal GlobalGlobalArtificial System System MajorMajorExtreme Asteroid Asteroid Future Bad Global Global System Major Asteroid IntelligenceIntelligence ClimateClimate Change Change GlobalGlobal Governance Governance PandemicPandemic CollapseCollapseIntelligence ImpactImpactClimate Change Global Governance Pandemic Collapse Impact ArtificialArtificial ExtremeExtreme FutureFuture Bad Bad EcologicalGlobalGlobal NanotechnologyGlobalGlobal System System NuclearMajorMajor Asteroid AsteroidWar Super-volcano Synthetic Unknown IntelligenceIntelligence ClimateClimate Change Change GlobalGlobal Governance Governance CatastrophePandemicPandemic CollapseCollapse ImpactImpact Biology Consequences EcologicalEcological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSynthetic UnknownUnknown CatastropheCatastrophe BiologyBiology ConsequencesConsequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact EcologicalEcological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSyntheticEcological UnknownUnknownNanotechnology Nuclear War Super-volcano Synthetic Unknown CatastropheCatastrophe BiologyBiologyCatastrophe ConsequencesConsequences Biology Consequences EcologicalEcological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSynthetic UnknownUnknown CatastropheCatastrophe BiologyBiology ConsequencesConsequences

Ecological Nanotechnology Nuclear War Super-volcano ArtificialSyntheticArtificial ExtremeUnknownExtreme FutureFuture Bad Bad GlobalGlobal GlobalGlobal System System MajorMajor Asteroid Asteroid Catastrophe IntelligenceBiologyIntelligence ClimateConsequencesClimate Change Change GlobalGlobal Governance Governance PandemicPandemic CollapseCollapse ImpactImpact

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 7

EcologicalEcological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSynthetic UnknownUnknown CatastropheCatastrophe BiologyBiology ConsequencesConsequences Executive Summary

The four main goals of this report are to acknowledge, The goals inspire, connect and deliver. of the report The first of the report’s goals – The third goal is to connect acknowledging the existence of different groups at every level, risks with potentially infinite impact – so that leaders in different sectors seeks to help key stakeholders to connect with each other to acknowledge the existence of the encourage collaboration. This will category of risks that could result need a specific focus on financial in infinite impact, and to show them and security policy, where significant that we can reduce or even eliminate risks combine to demand action most of them. beyond the incremental.

The second goal is to inspire by The fourth goal is to deliver actual showing the practical action that strategies and initiatives that is taking place today. This report produce actual results. The report seeks to show that helping to meet is a first step and its success will these global challenges is perhaps ultimately be measured only on how the most important contribution it contributes to concrete results. anyone can make today, and highlights concrete examples to The report will have achieved its inspire a new generation of leaders. goals when key decision-makers recognise the magnitude of the possible risks and our ability to reduce or even eliminate most of them.

The goals 1. to acknowledge the existence of risks with potentially infinite impact. 2. to inspire by showing the practical action that is taking place today. 3. to connect different groups at every level. 4. to deliver actual strategies and initiatives that produce actual results.

8 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Executive Summary

The first part of the report introduces and defines the Report global challenges and includes the methodology for structure selecting them. The third part of the report discusses The second part is an overview of the relationship between the different the twelve challenges and key challenges, as action to address events that illustrate strategic work one can increase the risk of to address them. It also lists for each another. Many solutions can also challenge five important factors that address multiple challenges, so influence its probability or impact. there are significant benefits from The challenges are divided into four understanding how they are linked. different categories: The fourth part is an overview, the first – current challenges includes those ever to the authors’ knowledge, of the which currently threaten humanity probabilities of global challenges with because of its economic and potentially infinite impacts. technological development; The fifth part presents some of the exogenic challenges – are those most important underlying trends where the basic probability of an that influence the challenges, which event is beyond human control, but often build up slowly to a threshold where the probability and magnitude where very rapid changes can of the impact can be influenced; ensue. – emerging challenges could both help reduce the risks associated The sixth part presents an overview with current challenges and also of possible ways forward. result in infinite impacts;

– the last of the twelve challenges are global policy challenges, threats arising from future global governance as it resorts to destructive policies in response to the categories of challenge listed above.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 9 Executive Summary

The idea that there may be risks where the impact can be A new category described as infinite, defined as the end of human civilisation of global risk or even human life, is not new. However, it excites relatively little political or academic interest, and the way it is treated in popular culture makes a serious discussion more difficult.

For several reasons the potentially The review of literature indicates infinite impacts of the challenges in that, under a business as usual this report are not as well known as scenario, new risks with potential they should be. One reason is the infinite impact are probably way that extreme impacts are often inseparable from the rapid masked by most of the theories and technological development in models used by governments and areas like synthetic biology, business today. nanotechnology and AI.

Climate change is a good example, Most risks are linked to increased where almost all of the focus is on knowledge, economic and technical the most likely scenarios, and there development that has brought many are few public studies that include benefits. E.g. climate change is a the low-probability high-impact result from the industrial revolution scenarios. In most reports about and fossil fuel based development. climate impacts, those caused by The increased potential for global warming beyond five or six degrees pandemics is one consequence of Celsius are omitted from tables an integrated global economy where and graphs. Other aspects that goods and services move quickly contribute to this relative invisibility internationally. Similar challenges can include the fact that extreme be expected for synthetic biology, impacts are difficult to translate nanotechnology and AI. into monetary terms, as they have a global scope and often require a There are remedies, including time-horizon of a century or more. technological and institutional, They cannot be understood simply by for all risks. But they will require linear extrapolation of current trends, collaboration of a sort humanity has and they lack historical precedents. not achieved before, and the creation There is also the fact that the of systems which can deal with measures required to significantly problems pre-emptively. It is important reduce the probability of infinite to understand that much of the impacts will be radical compared to knowledge and many tools that we a business-as-usual scenario. have, and will develop, can be both a risk and a solution to risks A scientific approach requires us depending on context. to base our decisions on the whole probability distribution.

Risk = Probability x Impact

10 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

Normal Risks New Category Traditional measures Requires new measures and tools and tools applicable

Threshold probability

0 impact Executive Summary

There is a clear ethical dimension to the concept of infinite Infinite impacts impact, because a very small group alive today can take and thresholds decisions that will fundamentally affect all future generations.

Using traditional economic tools events that could result first in a Risk = Probability x Impactis problematic and can generate civilisation collapse, and then later disagreement over issues such result in an infinite impact. Such as discounting, which the report thresholds are especially important examines in some detail, considering to recognise in a complex and for example the role of tipping points. interconnected society where resilience is decreasing. The report distinguishes between the concepts of infinite impact – where A collapse of civilisation is defined civilisation collapses to a state of as a drastic decrease in human great suffering and does not recover, population size and political/ or a situation where all human life economic/social complexity, globally ends – and infinite impact threshold – and for an extended time. an impact that can trigger a chain of

Normal Risks New Category Traditional measures Requires new measures and tools and tools applicable

Threshold probability

0 impact

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 11 Executive Summary

In order to establish a list of global challenges with Methodology potentially infinite impact, a methodological triangulation was used, consisting of:

– A quantitative assessment of Two workshops were arranged relevant literature. where the selection of challenges was discussed, one with risk experts – A strategic selection of relevant in Oxford at the Future of Humanity organisations and their priorities. Institute and the other in London – A qualitative assessment with the with experts from the financial sector. help of expert workshops. No challenge was excluded at the workshops, but one was added: the participants agreed to include Global System Collapse as a category.

Relevant literature Identification of credible sources: search relevant literature in academic literature included in World of Knowledge and Google Scholar. 1 Estimations of impact Only literature where there is some estimation of impact that indicates the possibility of an infinite impact is included.

3 Possibility of addressing the risk From the risks gathered from literature and organisations, only those where the probability or impact can be affected by human actions are included.

Expert review 4 Qualitative assessment: Expert review in order to increase the probability of covering all relevant global risks. 5 List of risks Result: List of risks with potentially infinite impacts. 6

12 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Executive Summary

Quick overview of each risk

Current Current Current risk risk risk

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial Extreme EcologicalFuture Bad NanotechnologyGlobal ArtificialNuclearGlobal War System ExtremeSuper-volcanoMajor Asteroid FutureSynthetic Bad GlobalUnknown Global System Major Asteroid Intelligence Climate Change CatastropheGlobal Governance Pandemic IntelligenceCollapse ClimateImpact Change GlobalBiology Governance PandemicConsequences Collapse Impact

Current Current Exogenic risk risk risk

Ecological Nanotechnology Nuclear War Super-volcano EcologicalSynthetic NanotechnologyUnknown Nuclear War Super-volcano Synthetic Unknown Catastrophe Artificial Extreme Future Bad CatastropheGlobalBiology ConsequencesGlobal System Major Asteroid Biology Consequences Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial ExtremeEcological ArtificialFutureNanotechnology Bad ExtremeGlobalNuclear War FutureGlobal BadSuper-volcano System GlobalMajorSynthetic Asteroid Global SystemUnknown Major Asteroid Intelligence ClimateCatastrophe Change IntelligenceGlobal Governance ClimatePandemic Change GlobalCollapse Governance PandemicImpactBiology CollapseConsequences Impact

Exogenic Emerging Emerging risk risk risk

Ecological Nanotechnology EcologicalNuclear War NanotechnologySuper-volcano NuclearSynthetic War Super-volcanoUnknown Synthetic Unknown Catastrophe Catastrophe Biology Consequences Biology Consequences Artificial Extreme ArtificialFuture Bad ExtremeGlobal FutureGlobal Bad System GlobalMajor Asteroid Global System Major Asteroid Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change IntelligenceGlobal Governance ClimatePandemic Change GlobalCollapse Governance PandemicImpact Collapse Impact Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology EcologicalNuclear War NanotechnologySuper-volcano NuclearSynthetic War Super-volcanoUnknown Synthetic Unknown Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Catastrophe Biology Consequences Biology Consequences Catastrophe Biology Consequences

Emerging Emerging Global Policy risk risk risk

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology NuclearArtificial War Super-volcanoExtreme SyntheticFuture Bad UnknownGlobal ArtificialGlobal System ExtremeMajor Asteroid Future Bad Global Global System Major Asteroid Catastrophe Intelligence Climate Change BiologyGlobal Governance ConsequencesPandemic IntelligenceCollapse Climate ImpactChange Global Governance Pandemic Collapse Impact

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 13

Ecological Nanotechnology Nuclear War Super-volcano EcologicalSynthetic NanotechnologyUnknown Nuclear War Super-volcano Synthetic Unknown Catastrophe CatastropheBiology Consequences Biology Consequences Executive Summary

Current risk Extreme 5 key Climate Change factors: Extreme1 The uncertainties in As for all risks there are uncertainties Mass deaths and famines, social Climate Change climate sensitivity models, in the estimates, and warming could collapse and mass migration are including the tail. be much more extreme than the certainly possible in this scenario. 2 The likelihood - or not - of middle estimates suggest. Feedback Combined with shocks to the global coordination on loops could mean global average agriculture and biosphere-dependent controlling emissions. temperatures increase by 4°C or industries of the more developed even 6°C over pre-industrial levels. countries, this could lead to global 3 The future uptake of Feedbacks could be the release conflict and possibly civilisation low carbon economies, of methane from permafrost or the collapse. Further evidence of the including energy, mobility dieback of the Amazon rainforest. risk comes from signs that past and food systems. The impact of global warming would civilisation collapses have been 4 Whether technological be strongest in poorer countries, driven by climate change. innovations will improve which could become completely or worsen the situation, uninhabitable for the highest range and by how much. of warming. 5 The long-term climate impact caused by global warming.

Current risk

5 key Nuclear War factors:

Nuclear1 How War relations between The likelihood of a full-scale nuclear plunge temperatures below freezing current and future war between the USA and Russia around the globe and possibly also nuclear powers develop. has probably decreased. Still, the destroy most of the ozone layer. 2 The probability of potential for deliberate or accidental The detonations would need to accidental war. nuclear conflict has not been start firestorms in the targeted removed, with some estimates cities, which could lift the soot up 3 Whether disarmament putting the risk in the next century into the stratosphere. The risks are efforts will succeed in or so at around 10%. A larger impact severe and recent models have reducing the number of would depend on whether or not the confirmed the earlier analysis. The nuclear warheads. war triggered what is often called a disintegration of the global food 4 The likelihood of a nuclear winter or something similar – supply would make mass starvation nuclear winter. the creation of a pall of smoke high and state collapse likely. 5 The long-term effects in the stratosphere that would of a nuclear war on climate, infrastructure and technology. A new category of global risk.

14 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Executive Summary

Current risk Global 5 key Pandemic factors:

Global1 What the true probability An epidemic of infectious disease (and influenza has demonstrated Pandemic distribution for pandemics that has spread through human antigenic shift, the ability to combine is, especially at the tail. populations across a large region or features from different viruses), its 2 The capacity of international even worldwide. There are grounds death toll would be extreme. health systems to deal for suspecting that such a high- The world has changed considerably, with an extreme pandemic. impact epidemic is more probable making comparisons with the past than usually assumed. All the problematic.Today it has better 3 How fast medical research features of an extremely devastating sanitation and medical research, as can proceed in an disease already exist in nature: well as national and supra-national emergency. essentially incurable (Ebola), nearly institutions dedicated to combating 4 How mobility of goods and always fatal (rabies), extremely diseases. But modern transport people, as well as population infectious (common cold), and and dense human population allow density, will affect pandemic long incubation periods (HIV). If infections to spread much more transmission. a pathogen were to emerge that rapidly, and slums can be breeding 5 Whether humans can somehow combined these features grounds for disease. develop novel and effective anti-pandemic solutions.

Current risk Ecological 5 key Collapse factors: Ecological Collapse1 The extent to which This is where an ecosystem suffers the damage and (unlike previous, humans are dependent on a drastic, possibly permanent, localised collapses) the whole world the ecosystem. reduction in carrying capacity for is potentially at risk. 2 Whether there will be all organisms, often resulting in It seems plausible that some human effective political measures mass extinction. Humans are part lifestyles could be sustained in a taken to protect the of the global ecosystem and so relatively ecosystem independent ecosystem on a large scale. fundamentally depend on it. Species way, at relatively low costs. Whether extinction is now far faster than the this can be achieved on a large 3 The likelihood of the historic rate, and attempts to scale in practice, especially during emergence of sustainable quantify a safe ecological operating a collapse, will be a technological economies. space place humanity well outside it. challenge and whether it is something 4 The positive and negative Many of the problems of ecological we want is an ethical question. impacts on the ecosystems degradation interact to multiply of both wealth and poverty. 5 The long-term effects of an ecological collapse on ecosystems.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 15 Executive Summary

Current risk Global System 5 key Collapse factors: Global1 WhetherSystem global system An economic or societal collapse structure of the network – even if Collapse collapse will trigger on the global scale. The term has each component of the network is subsequent collapses or been used to describe a broad reliable. This gives rise to systemic fragility in other areas. range of conditions. Often economic risk, when parts that individually may 2 What the true trade-off is collapse is accompanied by social function well become vulnerable between efficiency chaos, civil unrest and sometimes a when connected as a system to a and resilience. breakdown of law and order. Societal self-reinforcing joint risk that can collapse usually refers to the fall or spread from part to part, potentially 3 Whether effective disintegration of human societies, affecting the entire system and regulation and resilience often along with their life support possibly spilling over to related can be developed. systems. The world economic and outside systems. Such effects have 4 Whether an external political system is made up of been observed in ecology, finance disruption will trigger many actors with many objectives and critical infrastructure such a collapse. and many links between them. as power grids. The possibility of 5 Whether an internal event Such intricate, interconnected collapse becomes more acute when will trigger a collapse. systems are subject to unexpected several independent networks system-wide failures caused by the depend on each other.

Exogenic risk Major Asteroid 5 key Impact factors: Major Asteroid Impact1 Whether detection and Large asteroid collisions – with There has been some discussion tracking of asteroids and objects 5 km or more in size – about possible methods for other dangerous space happen about once every twenty deflecting asteroids found on a objects is sufficiently million years and would have an collision course with the planet. exhaustive. energy a hundred thousand times Should an impact occur the main 2 How feasible it is to deflect greater than the largest bomb ever destruction will not be from the an asteroid. detonated. A land impact would initial impact, but from the clouds destroy an area the size of a nation of dust projected into the upper 3 Whether measures such as like Holland. Larger asteroids could atmosphere. The damage from such evacuation could reduce be extinction-level events. Asteroid an “impact winter” could affect the damage of an impact. impacts are probably one of the best the climate, damage the biosphere, 4 The short- and long-term understood of all risks in this report. affect food supplies, and create climate consequences of a political instability. collision. 5 Whether our current civilisation could adapt to a post-impact world.

16 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Executive Summary

Exogenic risk

5 key Super-volcano factors: Super-volcano1 Whether countries will Any volcano capable of producing while the Toba eruption around coordinate globally an eruption with an ejecta volume 70,000 years ago is thought by some against super-volcano risk greater than 1,000 km3. This is to have cooled global temperatures and damage. thousands of times larger than for over two centuries. 2 The predictability of super- normal eruptions. The danger from The effect of these eruptions could volcanic eruptions. super-volcanoes is the amount of be best compared with that of a aerosols and dust projected into the nuclear war. The eruption would 3 How directly destructive upper atmosphere. This dust would be more violent than the nuclear an eruption would be. absorb the Sun’s rays and cause explosions, but would be less likely 4 The effectiveness of a global volcanic winter. The Mt to ignite firestorms and other general mitigation efforts. Pinatubo eruption of 1991 caused secondary effects. 5 How severe the long-term an average global cooling of surface climate effects would be. temperatures by 0.5°C over three years,

Emerging risk Synthetic 5 key Biology factors: Synthetic Biology1 The true destructive potential The design and construction of This could emerge through military of synthetic biology, especially biological devices and systems or commercial bio-warfare, bio- the tail risk. for useful purposes, but adding terrorism (possibly using dual-use 2 Whether the field will be human intentionality to traditional products developed by legitimate successfully regulated, or pandemic risks. Attempts at researchers, and currently successfully manage to regulation or self-regulation are unprotected by international legal regulate itself. currently in their infancy, and may regimes), or dangerous pathogens not develop as fast as research leaked from a lab. Of relevance is 3 Whether the field will usher does. One of the most damaging whether synthetic biology products in a new era of bio-warfare. impacts from synthetic biology become integrated into the global 4 Whether the tools of synthetic would come from an engineered economy or biosphere. This could biology can be used pathogen targeting humans or a lead to additional vulnerabilities (a defensively to create effective crucial component of the ecosystem. benign but widespread synthetic counter measures. biology product could be specifically 5 The dangers of relying targeted as an entry point through on synthetic biologists to which to cause damage). estimate the danger of synthetic biology.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 17 Executive Summary

Emerging risk

5 key Nanotechnology factors: Nanotechnology1 The timeline for nanotech Atomically precise manufacturing, Of particular relevance is whether development. the creation of effective, high- nanotechnology allows the 2 Which aspects of throughput manufacturing processes construction of nuclear bombs. nanotech research will that operate at the atomic or But many of the world’s current progress in what order. molecular level. It could create problems may be solvable with the new products – such as smart or manufacturing possibilities that 3 Whether small groups can extremely resilient materials – and nanotechnology would offer, such assemble a weapons would allow many different groups as depletion of natural resources, arsenal quickly. or even individuals to manufacture a pollution, climate change, clean 4 Whether nanotech tools wide range of things. This could lead water and even poverty. Some have can be used defensively to the easy construction of large conjectured special self-replicating or for surveillance. arsenals of conventional or more nanomachines which would be 5 Whether nanotech tools or novel weapons made possible by engineered to consume the entire weaponry are made to be atomically precise manufacturing. environment. The misuse of medical outside human control. nanotechnology is another risk scenario.

Emerging risk

Artificial And if these motivations do not detail the survival and value of 5 key Intelligence humanity, the intelligence will be factors: driven to construct a world without humans. This makes extremely Artificial Intelligence1 The reliability of AI AI is the intelligence exhibited by intelligent AIs a unique risk, in that predictions. machines or software, and the extinction is more likely than lesser 2 Whether there will be a branch of computer science that impacts. On a more positive note, single dominant AI or a develops machines and software an intelligence of such power could plethora of entities. with human-level intelligence. easily combat most other risks 3 How intelligent AIs will The field is often defined as “the in this report, making extremely become. study and design of intelligent intelligent AI into a tool of great 4 Whether extremely agents”, systems that perceive their potential. There is also the possibility intelligent AIs can be environment and act to maximise of AI-enabled warfare and all the controlled, and if so, how. their chances of success. Such risks of the technologies that AIs 5 Whether whole brain extreme intelligences could not would make possible. An interesting emulations (human minds easily be controlled (either by the version of this scenario is the in computer form) will groups creating them, or by some possible creation of “whole brain arrive before true AIs. international regulatory regime), emulations”: human brains scanned and would probably act to boost and physically represented in a their own intelligence and acquire machine. This would make the AIs maximal resources for almost all into properly human minds, possibly initial AI motivations. alleviating a lot of problems.

18 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Executive Summary

Emerging risk Unknown 5 key Consequences factors: Unknown1 Whether there will be These represent the unknown generic probability of intelligent life Consequences extensive research into unknowns in the family of global (self-)destruction, which includes unknown risks and their catastrophic challenges. They uncertain risks. Anthropic reasoning probabilities. constitute an amalgamation of all the can also bound the total risk of human 2 The capacity to develop risks that can appear extremely extinction, and hence estimate the methods for limiting unlikely in isolation, but can combine unknown component. Non risk- the combined probability to represent a not insignificant specific resilience and post-disaster of all uncertain risks. proportion of the risk exposure. One rebuilding efforts will also reduce 3 The capacity for estimating resolution to the Fermi paradox – the damage from uncertain risks, “out of-model” risks. the apparent absence of alien life as would appropriate national and 4 The culture of risk. in the galaxy – is that intelligent life international regulatory regimes. assessment in potentially destroys itself before beginning to Most of these methods would also risky areas. expand into the galaxy. Results that help with the more conventional, 5 Whether general, non- increase or decrease the probability known risks, which badly need risk-specific mitigation or of this explanation modify the more investment. resilience measures are implemented.

Global Policy risk Future Bad 5 key Global Governance factors: Future Bad Global1 HowGovernance the severity of non- There are two main divisions in Two issues with governance deadly policy failures governance disasters: failing to disasters are first, the difficulty can be compared with solve major solvable problems, and of estimating their probability, potential casualties. actively causing worse outcomes. and second, the dependence of 2 Whether poor governance An example of the first would be the impact of these disasters on will result in a collapse failing to alleviate absolute poverty; subjective comparative evaluations: of the world system. of the second, constructing a it is not impartially obvious how to global totalitarian state. Technology, rank continued poverty and global 3 How mass surveillance political and social change may totalitarianism against billions of and other technological enable the construction of new casualties or civilisation collapse. innovations will affect forms of governance, which may be governance. either much better or much worse. 4 Whether there will be new systems of governance in the future. 5 Whether a world dictatorship may end up being constructed.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 19 Executive Summary

Two things make the understanding of the relation Relations between between the global risks particularly important. global risks 1. Impacts: The risks are 2. Specific measures to address a interconnected in different ways. risk: Global risks often require Often the situation resembles a significant changes, which will set of dominoes: if one falls, many result in situations where measures to follow. Even small impacts can reduce the risk in one area affect start a process where different the probability and/or the impact in risks interact. other areas, for better or worse.

ALL RISKS

first risk worsens second risk

solving first risk improves second risk

both of the above

20 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Executive Summary

In order to better understand the relations between The technical different global risks, work could start to analyse difficulty of similarities and differences. reducing the risk Below is an example of an overview of how different global risks can be and the difficulty plotted depending on the technical difficulty of reducing the risk and the of collaboration difficulty of collaborating to reduce it. technical difficulty of reducing risk of reducing technical difficulty

collaboration difficulty of reducing risk

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 21 Executive Summary

As the different challenges are very different and the Uncertainties status of probability estimates varies significantly, the initial probability numbers are provided together with estimates regarding:

1. Understanding1. Understanding 2. Data2. Data 3. Existing probability 1. Understanding of 1. sequence ofUnderstanding sequence 2. availabilityData availability2. Data 3. estimationExisting probability of sequence of sequence availability availability estimation all parts all data all parts all data all parts all data all parts all parts all data all data calculations with small uncertainty calculations with small uncertainty calculations with small uncertainty most parts most data most parts most data most parts most data most parts most parts most data most data calculations with uncertainty large calculations with uncertainty large calculations with uncertainty large some parts some data some parts some data best guesses by experts some parts some data best guesses by experts some parts some parts some data some data best guesses by experts no estimates none at all no data none at all no data

degreedegreenone at all of events of events from today’sfrom today’s actions actions amountno data amount of data of todata make to make probability probability kindno estimates of estimation and uncertainty to infiniteto infinite impact impact assessmentassessment on all onrelevant all relevant steps steps no estimates none at all none at all ofno data theof sequence the sequenceno data

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps degree of eventsdegree from of events today’s from actions today’s actions amountof the sequence of dataamount to make of data probability to make probabilitykind of estimation and uncertainty to infinite impactto infinite impact assessmentassessment on all relevant on allsteps relevant steps of the sequenceof the sequence

22 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

n / a

0.00003% n / a 0.0001% 0.00013% 0.005% 0.01% 0.1% 0-10%

infinite impact %

n / a

n / a 0.002% 0.01% 0.5% 0.8% 1% 5% 0-10%

infinite treshold %

0.0000001% 0.00001% 0.0001% 0.001% 0.01% 0.1% 1% 10% 100% one in one one one in one in one one one one hundred in ten in a hundred ten in a in a in ten in one million million million thousand thousand thousand hundred Executive Summary

These estimates are an attempt to assemble existing Probability estimates in order to encourage efforts to improve the numbers. They express estimates of probabilities over 100 years, except in the case of extreme climate change, where the time frame is 200 years.

Global challenges need to be seen in Population growth – the UN’s estimates the light of trends which help to shape range from 6.8 billion people by 2100 the wider society. These include: to a high-variant projection of 16.6 bn (which would require the resources of Poverty – although it has fallen, 10 Earth-like planets to provide everyone it could increase again. This is with a modern Western lifestyle). especially relevant to climate change Other trends include technological and pandemics. development and demographic changes.

n / a

0.00003% n / a 0.0001% 0.00013% 0.005% 0.01% 0.1% 0-10%

Probability of infinite impact (%)

n / a

n / a 0.002% 0.01% 0.5% 0.8% 1% 5% 0-10%

Probability of reaching or exceeding the infinite threshold (%)

0.0000001% 0.00001% 0.0001% 0.001% 0.01% 0.1% 1% 10% 100% one in a one one one in a one in one one one one hundred in ten in a hundred ten in a in a in ten in one million million million thousand thousand thousand hundred

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 23

0.01% 0.01% 0.001% one in ten 0.1% one in hundred thousand one in thousand thousand 0.0001% 1% one in a one in n/a n/a 0.8% million hundred

0.00001% 10% one in ten one in million infinite impact % ten

0.0000001% infinite threshold % 100% one in hundred one in one 0.01% million 0.01% 0.01%

0.00013%

0.01% 0.0001% 0.002% 0.1%

5% 5% 0.00003% 5%

0.005% 0.5% 0.01%

5% n/a 1% n/a n/a Executive Summary

There are ten areas that could help mitigate immediate Possible threats while also contributing to a future global ways forward governance system capable of addressing global risks with a potential infinite impact:

Global challenges 1. leadership networks Better quality risk 2. assessment for global challenges

Development of 3. early warning systems Encouraging 4. visualisation of complex systems

Highlighting 5. early movers Including the whole 6. probability distribution Increasing the focus on 7. the probability of extreme events

Encouraging 8. appropriate language to describe extreme risks

Establishing a Global Risk and 9. Opportunity Indicator to guide governance

Explore the possibility of establishing a 10. Global Risk Organisation (GRO)

24 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Executive Summary

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 25 Preface

Preface

Over the last century the world has changed in ways that humanity has never experienced within our history. The changes are being caused by the extremely rapid development of science and technology, by the population explosion that has quadrupled the number of people on Earth, and by a greatly improved but very resource-demanding standard of living in developed countries.

The consequences of these changes are very diverse:

– Less poverty, better health and longer life in many countries. – Globalisation, the most important effect of which is the emergence of a shattered global community where all people’s behaviour affects each other’s vital interests. – New global risks of previously unseen scope.

This means that we are now forced to live with the risk of various kinds of extreme disaster with the potential of severely affecting billions of people.

26 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Preface

In this Yearbook from the Global of global catastrophes are we decision-making system be Challenges Foundation, “risk” is prepared to accept? This question created - with or without a global defined as the potential damage has not yet appeared on the legal system? that can be caused by an extreme political agenda. The reason is that disaster multiplied by the probability both scientific reports and the media We are also convinced that that it will occur. choose to focus on the most likely knowledge of these risks is not only outcome of these risks. a prerequisite for reducing them, but For the risk of exceptional damage, also a responsibility which we owe the probability of occurrence In the absence of risk analysis both to our children, grandchildren and to is usually small, or very small, decision-makers and the public remain all future generations. It is up to us compared with other risks in society, blissfully unaware that the probabilities to decide whether these threats can but the effects can be absolutely of certain global catastrophes are possibly be reduced or not! These dire, meaning they must be taken significantly higher than we would efforts do not only demand sacrifices very seriously. accept in our everyday lives, where on our part. They also create incomparably smaller values are at stake. opportunities for everyone to make a We do not know what the exact Another, very important reason for significant contribution to improving nature of what these risks are or how not acting against acknowledged the future of humanity: they may strike. Some are obvious, global risks is that they require global others may sound like pure science responses and therefore global – For world leaders this means fiction, but they have led many decisions. assuming their responsibility and scientists to regard them as real starting to work towards common, threats - and therefore it is best to Regrettably there is no global global decision-making. include them in the calculations. decision-making body capable of – Scientists need to focus their that, no globally functioning legal research on areas that will help us take With few exceptions, humans have system, and so there is a lack of effective measures against the risks. created these risks. There are only effective tools for dealing with these – Companies should make a few risks where we are not the challenges. The result: the risks are sustainability a business model. cause, for example natural disasters increased in the absence of effective – And there is a special opportunity such as an asteroid impact. measures to counter them. for all of us - that when choosing our politicians and suppliers (of We could eliminate some of these This report wants, on a strictly goods and services), we should risks (e.g. nuclear war). In other scientific basis, to identify and consider their ambition to eliminate cases, all we can do is minimise the describe the global risks of extreme or at least minimise global risks likelihood of damage, since we have disasters, and also to report the latest and to create an efficient decision- already crossed the threshold that developments affecting these risks making system that can manage can lead to serious consequences and measures to face up to them. these risks. (with climate change, for example, where we have already emitted such The Global Challenges Foundation’s Finally, I would on behalf of the high levels of greenhouse gases that goal in this report is to accelerate Global Challenges Foundation extend there are small but not insignificant effective counter-actions against my sincere gratitude to both Dennis likelihoods of significant damage). global events with the potential for Pamlin, editor of the report, and to all For other risks we cannot affect large-scale unwanted effects by the scientists and other experts who the likelihood of them occurring, deepening both decision makers’ and have contributed their research and / only minimise damage (with super- the public’s insights into the risks, and or valuable comments. volcanic eruptions, for instance). also to inspire both debate and well- However, here we can build social judged decisions on these questions: and ecological resilience so as to reduce the damage. – What probabilities of extreme disasters are acceptable? For decisions concerning – Which are the optimal Laszlo Szombatfalvy countermeasures the first important countermeasures? Founder and Chairman, question is: What level of probability – How can an effective global The Global Challenges Foundation

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 27 1. Twelve risks threaten human civilisation

1. Twelve risks that threaten human civilisation

“Tell me, and I’ll forget. Show me, and I may remember. Involve me, and I’ll understand.”

Xunzi

28 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 1. Twelve risks threaten human civilisation

Current Current Current risk risk risk

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Current Current Exogenic risk risk risk Ecological Nanotechnology Nuclear War Super-volcano EcologicalSynthetic NanotechnologyUnknown Nuclear War Super-volcano Synthetic Unknown Catastrophe CatastropheBiology Consequences Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Exogenic Emerging Emerging

Ecological Nanotechnology risk EcologicalNuclear War NanotechnologySuper-volcano Nuclear riskSynthetic War Super-volcanoUnknown Synthetic riskUnknown Catastrophe Catastrophe Biology Consequences Biology Consequences

Artificial Extreme ArtificialFuture Bad ExtremeGlobal FutureGlobal Bad System GlobalMajor Asteroid Global System Major Asteroid Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change IntelligenceGlobal Governance ClimatePandemic Change GlobalCollapse Governance PandemicImpact Collapse Impact Intelligence Climate Change Global Governance Pandemic Collapse Impact

Emerging Emerging Global Policy risk risk risk

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 29

2. Risks with infinite impact: A new category of risks

“Most risk management is really just advanced contingency planning and disciplining yourself to realise that, given enough time, very low probability events not only can happen, but they absolutely will happen.”

Lloyd Blankfein, Goldman Sachs CEO, July 2013 1

30 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2. Risks with infinite impact: A new category of risks

A new group of global risks What is risk? This is a report about a limited Risk is the potential of losing something Risk = Probability × Impact number of global risks – that can of value, weighed against the potential be identified through a scientific to gain something of value. Every Impacts where civilisation and transparent process – with day we make different kinds of risk collapses to a state of great impacts of a magnitude that pose assessments, in more or less rational suffering and do not recover, a threat to human civilisation, or ways, when we weigh different options or a situation where all human even possibly to all human life. against each other. life end, are defined as infinite as the result is irreversible and With such a focus it may surprise The basic idea of risk is that lasts forever. some readers to find that the report’s an uncertainty exists regarding essential aim is to inspire action and the outcome and that we must dialogue as well as an increased find a way to take the best The slightly tilted bell curve is a use of the methodologies used for possible decision based on our common probability distribution, risk assessment. understanding of this uncertainty.3 but the shape differs and in reality is seldom as smooth as the example. The real focus is not on the almost To calculate risk the probability of an unimaginable impacts of the risks outcome is often multiplied by the The total area under the curve the report outlines. Its fundamental impact. The impact is in most cases always represents 100 percent, i.e. purpose is to encourage global measured in economic terms, but it all the possible outcomes fit under collaboration and to use this new can also be measured in anything the curve. In this case (A) represents category of risk as a driver we want to avoid, such as suffering. the most probable impact. With a for innovation. much lower probability it will be a At the heart of a risk assessment close to zero impact, illustrated by The idea that we face a number is a probability distribution, often (B). In the same way as in case B of global challenges threatening described by a probability density there is also a low probability that the very basis of our civilisation at function4; see figure X for a the situation will be very significant, the beginning of the 21st century graphic illustration. illustrated by (C). is well accepted in the scientific community, and is studied at a number of leading universities.2

But there is still no coordinated approach to address this group of challenges and turn them into opportunities for a new generation of global cooperation and the creation of a global governance system capable of addressing the greatest challenges of our time. A

This report has, to the best of our knowledge, created the first science- based list of global risks with a potentially infinite impact and has made the first attempt to provide an initial overview of the uncertainties C related to these risks as well as rough quantifications for the D probabilities of these impacts. probability B

0 impact

Figure 1:Probability density function

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 31 2. Risks with infinite impact: A new category of risks

The impacts (A), (B) and (C) all On a probability curve the impacts in Not only could a better belong to the same category, normal this report are usually at the very far understanding of the unique impacts: the impacts may be more right with a relatively low probability magnitude of these risks help or less serious, but they can be dealt compared with other impacts, address the risks we face, it could with within the current system. illustrated by (D) in Figure 2. also help to create a path towards more sustainable development. The impacts in this report are Often they are so far out on the tail however of a special kind. These of the curve that they are not even The group of global risks discussed are impacts where everything will included in studies. in this report are so different from be lost and the situation will not most of the challenges we face that be reversible, i.e challenges with For each risk in this report the they are hard to comprehend. potentially infinite impact. probability of an infinite impact is very low compared to the most But that is also why they can help us In insurance and finance this kind of likely outcome. Some studies even to build the collaboration we need risk is called “risk of ruin”, an impact indicate that not all risks in this report and drive the development of further where all capital is lost.5 This impact can result in an infinite impact. But a solutions that benefit both people is however only infinite for the significant number of peer-reviewed and the planet. company that is losing the money. reports indicate that those impacts From society’s perspective, that is not only can happen, but that their As noted above, none of the risks in not a special category of risk. probability is increasing due to this report is likely to result directly unsustainable trends. in an infinite impact, and some are In this report the focus is on the “risk probably even physically incapable of ruin” on a global scale and on a The assumption for this report is that of doing so. But all are so significant human level, in the worst case this by creating a better understanding that they could reach a threshold is when we risk the extinction of our of our scientific knowledge regarding impact able to create social and own species. risks with a potentially infinite impact, ecological instability that could we can inspire initiatives that can turn trigger a process which could lead to these risks into drivers for innovation. an infinite impact.

For several reasons the potentially infinite impacts of the risks in this Figure 2: Probability density function with tail highlighted report are not as well known as they should be. One reason is the way that extreme impacts are often masked by most of the theories and models used by governments and business today.

For example, the probability of A extreme impacts is often below what is included in studies and strategies.

The tendency to exclude impacts below a probability of five percent is one reason for the relative “invisibility” of infinite impacts. C The almost standard use of a 95% confidence interval is one reason D why low-probability high-impact

probability B events are often ignored.6

0 impact

32 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2. Risks with infinite impact: A new category of risks

Climate change is a good example, There is also the fact that the It should also be stressed that where almost all of the focus is on measures required to significantly uncertainty is not a weakness in the most likely scenarios and there reduce the probability of infinite science; it always exists in scientific are few studies that include the low- impacts will be radical compared to work. It is a systematic way of probability high-impact scenarios. In a business-as-usual scenario with a understanding the limitations of the most reports about climate impacts, focus on incremental changes. methodology, data, etc.9 Uncertainty the impacts caused by warming is not a reason to wait to take action beyond five or six degrees Celsius The exact probability of a specific if the impacts are serious. Increased are even omitted from tables and impact is difficult or impossible to uncertainty is something that risk graphs even though the IPCC’s estimate.8 However, the important experts, e.g. insurance experts and own research indicates that the thing is to establish the current security policy experts, interpret as a probability of these impacts are magnitude of the probabilities and signal for action. often between one and five percent, compare them with the probabilities and sometimes even higher.7 for such impacts we cannot accept. A contrasting challenge is that our A failure to provide any estimate for cultural references to the threat of Other aspects that contribute to this these riks often results in strategies infinite impacts have been dominated relative invisibility include the fact and priorities defined as though the throughout history by religious groups that extreme impacts are difficult to probability of a totally unacceptable seeking to scare society without any translate into monetary terms, they outcome is zero. An approximate scientific backing, often as a way have a global scope, and they often number for a best estimate also to discipline people and implement require a time-horizon of a century makes it easier to understand that unpopular measures. It should not or more. They cannot be understood a great uncertainty means the have to be said, but this report is simply by linear extrapolation actual probability can be both obviously fundamentally different as of current trends, and they lack much higher and much lower than it focuses on scientific evidence from historical precedents. the best estimate. peer-reviewed sources.

Infinite impact

The concept infinite impact refers to two aspects in particular; the terminology is not meant to imply a literally infinite impact (with all the mathematical subtleties that would imply) but to serve as a reminder that these risks are of a different nature.

Ethical

These are impacts that threaten the very survival of humanity and life on Earth – and therefore can be seen as being infinitely negative from an ethical perspective. No positive gain can outweigh even a small probability for an infinite negative impact. Such risks require society to ensure that we eliminate these risks by reducing the impact below an infinite impact as a top priority, or at least do everything we can to reduce the probability of these risks. As some of these risks are impossible to eliminate today it is also important to discuss what probability can right now be accepted for risks with a possible infinite impact.

Economic

Infinite impacts are beyond what most traditional economic models today are able to cope with. The impacts are irreversible in the most fundamental way, so tools like cost-benefit assessment seldom make sense. To use discounting that makes infinite impacts (which could take place 100 years or more from now and affect all future generations) close to invisible in economic assessments, is another example of a challenge with current tools. So while tools like cost-benefit models and discounting can help us in some areas, they are seldom applicable in the context of infinite impacts. New tools are needed to guide the global economy in an age of potential infinite impacts.

See chapter 2.2.2 for a more detailed iscussion.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 33 2. Risks with infinite impact: A new category of risks

An additional challenge in This report emphasises the need for in most cases, but it is unscientific acknowledging the risks outlined an open and democratic process in and dangerous if different levels of in this report is that many of the addressing global challenges with probability are presented as equal. traditional risks including wars and potentially infinite impact. Hence, violence have decreased, even though this is a scientifically based invitation The tendency to compare the it might not always looks that way in to discuss how we as a global most probable climate impact media.10 So a significant number of community can address what could with the possibility of a low or no experts today spend a substantial be considered the greatest challenges impact also results in a situation amount of time trying to explain of our time. where low-probability high-impact that much of what is discussed as outcomes are often totally ignored. dangerous trends might not be as The difficulty for individual scientists An honest and scientific approach dangerous as we think. For policy to communicate a scientific risk is to, whenever possible, present makers listening only to experts in approach should however not be the whole probability distribution traditional risk areas it is therefore underestimated. Scientists who and pay special attention to easy to get the impression that global today talk about low-probability unacceptable outcomes. risks are becoming less of a problem. impacts, that are serious but still far from infinite, are often accused The fact that we have challenges The chain of events that could of pessimism and scaremongering, that with some probability might be result in infinite impacts in this even if they do nothing but highlight infinite and therefore fundamentally report also differ from most of the scientific findings.11 To highlight infinite irreversible is difficult to comprehend, traditional risks, as most of them impacts with even lower probability and physiologically they are are not triggered by wilful acts, can therefore be something that a something our brains are poorly but accidents/mistakes. Even the scientist who cares about his/her equipped to respond to, according to probabilities related to nuclear war reputation would want to avoid. evolutionary psychologists.12 in this report are to a large degree It is hard for us as individuals to related to inadvertent escalation. In the media it is still common grasp that humanity for the first As many of the tools to analyse and to contrast the most probable time in its history now has the address risks have been developed climate impact with the probability capacity to create such catastrophic to protect nations and states from that nothing, or almost nothing, outcomes. Professor Marianne attacks, risks involving accidents will happen. The fact that almost Frankenhaeuser, former head of tend to get less attention. nothing could happen is not wrong the psychology division, Karolinska

Roulette and Russian roulette

When probability and normal risks are discussed the example of a casino and roulette is often used. You bet something, then spin the wheel and with a certain probability you win or lose. You can use different odds to discuss different kinds of risk taking. These kinds of thought experiment can be very useful, but when it comes to infinite risks these gaming analogies become problematic. For infinite impact a more appropriate analogy is probably Russian roulette. But instead of “normal” Russian roulette where you only bet your own life you are now also betting everyone you know and everyone you don’t know. Everyone alive will die if you lose. There will be no second chance for anyone as there will be no future generations; humanity will end with your loss. What probability would you accept for different sums of money if you played this version of Russian roulette? Most people would say that it is stupid and – no matter how low the probability is and no matter how big the potential win is – this kind of game should not be played, as it is unethical. Many would also say that no person should be allowed to make such a judgment, as those who are affected do not have a say. You could add that most of those who will lose from it cannot say anything as they are not born and will never exist if you lose. The difference between ordinary roulette and “allhumanity Russian roulette” is one way of illustrating the difference in nature between a “normal” risk that is reversible, and a risk with an infinite impact.

34 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2. Risks with infinite impact: A new category of risks

Institute, Stockholm, put it this way: curve exists. With climate change The LA-602 document can be seen “Part of the answer is to be found in this includes discussions about how as the first scientific global risk psychological defence mechanisms. sensitive the climate is, how much report addressing a category of risks The nuclear threat is collectively greenhouse gas will be emitted, and where the worst possible impact in denied, because to face it would what impacts that different warmings all practical senses is infinite.19 Since force us to face some aspects of the will result in. the atomic bomb more challenges world’s situation which we do not have emerged with potentially want to recognise.” 13 Just as it is important not to ignore infinite impact. Allmost all of these challenges with potentially infinite new challenges are linked to the This psychological denial may be impacts, it is also important not to increased knowledge, economic one reason why there is a tendency use them to scare people. Dramatic and technical development that among some stakeholders to images and strong language are best has brought so many benefits. For confuse “being optimistic” with avoided whenever possible, as this example, climate change is the denying what science is telling us, group of risks require sophisticated result of the industrial revolution and and ignoring parts of the probability strategies that benefit from rational development that was, and still is, curve.14 Ignoring the fact that there is arguments. Throughout history based heavily on fossil fuel. strong scientific evidence for serious we have seen too many examples impacts in different areas, and when threats of danger have been The increased potential for global focusing only on selected sources damagingly used to undermine pandemics is the result of an which suggest that the problem may important values. integrated global economy where not be so serious, is not optimistic. It goods and services move quickly is both unscientific and dangerous.15 around the world, combined The history of infinite impacts: with rapid urbanisation and high A scientific approach requires us The LA-602 document population density. to base our decisions on the whole The understanding of infinite impacts probability distribution. Whether it is is very recent compared with most In parallel with the increased number possible to address the challenge or of our institutions and laws. It is only of risks with possible infinite impact, not is the area where optimism and 70 years ago that Edward Teller, our capacity to analyse and solve pessimism can make people look at one of the greatest physicists of his them has greatly increased too. the same set of data and come to time, with his back-of-the-envelope Science and technology today different conclusions. calculations, produced results that provides us with knowledge and differed drastically from all that tools that can radically reduce the Two things are important to keep had gone before. His calculations risks that historically have been in mind: first, that there is always a indicated that the explosion of a behind major extinctions, such as probability distribution when it comes nuclear bomb – a creation of some pandemics and asteroids. to risk; second, that there are two of the brightest minds on the planet, different kinds of impacts that are of including Teller himself – could result Recent challenges like climate interest for this report. The probability in a chain reaction so powerful that it change, and emerging challenges distribution can have different shapes would ignite the world’s atmosphere, like synthetic biology and but in simplified cases the shape thereby ending human life on Earth.16 nanotechnology, can to a large tends to look like a slightly modified degree be addressed by smart use clock (remember figure 1). Robert Oppenheimer, who led the of new technologies, new lifestyles Manhattan Project to develop the and institutional structures. It will be In the media it can sound as though nuclear bomb, halted the project to hard as it will require collaboration experts argue whether an impact, see whether Teller’s calculations were of a kind that we have not seen for example a climate impact or a correct.17 The resulting document, LA- before. It will also require us to pandemic, will be dangerous or not. 602: Ignition of the Atmosphere with create systems that can deal with But what serious experts discuss is Nuclear Bombs, concluded that Teller the problems before they occur. The the probability of different oucomes. was wrong, But the sheer complexity fact that the same knowledge and They can disagree on the shape of drove them to end their assessment tools can be both a problem and a the curve or what curves should be by writing that “further work on the solution is important to understand studied, but not that a probability subject [is] highly desirable”.18 in order to avoid polarisation.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 35 2. Risks with infinite impact: A new category of risks

Within a few decades, or even sooner, The point of departure of this report Creating innovative and resilient many of the tools that can help us is the fact that we now have the systems rather than simply solve the global challenges of today knowledge, economic resources and managing risk would let us focus will come from fields likely to provide technological ability to reduce most more on opportunities. But the us with the most powerful instruments of the greatest risks of our time. resilience needed require moving we have ever had – resulting in their away from legacy systems is likely own sets ofRisk challenges.= Probability x ImpactConversely, the infinite impacts to be disruptive, so an open and we face are almost all unintended transparent discussion is needed Synthetic biology, nanotechnology results of human ingenuity. The regarding the transformative and artificial intelligence (AI) are all reason we are in this situation is solutions required. rapidly evolving fields with great that we have made progress in potential. They may help solve many many areas without addressing of today’s main challenges or, if not unintended low-probability high- guided in a benign direction, may impact consequences. result in catastrophic outcomes.

Figure 3: Probability density function with tail and threshold highlighted

Normal Risks New Category Traditional measures Requires new measures and tools and tools applicable

Threshold probability

0 impact

36 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.1 Report structure

2.1 Report structure “Current challenges” is the first The third part of the report category and includes the risks that discusses the relationship currently threaten humanity due to between the different risks. Action The first part of the report is an our economic and technological to reduce one risk can increase introduction where the global risks development - extreme climate another, unless their possible links with potential infinite impact are change, for example, which depends are understood. Many solutions introduced and defined. This part on how much greenhouse gas we emit. are also able to address multiple also includes the methodology for risks, so there are significant selecting these risks, and presents “Exogenic challenges” includes benefits from understanding how the twelve risks that meet this risks where the basic probability one relates to others. Investigating definition. Four goals of the report of an event is beyond human these correlations could be a start, are also presented, under the control, but where the probability but correlation is a linear measure headings “acknowledge”, “inspire”, and magnitude of the impact can and non-linear techniques may “connect” and “deliver”. be influenced - asteroid impacts, be more helpful for assessing the for example, where the asteroids’ aggregate risk. The second part is an overview of paths are beyond human control the twelve global risks and key but an impact can be moderated by The fourth part is an overview, the events that illustrate some of the either changing the direction of the first ever to our knowledge, of the work around the world to address asteroid or preparing for an impact. uncertainties and probabilities of them. For each challenge five global risks with potentially infinite important factors that influence the “Emerging challenges” includes impacts. The numbers are only probability or impact are also listed. areas where technological rough estimates and are meant to development and scientific be a first step in a dialogue where The risks are divided into four assessment indicate that they methodologies are developed and different categories depending on could both be a very important estimates refined. their characteristics. contribution to human welfare and help reduce the risks associated The fifth part presents some of with current challenges, but could the most important underlying also result in new infinite impacts.20 trends that influence the global AI, nanotechnology and synthetic challenges, which often build up biology are examples. slowly until they reach a threshold and very rapid changes ensue. “Global policy challenge” is a different kind of risk. It is a The sixth and final part presents an probable threat arising from future overview of possible ways forward. global governance as it resorts to destructive policies, possibly in response to the other challenges listed above.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 37 2.2 Goals

2.2 Goals But we now face the possibility that even tools created with the best of Goal 1: Acknowledge intentions can have a darker side Establish a category of risks with too, a side that may threaten human That key stakeholders, potentially infinite impact. civilisation, and conceivably the influencing global challenges, Before anything significant can continuation of human life. acknowledge the existence of happen regarding global risks with the category of risks that could potentially infinite impacts, their result in infinite impact. They existence must be acknowledged. This is what all decision-makers need should also recognice that the to recognise. Rather than succumbing list of risks that belong to this Rapid technological development to terror, we need to acknowledge that category should be revised as and economic growth have delivered we can let the prospect inspire and new technologies are developed unprecedented material welfare to drive us forward. and our knowledge increases. billions of people in a veritable tide Regardless of the risks included, of utopias.21 the category should be given special attention in all processes and decisions of relevance. The report also seeks to demonstrate to all key stakeholders that we have the capacity to reduce, or even eliminate, most of the risks in this category.

Show concrete action that is It does so by combining information taking place today. about the risks with information about Goal 2: Inspire This report seeks to show that it is individuals and groups who has not only possible to contribute to made a significant contribution by That policy makers inspire reducing these risks, but that it is turning challenges into opportunities. action by explaining how the perhaps the most important thing probabilities and impacts anyone can spend their time on. By highlighting concrete examples can be reduced and turned the report hopes to inspire a new into opportunities. Concrete generation of leaders. examples of initiatives should be communicated in different networks in order to create ripple effects, with the long-term goal that all key stakeholders should be inspired to turn these risks into opportunities for positive action.

38 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.2 Goals

Support new meetings between Even with those risks where many interested stakeholders. groups are involved, such as climate Goal 3: Connect The nature of these risks spans change and pandemics, very few countries and continents; they today address the possibility of That leaders in different sectors require action by governments and infinite impact aspects. connect with each other to politicians, but also by companies, encourage collaboration. A academics, NGOs, and many other Even fewer groups address the links specific focus on financial and groups. The magnitude of the between the different risks. security policy where significant possible impacts requires not only risks combine to demand leaders to act but above all new There is also a need to connect action beyond the incremental models for global cooperation and different levels of work, so that local, is required. decision-making to ensure delivery. regional, national and international The need for political leadership is efforts can support each other when therefore crucial. it comes to risks with potentially infinite impacts.

Identify and implement strategies In order to deliver results it is and initiatives. important to remember that global Goal 4: Deliver Reports can acknowledge, inspire governance to tackle these risks is and connect, but only people can the way we organise society in order That concrete strategies are deliver actual results. The main to address our greatest challenges. developed that allow key focus of the report is to show that It is not a question of establishing a stakeholders to identify, quantify actual initiatives need to be taken “world government”, it is about the and address global challenges that deliver actual results. way we organise ourselves on all as well as gather support for levels, from the local to the global. concrete steps towards a well- Only when the probability of an functioning global governance infinite impact becomes acceptably The report is a first step and should system. This would include low, very close to zero, and/or when be seen as an invitation to all tools and initiatives that can the maximum impact is significantly responsible parties that can affect help identify, quantify and reduced, should we talk about the probability and impact of risks reduce risks with potentially real progress. with potentially infinite impacts. infinite impacts. But its success will ultimately be measured only on how it contributes to concrete results.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 39 2.3 Global challenges and infinite impact

2.3 Global challenges can be affected, for example with Over the last few years a greater nuclear war, where the number/size of understanding of low probability and infinite impact weapons influences the impact and or unknown probability events has tensions between countries affects helped more people to understand This chapter first introduces the the probability. the importance of looking beyond the concept of infinite impact. It then most probable scenarios. Concepts describes the methodology used to Other risks, such as a supervolcano, like “black swans” and “perfect identify challenges with an infinite are included as it is possible to affect storms” are now part of mainstream impact. It then presents risks with the impact through various mitigation policy and business language.27 potentially infinite impact that the methods, even if we currently cannot methodology results in. affect the probability. Risks that are Greater understanding of the susceptible to human influence are technology and science of complex indirectly linked, because efforts to systems has also resulted in a new 2.3.1 Definition of address one of them may increase or understanding of potentially disruptive infinite impact decrease the likelihood of another. events. Humans now have such an impact on the planet that the term The specific criterion for including a “the anthropocene” is being used, risk in this report is that well-sourced 2.3.2 Why use “infinite even by mainstream media like The science shows the challenge can have impact” as a concept? Economist.28 The term was introduced the following consequences: 22 in the 90s by the Nobel Prize winner The concept of infinity was chosen Paul Crutzen to describe how humans 1. Infinite impact: When civilisation as it reflects many of the challenges, are now the dominant force changing collapses to a state of great especially in economic theory, to the Earth’s ecosystems.29 suffering and does not recover, or a addressing these risks as well as the situation where all human life ends. need to question much of our current The idea to establish a well defined The existence of such threats is way of thinking. category of risks that focus on risks well attested by science.23 with a potentially infinite impact 2. Infinite impact threshold – an The concept of a category of risks that can be used as a practical tool impact that can trigger a chain of based on their extreme impact is by policy makers is partly inspired events that could result first in a meant to provide a tool to distinguish by Nick Bostrom’s philosophical civilisation collapse, and then later one particular kind of risk from others. work and his introduction of a risk result in an infinite impact. Such The benefit of this new concept taxonomy that includes an academic thresholds are especially important should be assessed based on two category called “existential risks”.30 to recognise in a complex and things. First, does the category exist, interconnected society where and second, is the concept helpful in Introducing a category with risks resilience is decreasing.24 addressing these risks? that have a potentially infinite impact is not meant to be a A collapse of civilisation is defined as a The report has found ample evidence mathematical definition; infinity is a drastic decrease in human population that there are risks with an impact thorny mathematical concept and size and political/economic/social that can end human civilisation and nothing in reality can be infinite.31 It complexity, globally for an extended even all human life. The report further is meant to illustrate a singularity, time.25 The above definition means the concludes that a new category of when humanity is threatened, when list of challenges is not static. When risk is not only meaningful but also many of the tools used to approach new challenges emerge, or current timely. We live in a society where most challenges today become ones fade away, the list will change. global risks with potentially infinite problematic, meaningless, or even impacts increase in both number counterproductive. An additional criterion for including and probability according to multiple risks in this report is “human studies. Looking ahead, many The concept of an infinite impact influence”. Only risks where humans emerging technologies which will highlights a unique situation where can influence either the probability, certainly provide beneficial results, humanity itself is threatened and the the impact, or both, are included. For might also result in an increased very idea of value and price collapses most risks both impact and probability probability of infinite impacts.26 from a human perspective, as the

40 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.3 Global challenges and infinite impact

price of the last humans also can be Ethical aspects of infinite impact represents 1/4200, or 0.024%, of our seen to be infinite. This is not to say The basic ethical aspect of infinite potential history. So our generation that those traditional tools cannot still impact is this: a very small group has the option of risking everything be useful, but with infinite impacts alive today can take decisions and annulling 99.976% of our we need to add an additional set of that will fundamentally affect all potential history. Comparing 0.024% analytical tools. future generations. with the days of a person living to 100 years from the day of conception, Some of the risks, including nuclear “All future generations” is not a this would equal less than nine war, climate change and pandemics, concept that is often discussed, and days and is the first stage of human are often included in current risk for good reason. All through human embryogenesis, the germinal stage.35 overviews, but in many cases history we have had no tools with a Two additional arguments to treat their possible infinite impacts are measurable global impact for more potentially infinite impacts as a excluded. The impacts which are than a few generations. Only in the separate category are: 36 included are in most cases still very last few decades has our potential serious, but only the more probable impact reached a level where all 1. An approach to infinite impacts parts of the probability distributions future generations can be affected, cannot be one of trial-and-error, are included, and the last part of the for the simple reason that we now because there is no opportunity long tail – where the infinite impact have the technological capacity to to learn from errors. The reactive is found – is excluded.32 end human civilisation. approach – see what happens, limit damage, and learn from Most risk reports do not differentiate If we count human history from the experience – is unworkable. Instead between challenges with a limited time when we began to practice society must be proactive. This impact and those with a potential for settled agriculture, that gives us requires foresight to foresee new infinite impact. This is dangerous, as it about 12,000 years.33 If we make a types of threat and willingness to can mean resources are spent in ways moderate assumption that humanity take decisive preventative action that increase the probability of an will live for at least 50 million more and to bear the costs (moral and infinite impact. years34 our 12,000-year history so far economic) of such actions.

Life Value

The following estimates have been applied to the value of life in the US. The estimates are either for one year of additional life or for the statistical value of a single life.

– $50,000 per year of quality life (international standard most private and government-run health insurance plans worldwide use to determine whether to cover a new medical procedure) – $129,000 per year of quality life (based on analysis of kidney dialysis procedures by Stefanos Zenios and colleagues at Stanford Graduate School of Business) – $7.4 million (Environmental Protection Agency) – $7.9 million (Food and Drug Administration) – $6 million (Transportation Department) – $28 million (Richard Posner based on the willingness to pay for avoiding a plane crash)

Source: Wikipedia: Value of life http://en.wikipedia.org/wiki/Value_of_life US EPA: Frequently Asked Questions on Mortality Risk Valuation

http://yosemite.epa.gov/EE%5Cepa%5Ceed.nsf/webpages/MortalityRiskValuation.html Posner, Richard A. Catastrophe: risk and response. Oxford University Press, 2004

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 41 2.3 Global challenges and infinite impact

2. We cannot necessarily rely on economic perspective it makes no In cases that do not include infinite the institutions, morality, social sense. Money helps us to prioritise, impacts, discounting “reflects the attitudes or national security but with no humans there would be no fact that there are many high-yield policies that developed from our economy and no need for priorities. investments that would improve the experience of other sorts of risk. quality of life for future generations. Infinite impacts are in a different Ignoring, or discounting, future The discount rate should be set so category. Institutions and individuals generations is actually the only way that our investable funds are devoted may find it hard to take these risks to avoid astronomical numbers to the most productive uses.” 44 When seriously simply because they lie for impacts that may seriously there is a potentially infinite impact, the outside our experience. Our collective affect every generation to come. In focus is no longer on what investments fear-response will probably be ill- Catastrophe: Risk and Response, have the best rate of return, it is about calibrated to the magnitude of threat. Richard Posner provides a cost avoiding the ultimate end. estimate, based on the assumption Economic aspects of infinite that a human life is worth $50,000, While many economists shy away impact and discounting resulting in a $300 tn cost for the from infinite impacts, those exploring In today’s society a monetary value whole of humanity, assuming a the potentially extreme impacts of is sometimes ascribed to human life. population of six billion. He then global challenges often assume Some experts use this method to doubles the population number infinite numbers to make their point. estimate risk by assigning a monetary to include the value of all future Nordhaus for example writes that “the value to human extinction.37 generations, ending up with $600 tn, sum of undiscounted anxieties would while acknowledging that “without be infinite (i.e. equal to 1 + 1 +1 + … = We have to remember that the discounting, the present value of the ∞). In this situation, most of us would monetary values placed on a human benefits of risk-avoidance measures dissolve in a sea of anxiety about all life in most cases are not meant would often approach infinity for the the things that could go wrong for to suggest that we have actually type of catastrophic risk with which distant generations from asteroids, assigned a specific value to a life. this book is concerned.” 40 wars, out-of-control robots, fat tails, Assigning a value to a human life is smart dust and other disasters.” 45 a tool used in a society with a limited Discounting for risks that include supply of resources or infrastructure the possibility of an infinite impact It is interesting that Nordhaus (ambulances, perhaps) or skills. In differs from risk discounting for less himself provides very good graphs such a society it is impossible to save serious impacts. For example the that show why the most important every life, so some trade-off must Stern Review41 prompted a discussion factor when determining actions be made.38 The US Environmental between its chief author, Nicholas is a possible threshold (see below Protection Agency explains its use like Stern, and William Nordhaus,42 each Figure 4 and 5). Nordhaus was this: “The EPA does not place a dollar of whom argued for different discount discussing climate change, but the value on individual lives. Rather, when levels using different arguments. But role of thresholds is similar for most conducting a benefit-cost analysis neither discussed a possible infinite infinite impacts. The first figure of new environmental policies, the climate impact. An overview of the is based on traditional economic Agency uses estimates of how much discussion by David Evans of Oxford approaches which assume that people are willing to pay for small Brookes University highlighted some Nature has no thresholds; the second reductions in their risks of dying from of the differing assumptions.43 graph illustrates what happens with adverse health conditions that may be the curve when a threshold exists. caused by environmental pollution.” 39 Two things make infinite impacts special As Nordhaus also notes, it is hard from a discounting perspective. First, to establish thresholds, but if they The fact that monetary values for there is no way that future generations are significant all other assumptions human lives can help to define can compensate for the impact, as they become secondary. The challenge priorities when it comes to smaller will not exist. Second, the impact is that Nordhaus does not address, and risks does not mean that they are something that is beyond an individual which is important especially with suitable for quite different uses. preference, as society will no longer exist. climate change, is that thresholds Applying a monetary value to the become invisible in economic whole human race makes little Discounting is undertaken to allocate calculations if they occur far into the sense to most people, and from an resources in the most productive way. future, even if it is current actions that

42 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.3 Global challenges and infinite impact

unbalance the system and eventually push it over the threshold.46 total discounted 6 discounted damages abatement (limited part.) Note that these dramatic illustrations rest on assumptions that the thresholds are still relatively 5 benign, not moving us beyond tipping points which result in an 4 accelerated release of methane that could result in a temperature increase of more than 8 °C, possibly 3 producing infinite impacts.47

Calculating illustrative numbers 2 By including the welfare of future generations, something that is important when their very of global income (%) Share 1 existence is threatened, economic discounting becomes difficult. In this chapter, some illustrative numbers are provided to indicate 0 1 2 3 4 5 6 the order of magnitude of the Global temperature limit (°C) values that calculations provide Figure 4: Nordhaus, The Climate Casino: Total cost of different targets assuming limited participation and when traditional calculations also discounting of future incomes. include future generations. These illustrative calculations are only illustrative as the timespans that must be used make all traditional total discounted assumptions questionable to say 20 damages with ripping costs the least. Still, as an indicator for abatement (limited part.) why infinite impact might be a good approximation they might help. 16 As a species that can manipulate our environment it could be argued that the time the human 12 race will be around, if we do not kill ourselves, can be estimated to be between 1-10 million years – the typical time period for the 8 biological evolution of a successful species48 – and one billion years, 49 the inhabitable time of Earth. of global income (%) Share 4

0 1 2 3 4 5 6 Global temperature limit (°C)

Figure 5: Nordhaus, The Climate Casino: Climate policy with a sharp tipping point at 3.5°C. This shows that the optimal temperature increase is very close to the threshold. It is constrained on the low side by abatement costs and on the high side by the sharp increase in damages.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 43 2.3 Global challenges and infinite impact

If we assume These numbers can be multiplied For those that are reluctant or unable many times if a more philosophical to use infinity in calculations and – 50 million years for the future of and technology-optimistic scenario is are in need of a number for their humanity as our reference, assumed for how many lives we should formulas, $86 sextillion could be a – an average life expectancy of 100 include in future generations. The good initial start for the cost of infinite years50, and following quote is from an article by impacts. But it is important to note – a global population of 6 billion Nick Bostrom in Global Policy Journal: that this number might be orders of people51 magnitude smaller than an estimate – all conservative estimate – , we “However, the relevant figure is not which actually took into account a have half a million generations how many people could live on more correct estimation of the number ahead of us with a total of 3 Earth but how many descendants of people that should be included in quadrillion individuals. we could have in total. One lower future generations as well as the price bound of the number of biological that should be assigned to the loss of Assuming a value of $50,000 per life, human life-years in the future the last humans. the cost of losing them would then be accessible universe (based on $1.5 ×1020, or $150 quintillion. current cosmological estimates) 2.3.3 Infinite impact is 1034 years. Another estimate, threshold (IIT) This is a very low estimate, and which assumes that future minds Posner suggests that maybe the will be mainly implemented in cost of a life should be “written up computational hardware instead As we address very complex systems, $28 million” for catastrophic risks52. of biological neuronal wetware, such as human civilisation and global 54 Posner’s calculations where only one produces a lower bound of 10 ecosystems, a concept as important future generation is included result in a human-brain-emulation subjective as infinite impact in this report is that cost of $336 quadrillion. If we include life-years.” 54 of infinity impact threshold. This is the all future generations with the same impact level that can trigger a chain value, $28 million, the result is a total Likewise the value of a life, $28 of events that results in the end of cost of $86 sextillion, or $86 × 1021. million, a value that is based on human civilisation. an assessment of how individuals This $86 sextillion is obviously a chose when it comes to flying, can The infinite impact threshold (IIT) very rough number (using one billion be seen as much too small. This concept represents the idea that years instead of 50 million would value is based on how much we long before an actual infinite impact for example require us to multiply value our own lives on the margin, is reached there is a tipping point the results by 20), but again it is the and it is reasonable to assume that where it (with some probability) is no magnitude that is interesting. As a the value would be higher than only longer possible to reverse events. reference there are about 1011 to a multiplication of our own value So instead of focusing only on the 1012 stars in our galaxy, and perhaps if we also considered the risk of ultimate impact it is important to something like the same number of losing our family, everyone we know, estimate what level of impact the galaxies. With this simple calculation as well as everyone else on the infinity threshold entails. you get 1022 to 1024, or 10 to 1,000 planet. In the same way as the cost sextillion, stars in the universe to increases when a certain product is The IIT is defined as an impact that put the cost of infinite impacts in short supply, the cost of the last can trigger a chain of events that when including future generations in humans could be assumed to be could result first in a civilisation perspective.53 very high, if not infinite. collapse, and then later result in an infinite impact. Such thresholds are Obviously, the very idea to put a especially important to recognise in a price on the survival of humanity complex and interconnected society can be questioned for good where resilience is decreasing. reasons, but if we still want to use a number, $28 million per life should at least be considered as a significant underestimation.

44 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.3 Global challenges and infinite impact

Social and ecological systems are Orrell and McSharry also noted The exact level for an infinite impact complex, and in most complex that “in orthodox economics, the threshold should not be the focus, but systems there are thresholds where reductionist approach means that rather the fact that such thresholds positive feedback loops become the economy is seen as consisting exists and that an order of magnitude self-reinforcing. In a system where of individual, independent agents should be estimated.57 During the resilience is too low, feedback loops who act to maximise their own process of writing the report, experts can result in a total system collapse. utility. It assumes that prices are suggested that a relatively quick death These thresholds are very difficult driven to a state of near-equilibrium of two billion people could be used as to estimate and in most cases it is by the ‘invisible hand’ of the a tentative number until more research possible only to estimate their order economy. Deviations from this state is available.58 With current trends of magnitude. are assumed to be random and undermining ecological and social independent, so the price fluctuations resilience it should be noted that the As David Orrell and Patrick McSharry are often modelled using the normal threshold level is likely to become wrote in A Systems Approach to distribution or other distributions with lower as time progress. Forecasting: “Complex systems have thin tails and finite variance.” emergent properties, qualities that cannot be predicted in advance from The drawbacks of an approach using knowledge of systems components the normal distribution, or other 2.3.4 Global F-N curves alone”. According to complexity distributions with thin tails and finite and ALARP scientist Stephen Wolfram’s principle variance, become obvious when the In the context of global risks with of computational irreducibility, the only unexpected happens as in the recent potentially infinite impact, the way to predict the evolution of such credit crunch, when existing models possibility of establishing global F-N a system is to run the system itself: totally failed to capture the true risks curves is worth exploring. One “There is no simple set of equations of the economy. As an employee of of the most common and flexible that can look into its future.” 55 Lehman Brothers put it on August 11, frameworks used for risk criteria 2007: “Events that models predicted divides risks into three bands: 59 would happen only once in 10,000 years happened every day for three days.” 56 1. Upper: an unacceptable/ intolerable region, where risks are intolerable except in extraordinary circumstances and risk reduction Figure 6: Normal risks and risks with potentially infinite impact. measures are essential. 2. Middle: Threshold an ALARP (“as low as reasonably practicable”) region, where risk reduction measures are desirable but may not be implemented if their cost is disproportionate to the benefit achieved. 3. Lower: a broadly acceptable/ Risks with Normal risks infinite impact negligible region, where no further Situation that requires risk reduction measures are needed. Traditional measures new measures and tools and tools applicable The bands are expressed by F-N curves. When the frequency of events which cause at least N fatalities is plotted against the number N on log–log scales, the result is called an

probability F-N curve.60 If the frequency scale is replaced by annual probability, then 0 impact the resultant curve is called an f-N curve.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 45 2.3 Global challenges and infinite impact

1.00E-01 Figure 7: Example of F-n curve showing different levels of risk 61 1.00E-02 Intollerable

1.00E-03

1.00E-04 ALARP 1.00E-05

1.00E-06

1.00E-07 Negligible

Frequency (Events > N per Yr) Frequency 1.00E-08 1 10 100 1000 10000 Fatalities (N)

Negligible Intolerable Detailed Analysis

The concept for the middle band when using F-N curves is ALARP. It Pros Cons is a term often used in the area of safety-critical and safety-involved systems.62 The ALARP principle is that – Clearly shows relationship – Cumulative expression makes the residual risk should be as low as between frequency and size it difficult to interpret, especially reasonably practicable. of accident by non-risk specialists

The upper band, the unacceptable/ – Allows judgement on relative – Can be awkard to derive intolerable region, is usually the area importance of different sizes above the ALARP area (see figure 8) of accident

By using F-N curves it is also – Slope steeper than -1 – May be difficult to use if criterion possible to establish absolute impact provides explicit consideration is exceeded in one area but levels that are never acceptable, of multiple fatality aversion otherwise is well below regardless of probability (Figure 7. and favours concepts with Based on an actual F-n Curve lower potential for large showing an absolute impact level that fatality events is defined as unacceptable). This has been done in some cases for local – Allows company to manage – Much debate about criterion lines projects. The infinite threshold could overall risk exposure from be used to create an impact limit on portfolio of all existing and global F-N curves used for global future facilities challenges in the future. Such an approach would help governments, companies and researchers when they develop new technical solutions and when investing in resilience. Instead of reducing risk, such an approach encourages the building of systems which cannot have negative impacts above a certain level.

Figure 9: Pros and cons of F-N curves 63

46 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.3 Global challenges and infinite impact

10 -2 macroeconomics and its challenges in discussed. In one way the name is

10 -3 relation to the kind of impacts that the not very important so long as people risks in this report focus on. understand the impacts and risks Unacceptable Further, the name clearly highlights associated with it. Still, a name is -4 10 the unique nature without any symbolic and can either help or make normative judgements. it more difficult to get support to 10 -5 establish the new category.

ALARP Still, infinity is an abstract concept region -6 and it might not be best communicate The work to establish a list of risks 10 the unique group of risks that it covers with infinite impact evolved from to all stakeholders. In the same way “existential risk”, the philosophical -7 10 as it can be hard to use singularity to concept that inspired much of the describe a black hole, it can be difficult work to establish a clearly defined 10 -8 to use infinity to describe a certain risk. group of risks. The reason for not If people can accept that it is only from using the concept “existential risk Acceptable

-9 a specific perspective that the infinity and impact” for this category, beside Frequency (F) of Accidents with N or More Fatalities (Per Year) (F) of Accidents with N or More Frequency 10 10 100 1000 10000 concept is relevant it could be used the fact that existential impact is Number of Fatalities (N) beyond the areas of macroeconomics. also used in academic contexts to refer to a personal impact, is that the Figure 8: Example of F-n curve showing an absolute Two other concepts that also have infinite category is a smaller subset impact level that is defined as unacceptable/ infinite. i.e no level of probability is acceptable been considered during the process of “existential risk” and this new above a certain level of impact, in this case 1000 dead 64 of writing this report are “xrisks” and category is meant to be used as a “human risk of ruin”. Xrisk has the tool, not a scientific concept. Not only advantage, and disadvantage, of not should the impacts in the category really saying anything at all about the potentially result in the end of all 2.3.5 A name for a clearly risk. The positive aspect is that the human life, it should be possible to defined group of risks name can be associated with the affect the probability and/or impact that can provide general concept of extinction and the of that risk. There must also exist an philosophical concept of existential risk agreed methodology, such as the one practical guidance as both have the letter x in them. The suggested in this report, that decides disadvantage is the x often represents what risks belong and not belong on Today no established methodology the unknown and can therefore relate the list. exists that provides a constantly to any risk. There is nothing in the updated list of risks that threaten human name that directly relates to the kind of Another concept that the category civilisation, or even all human life. Given impacts that the category covers, so relates to is “global catastrophic risk” as it that such a category can help society to it is easy to interpret the term as just is one of the most used concepts among better understand and act to avoid such unknown risks. academics interested in infinite impacts. risks, and better understand the relation However it is vague enough to be used between these risks, it can be argued Human risk of ruin has the advantage to refer to impacts from a few thousand that a name for this category would of having a direct link to a concept, deaths to the end of human civilisation. be helpful.65 risk of ruin, that relates to a very Already in use but not clearly defined, specific state where all is lost. Risk of it includes both the academic concept To name something that refers to the ruin is a concept in use in gambling, existential risks and the category of risks end of humanity is in itself a challenge, insurance, and finance that can all with infinite impacts. as the very idea is so far from our give very important contributions to usual references and to many the the work with this new category of intuitive feeling will be to dismiss any risk. The resemblance to an existing such thing. concept that is well established could be both a strength and a liability. The concept used in this report is “infinity”. The reson for this is that Below is an overview of the many of the challenges relate to process when different names were

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 47 2.3 Global challenges and infinite impact

The three concepts are very different. 1. Risk of ruin – is a concept in 9. Unacceptable risks in different Global catastrophic risk is possibly gambling, insurance and finance combinations, e.g. unacceptable the most used concept in contexts relating to the likelihood of losing global risks – This is probably where infinite impacts are included, all one’s capital or affecting one’s not appropriate for two main but it is without any clear definition. bankroll beyond the point of reasons. First, it is a normative Existential risk is an academic recovery. It is used to describe statement and the category aims concept used by a much smaller individual companies rather to be scientific; whether these group and with particular focus on than systems.66 risks are unacceptable or not is future technologies. The category in 2. Extinction risk – is used in up to the citizens of the world this report is a tool to help decision biology for any species that to decide. Second, the idea of makers develop strategies that help is threatened. The concept is risk is that it is a combination of reduce the probability that humanity also used in memory/cognition probability times impact. If a risk will end when it can be avoided. The research. It is a very dramatic term, is unacceptable is therefore also relation between the three concepts to be used with care. These factors usually related to how easy it is can be illustrated with three circles. make it probably unsuitable for use to avoid. Even if a risk is small, The large circle (1) represents global by stakeholders accustomed to due to relatively low probability catastrophic risks, the middle one (2) traditional risk assessment. and relatively low impact, but existential risks and the small circle 3. Astronomical risk – is seldom is very easy to address, it can (3) the list of twelve risks in this used scientifically, but when it is be seen as unacceptable, in the report, i.e. risks where there are peer used it is often used for asteroids same way a large risk can be seen reviewed academic studies that and is probably best reserved as acceptable if it would require estimate the probability of an infinite for them.67 significant resources to reduce. impact and where there are known 4. Apocalyptic risk – could have ways to reduce the risk. A list that been suitable, as the original There will not be a perfect concept could be called infinite risks, xrisks, or meaning is apocálypsis, from the and the question is what concept human risk of ruin. Greek ἀπό and καλύπτω meaning can find the best balance between ‘un-covering’. It is sometime used, being easy to understand, acceptable but in a more general sense, to where policy decisions needs to be mean significant risks.68 But through made and also acceptable for all key history and today it is mainly used groups that are relevant for work in for a religious end of time scenario. these area. During the process to find Its strong links to unscientific a name for this category inspiration doom-mongers make it probably has been found in the process when unsuitable for a scientific concept. new concepts have been introduced; 5. End-of-the-world risk - belongs to from irrational numbers and genocide 3. 2. 1. the irrational doomsday narratives to sustainable development and and so is probably unsuitable for the Human Development Index. So scientific risk assessments. far “infinite risk” can be seen as the 6. Extreme risk – is vague enough least bad concept in some areas and to describe anything beyond the “xrisks” and “human risk of ruin” the normal, so it is probably unsuitable least bad in others. for risk assessments of this magnitude. The purpose of this report is to 7. Unique risk – is even vaguer, establish a methodology to identify Other concepts that are related to as every risk is unique in some a very specific group of risks as infinite impacts that could potentially way. Probably best avoided in well as continue to a process where be used to describe the same risk assessments. these risks will be addressed in a category if the above suggestions 8. Collapse risk – is based on systematic and appropriate way. are not seen as acceptable concepts Jared Diamond’s thinking.69 The issue of naming this group are presented below, together with There are many different kinds of of risks will be left to others. The the main reason why these concepts collapse and only a few result in important is that the category gets were not chosen for this report. infinite impact. the attention it deserves.

48 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.4 Methodology

2.4 Methodology 70

This chapter presents the methodology used to identify global risks with potentially infinite impact.

Methodology overview In order to establish a list of global risks with potentially infinite impact a methodological triangulation was used, consisting of:

– A quantitative assessment of relevant literature. – A strategic selection of relevant organisations and their priorities. – A qualitative assessment with the help of expert workshops.

Leading organisations’ priorities 2 In order to increase the probability of covering all relevant risks an overview of leading organisations' work was conducted. This list was then compared with the initial list and subjected to the same filter regarding the possibility to affect the probability or impact. 3 Possibility of addressing the risk Possibility of addressing the risk: From the risks gathered from literature and organisations, only those where the probability or impact can be affected by human actions are included. 4 Expert review Qualitative assessment: Expert review in order to increase the probability of covering all relevant global risks. 5 List of risks Result: List of risks with potentially infinite impacts. 6

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 49 2.4 Methodology

2.4.1 A scientific A list of 178 relevant books and reports – Covers multiple global 71 was established based on what other challenges (at least two). review of key literature studies have referred to, and/or which Publications that discuss a are seen as landmark studies by variety of global challenges are of The scientific review of literature was groups interviewed during the process. particular importance because they led by Seth Baum, Executive Director They were selected for a closer aid in identifying and comparing the of the Global Catastrophic Risk examination regarding the challenges various challenges. This process Institute72 and research scientist at the they include.76 is essential for research on global Center for Research on Environmental risks to identify boundaries and Decisions, Columbia University.73 The full bibliography, even with its focus research priorities. on publications of general interest, is The methodology for including global still rather long. So it is helpful to have a In order to identify which global risks with a potentially infinite impact shorter list focused on the highlights; the challenges are most commonly is based on a scientific review of key most important publications based on discussed, key surveys were identified literature, with focus on peer-reviewed how often they are quoted, how well- and coded. First, a list of publications academic journals, using keyword spread the content (methodology, lists, that survey at least three global search of both World of Knowledge74 etc.) is and how often key organisations challenges was compiled, and they and Google Scholar75 combined with use them. The publications included must were then scanned to find which existing literature overviews in the meet at least one of the following criteria: challenges they discussed. area of global challenges. This also included a snowball methodology – Historical significance. This The publications that survey many where references in the leading studies includes being the first publication global challenges were identified from and books were used to identify other to introduce certain key concepts, the full bibliography. Publications scientific studies and books. or other early discussions of global from both the academic and popular challenges. Publications of historical literature were considered. Emphasis In order to select words for a literature significance are important for showing was placed on publications of repute search to identify infinite impacts, a the intellectual history of global or other significance.78 To qualify process was established to identify challenges. Understanding how the as a survey of global challenges, words in the scientific literature state of the art research got to where it the publication had to provide an connected to global challenges with is today can also help us understand explicit list of challenges or to be of potentially infinite impacts. Some where it might go in the future. sufficient length and breadth for it words generate a lot of misses, i.e. to discuss a variety of challenges. publications that use the term but – Influential in developing the field. Many of the publications are books are not the focus of this report. For This includes publications that are or book-length collections of articles example “existential risk” is used in highly cited77 and those that have published in book form or as special business; “human extinction” is used motivated significant additional issues of scholarly journals. Some in memory/cognition. Some search research. They are not necessarily individual articles were also included terms produced relatively few hits. the first publications to introduce because they discussed a significant the concepts they discuss, but for breadth of challenges. For example “global catastrophic whatever reason they will have proved risk” is not used much. Other words important in advancing research. A total of 40 global challenge survey are only used by people within a publications were identified. For specific research community: few use – State of the art. This includes authors with multiple entries (Bostrom “existential risk” in our sense unless publications developing new concepts with three and WEF with ten) each they are using Nick Bostrom’s work. at the forefront of global challenges challenge was counted only once to The term “global catastrophe” was research as well as those providing avoid bias. identified as a phrase that referred the best discussions of important almost exclusively to extremely established concepts. Reading these negative impacts on humans, by publications would bring a researcher a diversity of researchers, not just up to speed with current research people in one research community. on global challenges. So they are important for the quality of their ideas.

50 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.4 Methodology

In terms of authorship and audience, was necessary because many of the The coding presented here erred on the there are 17 academic publications, publications did not contain explicit side of greater inclusivity: if a portion 9 popular publications, 1 government lists of global challenges, and the ones of text was in the vicinity of a global report, 3 publications written by that did often mentioned additional challenge, then it was coded as one. academics for popular audiences. challenges separately from their lists. For example, some publications In terms of format, there are 15 So it was not required that a global discussed risks associated with books, 5 edited collections, 7 articles, challenge be mentioned in a list nuclear weapons in a general sense 3 of miscellaneous format. Of the for it to be counted – it only had to without specifically mentioning the 40 publications identified, 22 were be mentioned somewhere in the possibility of large-scale nuclear war. available at the time of coding. In publication as a challenge. These discussions were coded as addition, 10 Global Risks Reports mentions of nuclear war, even though from the World Economic Forum were Assessing whether a particular they could also refer to single usages coded and then gathered under one portion of text counts as a global of nuclear weapons that would not heading: “WEF Global Risk Report challenge and which category it rate as a global challenge. 2005-2014”. fits in sometimes requires some interpretation. This is inevitable This more inclusive approach is A list of 34 global challenges was for most types of textual analysis, warranted because many of the developed based on the challenges or, more generally, for the coding publications were not focused mentioned in the publications. A of qualitative data. The need for exclusively on global challenges. If spreadsheet containing the challenges interpretation in this coding was they were focused on them, it is likely and the publications was created to heightened by the fact that the that they would have included these record mentions of specific challenges publications often were not written risks in their global challenge form in each publication to be coded. with the purpose of surveying the (e.g., nuclear war), given that they breadth of global challenges, and even were already discussing something Then each publication was scanned the publications that were intended related (e.g., nuclear weapons). in its entirety for mentions of global as surveys did not use consistent Below are the results from the challenges. Scanning by this method definitions of global challenges. overview of the surveys.

18 17

15 15 14 14 13 13 13 13

11 11 11

8 8 8 8 7 7

5 5 5 5 5 4 4 4 4 3 3 2 1 1 1 1

0 Climate Change Nuclear War Pandemic Biodiversity loss / Comet Meteor Asteroid Volcano Genetic Engineering Physics High Energy Nanotech Depletion Resource Artificial Intelligence Chemical Pollution Ecological Catastrophe Biogeochem Government Failure Poverty System Failure Explosion Astronomic LULCC Biological Weapons Chemical Weapons Extraterrestrial Reject Procreation Computer Failure EM Pulse New Technology Ozone Depletion Dysgenics Ocean Acidification Interstellar Cloud Aerosols Atmosphere Phase Transition Simulation Unknown

Figure 9: Number of times global challenges are included in surveys of global challenges

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 51 52 Figure 11:Numberofacademicpapersonvarioustopics(listed inScopus,August2012) with infiniteimpactsandhaveused focus more ontheglobalchallenges urged policymakersanddonorsto challenges, likeNickBostrom, have Experts inthefieldofglobal human race. that itisaboutthesurvivalof impact isverysmall,giventhefact challenges withpotentiallyinfinite that includesmultipleglobal It shouldbenotedthattheliterature Figure 10:Theglobalchallenges includedtentimesormore insurveysofglobalchallenges 2.4 Methodology 5 10 15 20 25 1000 0 600 800 200 400 0 21 From thepaper“ExistentialRiskPrevention asGlobalPriority” Climate Change dung beetle

Nuclear War 18

Pandemic 17 star trek

Biodiversity loss 15 Global Challenges –Twelve risks thatthreaten human civilisation –Thecasefora newcategoryof risks therefore difficulttoassess. human extinctioninfiniteimpactis How muchworkactuallyexistson low-probability high-impactoutcomes. global risks,butoftenwithoutincluding more studiesexistthatfocusonindividual However, itisimportanttonotethatmany compared withotherareas. little research isbeingdoneonthem dramatic rhetorictoillustratehow 14

zinc oxalate Asteroid / Comet / Meteor 79

Volcano 14 extinction human Genetic Engineering 13

High Energy Physics 13 80

Nanotech 13 only one or two challenges may not only oneortwochallenges may not contrast, organisations thatfocus on they considerimportantand why. In is tounderstandwhichchallenges those thatworkonmultiplechallenges more. Thereason forfocusingon challenges –atleastthree, andideally if theyworkedonavarietyofglobal They were selectedfrom thedirectory on globalchallenges organisations working 2.4.2 Areview of the GlobalCatastrophic RiskInstitute. risk organisation directory publishedby identified viatheglobalcatastrophic publications. Organisations were paralleled thecodingofkeysurvey The codingofkeyorganisations was excludedfrom thelist. done toensure thatnoimportantrisk and ontheirwebpages.Thiswas challenges includeintheirmaterial organisations workingonglobal a review ofwhatchallengeskey literature wascheckedagainst The listofrisksfoundinthescientific

Resource Depletion 13

Artificial Intelligence 11

Chemical Pollution 11 81

Ecological Catastrophe 11 82

Global Challenges –Twelve risks thatthreaten human civilisation –Thecasefora newcategoryof risks Figure 12:Globalchallengesthatkeyorganisations work with are workingon. global challengestheseorganisations reasonably accuratepicture ofwhat coded. Sotheresults shouldgivea have probably beenidentifiedand challenges thattheyhaveworkedon that were examined,overallthemain not mentionedonthewebpages have workedonglobalchallenges that someoftheseorganisations recent publications.While itispossible browsing otherwebpages, including these challengeswere identified by these pages.Additionalreferences to to globalchallengeswere pulled from work on.Where possible, references web pageswhichlistthetopicsthey organisations’ websites.Manyhave challenges were obtainedfrom References tospecificglobal for codingsurveypublications. same codingschemedeveloped The organisation codingusedthe most important. to whichchallengestheyconsiderthe be abletoadjusttheirfocusaccording 10 12 14 0 2 4 6 8 13 Climate Change 13 Nuclear War 12 Pandemic 9 Resource Depletion 8 Biological Weapons 7 Computer Failure 7 Government Failure 7 Nanotech 6 Chemical Weapons 5 Artificial Intelligence 4 Genetic Engineering they work with. they workwith. examination regarding thechallenges organisations were selectedfor acloser during theprocess, ashort-listof as influentialbygroups interviewed to theorganisation, and/orare seen which otherorganisations referred production ofrelevant literature, was established.Basedonthe which workwithglobalchallenges An initiallistof180organisations used toidentifyorganisations. during theworkshopswere also between organisations andinput challenges”. Cross-references extinction” or“greatest global “global catastrophic risk”,“human information about“existentialrisk”, and/or theirwebpagecontained claimed toworkonglobalchallenges identified, according towhether they based onthelistoforganisations snowball samplingwasconducted the basisofliterature overview. A challenges were initiallyselectedon Organisations workingwithglobal 4 System Failure 2 Biodiversity loss 2 Ecological Failure 2 Poverty 2 Volcano 1 Asteroid / Comet / Meteor 1 Astronomic Explosion 1 Biogeochem 1 Chemical Pollution 1 Extraterrestrial 1 High Energy Physics 1 New Technology estimates of possible impacts. estimates ofpossibleimpacts. a frameworkforthechallengesand their Extreme RiskReport2013,have thought leaders,likeTower Watson and to selectthechallenges.Onlyafew explanations forthemethodologyused with globalchallengesthere are no For mostorganisations working 3. Theiremphasis 2. Howsystematicallytheyidentify 1. Whattheycountasaglobal challenges varywidelyin: The organisations workingonglobal global challenges. organisations workingwithmultiple results from theoverviewofkey Below istheoverviewof a listof19organisations. challenges were selected,resulting in Then thoseworkingwithmultiple 1 important global challenges important globalchallenges global challenges;and Ozone Depletion challenge 1 Atmospheric Aerosols 0 Dysgenics 0 EM Pulse 0 Interstellar Cloud 0 on the most on themost LULCC 2.4 Methodology 0 Ocean Acidification 83 0 Phase Transition 0 Reject Procreation 0 Simulation

53 0 Unknown shared effortstoaddress them.” stakeholders, andpotentiallygalvanise are todifferent ofmostconcern approach canhighlightareas that methodology asfollows:“This The WEFdescribesitsperception scientific assessmentoftheactualrisk. as discussingperception of risk,nota WEF, alsoclearlydefinewhattheydo more seriousorganisations, likethe definition ofprobability orimpact. The to estimaterisks,butwithoutanyclear assessment, where experts are asked of riskperception ratherthan risk described asbelongingtothecategory many othergroups’, isprobably best Risk Report.TheWEF’s riskwork,with probably bestknownisthe WEF Global that focusesonglobalriskwhichis definition oftheprobability. The report a a definitionoftheimpact,nor In mostcasesthere isneither 54 several countriesandindustries?” causes significantnegativeimpactfor to answeris:“Whatoccurrence The questionwhichpeopleare asked 2.4 Methodology 85 86

Global Challenges –Twelve risks thatthreaten human civilisation –Thecasefora newcategoryof risks traditional pandemic,are identified. failurechange, governmental and horizon, likethefirstsignsofclimate happen whenapplyingafiveyear the challengesthatlikelycould the lasttenyearsindicatesthat An overviewofWEFreports from company orcountry). risk inmonetarytermsfortheirown as manyrespondents likelydefine not theimpactonhumansuffering, an assessmentoftheimpact(and the actualprobability (0-100%) or does notprovide anassessment of important toremember thatthereport challenges are perceived. Still,itis will beinfluencedbyhowdifferent probability asthereaders’ decisions the WEFstates,caninfluenceactual obviously veryinteresting and, as Such perception approaches are changes inperception over theyears. value ofthereport istotrack the define what1-4means,sothemajor It isthenuptotherespondent to Figure 13:Thetop12globalchallengesthatkeyorganisations workwith likelihood (within10years). 1-4, oneforimpactandanother provide anumberontwo scalesfrom The respondents are thenaskedto 10 12 14 0 2 4 6 8

Climate Change 13

Nuclear War 13

Pandemic 12

Resource Depletion 9 87

Biological Weapons 8

Computer Failure 7 for newandmore seriouspandemics. time resulting inanincreased probability known pandemicswhileatthesame system canreduce theprobability for pandemics thatare known.Sucha equipment thatscansforthetypeof trade ofgoodswhileinvestingin example, byencouragingincreased is argued thatthiscouldhappen,for make amajorpandemicmore likely. It pandemics couldresult inactionsthat on commercially relevant smaller Other research indicatesthatfocus result inhigh-carbonlock-in. problem forclimatechange asitcould a of significantmitigationcouldbe on incremental adaptation instead are reasons tobelievethatafocus of infiniteimpacts.Forexample,there can result inanincreased probability serious impactsofglobalchallenges whether afocusonthesmallerbutstill An importantquestiontoexplore is get less,orno,attention. volcanoes, AI,andasteroids, tendto change, nanotechnology, major probability, likeextreme climate have verybigimpactsbutlower On theotherhand,challengeswhich

Government Failure 7

Nanotech 7

Chemical Weapons 6

Artificial Intelligence 5 88 Genetic Engeneering 4

System Failure 4 89

Atmospheric Aerosols 2.4 Methodology / 2.5 The rseulting list of global risks using this methodology

2.4.3 Workshops 2.5 The list of The technologies included in emerging challenges, including synthetic Two workshops were arranged global risks biology, nanotechnology and artificial where the selection of challenges intelligence (AI), will be critical to was discussed, one with risk experts Based on the risks identified in the finding solutions to infinite impacts. in Oxford at the Future of Humanity literature review and in the review of Including these technologies should Institute and the other in London organisations and applying the criteria not be seen as an attempt to arrest with experts from the financial for potentially infinite impact, these them. If anything, the development sector. See Appendix 2 for agenda risks were identified: of sustainable solutions should be and participants. accelerated. But it is equally important 1. Extreme Climate Change to create guidelines and frameworks In both workshops the list of global 2. Nuclear War to avoid their misuse, whether challenges was discussed to see if 3. Global Pandemic intentional or accidental. any additional challenges should be 4. Ecological Catastrophe included, or if there were reasons to 5. Global System Collapse The fourth category, future global exclude some from the list. 6. Major Asteroid Impact policy challenges, is of a different kind. 7. Supervolcano It includes challenges related to the No challenge was excluded at the 8. Synthetic Biology consequences of an inferior or destructive workshops, but one was added. 9. Nanotechnology global governance system. This is Although little research exists yet that 10. Artificial Intelligence (AI) especially important as well-intended is able to verify the potential impacts, 11. Unknown Consequences actions to reduce global challenges the participants agreed to include 12. Future Bad Global Governance could lead to future global governance Global System Collapse as a risk systems with destructive impact. with possible infinite impact. There This is an initial list. Additional risks was agreement that further research will be added as new scientific studies The first category, current is needed to clarify exactly what become available, and some will be challenges, includes: parts of the economic and political removed if steps are taken to reduce system could collapse and result in their probability90 and/or impact so 1. Extreme Climate Change a potentially infinite outcome. The that they no longer meet the criteria. 2. Nuclear War conclusion was that enough research 3. Global Pandemic exists to include such a collapse on Four categories of global challenges 4. Ecological Catastrophe the list. The challenges included in this report 5. Global System Collapse belong to four categories. The first, current challenges, includes those The second category, exogenous where decisions today can result challenges, covers: directly in infinite impacts. They are included even if the time between 6. Major Asteroid Impact action and impact might be decades, 7. Supervolcano as with climate change. Those in the third category, emerging The second category is exogenous challenges, are: challenges, those where decisions do not – currently – influence probability, 8. Synthetic Biology but can influence impact. 9. Nanotechnology 10. Artificial Intelligence (AI) The third category is emerging 11. Unknown Consequences challenges, those where technology and science are not advanced enough The fourth category, global policy to pose a severe threat today, but challenges, is: where the challenges will probably soon be able to have an infinite impact. 12. Future Bad Global Governance

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 55 2.5 The rseulting list of global risks using this methodology

2.5.1 Risks The challenges mentioned in the 3. Chemical pollution. reviewed literature and organisations Increasingly, there is particular not included which are not included in this report concern about three types of often refer to economic damage such chemicals: those that persist in Many risks could severely damage as “fiscal crises” or “unemployment”. the environment and accumulate humanity but have not been included While such impacts could have in the bodies of wildlife and in this report. They were excluded for far-reaching consequences they are people, endocrine disruptors that one or more of three reasons: obviously of another magnitude than can interfere with hormones, and those included here. chemicals that cause cancer or 1. Limited impact. Many challenges damage DNA. can have significant local negative Some of the risks that were suggested ͢ Not included due to: effects, without approaching the “2 and/or which exist in books and Limited impact billion negatively affected” criterion reports about global risks were - tsunamis, for example, and rejected according to the criteria chemical pollution. above. They include: 91 4. Dangerous physics experiments creating black holes/strangelets 2. No effective countermeasures. 1. Astronomical explosion/nearby including high energy physics. The report focuses on promoting gamma-ray burst or supernova.92 These risks are of low probability94 effective interventions and so These seem to be events of and have been subsumed under ignores challenges where nothing extremely low probability and which “Uncertain Risks”. useful can be done to prevent or are unlikely to be survivable. Milder ͢ Not included due to: mitigate the impact, as with nearby versions of them (where the source Included in other challenges gamma-ray bursts. is sufficiently far away) may be considered in a subsequent report. 3. Included in other challenges. ͢ Not included due to: 5. Destructive solar flares. Many challenges are already No effective countermeasures Though solar flares or coronal covered by others, or have a mass ejections could cause damage profile so similar that great economic damage to our there seemed no need to have 2. False vacuum collapse. technological civilisation,95 they a separate category. Population If our universe is in a false would not lead directly to mass growth, for one, is an underlying vacuum and it collapses at any casualties unless the system lacks driver significant for climate change point, the collapse would expand basic resilience. They have been and eco-system catastrophe, but at the speed of light destroying subsumed in the Global System without direct large-scale impacts. all organised structures in the Collapse category. universe.93 This would not ͢ Not included due to: be survivable. Limited impact/included in ͢ ͢ Not included due to: other challenges No effective countermeasures

56 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.5 The rseulting list of global risks using this methodology

6. Moral collapse of humanity. 8. New technological 11. Computer failure/Cyber- Humanity may develop along experimental risks. warfare. Though an area of great a path that we would currently It is possible and plausible that interest and research, cyber- find morally repellent. The new unexpected technological warfare has never caused mass consequences of this are not risks will emerge due to casualties and would be unlikely clear-cut, and depend on value experiments. However, until we to do so directly. It may be the judgements that would be know what such risks may be, they subject of a future report, but in contentious and unshared.96 Some are subsumed in the “Uncertain this report it is considered to be of these risks (such as global Risks” category. a subset of warfare and general totalitarianism or enduring poverty) ͢ Not included due to: destabilising risks. were included in the Governance included in other challenges ͢ Not included due to: Disasters category. Limited impact/Submersed in ͢ ͢ Not included due to: other challenges included in other challenges 9. Genocides. Though immense tragedies within 12. Underlying trends, e.g. specific areas, past genocides overpopulation. Though increased 7. Resource depletion/LULCC/ have remained contained in space population will put strains on Biodiversity loss. and time and haven’t spread resources and can contribute to It has often been argued that across the globe.98 increased probability for other declining resources will cause ͢ Not included due to: challenges included in this report increased conflict.97 Nevertheless Limited impact (such as climate change and such conflicts would not be ecosystem catastrophe), plausible sufficient in themselves to threaten population levels will not cause humanity on a large scale, 10. Natural disasters. any direct harm to humanity.101 without a “ System Collapse” or Most natural disasters, like Population growth is however an “Governance Disasters”. tsunamis and hurricanes, have important trend that is significantly ͢ ͢ Not included due to: no likelihood of causing the affecting several risks. included in other challenges extent of casualties100 needed ͢ Not included due to: for consideration on this list, as Limited impact/Submersed in they are geographically limited other challenges and follow relatively mild impact probability curves. Note: Important underlying trends are ͢ ͢ Not included due to: discussed in chapter 5. Limited impact

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 57 2.6 Relations between impact levels beyond the infinite threshold

2.6 Relationship between What makes things difficult is that None of the risks in this report is likely once a system is unstable, a small to result directly in an infinite impact, impact levels beyond disaster can have knock-on effects – and some cannot do so physically. the infinite threshold the death of one Austrian nobleman All the risks however are big enough can result in an ultimatum which to reach a threshold where the social Complex systems are often stable draws in neighbours until Australians and ecological systems become so only within certain boundaries. end up fighting Turks and the First unstable that an infinite impact could Outside these boundaries the World War is well under way, to be ensue, as the graph below shows. system can collapse and rapidly followed by communism, the Second change to a new stable state, or it World War and the Cold War. This graph and its accompanying text can trigger a process where change explain, how an event that reaches continues for a long time until a new The challenge of understanding a threshold level could cascade into stable state is found. complex systems includes the fact even worse situations, via civilisation that many of them have multiple collapse105 to human extinction. Sometimes it can take a very long attractors, including what are called time for a system to stabilise again. “strange attractors”.103 Changes are The graph also seeks to illustrate the Looking at all the biotic crises over close to linear as long as the system importance of ensuring ecological the past 530 million years, a research does not change very much, but once and social resilience, the two major team from Berkeley found an it is pushed out of balance it will get insurance policies we have against a average of 10 million years between closer to other attractors, and when negative spiral after a major impact that an extinction and a subsequent those become strong enough the takes us beyond the infinite threshold. flourishing of life.102 system will tend to move towards chaos until a new balance is achieved around the new attractor.104

Pre-risk rebuilding enablers (tech stores...)

Post-collapse Long term impact external threats and risks

Social and ecosystem resilience

Extinction

Post-risk Post-risk Post-collapse collapse politics politics countermeasures

General Total Civilisation Maintaining mitigation short term technology base and resilience casualties collapse

General pre-risk Long-term collapse countermeasures reconstruction Anthropic effect probability

58 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2.6 Relations between impact levels beyond the infinite threshold

1. Social and ecosystem resilience. 5. Long-term impact. Some risks 10. Post-collapse external threats Resilient systems are naturally (such as climate change) have and risks. Simply because a resistant to collapse, though strong long-term impacts after years risk has triggered the collapse of this often comes at the cost of or even decades. Others (such human civilisation, that does not efficiency.106 The more resilient as pandemics) are more likely to mean that other risks are no longer the system, the more likely it is have only a short-term impact. This present. Humanity will have much to be able to adapt to even large category includes only direct long- less resilience to deal with further disasters. Improving resilience term impacts. damage, so the probability of these ahead of time can improve risks is important to determine the outcomes, even if the nature of the 6. Post-risk politics. The political ultimate fate of humanity. disaster isn’t known. structures of the post-risk world (governmental systems, conflicts 11. Anthropic effects. We cannot 2. General pre-risk collapse between and within political observe a world incapable of countermeasures. This category groupings, economic and political supporting life, because we consists of all those measures put links between groups) will be could not be alive to observe it. into place ahead of time to prevent important in determining if a large When estimating the likelihood civilisation collapse. It could include, impact leads ultimately to civilisation of disasters and recovery it is for instance, measures to ensure collapse or if recovery is possible. very important to take this effect continuity of government or prevent into consideration and to adjust breakup of countries (or to allow 7. Post-risk collapse probability estimates accordingly.110 these breakups to happen with the countermeasures. These are the minimum of disruption). At the same countermeasures that the post- 12. Long-term reconstruction time it should be noted that these risk political structures are likely to probability. A post-collapse world kinds of measures could also trigger implement to prevent a complete will differ significantly from a pre- the breakdown. civilisation collapse. industrial revolution world. Easy access to coal and oil will no longer 3. General mitigation and resilience. 8. Maintaining a technology base. be possible. In contrast, much This category consists of all Current society is complex, usable aluminium will have been measures that can reduce the with part of the world’s excess extracted and processed and will impact of risks and prevent them production diverted into maintaining be left lying on the surface for easy getting out of hand (excluding social a population of scientists, use. Thus it will be important to and ecosystem measures, which engineers and other experts, establish how technically possible it are important and general enough to capable of preserving knowledge may be to have a second industrial deserve their own category). of technological innovations and revolution and further reconstruction developing new ones. In the simpler up to current capabilities without 4. Pre-risk rebuilding enablers. post-collapse societies, with creating the problems that the first On top of attempting to prevent possibly much lower populations, industrial revolution resulted in. collapses, measures can also it will be a challenge to maintain be taken to enable rebuilding current technology and prevent after a collapse.107 This could crucial skills from being lost.109 involve building stores of food, of technology, or crucial reconstruction 9. Post-collapse politics. Just as tools.108 Alternatively, it could post-risk politics are important for involve training of key individuals preventing a collapse, post-collapse or institutions (such as the crews of politics will be important in allowing nuclear submarines) to give them a recovery. The ultimate fate of useful post-collapse skills. humanity may be tied up with the preservation of such concepts as human rights, the scientific method and technological progress.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 59 3. Twelve Global Challenges

3. Twelve Global Challenges

“You may choose to look the other way but you can never say again that you did not know.”

William Wilberforce

60 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3. Twelve Global Challengesof risks

For the selection of events The list of events was then ranked Four categories were used to classify information from specialised based on their risk relevance, i.e. their the different events: bodies and scientific journals in the effect on the probability and/or the area of global risk was gathered.111 impact of the challenge. 1. Policy: Global or national policy Using keywords related to the initiatives that affect probability various risks, a global selection To finalise the list, a group of experts and/or impact of events was sought, along with was consulted by email and a draft 2. Event: The challenge is made original sourcing in academic or overview of the challenges was real in some way that is relevant for official sources. presented at a workshop at the Future probability and/or impact of Humanity Institute (FHI) in Oxford, 3. Research: New knowledge about where additional input was provided probability and/or impact on selection and content. Issue experts 4. Initiative: A stakeholder/group were then consulted before the final list addressing the challenge in concrete 112 Artificial Extreme Future Badof events was established.Global ways to reduceGlobalArtificial probability System and impact MajorExtreme Asteroid Future Bad Global Global System Major Asteroid

Intelligence Climate Change Global Governance Pandemic CollapseIntelligence ImpactClimate Change Global Governance Pandemic Collapse Impact

Global GlobalGlobal System System Major Asteroid ArtificialEcologicalArtificialArtificial ExtremeNanotechnologyExtremeExtreme FutureNuclearFutureFuture Bad BadWarBad GlobalSuper-volcanoGlobal GlobalSyntheticGlobalEcological System System MajorUnknownMajorNanotechnology Asteroid Asteroid Nuclear War Super-volcano Synthetic Unknown Pandemic CollapseCollapse Impact IntelligenceCatastropheIntelligenceIntelligence ClimateClimateClimate Change Change Change GlobalGlobalGlobal Governance Governance Governance PandemicPandemic CollapseBiologyCollapseCatastrophe ImpactConsequencesImpact Biology Consequences

Artificial Extreme ArtificialFutureEcologicalEcological Bad ExtremeGlobal FutureGlobal Bad System GlobalMajor Asteroid Global System Major Asteroid ArtificialArtificial ExtremeExtreme FutureFuture Bad Bad GlobalEcologicalGlobalEcological GlobalNanotechnologyGlobalNanotechnologyNanotechnology System System NuclearMajorMajorNuclearNuclear Asteroid Asteroid War War War Super-volcanoSuper-volcanoSuper-volcano SyntheticSyntheticSynthetic UnknownUnknownUnknown Intelligence Climate Change IntelligenceGlobalCatastropheCatastrophe Governance ClimatePandemic Change GlobalCollapse Governance PandemicImpact Collapse Impact IntelligenceIntelligence ClimateClimate Change Change GlobalGlobal Governance Governance PandemicCatastrophePandemicCatastrophe CollapseCollapse ImpactImpact BiologyBiologyBiology ConsequencesConsequencesConsequences

EcologicalEcological NanotechnologyNanotechnologyEcological NuclearNuclearNanotechnology War War Super-volcanoEcologicalSuper-volcanoNuclear War SyntheticNanotechnologySyntheticSuper-volcano NuclearUnknownUnknownArtificialSyntheticArtificial War Super-volcanoExtremeUnknownExtreme SyntheticFutureFuture Bad Bad UnknownGlobalGlobal GlobalGlobal System System MajorMajor Asteroid Asteroid CatastropheCatastrophe Catastrophe Catastrophe BiologyBiology ConsequencesConsequencesIntelligenceBiologyIntelligence ClimateConsequencesClimate Change Change BiologyGlobalGlobal Governance Governance ConsequencesPandemicPandemic CollapseCollapse ImpactImpact

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 61

3.1 Current risks 3.1.1 Extreme Climate Change

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Climate change is a significant and lasting change in the statistical distribution of weather patterns over periods ranging from decades to millions of years. Ecological It Nanotechnologymay be a change in average Nuclearweather Warconditions, Super-volcano Synthetic Unknown Catastrophe Biology Consequences or in the distribution of weather around the average conditions (i.e., more or fewer extreme weather events).

Extreme climate change is used to distinguish from the impacts beyond the dangerous climate that a 2° C temperature rise is expected to result in.113

62 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

3.1.1.1 Expected impact Combined with shocks to the If for any reason it stopped (such as a agriculture and biosphere-dependent civilisation collapse), warming would industries of the more developed resume at a significantly higher pace, Many of the expected impacts of countries, this could lead to global reaching the point where it would climate change are well known, conflict and possibly civilisation have been without geo-engineering. including a warming climate, more collapse – to the extent that many The speed of this rebound would severe storms and droughts, rising experts see climate change as a put extra pressure on the ecosystem sea levels, ocean acidification, and national security risk126. Further and the world’s political system. So damage to vulnerable ecosystems.114 evidence of the risk comes from the biggest challenge is that geo- As for all risks there are uncertainties indications that past civilisation engineering may backfire and simply in the estimates, and warming could collapses have been driven by make matters worse.134 be much more extreme than the climate change.127 middle estimates suggest. Models tend to underestimate uncertainty115 Extinction risk could develop from this 3.1.1.2 Probability (especially where impact on humanity if the remaining human groups were disaggregation is concerned,116 where the effect also vulnerable to other shocks, such as depends on modellers’ choices such pandemics, possibly exacerbated as the discount rate117), so there is by the changed climate.128 There is Five important factors in estimating a probability118 that humanity could some evidence of 6°C climate change the probabilities and impacts of the be looking at a 4°C119 or even 6°C120 causing mass extinction in the past,129 challenge: warming in the coming decades. This but a technological species such as could arise from positive feedback ourselves might be more resilient to 1. The uncertainties in climate loops, such as the release of methane such a shock. sensitivity models, including the tail. from permafrost121 or the dieback 2. The likelihood - or not - of of the Amazon rainforests,122 that A unique feature of the climate global coordination on controlling strengthen the warming effect. So far, change challenge is what is called emissions. efforts at curbing emissions have been geo-engineering.130 Though this could 3. The future uptake of low-carbon only moderately successful and are - if it works - reduce many impacts economies, including energy, still very far from what is needed.123 at a relatively low cost, it would mobility and food systems. not do so evenly. Geo-engineering 4. Whether technological The impact of global warming, would possibly reduce the impacts innovations will improve or worsen whether mild or severe, would of climate change in some countries, the situation, and by how much. be felt most strongly in poorer benefitting them while leaving others 5. The long-term climate impact countries. Adaptation that can to suffer.131 This could lead to greater caused by global warming. address significant warming is political instability. One of the most often very expensive,124 and many popular geo-engineering ideas – of the poorest countries are in the stratospheric sulphate aerosols tropics and sub-tropics that would – suffers from the weakness that it be hardest hit (they could become must be continuous. 132 completely uninhabitable for the highest range of warming125). Mass deaths and famines, social collapse and mass migration are certainly possible in this scenario.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 63 3.1 Current risks CLIMATE CHANGE GOVERNANCE DISASTERS

Research in Easily visible Technological Economic effects of Climate research Climate warfare innovations emmision-reducing transformations technologies climate change

Deliberate Collapse of New,Global polluting, Technological Research in Low-carbon Global attempts to Global povety Globalgeoengineering instability uses coordinationfor carbon innovations mitigation and Geoengineering economies coordination construct world projects products dictatorshipadaptation

Meta-uncertainty on tradeoffs between e.g. New system Smart sensors poverty, survival, of governance freedom

Extreme Carbon Moderate Climate change climate change emissions climate change Feedback loops Global poverty mitigation and adaptation

Failing to solve Improvements to Making important problems global governance things worse

Ecosystem Disruption to Political Increased storms, Agriculture damage Direct casualties world politics instability in flooding and disruption (e.g. ocean and economy vulnerable nations natural disaters acidification)

Not achieving Global Lack of human Enduring poverty important Climate change pollution flourishing ethical goals

Long-term climate effects Undesirable world system (e.g. global dictatorship)

Disruption to Pre-warming Collapse of Post-disaster Long-term world politics Post warming Forced migration collapse world system politics negative effects and economy politics countermeasures

Pre-warming Total short-term casualties Civilization mitigation efforts collapse Extinction General mitigation Civilisation Total short-term Extinction casualties effort collapse

Meta-uncertainty on how to predict the international Meta-uncertainty on the true uncertainty in climate political process change models

Key

Key Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

64 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

1. Research which further refines our temperatures and climates. The indirect determining whether carbon emissions understanding of climate change and effect can however be significant, e.g. are self-damping or self-forcing (i.e. geo-engineering ideas will be essential in migration, starvation, extreme weather. whether an extra ton of CO2 emissions predicting change, preparing for it, and is likely to result in more or less than a potentially reversing it. On the negative 11. Climate change is likely to cause ton in the atmosphere). side, climate science research may allow extensive ecosystem damage, such the possibility of climate change tools as ocean acidification and pressure 21. Transitioning to low carbon being used for warfare. on many sensitive species that cannot economies will be crucial for reducing easily adapt to temperature changes. emissions without disrupting the world’s 2. Global poverty will affect both the political or economic systems. vulnerability of many nations to the 12. Agriculture will be disrupted by effects of climate change, and the increased temperature. 22. Geo-engineering offers the possibility likelihood of achieving global co- of decreasing carbon concentration in ordination earlier rather than later. 13. The direct and indirect effects of the atmosphere alongside, or instead of, climate change will have a great impact emission reductions. But it may make 3. Pre-extreme warming mitigation on the world’s political and economic climate warfare a possibility. efforts will affect the level of impact from systems, which will in turn determine the climate change. severity of the changes. 23. If geo-engineering projects collapse in the middle of implementation, this 4. Pre-warming collapse 14. Many nations will be made could lead to strong warming over a countermeasures will affect the politically vulnerable to the direct and dangerously short period of time. likelihood of civilisation collapse. indirect impacts of climate change, putting great pressure on their political 24. Technological innovations will be 5. Research into mitigation and systems and institutions. crucial for transitioning to low carbon adaptation is necessary for effective economies or allowing geo-engineering. implementation of either approach. 15. Climate change will cause an But they may also result in new, increase in storms, floods, and other carbon-intensive innovations, which, 6. Research into emission-reducing natural disasters. If political stability is if sufficiently profitable, could push technologies (such as alternative maintained, most of the casualties are emissions up. energies) will be important for likely to result from these factors. transitioning to a low carbon economy. 25. Some level of changes to the 16. Forced migration from unstable or standard economic system may 7. Global coordination and cooperation disrupted areas will put further pressure be needed to transition to low will be key to funding mitigation/ on more stable areas. carbon economies. adaptation research and development, and for the global control of carbon 17. The long-term impact of 26. Easily visible impacts of climate emissions or transitioning to a global low climate change (including further change may be instrumental in pushing carbon economy. carbon emissions and warming) better global coordination on the issue. will be important for determining 8. Climate warfare is possible if geo- the risk of collapse and 27. The political systems in place as engineering and climate modification subsequent rebuilding possibilities. warming increases will determine how methods can be harnessed by nations well the world copes with a hotter planet. to harm others. 18. Attempts to mitigate and adapt to climate change will be important 28. Climate models are extremely 9. New, more polluting uses of carbon for reducing the severity of climate detailed and inevitably uncertain. would, if they had a strong economic change’s impact. But the real level of uncertainty rationale, put upwards pressure on includes uncertainties about the carbon emissions. 19. The level of carbon emissions is the models themselves. driver of climate change, and will be 10. The direct casualties of limited crucial in determining its ultimate impact. 29. The course of international global warming are likely to be few, as politics is extremely hard to predict, humans can adapt to many different 20. Feedback loops will be important in even for political scientists.135

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 65 3.1 Current risks

21-May-13: China agrees 3.1.1.3 Main events Prior to the Industrial Revolution, natural climate variations caused to impose carbon targets by 2016141 during 2013 atmospheric CO2 to vary between – Policy about 200 ppm during ice ages 19-Apr-13: Launch of the report and 300 ppm during the warmer Since China is the world’s greatest “Unburnable Carbon 2013: Wasted inter-glacial periods. The last time emitter of CO2,142 any reduction capital and stranded assets” 136 concentrations were as high as steps it takes can have a substantial – Research they are now seems to have been impact. It has announced a “National during the Mid-Pliocene, about 3 Low Carbon Day“,143 a “series of To constrain the rise in global average million years before the present when major promotional events to improve temperature to less than 2°C above temperatures were 2-3°C warmer, awareness and get the whole society pre-industrial levels, a maximum of and in which geological evidence and to address climate change.” More around 565 – 886 billion tonnes (Gt) isotopes agree that sea level was at practically, the Chinese government of carbon dioxide could be emitted least 15 to 25 m above today’s levels has agreed to impose carbon targets before 2050.137 The world’s proven with correspondingly smaller ice by 2016 - a ceiling on greenhouse fossil fuel reserves amount to 2,860 Gt sheets and lower continental aridity.140 gas emissions.144 of CO2, however, and are viewed as assets by companies and countries. Since it is likely that these assets Figure 14-15, Source: Scripps Institution of Oceanography, via http://blogs.scientificamerican.com/ cannot be realised, these entities observations/2013/05/09/400-ppm-carbon-dioxide-in-the-atmosphere-reaches-prehistoric-levels are over-valued at current prices – arguably, a “carbon bubble.”

The report provides evidence that serious risks are growing for high- carbon assets, and aims to help investors and regulators manage these risks more effectively and prepare for a global agreement on emissions reductions. It indirectly highlights part of the challenge of emissions reductions: they will mean the loss of highly valuable assets to corporations and governments.

02-May-13: CO2 at 400 PPM for the first time in > 800,000 years138 – Event

The Mauna Loa carbon dioxide record, also known as the “Keeling Curve,” is the world’s longest unbroken record of atmospheric CO2 concentrations. It recently reached 400 ppm (parts per million) of CO2. Such concentrations have not been reached for at least 800,000 years,139 placing humanity in a historically unprecedented situation.

66 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

22-May-13: Private Sector Initiative 27-Sep-13: IPCC report: “Climate Change – The rate of sea level rise since - database of actions on adaptation145 2013: The Physical Science Basis” 147 the mid-19th century has been – Initiative – Research larger than the mean rate during the previous two millennia (high Global warming is an externality146 – a The 5th IPCC report “considers new confidence). Over the period 1901 to consequence of business decisions evidence of climate change based 2010, global mean sea level rose by made by entities that do not bear the on many independent scientific 0.19 [0.17 to 0.21] m. full cost of what they decide – so the analyses from observations of the drive to mitigate its effects is more climate system, palaeoclimate – The atmospheric concentrations likely to come from governmental or archives, theoretical studies of climate of carbon dioxide, methane, and supra-governmental organisations. processes and simulations using nitrous oxide have increased to Nevertheless, the private sector has climate models.” It concludes that: levels unprecedented in at least the been involved in mitigation attempts for last 800,000 years. Carbon dioxide a variety of reasons, from investment – Warming of the climate system is concentrations have increased by opportunities to public relations. The unequivocal, and since the 1950s 40% since pre-industrial times, United Nations Framework Convention many of the observed changes primarily from fossil fuel emissions on Climate Change (UNFCCC) are unprecedented over decades and secondarily from net land use maintains a database of some of these to millennia. The atmosphere and change emissions. attempts, ranging from Ericsson’s oceans have warmed, the amounts enabling access to climate services in of snow and ice have diminished, The report further predicted, amongst Uganda, through BASF’s development sea level has risen, and the other points, that: of new technologies for food security, concentrations of greenhouse gases Allianz insurers rewarding sustainable have increased. – Continued emissions of greenhouse business practices, all the way to gases will cause further warming Chiles de Nicaragua’s attempts to – Human influence on the climate and changes in all components of enable small agro-exporters to adapt to system is clear. This is evident from the climate system. Limiting climate climate change – and many more. the increasing greenhouse gas change will require substantial and concentrations in the atmosphere, sustained reductions of greenhouse The potential opportunities for private positive radiative forcing, observed gas emissions. companies are listed as: warming, and understanding of the climate system. It is extremely likely – The oceans will continue to warm – New market opportunities and that human influence has been the during the 21st century. Heat will expansion; dominant cause of the observed penetrate from the surface to the deep – Development of climate-friendly warming since the mid-20th century. ocean and affect ocean circulation. goods and services; Further uptake of carbon by the oceans – Potential cost savings; – Each of the last three decades has will increase ocean acidification. – Risk reduction measures, including been successively warmer at the Global mean sea level will continue physical operations; Earth’s surface than any preceding to rise during the 21st century. – Climate proofing the supply chain; decade since 1850. – Enhanced corporate social – It is very likely that Arctic sea ice responsibility. – Over the last two decades, the cover will continue to shrink and Greenland and Antarctic ice sheets become thinner. Global glacier have been losing mass, glaciers volume will further decrease. have continued to shrink almost worldwide, and Arctic sea ice and – Most aspects of climate change Northern Hemisphere spring snow will persist for many centuries even cover have continued to decrease if emissions of CO2 are stopped. in extent.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 67 3.1 Current risks

27-Sep-13: Launch of the Global Risk 23-Nov-13: Limited progress at Warsaw 03-Dec-13 Abrupt Impacts of Climate and Opportunity Indicator (GROI) 148 COP 19 climate negotiations 149 Change: Anticipating Surprises155 – Research – Policy – Research

Launched by the Global Challenge The global environment can be Climate change has been developing Foundation, this Indicator is a web tool considered a global public good (i.e. gradually, at least on the human for illustrating quantified risks, with non-excludable and non-rivalrous).150 scale156 (though very rapidly on a the objective of increasing awareness Economic theory claims that such geological timescale157). This may about global risks and opportunities goods will be undersupplied by the not continue, however: this paper and helping guide the changes required market.151 Hence the importance of looks at the potential for abrupt in the global governance system. The trans-national negotiations to address changes in physical, biological, and site is still under construction; the climate change. human systems, in response to Foundation’s aims are to achieve, by steady climate change. It highlights the end of 2014: Despite the importance of the subject, two abrupt changes that are already the main achievement of the Warsaw under way: the rapid decline in sea 1. An interactive Global Risk & negotiations was to keep talks on ice158 and the extinction pressure on Opportunity Indicator that allows track for more negotiations in 2015.152 species.159 On the other hand, some users to calculate the probability Though there was general agreement widely discussed abrupt changes – for any global warming, between on the necessity of cutting carbon the rapid shutdown of the Atlantic one and ten degrees Celsius, emissions, the dispute was over Meridional Overturning Circulation160 at different greenhouse gas how to share the burden of doing and the rapid release of methane concentrations. The indicator so. In this instance, the debate was from either thawing permafrost161 or will then be further developed to between more- and less-developed methane hydrates162 – are shown to illustrate interdependencies with countries, with the latter demanding be unlikely to occur this century. other global risks and highlight compensation from the former to opportunities for minimising the help them cope with the burden of The report argues that large risks. Subsequent development reducing emissions. That particular uncertainties about the likelihood of will allow users to change different dispute was papered over,153 but some potential abrupt changes163 underlying assumptions and see similar ones will be likely in future due highlight the need for expanded the corresponding change in risk. to the range of different actors and research and monitoring, and propose their divergent agendas.154 an abrupt change early warning 2. Methodology and data to system. The aim would be to foresee estimate probabilities for a number abrupt change before it occurs, and of climate impacts at different reduce the potential consequences. temperature levels, e.g., sea level rise, droughts, flooding and heat waves, as well as to explore the risk of runaway global warming.

3. Methodology and data to estimate the probability of existential climate threats, i.e., to estimate the risk that climate change impacts pose a significant threat to human civilisation – defined as a serious negative impact on at least two billion people.

68 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 69 Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact 3.1 Current risks

3.1 Current risks 3.1.2 Nuclear War

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

After their use in Hiroshima and Nagasaki nuclear weapons have never been used in a conflict, but because they are extremely powerful and could cause destruction throughout the world, the possibility of nuclear war has had a great effect on international politics. 164

70 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

3.1.2.1 Expected impact There are some uncertainties about society to the point where recovery both the climate models and the becomes impossible178 before likelihood of devastating firestorms,174 humanity succumbs179 to other risks, The likelihood of a full-scale nuclear but the risks are severe and recent such as pandemics.180 war between the USA and Russia has models175 have confirmed the earlier176 probably decreased in recent decades analysis. Even a smaller nuclear 3.1.2.2 Probability due to some improvements in relations conflict (between India and Pakistan, between these two countries and for instance) could trigger a smaller disaggregation reductions in the size of their arsenals. nuclear winter which would place Still, the potential for deliberate or billions in danger.177 Five important factors in accidental165 nuclear conflict has not estimating the probabilities and been removed, with some estimates The disintegration of the global impacts of the challenge: putting the risk of nuclear war in the food supply would make mass next century or so at around 10%166 – it starvation and state collapse likely. 1. How relations between current may have been mostly down to luck As the world balance of power and future nuclear powers develop. that such a war did not happen in the would be dramatically shifted and 2. The probability of accidental war. last half century167. previous ideological positions 3. Whether disarmament efforts will called into question, large-scale war succeed in reducing the number of A nuclear war could have a range of would be likely. This could lead to a nuclear warheads. different impacts. At the lowest end civilisation collapse. 4. The likelihood of a nuclear winter. is the most obvious and immediate 5. The long-term effects of a nuclear impact: destruction and death in Extinction risk is only possible if war on climate, infrastructure major cities across the world, due to the aftermath of the nuclear war and technology. the explosions themselves and the fragments and diminishes human radioactive fallout. But even if the entire populations of Europe, Russia and the USA were directly wiped out in a nuclear war – an outcome that some studies have shown to be physically impossible168, given population dispersal and the number of missiles in existence169 – that would not raise the war to the first level of impact, which requires > 2 billion affected.170

A larger impact would depend on whether or not the war triggered what is often called a nuclear winter or something similar.171 The term refers to the creation of a pall of smoke high in the stratosphere that would plunge temperatures below freezing around the globe and possibly also destroy most of the ozone layer.172

The detonations would need to start firestorms in the targeted cities, which could lift the soot up into the stratosphere.173

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 71 3.1 Current risks NUCLEAR WAR GOVERNANCE DISASTERS

US-Russia relations

Number of Relations between Disarmament Full-scale future major future major Nuclear attack DeliberateNuclear War efforts nuclear powers Globalnuclear powers Technological attempts to Global povety Global instability coordination innovations construct world dictatorship

Meta-uncertaintyNumber of Proliferation: on tradeoffs future small Newdesire system for Nuclear terrorism Nuclear security betweennuclear e.g.powers nuclear weapons Smart sensors poverty, survival, of governance freedom

Relations Proliferation: Nuclear between future building nuclear accidents or nuclear powers weapons misunderstandings

Failing to solve Improvements to Making important problems global governance things worse

Relations Meta-certainty Meta-certainty between current of political of changes in the Small-scale nuclear powers predictions military technology Nuclear War

Disruption to Not achieving Global Lack of human Nuclear attack Firestorm risks world politics Firestorm risks Enduring poverty important Climate change pollution flourishing ethical goals and economy

Undesirable world system (e.g. global dictatorship)

War casualties Small Post-war politics Nuclear Winter War casualties Nuclear Winter Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy

Total Long-term short term impact casualties General mitigation Civilisation Total short-term Extinction casualties effort collapse

Pre-war casualty Extinction countermeasures Civisation (bunkers, food...) collapse

Key

72 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

1. The success or failure of 11. A small-scale nuclear war could 21. Since a major nuclear power disarmament will determine the start with an attack by one or more must be one of the parties to a major number of nuclear warheads available nuclear powers. nuclear war, the number of the former for a future nuclear conflict. affects the probability of the latter. 12. A full-scale nuclear war could start 2. The first step of proliferation with an attack by one or more major 22. Post-war politics will be is countries desiring to possess nuclear powers. determined by the war, the disruption nuclear weapons. Various political it caused, and the number of interventions may reduce or increase 13. Aside from attacks, the other casualties it inflicted. this desire. way a nuclear war could start would be through accidental firings or 23. Unlike other risks, nuclear 3. The second step of proliferation is misinterpretations of other incidents. weapons are targeted by humans, countries building nuclear weapons. so may take out important parts Various mechanisms, agreements and 14. Firestorms caused by burning of the world’s infrastructure (and inspections may be relevant cities are one of the main ways conventional weapons used in a a nuclear conflict could cause conflict may have the same effect). 4. Nuclear terrorism may be the trigger major climate disruption, and of a larger nuclear conflict, especially hence high casualties. 24. Unlike other risks, nuclear if the detonation is misinterpreted as a weapons are targeted by humans, so traditional attack. 15. The direct war casualties from may take out important parts of the a nuclear conflict are likely to be world’s technology and research base 5. The security of nuclear weapons small compared with the potential (and conventional weapons used in a and materials affects both the climate effects. conflict may have the same effect). probability of nuclear terrorism and the control likelihood of nuclear accidents. 16. A nuclear winter is the way 25. Maintaining a technology base in which a nuclear conflict could will be complicated by the possible 6. The relations between future have the most damaging effects targeting of infrastructure and the nuclear powers will be the major on the world. technology base during a conflict. determinant of whether a nuclear war breaks out. 17. Even a smaller nuclear conflict 26. The further development of military could trigger a smaller nuclear technology is hard to predict. The 7. The relations between current winter that could have major current balance of power under MAD nuclear powers will be a major disruptive effects on agriculture (mutually assured destruction) is determinant of the relations between and hence human survival. based on certain assumptions about future nuclear powers. the effectiveness of nuclear weapons, 18. Any war will have a disruptive such as second strike capability. If this 8. The relations between future major impact on the world’s politics and were removed (such as by effective nuclear powers will be the major economy. A nuclear conflict – possibly submarine detection, or anti-ballistic component of determining whether a accompanied by a nuclear winter – missile shields), the effect on the major nuclear war breaks out. even more so. balance of power is hard to predict.

9. Relations between the USA and 19. The long term impact of nuclear 27. The course of international politics Russia (the only current major nuclear winter, infrastructure disruption, and is extremely hard to predict, even for powers) will be a major determinant possibly radiation, will determine the political scientists.181 of the relations between future major likelihood of collapse and rebuilding. nuclear powers. 20. Since a nuclear power must be 10. Pre-war countermeasures (such one of the parties to a nuclear war, as nuclear bunkers and food stores) the number of the former affects the can help mitigate the casualties of a probability of the latter. smaller nuclear conflict.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 73 3.1 Current risks

3.1.2.3 Main events Though the situation remains a – It is unlikely that any state or potential flashpoint for conventional international body could address the during 2013 and nuclear conflict, and its collapse immediate humanitarian emergency could have disastrous consequences191 caused by a nuclear weapon 12-Feb-13: North Korea carries out (including the possibility of “loose detonation in an adequate manner third, largest nuclear test 182 nukes” becoming available to various and provide sufficient assistance to – Event groups), it should be noted that the those affected. Moreover, it might “North Korean problem” has existed not be possible to establish such On 12 February 2013, North Korea carried in one form or another since the end capacities at all. out its third nuclear test. The test was of the Korean War in 1953, without – The historical experience from the condemned across the world, 183 erupting into open conflict.192 use and testing of nuclear weapons and led to increased sanctions184 has demonstrated their devastating against the already isolated nation.185 immediate and long-term effects. North Korea is the only nation to have 04-Mar-13: Conference: Humanitarian While political circumstances have withdrawn from the Nuclear Non- Impact of Nuclear Weapons 193 changed, the destructive potential of Proliferation Treaty,186 and is the only – Policy nuclear weapons remains. country to have conducted nuclear – The effects of a nuclear weapon tests in the 21st century, starting On 4 and 5 March 2013, the detonation, irrespective of cause, in 2006, 187 as well as developing a Norwegian Minister of Foreign will not be limited by national ballistic missile capability.188 It has also Affairs, Espen Barth Eide, hosted borders, and will affect states and been involved in the export of weapons an international conference on the people to significant degrees, technology, undermining the Treaty.189 humanitarian impact of nuclear regionally as well as globally. Diplomatic attempts to deal with North weapons. The conference heard Korea (especially on the part of the presentations on the effects of A number of states wished to explore United States) have generally been nuclear weapons detonations. these issues further, and Mexico said it inconsistent and unsuccessful.190 Three key points emerged: would host a follow-up conference.194

Worldwide nuclear testing, 1945-2013 Figure 16, Source: Wikimedia Commons, http://en.wikipedia.org/wiki/ 16, Source: Figure license. CC-BY-SA File:Worldwide_nuclear_testing.svg

74 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

16-May-13: Revealed: The USSR 24-Jun-13: Report: “Analysing and Then it combined the fault tree-based and US Came Closer to Nuclear War Reducing the Risks of Inadvertent risk models with parameter estimates Than Was Thought 195 Nuclear War Between the United sourced from the academic literature, – Research States and Russia” 204 characterising uncertainties in the – Research form of probability distributions, Documents recently released under a with propagation of uncertainties FOIA (US Freedom Of Information Act) Though the end of the Cold War has in the fault tree using Monte Carlo request show that the risk of nuclear reduced the likelihood of deliberate simulation methods. Finally, it also conflict between the superpowers nuclear war, its impact on the risk performed sensitivity analyses to was higher than realised at the time. of accidental nuclear war is much identify dominant risks under various The large-scale 1983 NATO nuclear smaller. The arsenals remain on assumptions. This kind of highly exercises Able Archer 83” spurred “a “launch on warning”,205 meaning that disaggregated analysis is most likely high level of Soviet military activity, there is a possibility for a “retaliatory” to elicit the best performance and with new deployments of weapons strike before an attack is confirmed. estimates from experts.209 and strike forces.” This unprecedented The most likely cause of such an Soviet reaction in turn created a accident is either a false warning Their conclusion was that (under the series of introspective US intelligence (of which there have been many, more pessimistic assumption), there analyses and counter-analyses, with causes ranging from weather was a mean 2% risk of accidental debating whether US intelligence phenomena to a faulty computer nuclear war a year (a high risk when had actually understood Soviet chip, wild animal activity, and control- compounded over several decades), actions, perceptions, and fears – and room training tapes loaded at the with the risk from false alarm being acknowledging the danger of nuclear wrong time)206 or a misinterpreted orders of magnitude higher than that “miscalculation” if it had not.196 terrorist attack.207 from terrorist attacks. The analysis suggests that the most important This is but one of the many nuclear The report attempted a rigorous inadvertent nuclear war risk factor is the accidents197 and incidents that estimate of the numerical probability short launch decision times,210 inherent peppered the Cold War and its of nuclear war. Such numerical rigour in the “launch on warning” posture. aftermath, and which have been is rare, with the exception of Hellman’s Some ways of improving this were revealed only subsequently. We know estimates.208 This report applied risk suggested, for instance by moving each now that there were at least three analysis methods using fault trees and country’s strategic submarines away occasions – the Cuban missile crisis in mathematical modelling to assess the from the other’s coasts. 1962,198 the Petrov incident in 1983199 relative risks of multiple inadvertent and the Norwegian rocket incident in nuclear war scenarios previously 1995200 – where a full-scale nuclear identified in the literature. war was only narrowly averted.201 Further information on these incidents, and on how they were interpreted and misinterpreted202 by the great powers, will be important to estimate the probability of nuclear conflict in the coming decades. On a more positive note, efforts are being made to reduce the probability of inadvertent or accidental nuclear conflicts.203

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 75 3.1 Current risks

03-Sep-13: Report of the UN 15-Nov-13: International Physicians 24-Nov-13: Nuclear deal with Iran General Assembly working group on for the Prevention of Nuclear War may reduce risk of proliferation 224 “Taking Forward Multilateral Nuclear report: “Nuclear Famine: Two Billion – Policy Disarmament Negotiations” 211 People at Risk?” 218 – Policy – Research In November, Iran struck a deal with the so called “P5+1” (the five The working group had extensive This report is one of a series of permanent members of the security exchanges of view from different reports and publications in recent council, plus Germany). The deal, if participants, and reviewed existing years about the potential impacts of it holds, would allow Iran to continue disarmament commitments and nuclear conflicts.219 It looked at the some uranium enrichment, but it proposals, including international law. likely consequences of a “limited” would have to submit to inspections The issues surrounding disarmament nuclear war, such as between India to ensure it wasn’t developing a and treaties were analysed in depth, and Pakistan. nuclear weapons programme (the deal and several proposals were put would also result in eased sanctions forward, with an eye to the complete While previous papers had estimated in return). There have been long- elimination of nuclear weapons. that up to a billion people might running fears than Iran may have been be at risk in such a conflict,220 this attempting to construct a nuclear A key recognition was, however, that report increased the estimate to weapon225, resulting in sanctions “participants recognised the absence two billion. The main source of this being imposed on it.226 of concrete outcomes of multilateral increase is decreased agricultural nuclear disarmament negotiations production in the United States221 This event illustrates the surprising within the United Nations framework and in China.222 A key component success of the Non-Proliferation for more than a decade”. Indeed, of these estimates was the severe Treaty,227 which came into force in though the Nuclear Non-Proliferation agricultural impact of the relatively 1970. At the time it was proposed Treaty212 (NPT) is a multilateral treaty mild temperature reduction in 1816, there were fears of very rapid closely connected with the United the “year without a summer” 223, proliferation of nuclear weapons.228 Nations, and though it committed due mainly to the “volcanic winter” And though 40 countries or more the nuclear powers to reduce their caused by the eruption of Mount currently have the knowhow to arsenals, all the major nuclear arms Tambora. The report highlights some build nuclear weapons,229 only nine reduction deals have been bilateral significant areas of uncertainty, countries are currently known to treaties between the US and the such as whether a small nuclear possess them: the five security council USSR/Russia. These include the INF conflict and its consequences would members, India, Pakistan, and North treaty213, START I214, SORT215, and New lead to further conflicts across the Korea, plus Israel.230 START216, which have significantly world, and doubts whether markets, reduced the world’s stock of nuclear governments and other organisations weapons. It has also been argued could mitigate the negative impacts. that the NPT has been undermined The report is a reminder that even by a number of bilateral deals made small-scale nuclear conflict could by NPT signatories, most notably have severe consequences. the United States.217 This further serves to emphasise the weakness of international institutions where nuclear arms control is concerned.

76 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 77 Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact 3.1 Current risks

3.1 Current risks 3.1.3 Ecological Catastrophe

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Ecological collapse refers to a situation where an ecosystem suffers a drastic, possibly permanent, reduction in carrying capacity for all organisms, often resulting in mass extinction.

Usually an ecological collapse is precipitated by a disastrous event occurring on a short time scale. 231

78 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

3.1.3.1 Expected impact There is currently more than enough 3.1.3.2 Probability food for everyone on the planet to ensure the nutrition needed, 240 but disaggregation Humans are part of the global its distribution is extremely uneven ecosystem and so fundamentally and malnutrition persists. Thus Five important factors in depend on it for our welfare. ecological collapse need not have estimating the probabilities and a strong absolute effect in order to impacts of the challenge: Species extinction is proceeding at result in strong localised, or global, a greatly increased rate compared effects. Even a partial collapse could 1. The extent to which humans are with historic data232, and attempts to lead to wars, mass migrations, and dependent on the ecosystem. quantify a safe ecological operating social instability. It is conceivable that 2. Whether there will be effective space place humanity well outside such a scenario, if drawn out and political measures taken to protect it.233 Furthermore, there may be signs exacerbated by poor decision-making, the ecosystem on a large scale. of a “sudden” biosphere collapse, could eventually lead to mass deaths 3. The likelihood of the emergence possibly within a few generations.234 and even the collapse of civilisation. of sustainable economies. Many of the problems of ecological 4. The positive and negative degradation interact to multiply Extinction risk is possible only if the impacts on the eco systems of the damage and (unlike previous, aftermath of collapse fragments both wealth and poverty. localised collapses) the whole world and diminishes human society 5. The long-term effects of an is potentially at risk, 235 with severe so far that recovery becomes ecological collapse on ecosystems. challenges to countering this risk impossible241 before humanity through global policy.236 succumbs to other risks (such as climate change or pandemics). If animals are seen to have intrinsic value, 237 or if human quality of After a post-civilisation collapse, life is dependent on a functioning human society could still be suffering ecosystem, 238 the current situation from the effects of ecological already represents a large loss. collapse, and depending on what form it took, this could make the Whether such a loss will extend to recovery of human civilisation more human lives depends on technological challenging than in some of the other and political factors - technological, scenarios presented here. because it seems plausible that some human lifestyles could be sustained in a relatively ecosystem-independent way, at relatively low costs.239 Whether this can be implemented on a large scale in practice, especially during a collapse, will be a political challenge and whether it is something we want is an ethical question.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 79 3.1ECOLOGICAL Current risks CATASTROPHE GOVERNANCE DISASTERS

Post-eco-collapse Global Sustainability climate change Global poverty coordination research

Long-term ecological effects Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship

Quality of life Ecological Preservation Meta-uncertainty loss from collapse efforts on tradeoffs ecosystem loss between e.g. New system Smart sensors poverty, survival, of governance freedom

Threat to Moral tragedy from Pollution food supply ecosystem loss

Failing to solve Improvements to Making global governance things worse Loss of important problems Economic costs Pre-eco-collapse biodiversity climate change

Disruption to Rebuilding politics and Post-eco-collapse the ecosystem economy politics

Not achieving Global Lack of human Enduring poverty important Climate change pollution flourishing ethical goals

Vulnerabilities Sustainable or New, to flood and non-sustainable environmentally other disasters economies damaging industries Undesirable world system (e.g. global dictatorship)

Pre-eco-collapse Technological mitigation efforts innovations Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy

Human survivability Meta-uncertainty on the true dependence of in “closed” systems humanity on the ecosystem

General mitigation Civilisation Total short-term Extinction casualties effort collapse

Total short-term Civilisation Extinction casualties collapse

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

80 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

1. Global coordination and 9. New, profitable, but environmentally 19. Technological innovations cooperation will be important to any damaging industries could put extra may result in more sustainable attempt to control ecological damage strain on the ecosystem. economies, or in more on a large scale and prevent “races to environmentally damaging products. the bottom”. 10. According to some systems of value, the loss of certain animals 20. It may be possible to 2. Poverty is often seen as and ecosystems constitutes a moral ensure human survival in semi- exacerbating ecological damage tragedy in and of itself. “closed” systems (solar power, through unsustainable practices, hydroponic food, distilled water), while richer countries introduce 11. Humans derive much pleasure and with minimal dependency on the environmental regulations – but richer many benefits from various parts of external ecosystem. nations exploit many resources (such the ecosystem, and losing this would as fossil fuels) in non-sustainable and result in a loss to human quality of life. 21. Over the long term, it may become damaging ways. possible and necessary to go about 12. Ongoing and continuous rebuilding the ecosystem and healing 3. Transitioning to sustainable biodiversity loss is a clear its damage. economies, or sustainable consequence of ecological collapse. economic trajectories, could control 22. Political decisions will be the most ecological damage. 13. Ecological damage can put likely factors to exacerbate or mitigate the human food system in danger, an ecological disaster. 4. Research into sustainability could triggering famines. allow the construction of sustainable 23. It is unclear how dependent economies or environments at costs 14. Ecological damage increases humans truly are on the that people are willing to bear. vulnerability to floods and other ecosystem, and how much natural disasters. damage they could inflict without 5. Climate change exacerbates the threatening their own survival. pressure on the ecological system 15. Disruptions to the world’s political by changing weather patterns and and economic systems could trigger increasing natural disasters in ways further conflicts or instabilities, ecosystems find hard to adapt to. causing more casualties and impairing effective response. 6. Global pollution is a visible source of ecological damage, one that global 16. Since a lot of the world’s carbon is agreements have had moderate locked up in trees, ecological collapse success at tackling. could exacerbate climate change.

7. Truly global preservation efforts 17. The ecosystem is of great may be needed for some threatened economic benefit to humanity, ecosystems that stretch beyond so its loss would have large natural boundaries (e.g. in the seas economic costs. and oceans). 18. Ecological damage is likely to 8. Beyond general all-purpose be long-term: the effects will last for mitigation efforts, addressing many generations. this threat could include the preservation of ecosystems, species or genetic codes, to allow a subsequent rebuilding.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 81 3.1 Current risks

05-Apr-13: Ocean data added to 30-May-13: Improvement in 3.1.3.3 Main events Microsoft Eye on Earth project managed fisheries in Europe 252 during 2013 – Initiative – Research

22-Jan-13: Current extinctions In order to safeguard ecological Human action has been shown to probably the result of past actions; resources, it is important to be able to mitigate some ecosystem many future extinctions to come 242 track and quantify them. This damage. Overfishing is expected by – Research has traditionally been the role of standard economic theory: the sea’s governments or non-governmental resources are a (global) common, An estimated 40% of world trade organisations.248 Recently, where the rational behaviour of is based on biological products however, private organisations individual fishermen must lead to or processes such as agriculture, have started developing tools to dilapidation of the resource.253 Unlike forestry, fisheries and plant-derived enable companies and individuals on land, where nature reserves or pharmaceuticals, and biodiversity to track ecological damage and parks can be established, there comprises an invaluable pool make decisions in consequence. are no easy ways of establishing for innovations.243 And yet this One such tool was Eye on Earth, property rights in the sea254 (thus biodiversity is being lost at an developed by Microsoft in alliance privatising that “common”). A alarming rate – the rate of extinctions with the European Environment typical example of this behaviour for plants and animals is 100 to 1,000 Agency and Esri.249 It was launched is the collapse of the Grand Banks times higher than their pre-human with three services – WaterWatch, fisheries off Canada’s Atlantic coast levels.244 A variety of methods have AirWatch and NoiseWatch – keeping in the 1990s, where cod biomass fell been suggested to halt or slow track of the levels of different by over 95% from its peak and has this loss, ranging from putting an pollutants, using official sources currently not recovered.255 explicit value245 on biodiversity and and inputs from citizens.250 This was ecosystem services (human benefits subsequently expanded to include It is therefore significant that the from a multitude of resources and other environmentally sensitive European Union has been partly processes that are supplied by pieces of information, such as the successful in its attempts to control ecosystems), 246 to performing triage states of coral reefs and invasive over-fishing through legislation. on the most valuable species.247 This alien species. For instance, despite the fact that research paper suggests, however, North Sea cod remains vulnerable, that there is a lag of several decades It was primarily land-based, so there has been a recent increase between human pressure on the the oceans were missing from in stock size and a decrease in ecosystem and ultimate species this visualisation tool. This lack fish mortality. This may point to extinction. This suggests that many has been partially overcome the potential for further ecological extinctions will continue in decades with the inclusion of data from improvements through well-chosen to come, irrespective of current the MyOcean 2 project251 policy interventions. conservation efforts. (partly funded by the European Commission). The data cover sea surface temperature, salinity and currents for the Mediterranean Sea and the Black Sea.

82 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

02-Jul-13: About 21,000 Species Face Extinction, says International Union for Conservation of Nature (IUCN) 256 – Event

In 2013 the IUCN added an additional 4,807 species to its Red List of Threatened Species. This brings the total to about 21,000. Some have argued that we are entering a new geological era in Earth’s history: the Anthropocene257, when human actions are one of the major impactors on the planet’s biosphere.

The graph shows a fairly steady growth in the (estimated) number of threatened species. This steadiness may be illusory, as the biosphere shows signs that it may be Figure 17: Collapse of Atlantic cod stocks (East Coast of Newfoundland), 1992 approaching a planetary-scale tipping Source: http://en.wikipedia.org/wiki/File:Surexploitation_morue_surp%C3%AAcheEn.jpg) point, where it may shift abruptly and irreversibly from one state to another. As a result, the biological resources humans presently take for granted may be subject to rapid and unpredictable transformations within a few human generations.258 This could be seen as a great tragedy beyond purely human concerns, if animals (and animal welfare) are seen to have intrinsic value.259

Figure 18: Increase in the number of species assessed for the IUCN Red List of Threatened SpeciesTM (2000–2013.2). Source: http://www.iucnredlist.org/about/summary-statistics

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 83 3.1 Current risks

3.1 Current risks 3.1.4 Global Pandemic

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

A pandemic (from Greek πᾶν, pan, “all”, and δῆμος demos, “people”) is an epidemic of infectious disease that has spread through human populations Ecological Nanotechnology Nuclear War acrossSuper-volcano a large region; for instanceSynthetic Unknown Catastrophe Biology Consequences several continents, or even worldwide.

Here only worldwide events are included. A widespread endemic disease that is stable in terms of how many people become sick from it is not a pandemic. 260

84 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

3.1.4.1 Expected impact Many relevant features of the world Extinction risk is only possible if the have changed considerably, making aftermath of the epidemic fragments past comparisons problematic. and diminishes human society to Infectious diseases have been one The modern world has better the extent that recovery becomes of the greatest causes of mortality sanitation and medical research, as impossible277 before humanity in history. Unlike many other well as national and supra-national succumbs to other risks (such as global challenges pandemics have institutions dedicated to combating climate change or further pandemics). happened recently, as we can see diseases. Private insurers are also where reasonably good data exist. interested in modelling pandemic Plotting historic epidemic fatalities risks.273 Set against this is the fact 3.1.4.2 Probability on a log scale reveals that these that modern transport and dense disaggregation tend to follow a power law with a human population allow infections small exponent: many plagues have to spread much more rapidly274, Five important factors in been found to follow a power law and there is the potential for urban estimating the probabilities and with exponent 0.26.261 slums to serve as breeding grounds impacts of the challenge: for disease.275 These kinds of power laws are 1. What the true probability heavy-tailed262 to a significant Unlike events such as nuclear wars, distribution for pandemics is, degree.263 In consequence most of pandemics would not damage the especially at the tail. the fatalities are accounted for by the world’s infrastructure, and initial 2. The capacity of modern top few events.264 If this law holds for survivors would likely be resistant international health systems to future pandemics as well,265 then the to the infection. And there would deal with an extreme pandemic. majority of people who will die from probably be survivors, if only in 3. How fast medical research can epidemics will likely die from the isolated locations. Hence the risk proceed in an emergency. single largest pandemic. of a civilisation collapse would 4. How mobility of goods and come from the ripple effect of the people, as well as population Most epidemic fatalities follow a fatalities and the policy responses. density, will affect pandemic power law, with some extreme These would include political and transmission. events – such as the Black Death agricultural disruption as well as 5. Whether humans can develop and Spanish Flu – being even economic dislocation and damage novel and effective anti-pandemic more deadly.267 to the world’s trade network solutions. (including the food trade). There are other grounds for suspecting that such a high- impact epidemic will have a greater probability than usually assumed. All the features of an extremely devastating disease already exist in nature: essentially incurable (Ebola268), nearly always fatal (rabies269), extremely infectious (common cold270), and long incubation periods (HIV271). If a pathogen were to emerge that somehow combined these features (and influenza has demonstrated antigenic shift, the ability to combine features from different viruses272), its death toll would be extreme.

Influenza subtypes266

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 85 GLOBAL PANDEMIC 3.1GOVERNANCE Current risks DISASTERS

Contact with Small pandemic reservoir species Global poverty scares

Impact ofDeliberate GlobalDensity of GlobalTechnological attempts to Global povetyMedical research GlobalBio-terrorism instability increased coordinationpopulation coordinationinnovations movementconstruct of world goods anddictatorship people

Meta-uncertainty on tradeoffs between e.g. New system Smart sensors poverty, survival, of governance Pandemic freedom Impact of Healthcare in Accidental release Antibiotics combining from lab sanitation or resistance individual lack thereof countries different deadly features

Impact of FailingPandemic to solve Improvements to Making globalDeadly governance things worse Post-pandemic monoculture importantleaping problems the pandemic Smart sensors politics food supply species barrier

Impact on meat Disruption to Effectiveness of Pandemic production and world politics Direct casualties transmission countermeasures Not achievingfood supply and economy Global Lack of human Enduring poverty important Climate change pollution flourishing ethical goals

Undesirable Long-term fate world system of pandemic virus/ (e.g. global bacteria/parasite dictatorship)

Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy Pre-pandemic medical Total short-term contingency plans casualties

Civilisation Meta-uncertainty of how the changed world Extinction has affected pandemic probabilities collapse General mitigation Civilisation Total short-term Extinction casualties effort collapse

Meta-uncertainty of what probability distributions pandemics follow

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

86 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

1. Extensive medical research 9. Mass casualties and finger-pointing 20. The improvements to surveillance will be key to preventing and could destabilise the world political and sensing technologies (including combatting large scale pandemics. and economic systems. indirect detection via web queries or The drawbacks are the possibility social media) open the possibility of of accidental release of dangerous 10. If the pathogen is transmissible smarter interventions (such as micro- pathogens from laboratories and to farm animals, this could affect the quarantines) and faster understanding of bioterrorism. world food supply. of the pathogen’s transmissibility.

2. As so much is known about 11. It is unlikely the pathogen would 21. Post-pandemic politics will be pandemic risks compared with be a recurrent, long-term risk, but important for preventing a civilisation other risks, there are more variants of it could continue to affect collapse or enabling reconstruction. possibilities for specific pre- people and animals for many years, pandemic contingency plans. dependent on its transmissibility and 22. Many pathogens incubate in life cycle. species close to humans, before 3. The effectiveness of healthcare leaping the species barrier. systems will be important, especially 12. Small pandemic scares could in less developed nations where improve global coordination on 23. Monoculture food systems make the pandemic may overwhelm the the issue. it easier to transmit any pathogen system, and then transmit from there infecting human food animals. to other nations. 13. Increased population density causes increased transmissibility 24. The mode of transmission of the 4. Global coordination in detection, of the pathogen, especially in pathogen will be critical to its ultimate analysis and treatment are vital urban slums. reach and impact. for stopping a pandemic in its early stages, and for implementing 14. Some pathogens, such as bird flu, 25. Various countermeasures are measures such as quarantines and depend on regular contact between available in terms of detection, more advanced countermeasures. humans and “reservoir species” virus analysis, treatment, and in order to evolve into periodically quarantining. Future research, 5. Poverty will affect the quality dangerous strains. technological and political of national healthcare systems, developments may open population density and sanitation 15. If antibiotic resistance develops, up new methods of fighting quality, the movement of local goods humanity could see the resurgence of the pathogen. and people, and the effectiveness of bacteria-based pandemics. the political response. 26. Many of the current factors 16. The increased movement of determining pathogen transmission 6. Bioterrorists may unleash a people and products increases are unprecedented, such as pathogen held in storage, such the speed and spread of movements of goods and people, the as smallpox. pandemic transmission. quality of healthcare systems, and the existence of a centralised political 7. Laboratory security at the top labs 17. Sanitation or its lack will strongly response. This means that data from is insufficient for the danger at hand, affect the spread of certain pathogens past pandemics will not be as reliable and accidental release is a non- in key areas. for computing probability distributions. negligible possibility. 18. The efficiency of global reaction 27. The pandemic risk lies in the 8. Pandemics are one of the risks to a new pandemic will be strongly “tails” – the extreme events – and where there is a possibility for a determined by the speed of research these tails must be estimated from very large number of direct on the pathogen during the pandemic. few data points, making them tricky casualties, depending on the and uncertain. severity of the pathogen. 19. A great risk will arise if a pathogen combines the different dangerous features of current viruses or bacteria.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 87 3.1 Current risks

09-Aug-13: Epihack: Digital disease 3.1.4.3 Main events The guidance paper indicates simultaneously the weaknesses surveillance hack-a-thon 292 during 2013 of pandemic preparations, – Initiative the improvements in these 10-Jun-13: Pandemic Influenza preparations, and the continued Beyond the formal, top-down Risk Management: WHO Interim role of the WHO as global directing initiatives to deal with pandemics, Guidance 278 and coordinating authority. there are openings for bottom-up, – Policy innovative ideas. Epihack attempted to generate just such ideas, through This is an updated document 24-Jul-13: Bacteria become three days of designing and hacking that replaces the 2009 Pandemic resistant to some of the last in Cambodia. Descriptions of the Influenza Preparedness and remaining antibiotics 285 winning projects were given: Response: a WHO guidance – Event document.279 It updates its – CoPanFlu: This project included recommendations based on lessons Bacterial infections, such as the Black home visits to collect blood from the influenza A(H1N1) 2009 Death, 286 syphilis, 287 and tuberculosis, samples from 807 homes and pandemic (swine flu),280 the adoption 288 have been responsible for millions weekly follow-up phone calls by the Sixty-fourth World Health of deaths, over the thousands of years to document the occurrence of Assembly of the Pandemic Influenza they have co-existed with humanity. infectious respiratory symptoms. Preparedness Framework281 (for Though these diseases have not These visits and phone calls the sharing of influenza viruses been eradicated – overall, a third of caused disturbance to the and access to vaccines and other the world is currently infected with the participants. The new system uses benefits), and the States Parties’ tuberculosis bacillus289 – they have SMS for users to report symptoms. obligations on capacity strengthening been controlled since the introduction Chart and map visualisation of contained in the International Health of antibiotics, and prognostics have the data (with full case details) Regulations of 2005.282 improved tremendously. But recently and a fieldwork tracking tool were a rising number of bacteria have developed to help the research Of significance was the Report developed antibiotic resistance, due team analyse and monitor data. of the Review Committee on the mainly to antibiotic over-prescription290 Functioning of the International and use in livestock feed.291 This – DoctorMe: In addition to all of the Health Regulations (2005) on the Nature report highlights the worrying popular features of DoctorMe (free A(H1N1) 2009 pandemic,283 which way in which Enterobacteriaceae health information for the general concluded: “We were lucky this time, (bacteria with a 50% mortality public), the tool now features a but as the report concludes, the world rate) have become resistant to weekly survey for users. The survey is ill-prepared to respond to a severe carbapenems, one of the last will ask participants to select influenza pandemic or to any similarly remaining antibiotics that had been whether they are experiencing any global, sustained and threatening effective against them. symptoms from a list. public-health emergency.” This is reinforced by the fact that the 2009 pandemic is alleged to have infected 24% of the population.284

The main lesson the WHO drew from that epidemic was that member states generally had communication issues (between ministries of health and decision,makers, and with the public), and were prepared for a pandemic of high severity and appeared unable to adapt their national and subnational responses adequately to a more moderate event.

88 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

– ILI Surveillance, Bureau of 22-Sep-13: Research hints at This report uses an agent-based model Epidemiology Thailand: The old possibility for universal flu vaccine 293 to analyse whether the accidental system was web-based and had – Research laboratory release of pandemic flu no visual element. The new mobile viruses could be contained, and application and website provides a The Spanish flu outbreak was the concludes that controllability of escape map visualisation for the reported deadliest short pandemic in history, events is not guaranteed. cases of influenza-like illness (ILI) in infecting about a third of the world Thailand. The map shows hospital population (≈ 500 million people) ILI cases with colour-coded pins to and killing 50-100 million people.294 3-Dec-13: Global pandemic tops poll indicate the level of ILI and allows There have been numerous flu of insurance industry risks 306 for simple analysis of the situation. pandemics in the last few centuries, – Initiative with three others having around a – Mae Tao Clinic: The electronic million casualties (the 1889-1890 Academics and governmental307/ records for this healthcare clinic were Russian Flu,295 the 1957-1958 supra-governmental308 organisations very basic. During EpiHack, the data Asian Flu, and the 1968-1969 have long worried about the risks of was moved to the cloud and is now Hong Kong Flu296 outbreaks). The pandemics. But such organisations open-source. A data visualisation most recent pandemic was that in attract certain types of people with dashboard was created to allow for 2009, which killed 150,000-500,000 specific outlooks, who can be subject map visualisation of diagnoses. The people.297 Thus any move towards to further biases because of their staff at Mae Tao Clinic can now easily a universal flu vaccine would be profession and the social milieu view and analyse the data to spot of great importance to combating surrounding it.309 Insurers come from trends and send alerts. They plan to such recurring pandemics. This a different background, focusing on pilot this programme at their clinic paper, analysing the role of T cells in practical profitability in the business and, if successful, to replicate it with combating influenza, suggests a way world. It is therefore instructive that other clinics. that such a vaccine could be feasible. they too see pandemics as among the major threats in the world today. – Verboice: The technology platform of This also implies that combating Verboice is so user-friendly 28-Nov-13: Difficulties in containing pandemics is of use not only from it doesn’t require technical the accidental laboratory escape of a humanitarian but also from an developers to develop the systems. potential pandemic influenza viruses 298 economic standpoint. At EpiHack, project managers were – Research able to design and create systems to address needs in their work Biosafety laboratories experiment completely on their own. In just with some of the deadliest of the eight hours, four project managers world’s pathogens, and occasionally each completed their own voice- create new ones.299 Their number based participatory surveillance is increasing globally, and their systems to monitor One Health safety record is far from perfect, in Kenya and Tanzania; early with several pathogen leaks warning generation in South Sudan; reported300 and others suspected301 animal health in Laos; unexploded (the last smallpox fatality was due ordnance in Laos; child trafficking to a virus that escaped a lab302, in Cambodia. The project owners after eradication of the virus in the of these new systems will now wild). The rate of pathogen escape take them back to their countries has been estimated at 0.3% per and develop implementation and laboratory, per year303 – a very high sustainability plans. probability, given the 44 BSL-4304 labs and several thousands of BSL-3 labs. There have already been three known escapes from BSL-4 labs since 1990.305

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 89 3.1 Current risks

3.1 Current risks 3.1.5 Global System Collapse

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Global system collapse is defined here as either an economic or societal collapse on the global scale. There is no precise definition of a system collapse. Ecological Nanotechnology Nuclear War Super-volcano TheSynthetic term has been used to describeUnknown a broad range Catastrophe Biology Consequences of bad economic conditions, ranging from a severe, prolonged depression with high bankruptcy rates and high unemployment, to a breakdown in normal commerce caused by hyperinflation, or even an economically-caused sharp increase in the death rate and perhaps even a decline in population. 310

90 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

Often economic collapse is This trend is likely to be intensified reasons.327 And institutional collapses accompanied by social chaos, civil by continuing globalisation,320 while can create knock-on effects, such as unrest and sometimes a breakdown of global governance and regulatory the descent of formerly prosperous law and order. Societal collapse usually mechanisms seem inadequate to states to much more impoverished refers to the fall or disintegration of address the issue.321 This is possibly and destabilising entities.328 Such human societies, often along with their because the tension between processes could trigger damage on life support systems. It broadly includes resilience and efficiency322 can even a large scale if they weaken global both quite abrupt societal failures typified exacerbate the problem.323 political and economic systems to such by collapses, and more extended an extent that secondary effects (such gradual declines of superpowers. Here Many triggers could start such a failure as conflict or starvation) could cause only the former is included. cascade, such as the infrastructure great death and suffering. damage wrought by a coronal mass 3.1.5.1 Expected impact ejection,324 an ongoing cyber conflict, 3.1.5.2 Probability or a milder form of some of the risks presented in the rest of the paper. disaggregation The world economic and political Indeed the main risk factor with global system is made up of many actors systems collapse is as something Five important factors in estimating with many objectives and many which may exacerbate some of the probabilities of various impacts: links between them. Such intricate, the other risks in this paper, or as a interconnected systems are subject trigger. But a simple global systems 1. Whether global system collapse to unexpected system-wide failures collapse still poses risks on its own. will trigger subsequent collapses due to the structure of the network311 The productivity of modern societies or fragility in other areas. – even if each component of the is largely dependent on the careful 2. What the true trade-off is network is reliable. This gives rise to matching of different types of capital325 between efficiency and resilience. systemic risk: systemic risk occurs (social, technological, natural...) 3. Whether effective regulation and when parts that individually may with each other. If this matching is resilience can be developed. function well become vulnerable disrupted, this could trigger a “social 4. Whether an external disruption when connected as a system to a collapse” far out of proportion to will trigger a collapse. self-reinforcing joint risk that can the initial disruption.326 States and 5. Whether an internal event will spread from part to part (contagion), institutions have collapsed in the past trigger a collapse. potentially affecting the entire system for seemingly minor systemic and possibly spilling over to related outside systems.312 Such effects have been observed in such diverse areas as ecology,313 finance314 and critical infrastructure315 (such as power grids). They are characterised by the possibility that a small internal or external disruption could cause a highly non-linear effect,316 including a cascading failure that infects the whole system,317 as in the 2008-2009 financial crisis.

The possibility of collapse becomes more acute when several independent networks depend on each other, as is increasingly the case (water supply, transport, fuel and power stations are strongly coupled, for instance).318 This dependence links social and technological systems as well.319 Example of an interconnected network: the Internet. Each line is drawn between two nodes, representing two IP addresses. This is a small look at the backbone of the Internet. Source: http://en.wikipedia.org/wiki/File:Internet_map_1024.jpg

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 91 3.1GOVERNANCE Current risks DISASTERS GOVERNANCE DISASTERS

Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship Meta-uncertainty on tradeoffs between e.g. New system Smart sensors poverty, survival, of governance Meta-uncertaintyfreedom on tradeoffs between e.g. New system Smart sensors poverty, survival, of governance freedom

Failing to solve Improvements to Making important problems global governance things worse

Not achieving Global Lack of human Enduring poverty important Climate change pollution flourishing ethical goals

Undesirable world system (e.g. global dictatorship) Undesirable world system (e.g. global dictatorship) Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy

General mitigation Civilisation Total short-term Extinction casualties effort collapse

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

92 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

1. Increased global coordination 8. The system collapse may lead to 18. The world’s political and legal and cooperation may allow effective increased fragility in areas that it does systems are becoming more closely regulatory responses, but it also not directly damage, making them integrated as well. Any risk has not causes the integration of many vulnerable to subsequent shocks. been extensively researched yet, and different aspects of today’s world, there remain strong obstacles (mainly likely increasing systemic risk. 9. A collapse that spread to at the nation state level) slowing down government institutions would this form of integration. 2. Systemic risk is only gradually undermine the possibilities of becoming understood, and further combating the collapse. 19. The politics of the post-system research is needed, especially collapse world will be important in when it comes to actually reducing 10. A natural ecosystem collapse formulating an effective response systemic risk. could be a cause or consequence of a instead of an indifferent or counter- collapse in humanity’s institutions. productive one. 3. Since systemic risk is risk in the entire system, rather than in any 11. Economic collapse is an obvious 20. System collapses can be triggered individual component of it, only and visible way in which system internally by very small events, without institutions with overall views and collapse could cause a lot of damage. an apparent cause. effects can tackle it. But regulating systemic risk is a new and 12. In order to cause mass casualties, 21. External disruptions can trigger the uncertain task. a system collapse would need to collapse of an already fragile system. cause major disruptions to the world’s 4. Building resilience – the ability of political and economic system. 22. The trade-off between efficiency system components to survive shocks and resilience is a key source of – should reduce systemic risk. 13. If the current world system fragility in a world economy built collapses, there is a risk of casualties around maximising efficiency. 5. Fragile systems are often built through loss of trade, poverty, wars because they are more efficient and increased fragility. 23. Climate change, mass than robust systems, and hence movements of animals and more profitable. 14. It is not obvious that the world’s agricultural mono-cultures are institutions and systems can be put interlinking ecosystems with each 6. General mitigation efforts should together again after a collapse; they may other and with human institutions. involve features that are disconnected be stuck in a suboptimal equilibrium. from the standard system, and thus 24. There is a lot of uncertainty should remain able to continue being 15. Power grids are often analysed about systemic risk, especially in of use if the main system collapses as possible candidates for system the interactions between different collapse, and they are becoming fragilities that would not be sufficient 7. A system collapse could spread more integrated. to cause a collapse on their own. to other areas, infecting previously untouched systems (as the sub- 16. The world’s financial systems prime mortgage crisis affected the have already caused a system world financial system, economy, and collapse, and they are still growing ultimately its political system). more integrated.

17. The world’s economies are also getting integrated, spreading recessions across national boundaries.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 93 3.1 Current risks

14-Mar-13: Systemic sovereign credit 3.1.5.3 Main events It will bring together experts from finance, economics, computer risk has “deep roots in the flows and during 2013 science, political science, law and the liquidity of financial markets.”330 natural and mathematical sciences. – Research 16-Jan-13: Systemic Risk Centre This will allow researchers affiliated to founded at the LSE 329 the Centre to investigate how risk is It is important to estimate the source – Event created through feedback loops within of systemic risk. Different mitigation and between the financial, economic, policies should be implemented Effective interventions into systemic legal and political systems. Political if sovereign systemic risks spring risks depend on high quality research, decisions, for example, can directly from financial markets rather than which may be why the London affect people’s behaviour in the macroeconomic fundamentals. This School of Economics (LSE) founded financial markets, which in turn affects paper argues that systemic sovereign a £5 million research centre to study political decision-making and so on – risks spring from financial markets systemic financial risk. A press with the outcomes being unexpected (through capital flows, funding release said: and complex.” availability, risk premiums, and liquidity shocks331) rather than from “The Centre will undertake an Besides the research results produced fundamentals.332 It further estimates economic analysis of the fundamental by the centre, its very existence shows that systemic risks are three times risks to the financial system, based on that systemic risk is being taken larger in eurozone countries than in an interdisciplinary approach. seriously in academic quarters. US states.

Figure 19: Network Diagram of connections between, banks, brokers/dealers, insurers and hedge funds. Jan 1994-Dec 1996 Source: https://app.box.com/shared/oesro8zzco0mtvuymh3f

94 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.1 Current risks

17-Jul-13 IMF launches “Systemic – Fundamentals-based models 335 rely The paper concludes, however, that Risk Monitoring (“SysMo”) Toolkit—A on macroeconomic or balance sheet the systemic risk monitoring toolkit User Guide” 333 data to help assess macro-financial is incomplete and that “tools exist – Policy linkages (e.g., macro stress testing to assess most sectors and levels of or network models). By providing aggregation, but they provide only In order to mitigate or prevent vulnerability measures based on partial coverage of potential risks and systemic risk, it needs to be actual interconnectedness and only tentative signals on the likelihood monitored. In this paper, the exposures, these models may help and impact of systemic risk events. As authors set out to clarify the nature build a realistic “story”. However, such, they may not provide sufficient and use of the systemic risk they often require long-term data comfort to policymakers.” monitoring tools that are currently series, assume that parameters available, providing guidance on and relationships are stable under how to select the best set of tools stressed conditions, and produce 23-Dec-13: Citigroup analysis reports depending on the circumstances. only low-frequency risk estimates. reduced systemic political and The paper breaks down the tools financial risks in 2013 and 2014 338 into four categories, each with their – Market-based models. 336 These – Initiative strengths and weaknesses: models uncover information about risks from high-frequency Tracking the ebb and flow of – Single risk/soundness indicators. 334 market data and are thus suitable the likelihood of various risks is Indicators based on balance sheet for tracking rapidly-changing important for estimating where best data, such as financial soundness conditions of a firm or sector. to direct energy and resources. Even indicators (FSIs), are widely These approaches are more approximate, order of magnitude available and cover many risk dynamic, but their capacity to estimates are sufficient if they dimensions. However, they tend to reliably predict financial stress has establish that some risks are much be backward-looking and do not yet to be firmly established. more dangerous than others (order account for probabilities of default of magnitude estimates correspond or correlation structures. Moreover, – Hybrid, structural models. 337 These to the “Class 5 cost estimate”, 339 only some of these indicators models estimate the impact undertaken at the very beginning of can be used as early-warning of shocks on key financial and the project, between 0% and 2% of tools (e.g., indicators of funding real variables (e.g., default its completion). In 2013, Citigroup structures). Market data can be probabilities, or credit growth) by analysts predicted that (with caveats) used to construct complementary integrating balance sheet data and systemic risks would recede in indicators for higher-frequency market prices. Examples include Europe during the year, a prediction risk monitoring. the CCA and distance-to-default which seems to have been vindicated measures, which compare the by events. As for the future, Tina market value of an entity‘s assets Fordham, chief global political analyst to its debt obligations. at Citigroup Global Markets, predicted that “systemic political risks will decline in 2014, but country-level and geopolitical risks remain significant.” It seems positive both that market analysts are tracking systemic risks and that they see them as decreasing (though their focus is mainly on political and financial systemic risks).

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 95 3.2 Exogenic risks

3.2 Exogenic risks 3.2.1 Major Asteroid Impact

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

When large objects strike terrestrial planets like the Earth, there can be significant physical and biospheric consequences, though Ecological Nanotechnology Nuclear War Super-volcano Synthetic atmospheresUnknown mitigate many surface impacts Catastrophe Biology Consequences by slowing an object’s entry.

Impact structures are dominant landforms on many of the solar system’s solid objects and present the strongest empirical evidence for their frequency and scale. 340

96 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.2 Exogenic risks

3.2.1.1 Expected impact Should an impact occur, though, 3.2.1.2 Probability asteroid impact risks are similar to those of super-volcanoes, in that the disaggregation Asteroids have caused significant main destruction will not be wrought extinction events throughout the by the initial impact, but by the clouds Five important factors in Earth’s history. The most famous is the of dust projected into the upper estimating the probabilities and Chicxulub impactor, which probably atmosphere. The damage from such impacts of the challenge: helped cause the extinction of the an “impact winter” could affect the non-avian dinosaurs and more than climate, damage the biosphere, affect 1. Whether detection and tracking of 75% of all species.341 Large asteroid food supplies, and create political asteroids and other dangerous collisions – objects 5 km or more in instability. Though humanity currently space objects is sufficiently size – happen approximately once produces enough food to feed all exhaustive. every twenty million years and would humans,352 this supply is distributed 2. How feasible it is to deflect have an energy a hundred thousand extremely unevenly, and starvation still an asteroid. times greater342 than the largest bomb exists. Therefore a disruption that is 3. Whether measures such as ever detonated.343 A land impact small in an absolute sense could still evacuation could reduce the would destroy an area the size of a cause mass starvation in the future. damage of an impact. nation like Holland.344 Larger asteroids Mass starvation, mass migration, 4. The short- and long-term climate could be extinction level events. political instability and wars could consequences of a collision. be triggered, possibly leading to a 5. Whether our current civilisation Asteroid impacts are probably one civilisation collapse. Unless the impact could adapt to a post-impact world. of the best understood of all risks is at the extreme end of the damage in this report. Their mechanisms scale and makes the planet unviable, and frequencies are reasonably well human extinction is possible only as estimated.345 Recent ground- and a consequence of civilisation collapse space-based346 tracking projects and subsequent shocks.353 have been cataloguing and tracking the largest asteroids,347 and have discovered that the risks were lower than was previously feared.348 The projects are now cataloguing asteroids of smaller size and damage potential.

There has been some speculation about possible methods for deflecting asteroids350, should they be found on a collision course with the planet. Such means remain speculative, currently, but may become more feasible given technological progress and potentially more affordable access to space.351

Figure 20: How the Spaceguard Survey has reduced the short-term risk of impacts from near-Earth objects349

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 97 MAJOR3.2GOVERNANCE Exogenic risks ASTEROID DISASTERS IMPACT

Meta-uncertainty of Private space Easier access anthropic effects programs to space upon the true risk probability

National space programs Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship

Meta-uncertainty Making the risk Asteroid deflection on tradeoffs a priority technologies between e.g. New system Smart sensors poverty, survival, of governance freedom

Small asteroid Asteroid dection Asteroid impact impacts and tracking

Failing to solve Improvements to Making important problems globalDisruption governance to things worse Global Preemptive Long-term coordination evacuation world politics Climate impacts and economy climate impact

Post-impact politics

Not achieving Global Lack of human Enduring poverty important Climate change pollution flourishing ethical goals

Direct destruction Civilisation Extinction Total short-term collapse casuaities Undesirable world system (e.g. global dictatorship)

Disruption to Collapse of Post-disaster Long-term world politics General mitigation world system politics negative effects and economy efforts

General mitigation Civilisation Total short-term Extinction casualties effort collapse

Key

98 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.2 Exogenic risks

1. Competition between private 6. General mitigation efforts may 12. There are currently no asteroid space programmes could be a help reduce the direct and indirect deflection abilities, but there are determining factor in reducing the negative outcomes of an impact by, many plans that could conceivably be cost of space flight. for instance, equipping people to deal implemented in due course. with the changed climate. 2. National space programmes have 13. Small asteroid impacts could motivate always provided the impetus for space 7. Unlike many risks, there is no upper increased anti-asteroid precautions. flight projects, especially the more bound on how destructive an asteroid speculative and cutting-edge ones. impact could be, though the largest 14. With enough warning, it could be impacts are the rarest. possible to preemptively evacuate the 3. Protecting against asteroid impacts impact area. is already accepted as a project worth 8. The aftermath of an impact could funding, but increased focus on the greatly disrupt the world economic 15. Post-impact politics will be problem could increase the ability to and political system. important for reconstruction, predict and prevent such impacts. adapting to the changed climate, and 9. Climate changes would be the most prevention of further harm. 4. Asteroid detection and tracking destructive consequences of medium- continues to progress well currently, scale meteor impacts, with the world 16. Estimating the likelihood of and is key to preventing such plunged into an “impact winter”. asteroid impacts suffers from collisions in future. “anthropic shadow” effects:355 we 10. The effects of an impact winter may be underestimating the danger 5. Better global coordination is not could last for a long time. because if there had been many more strongly needed to track or deflect impacts in recent times, humans asteroids, but would be important if a 11. Easier access to space would be would not currently be around to large-scale evacuation was needed. important for any plans to actually observe their effects and take them deflect an asteroid. into account.

Source: http://en.wikipedia.org/wiki/File:Chelyabinsk_meteor_trace_15-02-2013.jpg

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 99 3.2 Exogenic risks

14-Nov-13: Risk of medium asteroid 3.2.1.3 Main events 17-Oct-13: The probability for “Asteroid 2013 TV135” to impact strike may be ten times larger than during 2013 Earth in 2032 is one in 63,000 363 previously thought 367 – Event – Research 15-Feb-13: Chelyabinsk meteor causes large fireball 356 NASA reports that a 400-metre This paper analyses in detail the – Event asteroid has one chance in 63,000 Chelyabinsk impact, estimated to have of impacting the Earth. An asteroid had an energy of 500 kilotonnes of The Chelyabinsk meteor was a this size would produce ocean- TNT. It demonstrates problems with near-Earth asteroid that entered wide tsunamis or destroy land areas the standard methods for estimating Earth’s atmosphere over Russia, with the size of a small state (Delaware, the energy of collisions – derived from an estimated speed of 18.6 km/s, Estonia).364 For comparison, the odds nuclear weapons results 368 – and almost 60 times the speed of sound. of dying from lightning strike are 1 from that deduces that the number of It exploded into a very visible air burst in 83,930, of a snake, bee or other impactors with diameters of tens of over Chelyabinsk Oblast, which was venomous bite or sting is 1 in 100,000, metres may be an order of magnitude recorded by numerous video sources. of an earthquake 1 in 131,890, and of higher than estimated. It argues that this The meteor was undetected before it a dog attack 1 in 147,717.365 So the demonstrates a deviation from a simple entered the Earth’s atmosphere, and risk of asteroid death, though low, is power law, and thus that there is a non- caused numerous injuries, extensive comparable to more common risks. equilibrium in the near-Earth asteroid damage, but no deaths. It was the population for objects 10 to 50 metres largest to crash to Earth since 1908, 28-Oct-13: United Nations to Adopt in diameter. This shifts more of the 357 when an object hit Tunguska in Asteroid Defence Plan 366 impact risk to asteroids of these sizes. Siberia.358 The meteor seemed ideal – Policy from the risk reduction perspective: 3-Dec-13: SpaceX launches into a large, visible impact that attracted The UN plans to set up an geostationary orbit 369 great attention, and a renewed International Asteroid Warning – Initiative commitment to asteroid precautions, 359 Group for member nations to but no actual fatalities. share information about potentially Easy access to space is important for all hazardous space rocks. If asteroid deflection proposals.370 Since 19-Jun-13: Space Research astronomers detect an asteroid that America retired the Space Shuttle, 371 Institute of Russian Academy of poses a threat to Earth, the UN’s it has been putting its hope in private Science presents a strategy to Committee on the Peaceful Uses of space companies.372 The success use small asteroids to deflect Outer Space will help coordinate a of SpaceX opens the possibility of hazardous objects from the mission to launch a spacecraft to slam eventual cheaper access to space. trajectory of collision with Earth 360 into the object and deflect it from its – Research collision course.

Though the analysis and tracking of This marks the first time an asteroids has progressed rapidly, 361 international body has assigned methods for deflecting a dangerous responsibility for tracking and asteroid, should it be detected, remain intercepting dangerous asteroids. speculative.362 The Space Research Institute of the Russian Academy of Science introduces another approach: selecting small (10-15m) near-Earth asteroids and causing them to strike a larger dangerous one, altering its trajectory. The more suggestions and ideas there are for such deflections, the more likely it is that one of them will yield an implementable approach.

100 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.2 Exogenic risks

NEO Yield Crater Average Consequences diameter Megatonnes diameter interval b/w (MT*) (km) impact (years)

<10 Upper atmosphere detonation of “stones” (stony asteroids) and comets; only “irons” (iron asteroids) <3% penetrate to surface.

75m 10 to 100 1.5 1,000 Irons make craters (Barringer Crater); Stones produce air-bursts (Tunguska). Land impacts could destroy areas the size of a city (Washington, London, Moscow).

160m 100 to 1,000 3 4,000 Irons and stones produce ground-bursts; comets produce air-bursts. Ocean impacts produce significant tsunamis. Land impacts destroy areas the size of a large urban area (New York, Tokyo).

350m 1,000 6 16,000 Impacts on land produce craters; ocean-wide tsunamis to are produced by ocean impacts. Land impacts destroy 10,000 areas the size of a small state (Delaware, Estonia).

700m 10,000 12 63,000 Tsunamis reach hemispheric scales, exceed damage to from land impacts. Land impacts destroy areas the 100,000 size of a moderate state (Virginia, Taiwan).

1.7km 100,000 30 250,000 Both land and ocean impacts raise enough dust to to affect climate, freeze crops. Ocean impacts generate 1 million global scale tsunamis. Global destruction of ozone. Land impacts destroy areas the size of a large state (California, France, Japan). A 30 kilometre crater pen- etrates through all but the deepest ocean depths.

3km 1 million 60 1 million Both land and ocean impacts raise dust, change to climate. Impact ejecta are global, triggering wide- 10 million spread fires. Land impacts destroy areas the size of a large nation (Mexico, India).

7km 10 million 125 10 million Prolonged climate effects, global conflagration, to probable mass extinction. Direct destruction ap- 100 million proaches continental scale (Australia, Europe, Usa).

16km 100 million 250 100 million Large mass extinction (for example K/T or Creta- to ceous-Tertiary geological boundary). 1 billion

>1 billion Threatens survival of all advanced life forms.

Figure 15: Impact effects by size of Near Earth Object354

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 101 Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact 3.2 Exogenic risks

3.2 Exogenic risks 3.2.2 Super-volcano

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

A super-volcano is any volcano capable of producing an eruption with an ejecta volume greater than 1,000 km3. This is thousands of times larger than normal eruptions.

Super-volcanoes can occur when magma in the mantle rises into the crust from a hotspot but is unable to break through it, so that pressure builds in a large and growing magma pool until the crust is unable to contain the pressure. 373

102 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.2 Exogenic risks

3.2.2.1 Expected impact The extent of the impact would Unless the eruption is at the extreme end thus depend on the severity of the of the damage scale and makes the planet eruption - which might or might not be unviable, human extinction is possible only The eruption which formed the foreseen, depending on improvements as a consequence of civilisation collapse Siberian Traps was one of the largest in volcanic predictions384 - and the and subsequent shocks.387 in history. It was immediately followed subsequent policy response. Another by the most severe wave of extinction Siberian Trap-like eruption is extremely Prof. Michael Rampino, New York in the planet’s history, 374 the Permian– unlikely on human timescales, but the University, has estimated that a large Triassic extinction event, 375 where damage from even a smaller eruption (1,000 cubic kilometres of magma) 96% of all marine species and 70% of could affect the climate, damage the super-eruption would have global terrestrial vertebrate species died out. biosphere, affect food supplies and effects comparable to an object 1.5km Recent research has provided evidence create political instability. in diameter striking the Earth.388 of a causal link: that the eruption caused the mass extinction.376 There A report by a Geological Society 3.2.2.2 Probability have been many other super-volcanic of London working group notes: eruptions throughout history.377 The “Although at present there is no disaggregation return period for the largest super- technical fix for averting super- volcanoes (those with a Volcanic eruptions, improved monitoring, Five important factors in Explosivity Index378 of 8 or above) has awareness-raising and research-based estimating the probabilities and been estimated from 30,000 years379 planning would reduce the suffering of impacts of the challenge: at the low end, to 45,000 or even many millions of people.” 385 700,000 years380 at the high end. 1. Whether countries will coordinate Though humanity currently produces globally against super-volcano risk Many aspects of super-volcanic activity enough food to feed everyone,386 and damage. are not well understood as there have this supply is distributed extremely 2. The predictability of super- been no historical precedents, and unevenly, and starvation still exists. volcanic eruptions. such eruptions must be reconstructed Therefore a disruption that is small in 3. How directly destructive an from their deposits.381 an absolute sense could still cause eruption would be. mass starvation. Mass starvation, mass 4. The effectiveness of general The danger from super-volcanoes migration, political instability and wars mitigation efforts. is the amount of aerosols and dust could be triggered, possibly leading to 5. How severe the long-term projected into the upper atmosphere. a civilisation collapse. climate effects would be. This dust would absorb the Sun’s rays and cause a global volcanic winter. The Mt Pinatubo eruption of 1991 caused an average global cooling of surface temperatures by 0.5°C over three years, while the Toba eruption around 70,000 years ago is thought by some to have cooled global temperatures for over two centuries.382 The effect of these eruptions could be best compared with that of a nuclear war. The eruption would be more violent than the nuclear explosions,383 but would be less likely to ignite firestorms and other secondary effects. Unlike nuclear weapons, a super-volcano would not be targeted, leaving most of the world’s infrastructure intact.

Satellite image of Lake Toba Source: NASA, http://en.wikipedia.org/wiki/File:Toba_overview.jpg

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 103 SUPER VOLCANO 3.2GOVERNANCE Exogenic risks DISASTERS

Medium Making the risk volcanic eruptions a priority

Predictability Supervolcano of eruptions research Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship

Meta-uncertainty on tradeoffs between e.g. New system Smart sensors Better monitoring ofSupervolcano governance poverty, survival, and prediction eruption freedom

Preemptive Global Direct destruction Climate impacts Long-term evacuation coordination climate impact

Failing to solve Improvements to Making important problems global governance things worse

Disruption to world politics and economy

Not achieving Global Lack of human Meta-uncertainty Enduring poverty important Climate change pollution flourishing ethical goals Post-eruption of anthropic effects politics upon the true risk probability

Undesirable world system (e.g. global dictatorship)

DisruptionCivilisation to General Total short-term Collapse of Post-disaster Long-term Extinction world collapsepolitics mitigation efforts casualties world system politics negative effects and economy

General mitigation Civilisation Total short-term Extinction casualties effort collapse

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

104 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.2 Exogenic risks

1. Whether super-volcano risk is made 6. A super-volcano’s main 10. Better volcano monitoring a priority will have a large impact on destructive impact is through its and prediction (if possible) will research and volcano monitoring. effect on the climate, akin to a allow such interventions as pre- nuclear winter cooling effect. This emptive evacuations. 2. Further super-volcano research will will strongly affect all impact levels, be important in any mitigation and and the disruption to the world’s 11. Evacuations are likely to be the monitoring efforts. political and economic system. only effective response to an imminent eruption, as super-volcanoes are 3. Global coordination and 7. The level of this disruption will unlikely to be controllable or divertible. cooperation between nations will determine how well countries cope determine research levels, the with the aftermath of the eruption and 12. Post-eruption politics will be a chances of evacuations, and post- subsequent climate changes, and consequence of the number of short- eruption disruption to the world whether subsequent conflicts or trade term casualties, and the disruption to political and economic system. wars will occur, adding to the damage. the world system.

4. General mitigation efforts may help 8. The long-term climate impact will 13. Medium scale volcanic eruptions reduce the direct and indirect negative determine in what state the post- may persuade leaders to make the impact of an eruption, by, for instance, eruption world will find itself, relevant risk more of a priority. equipping people to deal with the both for reconstruction after a collapse changed climate. and for preventing such a collapse. 14. Estimating the likelihood of super-volcanic eruptions suffers from 5. The direct destructive effect 9. Whether eruptions are “anthropic shadow” effects:390 we of a super-volcano can be fundamentally predictable or may be underestimating the danger extensive, especially in the area not, and how far in advance, because if there had been many more around the eruption. will be very important for many eruptions in recent times, humans mitigation strategies. would not currently be around to observe their effects and take them into account.

VEI Ejecta volume Classification Description Plume Frequency

0 < 10,000 m3 Hawaiian Effusive < 100 m constant

1 > 10,000 m3 Hawaiian / Strombolian Gentle 100-1000 m daily

2 > 1,000,000 m3 Strombolian / Vulcanian Explosive 1-5 km weekly

3 > 10,000,000 m3 Vulcanian / Peléan Catastrophic 3-15 km few months

4 > 0.1 km3 Peléan / Plinian Cataclysmic 10-25 km ≥ 1 yr

5 > 1 km3 Plinian Paroxysmic 20-35 km ≥ 10 yrs

6 > 10 km3 Plinian / Ultra-Plinian Colossal > 30 km ≥ 100 yrs

7 > 100 km3 Ultra-Plinian Mega-colossal > 40 km ≥ 1,000 yrs

8 > 1,000 km3 Supervolcanic Apocalyptic > 50 km ≥ 10,000 yrs

Figure 21: Volcanic Explosivity Index 389

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 105 3.2 Exogenic risks

3.2.2.3 Main events They put the risk of super-volcanic This is due to the normalcy bias, eruptions in the context of standard “the tendency to minimise the during 2013 volcanic eruptions, just on a larger probability of potential threats or scale (noting that super-volcanic their dangerous implications,”. 396 15-Mar-13: Climate impact of eruptions have affected Japan in super-volcanoes may be less than the past). Japan has been a very previously thought 391 seismically active country for its entire 27-Oct-13: Yellowstone super- – Research history,394 so it might be hoped that volcano larger than previously adequate volcanic mitigation measures thought 397 The Toba eruption around 70,000 would have been implemented. – Research years ago was one of the world’s largest super-volcanic eruptions. In But the report notes that “remarkably Another continuing development in contrast with some theories that claim few [of Japan’s local governments] the science of super-volcanoes, this it caused a volcanic winter that may have drafted volcanic disaster paper demonstrates that the crustal have lasted over two centuries, 392 countermeasure[s]”, 395 adding that magma reservoir under Yellowstone this paper claims that analysis of ash “Local governments that have actually was 50% larger than was previously from the Toba super-eruption in Lake experienced a volcanic disaster focus thought. However, despite this Malawi shows no evidence of volcanic attention on volcanic disaster-related increase, integrated probabilistic winter in East Africa. This further discussion, but most have not drafted hazard assessment shows that the illustrates the difficulty of establishing specific procedures for volcanic biggest Yellowstone Plateau threat the exact impact of large-scale disasters and seem to think that the is from large M7+ earthquakes - disasters when the evidence record is general disaster countermeasure significantly damaging398, but very poor. volume is adequate.” This provokes unlikely to threaten billions - not from some pessimism about the likelihood volcanic or super-volcanic eruptions. of effective planetary super-volcano 17-Jul-13: The Volcanological mitigation measures being implemented, Society of Japan looks at volcano especially in those areas with no and super-volcano mitigation 393 direct experience of volcanic risk. – Policy

Prevention of super-volcano eruptions is impossible with current technology, Figure 22: Location of Yellowstone hotspot over time (numbers indicate millions of years before the present). Source: http://en.wikipedia.org/wiki/File:HotspotsSRP.jpg but there may be some possibility of mitigating their effects. The Volcanological Society of Japan is one of the few organisations that have looked at such potential mitigation.

106 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.2 Exogenic risks

15-Nov-13: Insurance executives 20-Dec-13: Super-volcano confirmed rank super-volcanoes low on the list as responsible for one of the largest of extreme risks 399 extinctions in history 401 – Initiative – Research

Academics have long worried about The maximal destructive potential the probability of super-volcanic of super-volcanoes is uncertain. eruptions. But academia attracts There have been large super- certain types of people with specific volcanic eruptions throughout outlooks, who can be subject to history, 402 and many extinction further biases because of their events, but uncertainties in the profession and the social milieu geological record mean that it was surrounding it.400 Insurers come from hard to establish whether they a different background, focusing on were causally linked. One eloquent practical profitability in the business example was the eruption which world and using a relatively short formed the Siberian Traps403 (one time horizon. So it is instructive that of the largest in history), and the they do not see super-volcanoes as Permian–Triassic extinction, 404 a major threat in the world today: “Of where 96% of all marine species interest to us is the very low ranking and 70% of terrestrial vertebrates of the user-submitted idea of super- died out. The two events were volcanoes in the US”. close on the geological timeline, and this paper, using recent dating techniques, confirmed that the super-volcano erupted shortly before the extinction, making it the likely culprit. The risk of large impacts from super-volcanoes has thus gained in plausibility.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 107 Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact 3.3 Emerging risks

3.3 Emerging risks 3.3.1 Synthetic Biology

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Synthetic biology is the design and construction of biological devices and systems for useful purposes. It is an area of biological research and technology that combines biology and engineering, and so often overlaps with bioengineering and biomedical engineering.

It encompasses a variety of different approaches, methodologies, and disciplines with a focus on engineering biology and biotechnology. 405

108 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

3.3.1.1 Expected impact This could emerge through military Extinction risk is unlikely, 423 but bio-warfare, 415 commercial bio-warfare, possible if the aftermath of the 416 bio-terrorism417 (possibly using epidemic fragments and diminishes Pandemics are one of the dual-use products418 developed by human society to the extent that worst killers in human history. legitimate researchers, and currently recovery becomes impossible424 before Synthetic biology is the design unprotected by international legal humanity succumbs to other risks.425 and construction of biological regimes419), or dangerous pathogens devices and systems to leaked from a lab420. Of relevance is accomplish the specific goal of whether synthetic biology products 3.3.1.2 Probability 406 the synthetic biologist, adding become integrated into the global disaggregation human intentionality to traditional economy or biosphere. This could pandemic risks. lead to additional vulnerabilities (a benign but widespread synthetic Five important factors in The positive and negative potentials biology product could be specifically estimating the probabilities and of synthetic biology are unclear407 targeted as an entry point through impacts of the challenge: – much of the information currently which to cause damage). But such a comes from synthetic biologists, 408 development would lead to greater 1. The true destructive potential who may not be able to provide an industry and academic research, of synthetic biology, especially impartial overview (the problem is which could allow the creation of the tail risk. exacerbated by the decentralised reactive or pre-emptive cures.421 2. Whether the field will be nature of the field409). Attempts at successfully regulated, or regulation410 or self-regulation411 are The impact is very similar to that of successfully manage to currently in their infancy, and may not pandemics: mass casualties and regulate itself. develop as fast as research does.412 subsequent economic and political 3. Whether the field will usher in a instabilities leading to possible new era of bio-warfare. One of the most damaging impacts civilisation collapse. A bio-war would 4. Whether the tools of synthetic from synthetic biology would come contribute greatly to the resulting biology can be used defensively to from an engineered pathogen,413 instability. Even for the most perfectly create effective counter measures. targeting humans or a crucial engineered pathogen, survivors are 5. The dangers of relying on component of the ecosystem (such likely, if only in isolated or mainly synthetic biologists to estimate the as rice, which accounts for 20% of all isolated locations.422 danger of synthetic biology. calories consumed by humans).414

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 109 SYNTHETIC BIOLOGY 3.3GOVERNANCE Emerging risks DISASTERS

Global Knowledge coordination Military research control Smart sensors

Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship Meta-uncertainty Effective regulatory Continuing Knowledge Dual use of reying on framework synthetic biology leak research synthetic biologists research for risk estimates Meta-uncertainty on tradeoffs between e.g. New system Smart sensors poverty, survival, of governance freedom

Commercially Meta-uncertainty Small Pathogen Bio-terrorism targeted damaging of true potential security scares research synthetic biology of synthetic biology

Failing to solve Improvements to Making important problems global governance things worse

Engineered Long-term Bio-warfare ecosystem- ecosystem impact affecting pathogen

Not achieving Global Lack of human Enduring poverty Prevalenceimportant of Climate change pollutionEngineered flourishing syntheticethical biology goals New human-affecting Accidental products vulnerabilities pathogen release

Undesirable world system (e.g. global dictatorship) Pathogen Disruption to Agriculture Post-pathogen Direct world politics Extinction impact politics transmission casualties chains and economy Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy

Pre-release Civilsation medical Total short-term collapse contingency casualties plans

General mitigation Civilisation Total short-term Extinction casualties effort collapse

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

110 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

1. Global coordination and 10. Bio-terrorism has the potential 20. It may be possible to control direct cooperation will be important to to be the most destructive form of pathogen research through regulations preventing biowarfare and creating an terrorism of all, with a small group – certainly more so than dual use effective regulatory framework. causing billions of casualties. products. This kind of research is the most likely to lead to bio-weapons, or 2. Military research in synthetic biology 11. The various products produced by to accidental release. would be a direct risk for creating synthetic biology research could be dangerous bio-weapons. deadly if accidentally released. 21. If synthetic biology products are prevalent, this may introduce 3. Effective and intelligent regulatory 12. It is hard to estimate ahead of new vulnerabilities. frameworks are the great challenge for time, but the direct casualties of an controlling synthetic biology risks. The engineered pathogen could potentially 22. Post-pathogen politics will be field is currently self-regulated, and it include everyone infected, which important for preventing civilisation isn’t clear whether this is sufficient. could include almost everyone alive. collapses, and regulating further synthetic biology experiments 4. Synthetic biology is novel enough 13. The most devastating pathogen and developments. for some parts of the field potentially affecting the ecosystem would be one to be shut down if they are seen to be targeting food production in one form 23. The pathogen transmission too dangerous: continuing synthetic or another. chains are important in determining biology research is not a given. the transmissibility of the pathogens 14. The widespread use of synthetic in the human population, and 5. Of all technological fields, synthetic biology products could introduce new whether quarantine or similar biology could be one requiring vulnerabilities, if these products are methods will be effective. knowledge control: where dangerous specifically targeted. knowledge (such as how to synthesise 24. Synthetic biology research certain pathogens) is kept out of the 15. Human- or ecosystem-targeting may enable the construction of public domain. Other dangerous pathogens on a large scale could effective preventative measures or technologies (e.g. nuclear weapons) disrupt the world’s political and countermeasures to an outbreak require a large project or rare economic system, especially if one (both for a designed pathogen and a materials, and could be regulated at party is blamed for their release. natural one). that level instead. 16. Natural pathogens are unlikely to 25. The pathogen transmission chains 6. Mass surveillance and smart have a long-term devastating effect, are important in determining the sensors may be needed to ensure but human-designed ones could transmissibility of the pathogens in the dangerous synthetic biology projects – or they could be upgraded and ecosystem, and the effectiveness of are not carried out. changed regularly. various countermeasures.

7. Most of the pre-release mitigation 17. Small security scares could 26. Of all risks, those of synthetic efforts are similar to those for fighting provide impetus to the development biology are the most uncertain: they a conventional pandemic. of effective regulations. could turn out to be very high, or very low; it is currently not known. 8. Biowarfare is one major scenario 18. Knowledge leaks (such as in which synthesised biological genomes published online) could 27. Active synthetic biologists are agents are targeted at humans or at enable bioterrorism if the cost of the major source of information on the ecosystem. producing pathogens is low. synthetic biology risks, which calls the impartiality of their estimates 9. Commercial enterprises, especially 19. Much legitimate synthetic biology into question. those exploiting natural resources, research could have dual use for may be tempted to target their rival’s terrorists or as weapons. products with pathogens that may get out of control.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 111 3.3 Emerging risks

23-Jan-13: Work resumes on lethal 28-Feb-13: WHO report: Many 3.3.1.3 Main events flu strains 430 countries and institutions lack during 2013 – Event oversight of “dual use” biological research, and there is a lack of global 15-Jan-13: Improved bio-safety in In 2011, scientists working in frameworks on the issue 436 iGEM synthetic biology competition 426 avian flu research performed two – Policy – Initiative experiments431 showing how the flu virus could be made transmissible to Dual use biological research A significant part of synthetic ferrets (and, by extension, humans). concerns life sciences research biology is developed by “bio- This generated protests and calls for intended for benefit, but with results hackers“, 427 small-scale operations the papers to remain fully or partially which might easily be misapplied with a hobbyist or competitive unpublished, 432 because of the to produce harm when used by hacker ethos. This ethos would be potential for misuse by bio-terrorists bio-terrorists or in bio-weapon more attracted to self-regulation428 or bio-weapons programmes. In research. Examples of these included rather than outside governmental response, researchers in the field the experiments making avian ‘flu regulation. But industry self- declared a voluntary moratorium in transmissible to humans.437 But there regulations often fail in their January 2012.433 A year later, they were other examples too, including: 438 goals (especially without explicit decided to lift the moratorium. sanctions for malfeasance),429 so it – Accidentally increasing the virulence is currently unclear whether it can One cannot expect workers in a of mousepox as part of an be relied upon to reduce risk. The field to be unbiased about their experiment to control mice as International Genetically Engineered own research, 434 so it is significant pests in Australia.439 Machines (iGEM) competition is one that this decision was condemned – Variola virus immune of the largest in synthetic biology, by many scientists, including other evasion design.440 and has attempted to promote virologists.435 This provides strong – Chemical synthesis of bio-safety in its participants. It is evidence that ending the moratorium poliovirus cDNA.441 significant for the potential of self- was a dangerous decision. – Reconstruction of the 1918 regulation that such attempts have flu virus.442 been partially successful. – Creating and synthesising a minimal organism.443

As life science techniques develop, there is the potential for more such potentially dual use research in future. Yet, despite these dangers, the WHO reports that many countries and institutions lack oversight of such research, and that there is a lack of global frameworks on the issue.

112 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

01-Jun-13: Scientists create hybrid airborne H5N1 flu444 – Research

The H in H5N1 stands for “hemagglutinin”, as depicted in this molecular model. Permission is granted to copy, distribute and/ or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled GNU Free Documentation

Research continues into gain of function (GOF) for different influenza viruses. This report detailed methods for airborne mammal-to-mammal Hemagglutinin molecule transmission of the H5N1 flu virus, Source: http://en.wikipedia.org/wiki/ File:Hemagglutinin_molecule.png when hybridised with a highly transmissible (and human-infective) H1N1 virus. There is a possibility that such viruses (or those created in similar GOF experiments) could become transmissible to humans, and potentially cause a pandemic if they escaped from the lab.

A report from the Center for Arms Control and Non-Proliferation445 applied likelihood-weighted- consequence analysis to estimate the probability and impact of such escapes. It estimated that the risk was considerable: even if a rapid quarantine was instituted, each lab-year of such research carried an expected casualty rate of 180 to 1,100 fatalities, and $2.3 million to $390 million in economic damage.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 113 Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact 3.3 Emerging risks

3.3 Emerging risks 3.3.2 Nanotechnology

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Nanotechnology, 446 here defined as atomically precise manufacturing, 447 is the creation of effective, high-throughput manufacturing processes that operate at the atomic or molecular level.

There are many suggested designs, 448 but there are no immediately available methods to construct them. 449

114 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

3.3.2.1 Expected impact Some nanotechnology Extinction risk is only likely as pathways could mitigate a long-term consequence of these developments, however. civilisation collapse, if the survivors It is currently unclear whether Cheap mass surveillance, 456 are unable to rebuild and succumb nanotechnology would be a for instance, could catch such to other threats.464 The possibility revolution in manufacturing, or merely re-armament efforts (though of nanomachines or nanoweapons a continuation of current trends. surveillance could have its own remaining active after a civilisation Industry represents 30% of world detrimental effects). Many of the collapse may make the rebuilding GDP, 450 a declining fraction, so from world’s current problems may be more difficult, however, while the a narrow economic perspective it solvable with the manufacturing availability of atomically precise could be argued that the impact of possibilities that nanotechnology manufacturing systems, by contrast, nanotechnology would be relatively would make possible, such as could aid rebuilding. small. However, nanotechnology could depletion of natural resources, create new products – such as smart pollution, climate change, clean 3.3.2.2 Probability or extremely resilient materials451 – and water, and even poverty.457 There disaggregation would allow many different groups are currently few applicable or even individuals to manufacture a international legal regimes wide range of things. governing nanotechnology.458 Five important factors in This could lead to the easy In the media the label “grey estimating the probabilities and construction of large arsenals of goo” 459 is sometimes applied to impacts of the challenge: weapons by small groups.452 These nanotechnology. This is meant to might be masses of conventional describe a hypothetical situation 1. The timeline for nanotech weapons (such as drones or cruise where special self-replicating development. missiles), or more novel weapons nanomachines would be engineered 2. Which aspects of nanotech made possible by atomically precise to consume the entire environment. research will progress in what order. manufacturing. If this is combined It is unclear how effective they could 3. Whether it will be possible with a possible collapse in world trade be, and they play no role in atomically for small groups to assemble a networks453 – since manufacturing precise manufacturing.460 Mass self- weapons arsenal in a short period could now be entirely local – there replication would be detectable, and of time. would be a likely increase in the vulnerable to human-directed counter- 4. Whether nanotech tools can be number of conflicts throughout measures.461 However, it is possible effectively used defensively or for the world. Of particular relevance that such replicating machines surveillance. is whether nanotechnology allows could endure and thrive in particular 5. Whether nanotech tools rapid uranium extraction and isotope ecological niches, where the cost or weaponry are made to be separation454 and the construction of of removing them is too high.462 The independent of human control. nuclear bombs, which would increase misuse of medical nanotechnology463 the severity of the consequent is another risk scenario. conflicts. Unlike the strategic stalemate of nuclear weapons, nanotechnology arms races could involve constantly evolving arsenals and become very unstable.455 These conflicts could lead to mass casualties and potentially to civilisation collapse if the world's political and social systems were too damaged.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 115 3.3 EmergingNANOTECHNOLOGY risks GOVERNANCE DISASTERS

Meta-uncertainty: Effective Minor negative nanotech revolution Military Continuing regulatory impacts of early or evolution, speed nanotech research nanotech research framework nanotechnology of implementation?

Deliberate GlobalPrevalence of Meta-uncertainty Ultra- Technological attemptsGlobal to Global povety Global instability coordinationnanotechnology Smart sensors of political manufacturing innovations constructcoordination world predictions products dictatorship

Meta-uncertainty on tradeoffs between e.g. New system SmartBreakdown sensors poverty, survival, of governanceNew vulnerabilities of trade freedom

International relations

Failing to solve Improvements to Making important problems global governance things worse

Nano-terrorism AI Nano-warfare

Loss of control Not achieving over aggressive Global Lack of human Enduring poverty important Climatenanotechnology change pollution flourishing ethical goals

Uncontrolled Post-nanotech Large-scale UndesirableLong-term nano-ecology politics nano-war worldecosystem system impact (e.g. global dictatorship)

Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy

Damage to the Disruption to Direct world’s ecosystem world politics Direct Effectiveness of casualties and agriculture and economy casualties countermeasures

Total short- General Total short-termCivilsation General mitigation Civilisation Extinction term casualties Extinctionmitigation effort casualtiescollapse effort collapse

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

116 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

1. An effective regulatory framework 9. The direct casualties of an 17. A nano-ecology becomes could control the potential dangers uncontrolled nanotechnology are hard considerably more dangerous if of nanotechnology, though this to estimate, as they depend critically there is an intelligence controlling it depends very much on the nature of on the nature of the nanotechnology, (or pieces of it). Successful artificial the problems and the design of the the countermeasures used, and the intelligence could allow this to happen. regulations. Regulating the potentially general technological abilities of the pollutant aspects of nanotechnology human race after nanotechnology 18. It is in its potential for extreme – such as micro-particles – would development. The casualties from manufacturing that the promise and be more feasible under traditional nanowarfare are similarly hard to perils of nanotechnology lie. frameworks, but somewhat tangential determine, as it is unclear what would to the main issues. be the most effective military use of 19. Nanoweapon proliferation could nanoweapons, and whether this would completely destabilise international 2. Continuing research – into the involve high or low casualties (contrast relations and arms control treaties, transformative aspects, not just mass nuclear weapons with targeted by allowing small groups to rapidly standard materials science – is required shutdown of information networks). construct large arsenals. for nanotechnology to become a viable option for manufacturing. 10. Disruption of the world 20. One of the greatest threats of political and economic system nanotechnology is the possibility 3. Military nanotechnology (exacerbated by the collapse of that it could result in a breakdown research increases the chance that trade routes or nanowarfare) could of trade between currently nanotechnology will be used for lead to further casualties. interdependent nations. effective weapons production, and may lead to an arms race. 11. A nano-ecology could disrupt and 21. International relations could break undermine the standard biological down if trade does, leading to much 4. Global coordination allows for ecology, including food production. potential for conflict. regulatory responses, and may mitigate the effect of possible 12. The widespread use of 22. The effectiveness of collapse of trade routes. nanotechnology could generate new countermeasures is extremely vulnerabilities (just as modern cities are hard to judge, as is the balance 5. The general mitigation efforts of vulnerable to EMP (electro-magnetic between “defensive” and “offensive” most relevance to nanotechnology are pulse) weapons that would have had nanoweaponry. Nanotechnology could probably in surveillance and improved no effects in previous eras). allow novel approaches to controlling international relations. the problem, such as extremely 13. Over the long term, a nano-ecology effective sensors. 6. Nanoterrorism is one way in could spread and develop in ways that which humanity could lose control of are hard to predict or control (especially 23. Post-nanotech politics will aggressive nanotechnology. if there are new vulnerabilities to it). determine the risk of collapse and the potential for reconstruction. 7. Nanotechnology-empowered 14. Any problems with early warfare could spiral out of control, nanotechnology could provide impetus 24. Much of the analysis of the impact or could lead to the deployment for a regulatory or political response. of nanotechnology proceeds by of uncontrolled aggressive analogy with previous discoveries nanotechnology. The risk would be 15. The prevalence of or economic changes. It is unclear acute if small groups were capable of nanotechnology products could whether evolution or revolution is the effective nanowarfare on their own. introduce new vulnerabilities. better analogy, and what the speed of implementation of nanotechnological 8. Uncontrolled aggressive 16. Smart sensors of all kinds would discoveries will be. nanotechnology is a scenario in which be very important to either controlling humanity unleashes weapons that it a nano-ecology or preventing small 25. The course of international politics cannot subsequently bring under control, groups from rapidly constructing is extremely hard to predict, even for which go on to have independent arsenals of weapons. political scientists.465 negative impacts on the world.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 117 3.3 Emerging risks

07-May-13: Publication of Eric 3.3.2.3 Main events range of products without requiring Drexler’s book “Radical Abundance: large amounts of capital or long during 2013 How a Revolution in Nanotechnology supply chains. The risks of social Will Change Civilization” 473 and political disruption are then 11-Jan-13: Artificial molecular – Research examined. The disruptions that assembly device created 466 can be anticipated include “falling – Research Eric Drexler is one of the pioneers of demand for conventional labor, nanotechnology, and introduced the resources, and capital in physical A functional and practical design concepts to the general public with production, with the potential for assembling molecules is an his book “Engines of Creation”.474 for cascading disruptive effects essential feature for successful Twenty seven years later, he presents throughout the global economy”, nanotechnology. There have been a history, progress report, and as well as disruptions in supply many designs proposed, 467 and updated version of his vision, the chains, trade, dependence, and some constructed, but not yet a central theme of which is to “imagine the revaluation of assets (mineral fully functional molecular assembly a world where the gadgets and goods resources and large industrial device.468 This design, based on that run our society are produced not facilities, for example, will lose much principles from biology (it uses in far-flung supply chains of industrial of their value). messenger RNA as its input code, facilities, but in compact, even and synthesises peptides) represents desktop-scale, machines.” This would go together with an another step towards that important goal. increase in surveillance capability The revolution in manufacturing and a potential nanotechnology would produce the “radical arms race. The book recommends 06-May-13: First weapon made with abundance” of the title, with small taking pre-emptive action at the 3D printer 469 groups and individuals capable of international level to prepare for – Event producing an extraordinarily wide these disruptions.

It is the ability to make weapons en masse that represents one of the dangers of nanotechnology.470 3D printing (or additive manufacturing)471 is not nanotechnology, but can be considered a precursor, as it similarly allows small groups to design and manufacture their desired products themselves. That one of the early designs has been a functioning weapon, and that such weapon design was justified on moral grounds,472 indicates a very high probability that nanotechnology will be used for weapon production.

118 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

01-Jun-13: Nanostart AG: Venture 16-Dec-13: Nanotechnology: A Policy Capital Investments in Nanotech 475 Primer, CRS report of Congress 478 – Initiative – Policy

A key sign of a developing technology Governmental and supra- is interest from investment companies. governmental policies will be Nanostart AG is an example of such a key to dealing with the dangers company, with extensive investments and destabilising influences of in various nanotechnology projects. nanotechnology, through regulation, Interestingly, their interests are treaties, redistributive efforts or not limited to more conventional simply through preparing their nanotech projects, but extend to populations for the change. such speculative endeavours as And institutions such as the US space elevators.476 This serves as Congress are keeping an eye a reminder of the potentially large on nanotechnology, in this case profits available in nanotechnology. through the Congressional Research Thus it seems likely that when the Service. This report, however, technology matures sufficiently to does not delve into the major risks cause increased risks, there will be of nanotechnology, but restricts many commercial entities heavily itself to minor subjects such as investing in the technology, which the safety of nanomaterials and will make the process of regulation US competitiveness in that field. more contentious, possibly leading War, trade disruption and potential to “regulatory capture” 477 by development and misuse of nano- these entities, with their interests replicators479 are not discussed. represented rather than those of the This seems to reflect a certain lack broader community. of prioritisation and perhaps even a misplaced focus on the less important risks.

The World's First 3D-Printed Gun Source: http://upload.wikimedia.org/wikipedia/commons/thumb/b/be/XP002.jpg/1279px-XP002.jpg

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 119 3.3 Emerging risks

3.3 Emerging risks 3.3.3 Artificial Intelligence

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences Artificial intelligence (AI) is the intelligence exhibited by machines or software, and the branch of computer science that develops machines and software with human-level intelligence.

120 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

Major AI researchers and textbooks Such extreme intelligences could define the field as “the study and not easily be controlled (either by the design of intelligent agents”, where groups creating them, or by some an intelligent agent is a system that international regulatory regime),485 perceives its environment and takes and would probably act in a way actions that maximise its chances to boost their own intelligence and of success.480 acquire maximal resources for almost all initial AI motivations.486 And if these motivations do not detail487 3.3.3.1 Expected impact the survival and value of humanity in exhaustive detail, the intelligence will be driven to construct a world Artificial Intelligence (AI) is one of the without humans or without meaningful least understood global challenges. features of human existence. There is considerable uncertainty on what timescales an AI could be built, This makes extremely intelligent AIs 488 Source: http://commons.wikimedia.org/wiki/ if at all, with expert opinion shown to a unique risk, in that extinction is File:Artificial.intelligence.jpg be very unreliable in this domain.481 more likely than lesser impacts. An This uncertainty is bi-directional: AIs AI would only turn on humans if it could be developed much sooner or foresaw a likely chance of winning; much later than expected. otherwise it would remain fully integrated into society. And if an An interesting version of this scenario Despite the uncertainty of when AI had been able to successfully is the possible creation of “whole brain and how AI could be developed, engineer a civilisation collapse, for emulations,” human brains scanned there are reasons to suspect that instance, then it could certainly drive and physically instantiated - physically an AI with human-comparable the remaining humans to extinction. represented - in a machine. This skills would be a major risk factor. would make the AIs into what could AIs would immediately benefit On a more positive note, an be called properly human minds, from improvements to computer intelligence of such power could easily possibly alleviating a lot of problems. speed and any computer research. combat most other risks in this report, They could be trained in specific making extremely intelligent AI into professions and copied at will, thus a tool of great positive potential as 3.3.3.2 Probability replacing most human capital in well.489 Whether such an intelligence disaggregation the world, causing potentially great is developed safely depends on how economic disruption. Through much effort is invested in AI safety their advantages in speed and (“Friendly AI”)490 as opposed to simply Five important factors in performance, and through their building an AI.491 estimating the probabilities and better integration with standard impacts of the challenge: computer software, they could If the returns from increased quickly become extremely intelligence are low, intelligence 1. The reliability of AI predictions. intelligent in one or more domains explosions and extreme intelligence 2. Whether there will be a single (research, planning, social skills...). may not be possible. In that case, dominant AI or a plethora of entities. If they became skilled at computer there would probably be an ecology 3. How intelligent AIs will become. research, the recursive self- of AIs of different levels of intelligence, 4. Whether extremely intelligent AIs improvement could generate what performing different tasks. In this can be controlled, and how. is sometime called a “singularity”, scenario, apart from the economic 5. Whether whole brain emulations 482 but is perhaps better described dislocation already noted, there is also (human minds in computer form) as an “intelligence explosion”, 483 the possibility of AI-enabled warfare will arrive before true AIs. with the AI’s intelligence increasing and all the risks of the technologies very rapidly.484 that AIs would make possible.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 121 AI 3.3GOVERNANCE Emerging risks DISASTERS

AI AI arms race amalgamation Extinction

Global Effective Friendly Long-term coordination regulatory AI projects impact framework Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship

Military Extreme Meta-uncertaintyAI-warfare AI research intelligence AI on tradeoffs between e.g. New system Smart sensors poverty, survival, of governance freedom

Meta-uncertainty of what an AI is Reduced impact AI attack Diminishing return likely to be AI/Oracle AI contained to intelligence

Failing to solve Improvements to Making Meta-uncertainty global governance things worse important problems Partial on the reliability of Uploads Plethora of AIs friendliness AI predictions

Uncontrolled Copyable AI research human capital

Not achieving Global Lack of human Enduring poverty important Climate change pollution flourishing ethical goals

Suffering of artifical agents Undesirable world system (e.g. global dictatorship)

Economic Human collapse redundancy Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy

General mitigation Total short-term Civilisation effort casualties Collapse

General mitigation Civilisation TotalAI short-term Post-eruption Extinction casualtiesamalgamation politicseffort collapse

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

122 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

1. The advantages of global 7. General mitigation methods will be 15. Disparate AIs may amalgamate coordination and cooperation are of little use against intelligent AIs, but by sharing their code or negotiating to clear if there are diminishing returns may help in the aftermath of conflict. share a common goal to pursue their to intelligence and a plethora of AIs, objectives more effectively. but less clear if there is a strong first 8. Copyable human capital – mover advantage to the first group to software with the capability to 16. There may be diminishing returns produce AI: then the decisions of that perform tasks with human-like skills to intelligence, limiting the power first group are more relevant than the – would revolutionise the economic of any one AI, and leading to the general international environment. and social systems. existence of many different AIs.496

2. Military AI research will result in 9. Economic collapse may follow from 17. Partial “friendliness” may AIs built for military purposes, but mass unemployment as humans are be sufficient to control AIs in possibly with more safeguards than replaced by copyable human capital. certain circumstances. other designs. 10. Many economic and social 18. Containing an AI attack may be 3. Effective regulatory frameworks set-ups could inflict great suffering possible, if the AIs are of reduced would be very difficult without on artificial agents, a great moral intelligence or are forced to attack knowledge of what forms AIs will negative if they are capable of feeling before being ready. ultimately take. such suffering.494 19. New political systems may emerge 4. Uncontrolled AI research (or 11. Human redundancy may follow the in the wake of AI creation, or after an research by teams unconcerned creation of copyable human capital, as AI attack, and will profoundly influence with security) increases the risk of software replaces human jobs. the shape of future society. potentially dangerous AI development. 12. Once invented, AIs will 20. AI is the domain with the largest 5. “Friendly AI” projects aim to directly be integrated into the world’s uncertainties; it isn’t clear what an AI produce AIs with goals compatible economic and social system, is likely to be like. with human survival. barring massive resistance. 21. Predictions concerning 6. Reduced impact and Oracle AI 13. An AI arms race could result in AI are very unreliable and are examples of projects that aim AIs being constructed with pernicious underestimate uncertainties. to produce AIs whose abilities and goals or lack of safety precautions. goals are restricted in some sense, to prevent them having a strong negative 14. Uploads – human brains impact on humanity.493 instantiated in software – are one route to AIs. These AIs would have safer goals, lower likelihood of extreme intelligence, and would be more likely to be able to suffer.495

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 123 3.3 Emerging risks

development. 3.3.3.3 Main events 25-Apr-13: Kurzweil plans to help 13-Sep-13: Publication: “Responses to during 2013 Google make an AI brain 501 Catastrophic AGI Risk: A Survey” 507 – Initiative – Research 19-Mar-13: DARPA Sets Out to Make Computers That Can The idea of creating a fully general Since the recognition of the potential Teach Themselves 497 AI, an AI that is capable of all tasks risk with AGI (Artificial General – Policy requiring intelligence, went into Intelligence),508 various proposals abeyance during the AI winter,502 have been put forward to deal with The amount of information stored a period of reduced interest and the problem. After arguing that in a human brain is extremely funding in AI. The term AI itself fell into uncertainty about a timeline to AI509 large. Similarly, the amount of disfavour.503 But recent AI successes does not translate into a certainty information needed to perform such as Watson’s triumph on that AIs will take a long time, the adequately at complex human tasks “Jeopardy!” 504 (demonstrating a certain paper analyses why AIs could be an is considerable – far more than is level of natural language recognition existential risk. It argues that a trend easily programmable by hand (the and processing) and Google’s self- toward automatisation would give Cyc project,498 for instance, started driving car505 (demonstrating spatial AIs increased influence in society, in 1984, aiming to rapidly formally awareness and movement) have as such systems would be easier codify all human common sense – revived interest in constructing a to control, and there could be a and is still running). Thus the interest human-like mind in digital form. discontinuity in which they gained in the field of machine learning, Kurzweil, hired by Google at the end power rapidly.510 This could pose a and in algorithms that can teach of 2012, reveals in this interview his great risk to humanity if the AIs did not themselves skills and knowledge interest in doing just that. A notable share human values (intelligence and from raw data. With the rise of feature of Kurzweil is his optimism values are argued to be independent “Big Data”,499 vast databases and about the consequences of creating for an AI),511 a task which seems increased computer power, there has AIs,506 which could affect the level of difficult to achieve if human values are been a flowering of applications of precautions his team would include in complex and fragile,512 and therefore computer learning.500 This has caught its design. problematic to specify. the eye of the Defense Advanced Research Projects Agency (DARPA), The authors then turned to analysing a research arm of the US defense the AI safety proposals, dividing them department responsible for the into proposals for societal action, development of new technologies. external constraints, and internal In this project, DARPA aims both constraints. They found that many to “enable new applications that proposals seemed to suffer from are impossible to conceive of using serious problems, or to be of limited today’s technology” and to simplify effectiveness. They concluded by machines so that non-experts reviewing the proposals they thought can effectively use them and build most worthy of further study, including applications for them. This most AI confinement, Oracle AI, and recent project confirms the interest motivational weaknesses. For the long of the military in artificial intelligence term, they thought the most promising approaches were value learning (with human-like architecture as a less reliable but possibly easier alternative). Formal verification was valued, whenever it could be implemented.

124 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

for the human race. even of their own. 01-Oct-13: Publication of “Our Final 15-Oct-13: “Racing to the precipice: 24-Oct-13: Growing researcher Invention: Artificial Intelligence a model of artificial intelligence awareness of the threat of artificial and the End of the Human Era” development” lays out the dangers intelligence 525 by James Barrat, warning of the of AI arms races 522 – Research dangers of AI 519 – Research – Research, Much more effort is devoted to AIs may be developed by different creating AI than to ensuring that In this book, James Barrat argues groups, each desiring to be the it is developed safely.526 Those for the possibility of human-level AI first to produce an artificial mind. working in developing AI could be being developed within a decade, The competitive pressure will be motivated to minimise the extent based on the current progress in stronger the more powerful AIs are their creation represented a potential computer intelligence and the large believed to be, thus maximising danger.527 It is therefore significant sums invested by governments and the danger in those situations. when a researcher focused on the corporations into AI research. Once This paper considers an AI arms danger of AI is invited to speak at this is achieved, the AI would soon race,523 where different teams have a mainstream AI conference, as surpass human intelligence, and the option of reducing their safety Dr. Anders Sandberg of the Future would develop survival drives similar precautions in order to perfect of Humanity Institute was, at the to humans (a point also made in their device first – but running the 23rd International Joint Conference Omohundro’s “AI drives” thesis).520 risk of creating a dangerous and on Artificial Intelligence in Beijing. The book then imagines the uncontrollable AI. In the absence of He took part in a panel discussion competition between humanity and enforceable agreements between the entitled “The Future of AI: What if a cunning, powerful rival, in the form teams, this dynamic pushes each to we succeed?”, along with Joanna of the AI – a rival, moreover, that may take on more risk than they would Bryson, Henry Kautz and Sebastian not be “evil” but simply harmful to want (similarly to the “prisoner’s Thrun. He argued that though humanity as a side effect of its goals, dilemma”),524 potentially causing an current AI research does not appear or simply through monopolising extremely damaging outcome. to directly lead to dangerous AIs, scarce resources. the time to design and implement The situation is improved if risk- safety measures is now. This is Along with many interviews of taking makes little difference to both because of the time needed researchers working in the forefront speed of development, if the teams to develop such safety measures, of current AI development, the have reduced enmity between which could necessitate solving book further claims that without them, or if there are fewer teams hard philosophical problems,528 and extraordinarily careful planning,521 involved (those last two factors also because of the potential for sudden powerful “thinking” machines present help with reaching agreements). increases in AI skill and intelligence. potentially catastrophic consequences Somewhat surprisingly, information Further, security precautions would has a negative impact: the outcome be easier to implement if they is safer if the teams are ignorant of were integrated into the design by each other’s rate of progress, and the designers themselves (or by researchers intimately aware of the properties of the design). Further evidence of the increased awareness of risks was Stuart Russell’s529 joining of the board of the Cambridge Centre for the Study of Existential Risks (CSER).530

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 125 Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact 3.3 Emerging risks

3.3 Emerging risks 3.3.4 Unknown Consequences

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Uncertain risks represent the unknown unknowns in the family of global catastrophic challenges.

They constitute an amalgamation of all the risks that can appear extremely unlikely in isolation, but can combine to represent a not insignificant proportion of the risk exposure. 531

126 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

3.3.4.1 Expected impact But in these situations, the main 3.3.4.2 Probability source of probability of the risk is not the quoted number, but the disaggregation There are many different possible risks much greater probability that the that seem individually very unlikely experts’ models and world views Five important factors in and speculative. Could someone are wrong.537 If marginal scientific estimating the probabilities and develop a super-pollutant that renders theories predict large risks, the impacts of the challenge: the human race sterile? Could the probability is concentrated in the LHC have created a black hole likelihood that the theory might 1. Whether there will be extensive that swallowed the Earth? 532 Might be correct.538 Conversely, if many research into unknown risks and computer games become so addictive independent models, theories, and their probabilities. that large populations will die rather arguments all point in the direction 2. The capacity to develop than ceasing to indulge in them? 533 of safety, then the conclusion is methods for limiting the Could experiments on animals more reliable. combined probability of all lift them to a level of intelligence uncertain risks. comparable with humans? 534 Might There are methods to estimate 3. The capacity for estimating some of the people sending signals uncertain risks without needing to be “out-of-model” risks. to extra-terrestrial intelligences attract explicit about them. One resolution 4. The culture of risk assessment deadly alien attention? 535 What are to the Fermi paradox – the apparent in potentially risky areas. the risks out there that we can’t yet absence of alien life in the galaxy – is 5. Whether general, non-risk- conceive of? that intelligent life destroys itself before specific mitigation or resilience beginning to expand into the galaxy. measures are implemented. These risks sound unlikely and for Results that increase539 or decrease many possibly ridiculous. But many the probability of this explanation of today’s risks would have sounded modify the generic probability of ridiculous to people from the past. If intelligent life (self-)destruction, which this trend is extrapolated, there will be includes uncertain risks. Anthropic risks in the future that sound ridiculous reasoning540 can also bound the total today, which means that absurdity is risk of human extinction, and hence not a useful guide to risk intensity. estimate the unknown component.

Expert opinion provides some Non-risk-specific resilience and information on specific speculative post-disaster rebuilding efforts541 risks. But it will tend to give them will also reduce the damage from extremely low probabilities – after all, uncertain risks, as would appropriate the risks are highly speculative, which national and international regulatory also means the expert’s judgement is regimes.542 Most of these methods less reliable.536 would also help with the more conventional, known risks, and badly need more investment.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 127 3.3 EmergingECOLOGICAL risks CATASTROPHE GOVERNANCE DISASTERS

Post-eco-collapse Global Sustainability climate change Global poverty coordination research

Long-term ecological effects Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship

Quality of life Ecological Preservation loss from collapse efforts Meta-uncertainty ecosystem loss on tradeoffs between e.g. New system Smart sensors poverty, survival, of governance freedom

Threat to Moral tragedy from Pollution food supply ecosystem loss

Loss of Economic costs Improvements to MakingPre-eco-collapse biodiversity Failing to solve climate change important problems global governance things worse

Disruption to Rebuilding politics and Post-eco-collapse the ecosystem economy politics

Not achieving Global Lack of human Enduring poverty important Climate change pollution flourishing ethical goalsVulnerabilities Sustainable or New, to flood and non-sustainable environmentally other disasters economies damaging industries

Undesirable world system (e.g. global Pre-eco-collapse Technological dictatorship) mitigation efforts innovations

Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy Human survivability Meta-uncertainty on the true dependence of in “closed” systems humanity on the ecosystem

Total short-term General mitigation Civilisation Civilisation Total short-term Extinction Extinction casualties casualties effort collapse collapse

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

128 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

1. Smart sensors and surveillance 6. Some institutions may 12. The world’s political structure, could detect many uncertain risks deliberately pursue dangerous after an unknown risk is triggered, will in the early stages, and allow technologies or experiments, or determine whether humanity improves researchers to grasp what is going on. may convince themselves that their or worsens the situation. research is not dangerous. 2. Proper risk assessment in 13. Some methods (such as domains where uncertain risks are 7. Unforeseen accidents could be the considering the Fermi paradox) possible could cut down on the trigger for many uncertain risks. may bound the total probability of risk considerably. destructive uncertain risks, but these 8. The amount of direct casualties are very speculative. 3. Global coordination would aid risk varies wildly depending on the assessment and mitigation. risk involved. 14. Trying to estimate unknown or out of model risks is by definition very 4. Specific research into uncertain and 9. The disruptions to the world’s difficult and uncertain. unknown risks would increase our economic and political system vary understanding of the risks involved. wildly depending on the risk involved.

5. General mitigation efforts are mostly 10. The uncertain risk may have other general resilience building. disruptive effects (such as loss of trust in certain technologies).

11. The long-term impact varies wildly depending on the risk involved.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 129 3.3 Emerging risks

01-Aug-13: The Fermi paradox 3.3.4.3 Main events Such an explanation gives a bound provides an estimate of total to existential risk from all sources, during 2013 existential risk (including uncertain including uncertain risks. risks) 548 28-Mar-13: Paper on Adaptation – Research This paper demonstrates the to and recovery from global relative ease with which a space- catastrophes in general 543 The Fermi paradox is the seeming faring civilisation could cross – Research contradiction between the apparent between galaxies. ease with which intelligent life could One approach to dealing with arise in the galaxy, and the lack of Combined with recent evidence uncertain risks is to build general evidence of any such life. that the majority of Earth-like adaptation and recovery methods that planets formed before the Earth,551 would be relevant to a wide class of Many explanations have been this makes the absence of visible potential disasters. This paper notes proposed to resolve the paradox,549 intelligent life more inexplicable, the absence of published research one of which is relevant to and worsens the Fermi paradox, in this area,544 and seeks to begin to existential risks: the “Late Great increasing the probability of a Late fill the gap. It identifies methods for Filter” explanation.550 This posits Great Filter and thus of existential increasing survivor resilience and that intelligent life is inevitably risk from all sources. promoting successful adaptation destroyed before it can expand and recovery, even for isolated through the galaxy. communities. It recognises that the process is highly complex, and needs further research.

Figure 23: Number of galaxies that can reach us with speeds of 50%c, 80%c, 99%c and c, from different 28-Mar-13: Paper Evaluating Methods starting moments 552 for Estimating Existential Risks 545 – Research

It would be advantageous to have a rigorous approach for estimating severe risks, including uncertain and unknown ones. This paper reviews and assesses various methods for estimating existential risks, such as simple elicitation; whole evidence Bayesian; evidential reasoning using imprecise probabilities; Bayesian networks; influence modelling based on environmental scans; simple elicitation using extinction scenarios as anchors; and computationally intensive possible-worlds modelling.546 These methods can be applied rigorously to uncertain risks, assessing them in the same way as more standard risks. Influence modelling based on environmental scans547 can even suggest some new as yet unknown risks.

130 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.3 Emerging risks

01-Nov-13: Top conglomerates spearhead creation of private sector disaster response body 553 – Initiative

One of the most effective tools against uncertain risks is to adopt general disaster recovery measures. Anything that enables the preservation of resources or knowledge and the rapid reconstruction of key infrastructure will be of use against a wide variety of risks. Though governments and supra-governmental organisations play a vital role in this, it would be beneficial to get the private sector, with its funds and its expertise, involved too. The private sector has played a key role in recovery from many disasters (such as the Japanese 2011 earthquake/ tsunami).554 This news report shows that the private sector aims to take on a larger role in disaster relief in the Philippines. More importantly, the key players aim for the creation of a private sector disaster response body, so as to have a better organised private sector response during disasters. This is significant as it disperses disaster recovery expertise to a wider group of individuals, and suggests that private companies may be alternate entities capable of providing relief after a major disaster. Thus preparations for post-disaster recovery could include building up private sector capacity as well as other measures.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 131 3.4 Global Policy risk

3.4 Global Policy risk 3.4.1 Future Bad Global Governance

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

“Global governance refers to the way in which global affairs are managed. As there is no global government, global Ecological Nanotechnology governanceNuclear War typically involves aSuper-volcano range of Synthetic Unknown Catastrophe Biology Consequences actors including states, as well as regional and international organisations.

However, a single organisation may nominally be given the lead role on an issue.” 485

132 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.4 Global Policy risk

Often global governance is confused An example of the second would 3.4.1.2 Probability with global government, but they be constructing a global totalitarian are two very different things. Global state.557 In general, technology, disaggregation governance is just a term to describe political and social change may the way global affairs are managed, enable the construction of new forms Five important factors in estimating or not managed. Global government of governance, which may be either the probabilities of various impacts: is the idea that the world should be much better or much worse. run like a country with a government. 1. How the severity of non-deadly The global governance system These examples immediately policy failures can be compared will inevitably have pros and cons, illustrate two issues with governance with potential casualties. depending on the political decisions disasters. First, the task of estimating 2. Whether poor governance will result that are made. their probability is difficult. Long- in a collapse of the world system. term political predictions are of 3. How mass surveillance and questionable validity and subject to other technological innovations 3.4.1.1 Expected impact strong biases,558 especially where will affect governance. strongly-held values are concerned.559 4. Whether there will be new systems Second, the impact of these of governance in the future. This section looks at global governance disasters depends to a 5. Whether a world dictatorship governance disasters. Though all the large extent on subjective comparative may end up being constructed. risks in this report can be exacerbated evaluations. It is not impartially by poorly chosen policy decisions, this obvious how to rank continued classification contains those problems poverty and global totalitarianism that arise almost exclusively from bad versus billions of casualties or policy choices. civilisation collapse.560 The long term impact needs also to be considered: There are two main divisions in how will poverty and global governance disasters: failing to solve governance change? If there are many major solvable problems, and actively generations ahead of us, then the causing worse outcomes. An example long term state of humanity’s policy561 of the first would be failing to alleviate becomes much more important than absolute poverty.556 the short term one.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 133 3.4GOVERNANCE Global Policy risk DISASTERS GOVERNANCE DISASTERS

Failing to solve Improvements to Making important problems global governance things worse

Not achieving Global Lack of human Enduring poverty important Climate change pollution flourishing ethical goals

Not achieving Global Lack of human Enduring poverty important Climate change pollution flourishing ethical goals Undesirable world system (e.g. global dictatorship) Undesirable world system (e.g. global dictatorship) Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy

Disruption to Collapse of Post-disaster Long-term world politics world system politics negative effects and economy

General mitigation Civilisation Total short-term Extinction casualties effort collapse

Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts Key

Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts intervention areas Bad decisions Accidents Severe impacts

134 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.4 Global Policy risk

1. Global coordination between 6. Some groups may deliberately seek 14. Governance decisions taken at nations is essential for building a good to construct a world dictatorship, the global level have a high potential global governance system – but also either through self-interest or because to cause disruptions to the world’s essential for building a bad one. they believe it would be the best political and economic systems. design for global governance. 2. Global poverty is one of the 15. Bad governance at the global important problems that are being 7. Undesirable world systems (such as level may not be susceptible to only partially solved by current global dictatorships) could result from improvements and could cause policies. In turn, it can contribute to a worsening of global governance. problems for a considerable amount global instability, worsening likely of time. governance outcomes. 8. Many value systems do not distinguish between action and 16. Technological innovations could 3. Smart sensors and mass inaction, so a global system that allow completely new models of surveillance can contribute to new didn’t positively encourage human government, but could also facilitate systems of governance, but also to flourishing would be almost as surveillance dictatorships. large-scale dictatorships. pernicious as one that blocked it. 17. Global instability could result 4. The global system of governance 9. Global pollution is a problem in more pernicious systems of consists of the UN and a wide requiring solutions at the global governance, as well as an increased variety of bilateral or multilateral governance level. failure to solve important problems. agreements and norms, constructed mainly according to national 10. Climate change is a problem 18. New systems of governance self-interests. Thus significant requiring solutions at the global could be developed, using improvements to global governance governance level. modern communication are currently possible. technology for instance. 11. Various ethical systems have 5. General mitigation efforts against desirable goals that could be achieved 19. The political landscape after governance disasters are tricky – in theory, but would not be achieved a disaster will be important in most mitigation efforts are the results under suboptimal governance. determining whether governance of governance decisions! However, disasters could cause civilisation some efforts can be made – for 12. It would be a tragedy if absolute collapses or mass casualties. instance, an increase in recognised poverty were to endure over the human rights across the globe could generations to come, especially if this 20. How to compare enduring poverty, militate against certain pernicious outcome were avoidable. actual casualties, and repressive governance directions. These efforts governance is a question of values are of a very different nature to 13. A collapse of the world system, for and not just of direct comparison of mitigating other risks. any reason (including revolution) is the lives lost. most direct way a governance disaster could result in mass casualties.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 135 3.4 Global Policy risk

3.4.1.3 Main events – Perhaps the most cost-effective This confirms other studies, by way to reduce existential risks the World Bank564 and others:565 during 2013 today is to fund analysis of a poverty reduction is possible, and wide range of existential risks and has been successfully implemented 15-Feb-13: Existential risk reduction potential mitigation strategies, with in many countries. as a global priority 562 a long-term perspective. – Research If this paper is right, a general lack 05-Jun-13: Guardian leaks NSA In this paper Nick Bostrom, the of focus on existential risks by spying programme 566 director of the Future of Humanity governments and other agents can – Initiative Institute, lays out the case for making be considered a governance disaster existential risk reduction a global in itself. A significant event was the revelation priority. Existential risks (Xrisks) are the by Edward Snowden of the extent of highest category of negative impact the NSA’s surveillance programme. in this report, those that threaten the 19-Apr-13: Multidimensional poverty This included the mass recording entire future of humanity. The policy index diminishes in 18 out of 22 and mining of data across the implications of the paper are: analysed countries 563 United States and the interception of – Event foreign politicians’ data. – Existential risk is a concept that can focus long-term global efforts and Of 22 countries for which the Oxford The revelations caused great sustainability concerns. Poverty and Human Development controversy567 and raised questions – The biggest existential risks Initiative analysed changes in MPI about the NSA’s surveillance are anthropogenic and related to (Multidimensional Poverty Index) oversight.568 The episode established potential future technologies. poverty over time, 18 reduced that discrete mass surveillance – an – A moral case can be made that poverty significantly. important component of potential existential risk reduction is strictly more important than any other global public good. – Sustainability should be rethought Less poverty of nations in dynamic terms, as aiming for a sustainable trajectory rather than a 100 sustainable state. Gini index* – Some small existential risks can 29.1 be mitigated today directly (e.g. 80 asteroids) or indirectly (by building resilience and reserves to increase survivability in a range of extreme China 35.5 scenarios) but it is more important 35.1 60 to build capacity to improve humanity’s ability to deal with the India larger existential risks that will arise 30.8 33.4 later in this century. This will require 40 collective wisdom, technology foresight, and the ability when necessary to mobilise a strong 20 global coordinated response to 57.5 41.5

expected existential risks. Brazil 59.7 57.6 0

1981 81 90 95 2000 05 93

Figure 24: Less poverty of nations - Population living below $1.25 a day at 2005 Purchasing-pover parity, % * 0=absolute equality, 100=absolute inequality Source: Economist, originally World Bank, http://www.economist.com/node/14979330

136 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3.4 Global Policy risk

totalitarianism – was already possible using current technology and political organisation.

14-Jun-13: UNDP-UNEP Poverty- Environment Initiative Launches New Five-Year Phase to Meet Growing Demand from Member States 569 – Policy

To reduce poverty in the future, it is important to maintain and extend past trends in poverty mitigation. The United Nations’ Poverty-Environment Initiative (PEI), launched in 2008, has had a number of success stories from Uruguay570 to Malawi.571 Due to increased demand from member states, the programme has been extended for another five years, 2013-2017, and may add countries such as Myanmar, Mongolia, Indonesia, Albania, Peru and Paraguay. Such programmes demonstrate that the bureaucratic/policy side of poverty reduction is supported by an international infrastructure with a strong emphasis on assessments. The effect of such approaches on overall poverty will depend on the interplay between these policies and the other side of poverty reduction: economic growth572 and trade.573

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 137 4. Relations between global risk

4. Relations between global risks

“We have some idea what might happen if, in the face of other pressing global challenges, we divert our focus from making systemic improvements in public health and veterinary services — and that prospect is frightening.”

The World Bank 574

138 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 4. Relations between global risk / 4.1 General relations between global risks and their potential impacts

Two things make the understanding 4.1 General relations Nuclear war,577 asteroid impacts578 of the relation between the global and super-volcanoes579 have direct risks particularly important. between global risks impacts on the climate, and, through and their potential impacts that, on the ecosystem.580 1. Impacts: The global risks are interconnected in different Relations between global risks is an The kinds of mitigation efforts capable ways. Often the situation can be area where surprisingly little work is of containing the damage from a described as a set of dominoes: if being done. Most research focuses super-volcano would most likely be one falls, many others follow. Even on individual or closely related effective against asteroid impact small impacts can start a process groups of challenges. Organisations damage, because of the similar where different challenges interact. working on global challenges are nature of the impacts. The converse Higher temperatures due to global almost always working on individual is not true, since one major method warming can result in the spreading risks. The initial overview below is of reducing asteroid impact – space- of pandemics which increase based on individual studies where based deflection581 – would have no tensions between countries, and different relations are analysed, but impact on super-volcano risk. so on. no work has been identified where the relations between all twelve Solving climate change would help 2. Specific measures to address challenges have been analysed. reduce current ecological pressure.582 a risk: Global risks often require International agreements to reduce significant changes in our current A risk that is natural to start with is ecological damage could be extended society, from how we build cities future bad global governance, as all to combating climate change as to how food is produced and other global challenges exacerbate well, by establishing structures provided. Such significant changes governance disasters,575 and all other for international collaboration and will result in situations where global challenges can potentially be encouraging resource-efficient measures to reduce the risk in exacerbated by governance disasters. solutions. Climate change also creates one area affect the probability A well functioning global governance conditions more suitable for the and/or the impact in other areas. system is therefore a key factor to spread of pandemics.583 Measures to Depending on the measure chosen address global catastrophic risks. combat global pandemics, such as to reduce the risk, and other strengthened outbreak coordination complementary measures, the Conversely, avoiding governance and statistical modelling,584 could be effect can be positive or negative. disasters improves all risks, as used to combat synthetic pathogens better institutions are better able to as well. mitigate risks. Governance disasters directly increase the problems of If a safe artificial intelligence is climate change (through a lack of developed, this provides a great coordination between countries), resource for improving outcomes the risk of nuclear war (by stoking and mitigating all types of risk.585 conflict between nuclear powers) and Artificial intelligence risks worsening global system collapse (by weakening nanotechnology risks, by allowing global responses to systemic risks). nanomachines and weapons to be All risks exacerbate global system designed with intelligence and without collapse, by putting extra stress on an centralised control, overcoming the interconnected system.576 Conversely, main potential weaknesses of these a resilient governance system is machines586 by putting planning better able to cope with all risks, and abilities on the other side. a collapsed global system is more vulnerable to all risks.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 139 4.1 General relations between global risks and their potential impacts

Conversely, nanotechnology Nanotechnology and synthetic biology controlled and benevolent synthetic abilities worsen artificial intelligence are sufficiently closely related590 (both creations could act to improve and risks, by giving AI extra tools which dealing with properties on an atomic heal current ecological damage. it could use for developing its scale) for methods developed in power base.587 Nanotechnology one to be ported over to the other, There are many secondary effects and synthetic biology could allow potentially worsening the other risk. that are not covered here. Increasing the efficient creation of vaccines They are sufficiently distinct though nuclear power could for instance and other tools to combat global (a mainly technological versus a improve the outlook for climate pandemics.588 Nanotechnology’s mainly biological approach) for change,591 while increasing the risk of increased industrial capacity could countermeasures in one domain not proliferation592 and thus of nuclear war. allow the creation of large amounts necessarily to be of help in the other. There are many such effects between of efficient solar panels to combat various strategies for addressing climate change, or even potentially Uncontrolled or malicious synthetic different risks, but they are specific the efficient scrubbing of CO2 from pathogens could wreak great damage enough for there to be no simple the atmosphere.589 on the ecosystem; conversely, arguments of the type which says that mitigating risk X worsens risk Y.

ALL RISKS

first risk worsens second risk

solving first risk improves second risk

both of the above

140 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 4.1 Specific relations between global risks

4.2 Specific Below is an example of an overview An international initiative should start of how different global challenges to achieve better understanding relations between can be plotted depending on the of the relations between global global risks technical difficulty of reducing challenges in order to ensure the risk and the difficulty of synergies and avoid strategies that In parallel with work to increase our collaborating to reduce it. will undermine other challenges. understanding about the general relations between global risks, work to identify more specific relations should people die / suffer also be initiated. This is an area where many pieces of research exist. But very little work has been done to combine them and assess different strategies to address specific global risks and understand how these strategies will affect other global risks. It is important support use to distinguish between two different people afraid of infections short term thinking kinds of specific relations. Extreme Climate Change Global Pandemic First, there are solution strategies for one global risk and the ways it affects other global risks. For example, using video conferences can reduce the probability of pandemics by reducing unnecessary travel. On the other hand, unsustainable use of bio-energy could more more energy renewable nature from accident attack

increase spillover opportunities when video more meetings

a zoonosis (a disease transmitted less meat consumption from animals to humans) increases reduce risk the spread of pandemics due to an increased number of contacts between increase risk humans and infected animals in forests around the world.593

Second, how society reacts to the very threat of different risks can affect other challenges. For example, if people are afraid of pandemics they might use more video meetings and in that way help reduce carbon emissions.

Attempts to develop solutions for specific global challenges should assess their impacts, positive and negative, on other challenges. In order to better understand the relations between different global challenges, work could start to analyse similarities and differences. technical difficulty of reducing risk of reducing technical difficulty

collaboration difficulty of reducing risk

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 141 5. Probabilities and uncertainties – an initial overview

5. Probabilities and uncertainties – an initial overview

“Uncertainty is an uncomfortable position. But certainty is an absurd one.”

Voltaire

142 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 5. Probabilities and uncertainties – an initial overview

During the process of identifying almost all steps of the assessment. These estimates are only an attempt risks that could have an infinite Not only do great uncertainties exist to assemble existing estimates in impact it became evident that for all the risks, but the difficulties of order to encourage a process to the most common question estimating probabilities are also very improve these numbers. among people interested in different. At one end of the spectrum the global challenges is this: “How probability of a nuclear war can change These estimates range from rigorous probable is it that this impact dramatically from one day to another calculations based on large amounts will ever happen?” For those due to political decisions. Much of the of high-quality data (asteroids) to with expert knowledge in one uncertainty is related to psychological guesstimates by interested experts area the first question is often: assumptions of how different individuals (AI). The result is that some have a “How does the probability and will react under stress. more rigorous methodology behind magnitude of impact in this area them, and others should be taken compare with the probability and At the other end of the spectrum with a large grain of salt, but all are magnitude of impact in other there is AI, where there is not even a still very rough estimates. As science areas?” Finally, those who have generally accepted understanding of progresses they will be updated. tried to estimate probabilities for the possibility of the impacts capable It is even possible that some will global challenges ask: “What is the of creating the risks covered in this change by orders of magnitude. status of knowledge in other areas report. There are challenges with But instead of no estimate at all, we compared to mine?” very much data, including asteroids, now have an initial reference that we and other challenges with very little hope will trigger a discussion and These are all very important questions, relevant data, such as bad future collaboration that will help improve and this chapter is not an attempt global governance. what we have already. to answer them. But, as there is no organisation, process or report that Obviously the risks also share a As many of the challenges are long- has provided an overview of quantified number of characteristics: they all term and require early action to be assessment for global challenges with have potentially extreme outcomes avoided or mitigated, the probability potential infinite impact, the chapter and have never been experienced is provided for the next 100 years, does try to present the current state before. The possibility of studying instead of the annual probability that of knowledge in order to inspire series of data, exploring how the is often provided. The reason for this further work. outcome will change with incremental is that a 100-year perspective helps changes in input data, and testing us understand that even relatively It is easy to argue that it is too difficult, conclusions on similar events are just small probabilities can become or even impossible, to assess the a few examples of things that in most significant over a century. Say that probabilities that are at all meaningful cases cannot be done. Estimating it is a one in 100 probability (1%) for for the risks in this report, and probabilities in traditional ways is an impact to occur. Over a century therefore to exclude them. There are therefore very difficult.594 there is a 63.4% probability of one many good reasons for not trying, or more such impacts.595 Further, including significant uncertainty in However, as the current lack of structures that need to change require interest in global risks with potentially us to look beyond the immediate infinite impacts may in part be due to and incremental changes that most the lack of actual numbers, the best discussions focus on today. estimates that could be found are presented below with explanations.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 143 5. Probabilities and uncertainties – an initial overview

Structure of the probability estimates 2. Data availability One of the key guiding rules they As the different challenges are very This is an estimate of the amount of follow is to ensure a capital adequacy different and the status of probability data available to make probability at a 1-in-200 level. estimates varies significantly, the initial assessments on all relevant steps probability numbers are provided of the sequence. In some areas a This rule, which is included in for together with estimates regarding: lot of hard-to-get data is needed example ICA596 and Solvency II,597 to make an assessment (e.g a provides an opportunity to discuss 1. The understanding of sequence global pandemic); in other areas risks with a possible infinite impact. This is an estimation of how well the the data is related to secret and/ sequence from today to a possible or psychological factors (e.g. One contribution could be to discuss infinite impact is understood. At large-scale nuclear war). In others the pros and cons with different one extreme all the different paths relatively little data is needed definitions of the 1-in-200 level. For from today to an infinite impact are (asteroids), or a lot has been done example, one definition is that “each understood. At the other extreme, to gather data (e.g. climate change). company holds enough capital to there is only a theoretical idea that withstand the events of the next one is coherent and does not break any 3. Existing probability estimates year with a probability of 199 out of natural laws. In the latter case there These form an estimate of the 200.” 598 This would exclude many would be no understanding of how kind of uncertainty that exists. of the risks in this report and could it is possible to get from where we This obviously depends on an even result in the risks increasing, are today to an infinite impact. A understanding of sequence as the time perspective is so short. sequence is required to calculate and data availability, but it also Investments could help reduce short- an estimate instead of only having depends on resources and interest term risks at the same time as they educated guesses. in communicating with the rest of increase long-term risks. the world. Another definition is that “a company The estimates below are preliminary, should hold enough capital to be but a sound risk approach requires able to withstand a ‘reasonably stakeholders to begin to include them foreseeable’ adverse event”.599 This in strategic assessments. highlights the challenge of determining what “reasonably foreseeable” is. One group in particular is of interest Hopefully all the risks in this report and that is actuaries, the professionals could be included on such a list. Then who deal with the financial impact of the questions would be regarding risk and uncertainty. what we can do about it.

0.01% 0.01% 0.001% one in ten 0.1% one in a hundred thousand one in a thousand thousand 0.0001% 1% one in a one in a n/a n/a 0.8% million hundred

0.00001% 10% one in ten one in million infinite impact % ten

0.0000001% infinite threshold % 100% one in a hundred one in one 0.01% 0-10% million

0-10% 0.00013%

144 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

0.01% 0.0001% 0.002% 0.1%

5% 5% 0.00003% 5%

0.005% 0.5% 0.01%

5% n/a 1% n/a n/a 5. Probabilities and uncertainties – an initial overview

1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with uncertainty large some data some parts best guesses by experts none at all no data no estimates

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 145 5. Probabilities and uncertainties – an initial overview

Extreme Climate Change

Extreme climate change is one of The IPCC process ensures that Most models indicate that it will be the risks where global research data is widely available and of decades before the Earth reaches collaboration has taken place on a good quality, thanks to intensive equilibrium, and some impacts, significant scale. peer review in the natural science like sea-level rise, will happen 0.01% area. Estimates of political and over millennia.602 This long interval 0.01% The IPCC process is an technological development exist, but between action0.001% and impact meansone in ten 0.1% unprecedented scientific achievement they are more rudimentary compared the probabilityone of inclimate a hundred change isthousand one in a thousand thousand that has helped provide a unique level to the natural sciences.601 expressed on a 200-year timescale, of understanding for such a complex compared with 100 years for the area. Even so, the understanding of With such a high-profile area there0.0001% are other challenges. 1% one in a one in a n/a n/a 0.8% tipping points and collapses is still also a number of quantified estimatesmillion hundred rudimentary. From a risk perspective it in key areas such as emissions Based on available assessments603 is important to know that many factors trajectories, climate sensitivity, the best current estimate for that could result in infinite impacts impacts and thresholds that allow for extreme climate change in the are excluded from most studies, for approximations of probabilities.0.00001% But next 200 years is: 10% one in ten one in example the rapid release of methane only a few estimates millionexist that provide ten clathrates.600 Similarly, significant probability assessments, as there is a 5% for infinite threshold,infinite impact % uncertainty exists about political tendency to use scenarios instead. 0.01% for infinite impact decisions in many countries, about the implementation of new solutions and One aspect 0.0000001%that makes climate infinite threshold % 100% about what lifestyles will dominate. change differentone fromin a hundredall other risks one in one 0.01% 0-10% is that the timemillion from initial action to impact is very long. The great uncertainty is where the threshold lies where the planet begins to 0-10% emit greenhouse gases that start irreversible feedbacks. 0.00013%

0.01% 0.0001% 0.002% 0.1%

5% 5% 0.00003% 5%

146 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

0.005% 0.5% 0.01%

5% n/a 1% n/a n/a 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 147 5. Probabilities and uncertainties – an initial overview 0.01% 0.01% 0.001% one in ten 0.1% one in a hundred thousand one in a thousand thousand 0.0001% 1% Nuclear War one in a one in a n/a n/a 0.8% million hundred

0.00001% 10% one in ten one in million ten Nuclear war is the risk that started The data availability is relatively There are some estimatesinfinite of key impact % the work with scientific assessments low as much of the probability is aspects, such as the probability of related to infinite impact.604 decided by factors that are secret accidental initiation of a nuclear war, (e.g. the targets0.0000001% for nuclear weapons). but few estimates of theinfinite probability threshold of % 100% The understanding of the sequence It depends onone knowledge in a hundred that by a full-scale nuclear war. one in one 0.01% 0-10% million is relatively well known. Still, the definition is unavailable (no nuclear fact is that the impact will depend explosions, for example, have taken Based on available assessments607 significantly on how serious the place in a modern city); on human the best current estimate for nuclear 0-10% nuclear winter will be as the result of factors (e.g. stress tolerance and war in the next 100 years is: a war (if there is any nuclear winter aggressive tendencies among those at all). The probability of a nuclear who will have to decide whether 5% for infinite threshold, 0.00013% winter will depend on when during the or not to launch nuclear weapons); 0.005% for infinite impact year the war happens, and what the and on the effectiveness of current weather is during this time. The result policies (e.g. how efficient current is that the probability of an infinite deterrence policies are). The fact that impact has an inherent uncertainty climate change research has provided and can be estimated only once a war better scientific understanding of the has already started. probability and nature of a nuclear winter, thanks to better climate modelling, is worth noting as it shows how research programmes 0.01% 0.0001% 0.002% 0.1% on different global challenges can be mutually supportive.605

5% 5% 0.00003% 5%

0.005% 0.5% 0.01%

5% n/a 1% n/a n/a

148 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Global Challenges –Twelve risks thatthreaten human civilisation –Thecasefora newcategoryof risks ofsequence 1. Understanding to infiniteimpact degree ofeventsfrom today’s actions none at all some parts most parts all parts availability 2. Data of thesequence assessment onallrelevant steps amount ofdatatomakeprobability no data some data most data all data 5. Probabilities anduncertainties –aninitialoverview estimation 3. Existingprobability

kind ofestimationanduncertainty best guesses calculations with calculations with no estimates by experts large uncertainty small uncertainty 149 5. Probabilities and uncertainties – an initial overview

Global Pandemic

On a general level the sequence, or When it comes to data availability for Most of the probability estimates rather sequences, for a pandemic pandemics the situation is different made for pandemics are for their more are relatively well-known. The compared with nuclear war or benign versions. For the possible challenge here is that there are climate change, where the impact pandemic that could kill two billion or 0.01%0.01% so many different scenarios that depends0.01%0.01% on something that can more there are very few estimates. it is very difficult to calculate0.001%0.001% all be removedoneone in(nuclearin tenten weapons0.1%0.1% and the different possibilities evenoneone inin aa hundredhundredgreenhousethousandthousand gases). It is notoneone possible inin aa Based on available assessments608 thousandthousand thousandthousand if the sequence is well-known. to get rid of mutating viruses and the best current estimate of a global Compared with climate change other organisms, so the data needed pandemic in the next 100 years is: where greenhouse0.0001%0.0001% gases are a is of another kind and magnitude. 1%1% oneone inin aa 5%oneone for inin infinite aa threshold, nn/a/a nn/a/a 0.8%0.8% small group,millionmillion and nuclear war There will always be new diseases hundredhundred where the number of warheads emerging, because there is constant 0.0001% for infinite impact is relatively limited, the number of evolution, resulting from microbes ways that a global pandemic can looking for ecological niches. 0.00001%0.00001%start is almost unlimited. The reason10% for10% the big difference oneone inin tenten oneone inin millionmillion With many of the spillover effects between thresholdtenten and impact is Making things worse too is the fact infiniteinfiniteoccurring impactimpact in remote areas, %% even mainly that a pandemic will not that a global pandemic that reached basic data is still very rudimentary. directly affect infrastructure or the the infinite threshold would most Scientists who collect data relevant rest of the ecosystem in the way that 0.0000001%0.0000001%certainly be very different from almostinfiniteinfinite for pandemics thresholdthreshold are often working %% with extreme climate100%100% change or nuclear oneone inin aa hundredhundredall earlier pandemics, and maybe very small resources, and there is no war would. Thisoneone means inin oneone that resilience 0.01%0.01% 0-10%0-10% millionmillion something totally new that has never systematic way of collecting data on will be relatively better after the infinite happened before. Understanding a global scale, although interesting threshold is crossed. more than the most basic sequence initiatives are under way.607 While therefore becomes a challenge. an early warning system would be 0-10%0-10% comparatively inexpensive, there are still no resources available. 0.00013%0.00013%

0.01%0.01% 0.0001%0.0001% 0.002%0.002% 0.1%0.1%

5%5% 5%5% 0.00003%0.00003% 5%5%

150 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

0.005%0.005% 0.5%0.5% 0.01%0.01%

5%5% nn/a/a 1%1% nn/a/a nn/a/a 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 151 5. Probabilities and uncertainties – an initial overview0.01%0.01% 0.01%0.01% 0.001%0.001% oneone in in ten ten 0.1%0.1% oneone in in a a hundred hundred thousandthousand oneone in in a a thousandthousand thousandthousand 0.0001%0.0001% 1%1% Ecologicaloneone in in a a oneone in in a a nn/a/a nn/a/a 0.8%0.8% millionmillion hundredhundred Catastrophe 609 0.00001%0.00001% 10%10% oneone in in ten ten oneone in in millionmillion tenten This is one of the more complex infiniteinfiniteThere are impactimpact many studies %% about the Regarding probability estimates, it is risks as it can be seen more as a stability and possible collapse of only ecological collapse and global heading than a description of a different ecosystems, but there are system collapse of the current man- 0.0000001%0.0000001%specific challenge with a well-definedinfiniteinfinite few that thresholdthreshold look into the possibility %% made global100% challenges100% that have no oneone in in a a hundred hundredsequence. In other words it is not for a full ecological collapse that estimates forone oneinfinite in in one oneimpact. 0.01%0.01% 0-10%0-10% million million one sequence, but very many still would result in at least two billion unknown sequences. The concept people suffering. Based on available assessments611 of ecological collapse usually refers the best current estimate of an 0-10%0-10% to a situation where some part of the Data availability is good in many ecological catastrophe in the next 100 ecological web becomes so weak that areas, but the challenge is that years is: it collapses. without an understanding of the 0.00013%0.00013% system dynamics, and because of its 0.5% for infinite threshold, complexity, there are inherent limits to Not available for infinite impact how exact the knowledge is that can be achieved.610

0.01%0.01% 0.0001%0.0001% 0.002%0.002% 0.1%0.1%

5%5% 5%5% 0.00003%0.00003% 5%5%

0.005%0.005% 0.5%0.5% 0.01%0.01%

5%5% nn/a/a 1%1% nn/a/a nn/a/a

152 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 153 5. Probabilities and uncertainties – an initial overview

Global System Collapse

Since the financial crisis the Second, it is a recent system that The data availability of global system possibility of a global collapse of has been so interconnected for collapse is something of a paradox. the current political, economic and only a few years, as it depends on On the one hand the system is almost financial system has been discussed an infrastructure that did not exist nothing but information, but at the intensively. A rapidly evolving and before the internet, so there is little same time data about how the system increasingly interconnected system, it experience of how it works. itself operates, what algorithms are is subject to unexpected, system-wide used, and so on, are not well known. failures because of the structure of the Third, the system is rapidly network – it faces a systemic risk. changing and becoming No estimate of the probability of a even more complex as more global system collapse that would Possible sequences for a global connections are added and result in an impact beyond the infinite system collapse resulting in infinite its speed increases. Better threshold has been identified during impacts are very hard to establish, understanding of complex the project. for three reasons. First, it is a very systems with multiple complicated system, with many attractors and bifurcation Based on available assessments the dynamic interactions, as there are behaviour will help improve the best current estimate of a global many people who, together with possibility of understanding system collapse in the next 100 machines, react to each other. The the possible sequences.613 years is: current global system shows a lot of complex dynamic phenomena, An additional challenge for the Not available for infinite threshold, such as business cycles, financial understanding of sequences that Not available for infinite impact crises, irregular growth, and bullwhip can result in impacts beyond the effects.612 Many nonlinear dynamic infinite threshold is that almost all models of economics and finance research being done in the area of present various complex dynamic global system collapse focuses on its behaviours such as chaos, fractals, economic or geopolitical implications, and bifurcation. not on a full system collapse and not on human suffering.

0.01%0.01% 0.01%0.01% 0.001%0.001% oneone inin tenten 0.1%0.1% oneone inin aa hundredhundred thousandthousand oneone inin aa thousandthousand thousandthousand 0.0001%0.0001% 1%1% oneone inin aa oneone inin aa nn/a/a nn/a/a 0.8%0.8% millionmillion hundredhundred

0.00001%0.00001% 10%10% oneone inin tenten oneone inin millionmillion infiniteinfinite impactimpact %% 154 tenten Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

0.0000001%0.0000001% infiniteinfinite thresholdthreshold %% 100%100% oneone inin aa hundredhundred oneone inin oneone 0.01%0.01% 0-10%0-10% millionmillion

0-10%0-10% 0.00013%0.00013%

0.01%0.01% 0.0001%0.0001% 0.002%0.002% 0.1%0.1%

5%5% 5%5% 0.00003%0.00003% 5%5%

0.005%0.005% 0.5%0.5% 0.01%0.01%

5%5% nn/a/a 1%1% nn/a/a nn/a/a 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 155 5. Probabilities and uncertainties – an initial overview

Major Asteroid Impact

The understanding of sequence Other initiatives, like the the Based on available assessments618 when it comes to asteroid impacts Sentinel Mission by the B612 the best current estimate of a major is relatively straightforward, and our foundation, are under way that will asteroid impact in the next 100 planet is constantly experiencing further improve availability. years is: asteroids, so assumptions can be tested. This, combined with the fact Compared with most other 0.01% for infinite threshold, that there has been a number of global challenges there are many 0.00013% for infinite impact major impacts in the Earth’s history, probability estimates with transparent makes the sequence reasonably methodology, and the degree of well known.614 uncertainty is relatively low compared with other challenges. NASA even The data availability is still far from has an overview of the probability of perfect, but it is rapidly improving. individual objects hitting Earth.617 For Currently NASA has a table with the most severe impacts, the size of potential future Earth impact the asteroid will make it visible years in events that the JPL Sentry System advance, and this will only improve as has detected, based on currently our capacity to scan the space around available observations.615 us increases.

0.00001%0.00001% 10%10% oneone inin tenten oneone inin millionmillion infiniteinfinite impactimpact %% tenten

0.0000001%0.0000001% infiniteinfinite thresholdthreshold %% 100%100% oneone inin aa hundredhundred oneone inin oneone 0.01%0.01% 0-10%0-10% millionmillion

0-10%0-10% 0.00013%0.00013%

156 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

0.01%0.01% 0.0001%0.0001% 0.002%0.002% 0.1%0.1%

5%5% 5%5% 0.00003%0.00003% 5%5%

0.005%0.005% 0.5%0.5% 0.01%0.01%

5%5% nn/a/a 1%1% nn/a/a nn/a/a 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 157 5. Probabilities and uncertainties – an initial overview

Super-volcano

The super-volcano risk has many There is data available for different Based on available assessments619 similarities with a major asteroid risk. impacts, and knowledge of where the best current estimate of a super- Both have happened a number of super-volcanoes might erupt is volcano in the next 100 years is: times through our planet’s history, and increasing, but due to the lack of 0.01% 0.01% both have had major consequences. understanding when it comes to the 0.002% for infinite threshold, 0.001% one in ten 0.1% sequence, the probability estimations 0.00003% for infinite impact one in a hundred thousand oneThe understandingin a of the sequence is are still very rudimentary. thousand thousand however a lot lower than for asteroids, as the mechanisms behind volcano A number of estimates exist where 0.0001% eruptions are not1% very well known. The the probability is assessed, but they one in a one in a n/a n/a 0.8% million possibility of foreseeinghundred when a super- are quite rudimentary, based on the volcano will erupt and how big the historic frequency of earlier super- impact will be is therefore low. Compared volcano eruptions. As these are so with a major asteroid, there will therefore infrequent, the uncertainty becomes 0.00001% be much less time to prepare.10% very significant. one in ten one in million infinite impact % ten

0.0000001% infinite threshold % 100% one in a hundred one in one 0.01% 0-10% million

0-10% 0.00013%

0.01% 0.0001% 0.002% 0.1%

5% 5% 0.00003% 5%

158 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

0.005% 0.5% 0.01%

5% n/a 1% n/a n/a 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 159 0.01%0.01% 5. Probabilities and uncertainties – an initial overview 0.01%0.01% 0.001%0.001% oneone inin tenten 0.1%0.1% oneone inin aa hundredhundred thousandthousand oneone inin aa thousandthousand thousandthousand 0.0001%0.0001% 1%1% oneone inin aa Syntheticoneone inin aa nn/a/a nn/a/a 0.8%0.8% million hundred million Biologyhundred 0.00001%0.00001% 10%10% oneone inin tenten oneone inin millionmillion tenten infiniteinfinite impactimpact %% Many experts see synthetic biology One of the challenges to Based on available assessments622 as the most serious future risk. The understanding the sequence is that the best current estimate of an ability already exists to develop very the spreading of synthetic biology impact from synthetic biology in the 0.0000001%0.0000001% infiniteinfinite thresholdthreshold %%deadly viruses, and as knowledge100%100% will come either from a wilful act next 100 years is: oneone inin aa hundredhundred and technology develop further,oneone inmorein oneone (e.g. terrorism) or an accident 0.01%0.01% 0-10%0-10% million million deadly pandemics can be developed (e.g. unintentional release from a 1% for infinite threshold, by an increasing number of people. laboratory). This also makes data 0.01% for infinite impact hard to get. There are some numbers 0-10%0-10% The basic sequence is relatively for accidents in labs, but they are The probability numbers for synthetic well-known, given that it would be a available in only a few countries and biology are very high and can more deadly version of a current virus, there are probably0.00013%0.00013% many more than hopefully be reduced once better but there is also the possibility that those reported.620 With terrorist acts monitoring is in place, together with a new virus (or other organism) may there are probability estimates that increased global collaboration. be found where the sequence will be can be used as a basis for the use of unknown and therefore also much synthetic biology as well.621 more dangerous. There are some existing estimates for synthetic biology, but these are based on possible use in war, where calculations depend on some specific 0.01%0.01% 0.0001%0.0001% differences from existing pathogens 0.002%0.002% 0.1%0.1% that are assumed to be necessary for a pandemic with an infinite impact.

5%5% 5%5% 0.00003%0.00003% 5%5%

0.005%0.005% 0.5%0.5% 0.01%0.01%

5%5% nn/a/a 1%1% nn/a/a nn/a/a

160 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 161 5. Probabilities and uncertainties – an initial overview

Nanotechnology

Nanotechnology is best described Compared with many other global Admiral David E. Jeremiah, as a general capacity, rather than challenges, nanotechnology could for example, said at the 1995 a specific tool. In relation to infinite result in many different risks and Foresight Conference on Molecular impacts this is a challenge, as there opportunities, from an accelerated Technology: “Military applications are many ways that nanotechnology ability to manufacture (new) weapons623 of molecular manufacturing have can be used that could result in infinite to the creation of new materials and even greater potential than nuclear impacts, but also many others where substances. These are certainly orders weapons to radically change the it can help reduce infinite impacts. of magnitude more likely, far likelier balance of power.” 624 A systems- than any probability of the “grey forecasting approach could probably Different possible sequences from goo” that has resulted in significant provide better estimates and help today’s situation to precise atomic misunderstanding. develop complementary measures manufacturing are well documented, that would support the positive parts and the probability that none of The data availability is difficult to of nanotechnology while reducing the possible paths would deliver estimate as there are very different the negative. results is very small. What specific kinds of data, and also an obvious sequence and with what results is lack of data, as nanotechnology is in Based on available assessments625 however very uncertain. its very early days. the best current estimate of an impact from nanotechnology in the There are some estimates from next 100 years is: experts, but the uncertainty is significant. A relative probability 0.8% for infinite threshold, estimate is a possible first step, 0.01% for infinite impact comparing nanotechnology solutions with existing systems where the probability is better known.

0.01%0.01% 0.01%0.01% 0.001%0.001% oneone in in ten ten 0.1%0.1% oneone in in a a hundred hundred thousandthousand oneone in in a a thousandthousand thousandthousand 0.0001%0.0001% 1%1% oneone in in a a oneone in in a a nn/a/a nn/a/a 0.8%0.8% millionmillion hundredhundred

0.00001%0.00001% 10%10% oneone in in ten ten oneone in in millionmillion infiniteinfinite impact impact % % tenten 162 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

0.0000001%0.0000001% infiniteinfinite threshold threshold % % 100%100% oneone in in a a hundred hundred oneone in in one one 0.01%0.01% 0-10%0-10% millionmillion

0-10%0-10% 0.00013%0.00013%

0.01%0.01% 0.0001%0.0001% 0.002%0.002% 0.1%0.1%

5%5% 5%5% 0.00003%0.00003% 5%5%

0.005%0.005% 0.5%0.5% 0.01%0.01%

5%5% nn/a/a 1%1% nn/a/a nn/a/a 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 163 5. Probabilities and uncertainties – an initial overview

Artificial Intelligence

Artificial Intelligence is the global risk What is possible is to define a number Based on available assessments627 where least is known. Not even those of general factors determining the best current estimate of an who see the possibility of developing risk. These include Capability and impact from AI in the next 100 years an AI claim to be able to describe Compatible goals.626 For global is: what a working AI is in detail, let alone challenges in rapidly evolving areas provide a description of the sequence where incremental development might 0-10% for infinite threshold, from where we are today to an AI that not happen and little is known about 0-10% for infinite impact could result in infinite impact. the sequence, the only way to reduce risks with possible infinite impacts The reason for 0-10% on both impact The assumptions for an AI are based might be to ensure focus on these levels is that most experts assume on the current rapid technological general factors. that the kind of AI capable of impacts development, but as it is not even beyond the infinite threshold is likely possible to simulate a simple version The only estimates of probabilities to be one that also can result in an of AI it is hard to get any data. that exist so far have been made infinite impact. If we succeed it will by a small group with a significant move beyond control very rapidly. proportion of people with a passion Due to the significant impact it would for AI. Compared with many other have if it worked, there is no difference challenges the possibility of an AI between the two impact levels. capable of infinite impact can almost be described as all or nothing.0.01% This 0.01%0.01% is also why the estimates are the0.01% 0.001%0.001% oneone in in ten ten 0.1%0.1% same for the infinite threshold and the oneone in in a a hundred hundred thousandthousand oneone in in a a infinite impact. thousandthousand thousandthousand 0.0001%0.0001% 1%1% oneone in in a a oneone in in a a nn/a/a nn/a/a 0.8%0.8% millionmillion hundredhundred

0.00001%0.00001% 10%10% oneone in in ten ten oneone in in millionmillion infiniteinfinite impact impact % % tenten

0.0000001%0.0000001% infiniteinfinite threshold threshold % % 100%100% oneone in in a a hundred hundred oneone in in one one 0.01%0.01% 0-10%0-10% millionmillion

0-10%0-10% 0.00013%0.00013%

164 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

0.01%0.01% 0.0001%0.0001% 0.002%0.002% 0.1%0.1%

5%5% 5%5% 0.00003%0.00003% 5%5%

0.005%0.005% 0.5%0.5% 0.01%0.01%

5%5% nn/a/a 1%1% nn/a/a nn/a/a 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 165 5. Probabilities and uncertainties – an initial overview

Unknown Consequences

Uncertain risks must be included But we know from history that these Based on available assessments628 in any project addressing low- kinds of events happen over and the best current estimate of an probability high-impact events. The over again. With rapid technological uncertain risk in the next 100 years is: way to approach uncertain risks is, by development and increased tensions0.01%0.01% 0.01%0.01% definition, uncertain. over coming decades, the magnitude 5% for infinite threshold, 0.001%0.001% oneone in in ten ten 0.1%0.1% of the impacts can be assumed to 0.1% for infinite impact oneone in in a a hundred hundred thousandthousand oneone in in a a The sequence can only be assessed increase. The probability estimates thousandthousand thousandthousand on the basis of experience of exist only as best guesses by experts, unexpected events, so actual data and while it is possible to make 0.0001%0.0001% 1%1% does not exist. the assessments more formal, it is oneone in in a a oneone in in a a nn/a/a nn/a/a 0.8%0.8% millionmillion hundredhundred currently the best existing estimates that at least provide a preliminary order of magnitude for these risks. 0.00001%0.00001% 10%10% oneone in in ten ten oneone in in millionmillion infiniteinfinite impact impact % % tenten

0.0000001%0.0000001% infiniteinfinite threshold threshold % % 100%100% oneone in in a a hundred hundred oneone in in one one 0.01%0.01% 0-10%0-10% millionmillion

0-10%0-10% 0.00013%0.00013%

0.01%0.01% 0.0001%0.0001% 0.002%0.002% 0.1%0.1%

5%5% 5%5% 0.00003%0.00003% 5%5%

166 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks

0.005%0.005% 0.5%0.5% 0.01%0.01%

5%5% nn/a/a 1%1% nn/a/a nn/a/a 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 167 0.01%0.01% 5. Probabilities and uncertainties – an initial overview 0.01%0.01% 0.001%0.001% oneone in in ten ten 0.1%0.1% oneone in in a a hundred hundred thousandthousand oneone in in a a thousandthousand thousandthousand 0.0001% 1% 0.0001% 1% n/a n/a 0.8% oneone in in a a oneone in in a a nFuture/a Badn/a 0.8% million hundred million hundred Global Governance 0.00001%0.00001% 10%10% oneone in in ten ten oneone in in millionmillion tenten infiniteinfinite impact impact % % As there is no global government, The probability of a bad global Based on available assessments the global governance today typically governance system will increase the best current estimate of a future bad involves a range of actors including longer it takes to implement solutions global governance with potentially 0.0000001%0.0000001% infinite threshold % 100%100% states, regional and international to address global challenges. infinite impacts in the next 100 years is: one in a hundred infinite threshold % one in one one in a hundred one in one organisations. A single0.01%0.01% organisation When the world experiences 0-10%0-10% million million may nominally be given the lead role significant negative impacts, the Not available for infinite threshold, on an issue.629 time for reflection will be shorter and Not available for infinite impact polarisation is likely to increase. 0-10% Future bad global governance is 0-10% an important challenge, although To establish a failed governance 0.00013%0.00013%it is totally different from the other system on a global level will require challenges. We should remember that something extraordinary, as nations at present about two billion people tend to protect their national live in poverty, and the risks from sovereignty at almost any cost. global challenges are increasing. No governance system is perfect, but it What can help in understanding is possible to improve significantly on possible sequences is an increasing the current system without increasing amount of data on how complex the risks. organisations work and the psychology of organisations that can 0.01%0.01% 0.0001%0.0001% 0.002%0.002% create destructive patterns.630 0.1%0.1%

5%5% 5%5% 0.00003%0.00003% 5%5%

0.005%0.005% 0.5%0.5% 0.01%0.01%

5%5% nn/a/a 1%1% nn/a/a nn/a/a

168 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 5. Probabilities and uncertainties – an initial overview

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps of the sequence

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 169 6. Underlying trends of key importance

6. Underlying trends of key importance

“Those who do not remember the past are condemned to repeat it.”

George Santayana

170 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 6. Underlying trends of key importance / 6.1 Poverty

To address the global challenges 6.1 Poverty There are reasons to celebrate this means recognising several development as more people than underlying trends. These will ever live a life where they do not have influence the challenges by Global poverty has fallen dramatically to constantly worry about their most affecting society as a whole as over the last two centuries, and the basic needs. But there are two things well as more directly affecting the fall has intensified in recent decades, worth remembering: challenges. Currently most risk raising hopes that poverty, defined assessments and other studies by the World Bank as an income 1. Poverty could increase again. related to global challenges below US $1.25 per day, may be 2. Defining poverty is difficult. neglect the fact that these trends eliminated within the next 50 years. can have very different outcomes. The Economist even had a cover, in Today very few people assume that June 2013, with the title “Towards the poverty could increase again. But The most common mistake is that end of poverty”.631 The World Bank everything from economic crisis and the most likely development of these has set an interim target of reducing pandemics to climate change and underlying trends is taken for granted global extreme poverty to 9% of the wars could change that. The situation as the only possible outcome. world’s population by 2020, which, if after the fall of the Soviet Union Most of the trends have probability achieved, would mark the first time resulted in increased poverty.634 distributions where low-probability/ the rate has fallen to single digits.632 high-impact possibilities are often The milestone is based on a World Even conservative estimates show ignored. In this chapter some of Bank economic analysis of global that the percentage of people in the most important trends, where poverty trends aimed at the goal of poverty by 2030 could range from the possible outcomes can differ ending extreme poverty by 2030. almost zero to nearly 20%.636 significantly, are described through a global risk perspective. Reaching 9% in 2020 would mean an estimated 690m people would For each of the trends the simple still be living in extreme poverty by rule based on a risk perspective is: then, 510m fewer in poverty than a “Aim for the best, but be prepared decade earlier. That would be the for the worst”. equivalent of half the population of Africa, or more than double the population of Indonesia.633

Figure 25: Russian Male Life Expectancy635 Figure 26: Prospects for Ending Extreme Poverty by 2030.637

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 171 6.1 Poverty

Even more complicated is the Understanding the relationship The conclusion is that climate definition of poverty. The level between poverty and global strategies should prepare for of $1.25 per day is a very rough challenges requires us to develop successful poverty reduction by indicator and does not say very much strategies that help ensure poverty setting targets and developing about the situation of the people reduction. Planning means it is solutions that work in a world with low living life at that level. How desperate important to assume different levels poverty, and should not assume high are they (important for war/terrorism)? of poverty reduction - for example: levels of poverty. At the same time What risks do they feel they must For successful poverty reduction, our strategies for pandemics must take (important for climate change low-carbon solutions are crucial, as assume that poverty reduction could and ecological collapse as people it is rich people who are the main fail and develop solutions accordingly. will engage in illegal deforestation)? emitters on the planet because of Poverty is also important in their unsustainable lifestyles. pandemics, as reduced income can If poverty reduction is unsuccessful, result in increased migration and also structures to address a higher increased hunting of wild animals. likelihood of outbreaks that can turn into pandemics are required as poor people tend to live in societies where they are more likely to get infected and where often even basic health service is lacking.

Number of people in poverty, m 1990 2010 2030

China India Sub-Saharan Africa

$ 1.25 $ 1.25 $ 1.25 30 30 30

25 25 25

20 20 20

15 15 15

10 10 10

5 5 5

0 0 0 0 1 2 3 4 5 6 0 1 2 3 4 5 6 0 1 2 3 4 5 6

Mean daily consumption (PPP $) Mean daily consumption (PPP $) Mean daily consumption (PPP $)

Figure 27: Different kinds of poverty - Number of people in poverty638

172 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 6.2 Population growth

6.2 Population The high-variant projection depicted The fact that projections can change in the figure below assumes an extra is clearly demonstrated by the growth half a child per woman (on average) difference between the current (2012) compared with the medium variant, Revision, and the 2010 Revision of Population growth is a trend that implying a world population of 10.9 World Population Prospects. The has been discussed intensively bn in 2050 and 16.6 bn in 2100.645 latter was published only two years from a sustainability and risk That is equal to a 133% population earlier and projected world population perspective since Malthus did his increase in just 86 years . reaching 9.3 bn in 2050 and 10.1 bn famous projection.639 in 2100 (medium variant).647 This is The difference between projections almost a 10% difference in the space A “natural population increase” for 2100, from 10.9 bn people in of two years. occurs when the birth rate is the medium scenario, to 16.6 bn in higher than the death rate. While the high scenario, equals the world Current ways to provide the lifestyles a country’s population growth rate population in 1995. There is also a enjoyed in countries like the UK depends on this natural increase credible low scenario with 6.8 bn and US today would require 3.5-5 and on migration, global population by 2100.646 A strategic approach planets, while the global population growth is determined exclusively by must be based on all possible is about 7 bn people.648 Under the natural increase.640 outcomes. Planning as though the high-variant projection, more than 10 world population will be only 6.8 bn planets would be needed. Around the world, death rates is not optimistic: it is unscientific gradually decreased in the late 19th and dangerous. Even to plan for a With other underlying trends, and the 20th centuries, with those world with 10.9 bn is not strategic technology breakthroughs and in developing countries plummeting as this would ignore the significant institutional changes can result after World War II thanks to the spread probability that the world’s population in very rapid changes. Global of modern medicine which allowed would be much larger. There should population growth cannot change control of infectious diseases.641 be a plan for a world with 16.6 bn as rapidly. And as it is related people, combined with a long-term to many factors, including other According to the 2012 Revision of strategy to ensure a sustainable underlying trends such as income the official United Nations population population level. levels, education, access to health estimates and projections, the world services and cultural values that population of 7.2 billion in mid-2013 It is also important to ensure that are all assumed to be undergoing is projected to increase by almost more attention is paid to early significant changes over coming one billion people within twelve warning systems that allow us to decades, population growth over years, reaching 8.1 bn in 2025, and to influence population development in long time periods is even more further increase to 9.6 bn in 2050 and a sustainable direction. difficult to estimate. 10.9 bn by 2100.642

These results are based on the medium-variant projection, which assumes a decline of fertility for countries where large families are still prevalent as well as a slight increase of fertility in several countries with fewer than two children per woman on average.643

The medium projection is still dramatic as it assumes another four bn people on the planet, more than a 50% increase in population, equal to the Earth’s entire population in 1975, in just 86 years.644 Figure 28: Population of the world, 1950-2100, according to different projections and variants Source: http://esa.un.org/wpp/documentation/pdf/WPP2012_Volume-I_Comprehensive-Tables.pdf, p. xv

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 173 6.3 Technological development

6.3 Technological On the other hand there are natural forever. Second, nature itself may limits that could begin to constrain set limits. We may choose to take development technological development in more care of the planet, or limits to two ways. The technology itself materials like rare earths may begin Technological development since the may hit a barrier. For example, at to slow technology.655 But regardless industrial revolution has been faster some stage a processor may not of ultimate limits, many exponential than most experts expected. For continue to become smaller and trends are likely to continue over the material welfare this has been very faster, as the speed of light and coming decades and will present us positive; average longevity and health quantum mechanics will limit its with new opportunities as well as improvements in general have all development.654 There might be other risks in the 21st century, as these shown dramatic positive development. ways to overcome such boundaries, trends converge in a society with During the second half of the 20th but no exponential trend can last 20th century institutions. century global health improved more than in all previous human history. Average life expectancy at birth in low- and middle-income countries increased from 40 years in 1950 to 65 years in 1998.649

While weapons have become more deadly the death toll from wars has actually decreased over time.650 How big a part technology has played by creating greater transparency, or increasing the fear of using weapons which have become too powerful (for example nuclear bombs), is disputed. But most experts agree that technology has played an important 651 role. This is not the same as saying Figure 29: Moore’s forecast for PV656 that this development will continue. Figure 30: Global ICT development 2000-2013657

Estimating the future development of technology is very difficult. On 100 the one hand there is evidence that 95.5 technology will continue to accelerate 90 mobile-cellular telephone subscriptions individuals usign the Internet at the pace it has achieved so far. 80 fixed-telephone subscriptions Researchers at MIT and the Santa Fe Institute have found that some widely 70 used formulas for predicting how rapidly technology will advance — 60 notably, Moore’s and Wright’s Laws 50 — offer superior approximations of the pace of technological progress.652 40 40.4 Experts like Ray Kurzweil, who was recently hired by Google, is one of 30 those who think that most people 20 do not understand the implications 15.8 of exponential growth in the area 10 of technology and the results it generates in price, capacity and Per 100 inhabitants 0 2000 2002 2004 2006 2008 2010 2012 2014 overall transformation of society.653 Global ITC developments, 2000-2014*

174 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 6.3 Technological development / 6.4 Demographic changes

How technological development can 6.4 Demographic In 2000 there were an estimated be supported in order to increase 180,000 centenarians globally. By opportunities and reduce risks will be changes 2050 they are projected to number increasingly important to discuss. 3.2 m, an increase of about eighteen The world’s population is undergoing times.659 As technology in many areas is a massive demographic shift. Fertility developing exponentially, it is rates have fallen and the number of Within the more developed regions, important to analyse its development children has stopped growing. It is a Japan, in particular, will experience very carefully. The potential for historic shift. a remarkable increase in the number technology to help solve existing and of centenarians over the next half future global challenges is almost Those who are 80 or more now make century, from fewer than 13,000 limitless. And so unfortunately is its up only slightly more than 1% of the in 2000 to almost 1 m in 2050. By potential to accelerate existing risks total human population. This proportion then Japan is expected to have by and create new ones. Too many is projected to increase almost fourfold far the world’s largest number and initiatives today focus on only one over the next 50 years, to reach 4.1% proportion of centenarians, nearly side of technology, either the positive in 2050. Currently, only one country, 1% of its population.660 or the negative. Acknowledging both Sweden, has more than 5% in this sides is necessary in order to ensure age group. By 2050 the over-80s are The stagnating and ageing a strategic response. projected to number almost 379 m population in many OECD countries people globally, about 5.5 times as and China will put pressure on many as in 2000 (69 m). In 1950, the current systems, which were not over-80s numbered under 14 m. designed to deal with a situation of ageing and often shrinking Although the proportion of people who populations in many parts of the live beyond 100 is still very small, their world, while the populations in other number is growing rapidly. parts of the world are rapidly growing.

2050 379.0

2025 153.4

2000 69.2

1975 31.4

1950 13.8

population aged 80 or over: World, 1950-2050 millions

Figure 24: Population aged 80 or over661

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 175 7. Possible ways forward

7. Possible ways forward

“Our very survival depends on our ability to stay awake, to adjust to new ideas, to remain vigilant and to face the challenge of change.”

Martin Luther King Jr.

176 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 7. Possible ways forward

To better address global challenges with a potentially infinite impact, Global challenges both immediate action and long- leadership networks term work are needed. Below are 1. ten areas that could help mitigate immediate threats while also Better quality risk contributing to a future global assessment for governance system capable of 2.

addressing global challenges with global challenges infinite impacts. For all these areas more research is needed. Development of 3. early warning systems Encouraging 4. visualisation of complex systems

Highlighting 5. early movers Including the whole 6. probability distribution Increasing the focus on 7. the probability of extreme events

Encouraging 8. appropriate language to describe extreme risks

Establishing a Global Risk and 9. Opportunity Indicator to guide governance

Explore the possibility of establishing a 10. Global Risk Organisation (GRO)

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 177 7. Possible ways forward

Global challenges 1. leadership networks

The long-term goal Betterneeds to qualitybe riskinfinite impacts, and could work Four groups are of particular the2. establishment ofassessment a transparent for on a roadmap for a future global importance: experts in finance, and democratic globalglobal governance challenges governance system that can address experts in security policy, lawyers system that can address global existing and new global challenges. with knowledge of global risks and challenges with infiniteDevelopment impacts. To of international law, and finally a group support this leadership, networks can The networks should be as consisting of clusters of stakeholders early warning systems be3. established involving interested transparent and inclusive as possible, with solutions that can reduce the governments, major companies, especially as global collaboration is risks. Leadership networks that NGOs, researchers Encouragingand other relevant needed. The use of new collaboration include participants from all four stakeholders.4. Thesevisualisation networks could of tools and principles, such as wiki- groups are of particular interest. develop strategies tocomplex address multiple systems processes and creative commons, challenges with potentiallyGlobal challengesshould be encouraged. 1. Highlightingleadership networks early movers 5. Better quality risk 2. Includingassessment the for whole global challenges 6. probability distribution

There is currently noDevelopmentIncreasing global of methodology development could Institutions and universities coordination3. when earlytheit comes focus warning on systemsbe accelerated and improved, as engaged in developing new to7. risk assessmentsthe of global probability the possibility of learning from methodologies to assess global challenges. DifferentEncouragingof expertsextreme use events different areas would increase. Such risks have a particular responsibility different4. methodologies,visualisation data of a process could also encourage for developing and refining risk and ways to presentcomplexEncouragingGlobal their results, challenges systems increased investments in methodology assessments for global challenges. making1. it very difficultappropriateleadership to compare networks languagedevelopment based on the latest the8. risk assessments that exist. innovations, such as systems- Highlightingto describe extreme risks By establishing a process that forecasting approaches. earlyBetter movers quality risk coordinates5. and encouragesassessment risk for assessments2. of globalEstablishing challenges, 9. Includingaglobal Global challenges Risk the wholeand probabilityOpportunity distribution Indicator 6. Developmentto guide governance of 3. Increasingearly warning systems theExplore focus the on possibility 7. 662 The rapid technologicaltheEncouragingof establishingdevelopment probability a “big data”. These opportunities be assumed to increase or decrease has14. 0.many benefits,ofvisualisationGlobal but extreme also Risk eventsOrganisation of include both (GRO) new ways of collecting global risks. Such a system would challenges, as riskscomplex can rapidly systemslarge amounts of high-quality data, allow more time to develop strategies become very seriousEncouraging and reach infinite and new ways to analyse them. to mitigate risks and turn global thresholds. To developappropriateHighlighting early warning language challenges into opportunities for systems8. that can gather and process Early warning systems should be innovation and collaboration. toearly describe movers extreme risks data5. transparently is therefore of the built to ensure that data is collected utmost importance. Technological and analysed in ways that can be The warning system would require progress, from smartEstablishingIncluding phones and the wholeuseful for multiple global challenges. significant research into infinitey probability distribution sensors6.9. to significanta Global processing Risk andThe warnings should not only include thresholds. Both traditional as well as power and networks,Opportunity allows for totally Indicator changes in the physical world, more recent methodologies based on new ways of establishingtoIncreasing guide early warning governance but also indicate when decisions, understanding of complex systems systems7. based on theso-called focus on investments and legal changes can should be encouraged. Explorethe probability the possibility of establishingextreme events a 10. Global Risk Organisation (GRO) Encouraging 1788. appropriate languageGlobal Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks to describe extreme risks

Establishing a Global Risk and 9. Opportunity Indicator to guide governance

Explore the possibility of establishing a 10. Global Risk Organisation (GRO) Global challenges 1. leadership networks Better quality risk 2. assessment for global challenges 7. Possible ways forward

Development of 3. early warning systems Encouraging 4. visualisation of complex systems Global challenges leadership networks 1.The global challengesHighlighting depend on New visualisation tools could a very complex ecosystemearly movers and help make complex systems easier social5. system. Better quality risk to understand and also help the assessmentIncluding the for whole communication of challenges 2. 663

With a global economicglobalprobability and challenges distributionand opportunities. technological6. system, which both helps and creates DevelopmentrisksIncreasing that are of Visualisation tools are needed both increasingly interconnected and for decision makers to highlight the earlythe focus warning on systems difficult3.7. to understand, there is a consequences of different strategies challenge to understandthe probability and for citizens to increase their basic the challenges. Encouragingof extreme events understanding of infinite impacts. 4. visualisation of complexGlobalEncouraging challenges systems 1.8. leadershipappropriate networks language Highlightingto describe extreme risks Better quality risk 5. early movers 2. assessmentEstablishing for Includingglobala Global challenges Riskthe whole and Governments,9. companies, In particular, leadership with a focus probabilityOpportunity distribution Indicator organisations6. and networksDevelopment working of on multiple global challenges and the on global challengesto should guide governancerelationship between them should be early warning systems increase3. their effortsIncreasing to reward highlighted, as very little is being done leadership when theytheExplore find focus it. the Majoron possibility in this area. news7. outlets can alsotheEncouragingof establishing reportprobability when a significant14.0. positiveofvisualisationGlobal steps extreme are Risk taken eventsOrganisation of (GRO) to reduce global riskscomplex with potential systems infinite impacts. Encouraging 8. appropriateHighlighting language 5. toearly describe movers extreme risks EstablishingIncluding the whole aprobability Global Risk distribution and 9.6. Opportunity Indicator Governments, majortoIncreasing companies, guide governance Current lack of data and of scientific Tables, graphs and key conclusions NGOs,7. researchersthe and focusother on studies regarding low-probability in reports related to global challenges relevant stakeholdersExplorethe should probability addressthe possibility high-impact outcomes in many areas should, when possible, include the the whole probabilityofof distribution, establishingextreme events a should not be used as an excuse to whole probability distribution.664 1including0. low-probabilityGlobal high- Risk Organisationignore the (GRO) probability distribution. This impact scenarios. ThisEncouraging would ensure is especially important when many of that serious risks areappropriate not disregarded language the global challenges have a very long or8. obscured. and fat “tail”. to describe extreme risks

Establishing a Global Risk and 9. Opportunity Indicator to guide governance

Explore the possibility of establishing a Global10. Challenges – TwelveGlobal risks thatRisk threaten Organisation human civilisation (GRO) – The case for a new category of risks 179 Global challenges 1. leadership networks Better quality risk 2. assessment for global challenges

Development of 3. early warning systems Encouraging 4. visualisation of complex systems Global challenges 1. leadershipHighlighting networks 7. Possible ways forwardearly movers 5. Better quality risk 2. assessmentIncluding the for whole 6. globalprobability challenges distribution DevelopmentIncreasing of earlythe focus warning on systems 3.7. the probability Encouragingof extreme events 4. visualisation of When the impact iscomplex Encouraginginfinite it is notsystems Stakeholders should include the most The use of methodologies and enough8. only to revealappropriate the whole languageextreme impacts in all relevant work. approaches from security policy probability distribution.Highlightingto Itdescribe is important extreme If the risksprobability of infinite impacts and the financial sector that focus also to avoid confusingearly uncertain movers risk increases instead of decreasing on extreme events could be used to with5. low risk. Establishing because of new scientific findings or develop strategies for rapid action Includinga Global Riskthe whole andlack of action, strategies should be beyond the incremental approaches Infinite9. impacts render many of the prepared to allow more decisive action. that dominate today. probabilityOpportunity distribution Indicator traditional6. models forto risk guide management governance almost meaningless. Monetary calculations are oftenIncreasing useless, and discounting is not alwaystheExplore focus advisable. the on possibility 7. theof establishing probability a 10. ofGlobal extreme Risk eventsOrganisation (GRO)

Encouraging 8. appropriate language to describe extreme risks

The IPCC uses specific and defined Average: 2-9 dead and/or 10-49 Often words like “unlikely”,Establishing “negligible” language in its reports to describe seriously injured and9. “insignificant” aare Global used to describeRisk and different probabilities and thus ensure a risk when the probabilityOpportunity is considered Indicator clarity, but taken out of context and Large: 10-49 dead and/or 50-100 low. What is low is howeverto guide relative; governance without supporting definitions this seriously injured a low probability in one area can be language can be misleading. extremely high in another.Explore If I attend the onepossibility Very large: >50 dead and/or >100 of ten lectures - 10%of - peopleestablishing might say a For example, the term “very unlikely” seriously injured 1there0. is a low probabilityGlobal that RiskI will be Organisation is used by (GRO) the IPCC to describe a there. But if someone says that a new probability of between 0-10%,665 but The use of terms that can be aircraft crashes once in every ten flights, out of context its use could easily be interpreted as having normative most people will say that is an extremely understood as a normative judgement values to explain probability is high probability and will be likely to suggesting that we do not need to problematic and in future all bodies, assume it is an early prototype that is engage with the risk. including the IPCC, should explore nowhere close to commercial success. the possibility of using only numbers The language of the IPCC can be in external communications, at least A major problem is that probabilities compared with that used in the in the summary for policy makers, to that ought to be seen as very high for Swedish National Risk Assessment help everyone understand the reality risks with potentially infinite impact are (SNRA).666 The scale of impact is of the situation. described in a way that makes them not defined for the IPCC, but for the sound less urgent than they are - by the Swedish Assessment it is: Stakeholders should explore media, business, politicians and even ways to use language that better by scientists. Very small: no deaths or communicates how serious extreme serious injuries risks are in the case of climate One example is how probabilities are change, and where possible compare described by the Intergovernmental Small: One dead and/or 1-9 this with other risk areas to help Panel on Climate Change. seriously injured illustrate the situation.

180 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 7. Possible ways forward

IPCC Term SNRA Term Likelihood of the Likelihood of the Outcome: IPCC Outcome: SNRA

Very certainly Very high 99-100% probability > 20% [more than once every 5 years]

Very likely 90-100% probability

Likely 66-100% probability

About as likely as not 33-66% probability

Unlikely

High 20%-2% [Between once every 5 years and once every 50 years]

Very unlikely Average 0-10% probability 2%-0.02% [Between once every 50 years and once every 500 years]

Exceptionally unlikely 0-1% probability

Low 0.02%-0.002% [Between once every 500 years and once every 5000 years]

Very low < 0.002% [Less than once every 5000 years]

Figure 31: Comparing the probability scale in the Swedish National Risk Assessment667 and the Likelihood Scale used by the IPCC668

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 181 Global challenges 1. leadership networks Better quality risk 2. assessment for global challenges

Development of 3. early warning systems Encouraging 4. visualisation of complex systems

GlobalHighlighting challenges 1.5. leadershipearly movers networks BetterIncluding quality the wholerisk 2.6. assessmentprobability distribution for global challenges Increasing 7. Developmentthe focus on of earlythe probability warning systems 3. of extreme events 7. Possible ways forwardEncouraging visualisationEncouraging of 4. appropriate language 8. complex systems to describe extreme risks Highlighting Establishing early movers 5. a Global Risk and 9. Opportunity Indicator Includingto guide governancethe whole 6. probability distribution No mechanisms currentlyIncreasingExplore exist the to possibilitycompanies, influence the probability has responsibility for increasing and provide10. updated andtheof comprehensive establishing focus on a of different impacts and outcomes. decreasing the risk; who will suffer its global7. risk assessmentstheGlobal probability for Risk Organisation (GRO) consequences, and who will benefit. phenomena capable of threatening Establishing a global risk indicator, of extreme events human civilisation. with sub-indicators for different areas, Stakeholders should explore the as part of the UN system would help establishment of a global risk With many unsustainableEncouraging trends create a better understanding of indicator that will help guide priorities converging,8. it is crucialappropriate that leaders language extreme global risks individually and and inform society about different are able to act beforeto itdescribe is too late extremeof their risks interconnection, and it should risks, and about the relationship and to assess how actions, such as track both. An important feature between them. political decisions orEstablishing investments by would be its ability to illustrate who a Global Risk and 9. Opportunity Indicator to guide governance

Explore the possibility of establishing a 10. Global Risk Organisation (GRO)

There is currently no international or In addition, and probably equally global body that is coordinating work important, is the fact that a body set up on global risks with a potentially infinite to deal with such challenges could also impact. The following areas would ensure that the links between them benefit from global coordination: could be better understood.

– Probability estimations A first step could be to establish – Early warning systems, a centre for global risks and – Global coordination of solutions opportunities,669 focusing initially – Legal development only on knowledge-gathering and development of proposals, and with no mandate to implement any solutions.

182 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 7. Possible ways forward

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 183 Endnotes

Endnotes

1 http://www.cnbc.com/id/100915696 Ridley cherrypicks data and tends to avoid the 27 E.g. HBR blog about “black swans”: http://blogs. 2 http://en.wikipedia.org/wiki/Global_catastrophic_risks probability that we will see significant warming. hbr.org/2010/09/the-competing-black-swans-of-s/ 3 http://www.merriam-webster.com/dictionary/risk http://www.mattridley.co.uk/blog/the-probable-net- and The Guardian about “perfect storms” http://www. benefits-of-climate-change-till-2080.aspxhttp://www. theguardian.com/environment/earth-insight/2014/ 4 http://en.wikipedia.org/wiki/Probability_density_ rationaloptimist.com/ apr/03/ipcc-un-climate-change-perfect-storm-zombie- function 15 Climate Impacts: http://www.ipcc.ch/report/ar5/wg2/ oil 5 Dickson, David CM. Insurance risk and ruin. 28 http://www.economist.com/node/18744401 Cambridge University Press 2005. Pandemics: http://www.cmu.edu/dietrich/sds/docs/ fischhoff/AF-GPH.pdf 29 http://e360.yale.edu/feature/living_in_the_ http://books.google.se/ anthropocene_toward_a_new_global_ethos/2363/ books?id=9zPOMMqJXcUC&printsec Nuclear war: http://www.ippnw.org/nuclear-famine.html 16 30 http://www.nickbostrom.com/existential/risks.html =frontcover&source=gbs_ge_summary_r&redir_ http://news.stanford.edu/news/2003/september24/ 31 esc=y#v=onepage&q&f=false tellerobit-924.html https://www.youtube.com/watch?v=VjZyOTES6iQ 17 32 6 http://www.itl.nist.gov/div898/handbook/eda/ http://www.fas.org/sgp/othergov/doe/lanl/ http://en.wikipedia.org/wiki/Long_tail 33 section3/eda352.htm docs1/00329010.pdf http://en.wikipedia.org/wiki/Human 18 34 7 A recent example is IPCC WGII Summary for policy While the LA-602 document is relatively well-known, Mammals have an average species lifespan from makers. In this report http://ipcc-wg2.gov/AR5/ IPCC its final paragraph is often forgotten. The scientists origin to extinction of about 1 million years, but that acknowledges the need to include low-probability high- say: “However, the complexity of the argument and the will not necessarily apply to humans today because of impact scenarios: “assessment of the widest possible absence of satisfactory experimental foundations makes factors like local climate change or new species in the range of potential impacts, including low-probability further work on the subject highly desirable.” same ecological niche. The dinosaurs were around for 19 outcomes with large consequences, is central to Since Teller’s calculations the science has developed 135 million years and if we are intelligent, there are good understanding the benefits and trade-offs of alternative and is now clear that the main nuclear threat that chances that we could live for much longer. risk management actions.” Yet nothing is included in the potentially could threaten human civilisation is a full 35 http://en.wikipedia.org/wiki/Human_embryogenesis report about impacts above 4 degrees. nuclar war and the consequenses that this could result in 36 These four points are slightly rewritten versions of a 8 As an infinite impact by definition can’t have from effects like a nuclear winter. list of Nick Bostrom’s in the text “Existential Risks: 20 happened before, models are needed. For an overview of positive and negative aspects Analyzing Human Extinction Scenarios and Related 9 Making Sense of Uncertainty: Why uncertainty is part of nanotechnology see for example Drexler, Eric and Hazards” http://www.nickbostrom.com/existential/risks. of science http://www.lse.ac.uk/CATS/Media/SAS012- Pamlin, Dennis (2013): “Nano-solutions for the 21st html. Note that these four points were originally century. Unleashing the fourth technological revolution” MakingSenseofUncertainty.pdf developed for “existential risks”, those that threaten the http://www.oxfordmartin.ox.ac.uk/downloads/ 10 Pinker, Steven. The better angels of our nature: The extinction of intelligent life originating on Earth or the academic/201310Nano_Solutions.pdf decline of violence in history and its causes. Penguin UK, permanent destruction of its potential for desirable future 21 2011. This is true for utopias ranging from Plato’s Republic, development. via Thomas More’s book Utopia, to Edward Bellamy’s 11 Giddens, Anthony. “Risk and responsibility.” The 37 Posner, Richard A. Catastrophe: Risk and Response. Looking Backward and William Morris’ News from modern law review 62.1 (1999): 1-10. Oxford University Press, 2004, Nowhere. 12 http://blogs.ei.columbia.edu/2012/01/09/evolutionary- 38 http://en.wikipedia.org/wiki/Value_of_life 22 See next chapter for the methodology. psychology-of-climate-change/ 39 http://yosemite.epa.gov/EE%5Cepa%5Ceed.nsf/ 23 A list of organisations and studies that discuss http://www-ee.stanford.edu/~hellman/Breakthrough/ webpages/MortalityRiskValuation.html challenges that threaten human civilisation can be found book/chapters/frankenhaeuser.html 40 Posner, Richard A. Catastrophe: risk and response. here: http://en.wikipedia.org/wiki/Global_catastrophic_ “The challenge now is to help people extend their Oxford University Press, 2004. Loc 2363 risks attachments, their loyalties, and their engagement, to 41 http://webarchive.nationalarchives.gov.uk/+/http:/ 24 The number two billion was established during the include people outside their own narrow circle, their www.hm-treasury.gov.uk/sternreview_index.htm workshop in Oxford and is not an exact number. Further own country, their own imminent future. This global 42 research is needed to establish a better understanding William Nordhaus , The Stern Review on the reorientation is a prerequisite for changing the present of thresholds that can result in an infinite impact, Economics of Climate Change http://www.econ.yale. fatal course of development.” edu/~nordhaus/homepage/stern_050307.pdf 13 depending on what challenge resulted in the two billion http://www-ee.stanford.edu/~hellman/Breakthrough/ 43 impact and how the estimate for an infinite impact was http://www.res.org.uk/view/art3Apr08Features.html book/chapters/frankenhaeuser.html assumed to be between 0.01% and 10%. 44 Nordhaus, William D. The Climate Casino: Risk, 14 Two of the most famous “optimists”, who tend to 25 The definition is based on the definition used by Uncertainty, and Economics for a Warming World. Yale look at only the parts of the probability distribution Jared Diamond: http://www.jareddiamond.org/Jared_ University Press, 2013. Loc 2895 that support their opinion, are the Danish writer Bjorn Diamond/Collapse.html 45 Nordhaus, William D. The Climate Casino: Risk, Lomborg and the British journalist Matt Ridley. While 26 For examples see the recently established Centre Uncertainty, and Economics for a Warming World. Yale scientists in the areas he writes about constantly refute for Study of Existential Risk at Cambridge. It is an University Press, 2013. Loc 2895 Lomborg, his message of optimism is well received by interdisciplinary research centre focused on the study 46 Nordhaus, William D. The Climate Casino: Risk, many policy makers and business leaders. https://www. of human extinction-level risks that may emerge from Uncertainty, and Economics for a Warming World. Yale ma.utexas.edu/users/davis/375/reading/sciam.pdf technological advance University Press, 2013. Loc 3176

184 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Endnotes

47 Timothy M. Lenton, Juan-Carlos Ciscar: Integrating criteria%20-%20when%20is%20low%20enough%20 http://gcrinstitute.org/research/ tipping points into climate impact assessments. Springer good%20enough%20-%20saudi.pdf Institute for Defence Studies and Analyses 64 Netherlands 2013-04-01 http://link.springer.com/ SO 16732-1, Fire Safety Engineering – Guidance http://www.idsa.in/topics article/10.1007%2Fs10584-012-0572-8 on Fire Risk Assessment, International Organization for International Risk Governance Council 48 Carl Sagan use 10 million years in: Sagan, Carl Standardization, Geneva, Switzerland, 2012. http://www.irgc.org/issues/ (1983). “Nuclear war and climatic catastrophe: Some 65 An example of a concept that helped increase the Lifeboat Foundation policy implications”. Foreign Affairs 62: 275, and The focus on an important category is Genocide, which was http://lifeboat.com/ex/programs Blackwell’s Concise Encyclopedia of Ecology and other introduced by Raphael Lemkin. http://en.wikipedia.org/ sources provide an estimate of about 1 million years wiki/Genocide Nuclear Threat Initiative for mammals. http://eu.wiley.com/WileyCDA/WileyTitle/ 66 E.g. Vinso, Joseph D. “A Determination of the Risk http://www.nti.org/ productCd-0632048727.html of Ruin.” Journal of Financial and Quantitative Analysis Saving Humanity from Homo Sapiens 49 One billion years is used by Bruce E. Tonn in the 14.01 (1979): 77-100. http://shfhs.org/whatarexrisks.html paper Obligations to future generations and acceptable 67 E.g. Reed, Christina. Earth Science: Decade by Skoll Global Threats Fund risks of human extinction: “ Futures 41.7 (2009): 427- Decade. Infobase Publishing, 2009. http://www.skollglobalthreats.org/ 435. 68 Culpitt, Ian. Social policy and risk. Sage, 1999. Stimson Center 50 In Japan in Japan the life expectancy at age zero, that 69 http://en.wikipedia.org/wiki/Jared_Diamond is, at birth (LEB), is 83 years. In 2010 the world LEB was http://www.stimson.org/programs/ 70 The methodology applied in this study has been 67.2. Risk Response Network developed by Dennis Pamlin from Global Challenges 51 This is based on a low estimate for the planet’s http://forumblog.org/communities/risk-response- Foundation and Stuart Armstrong, James Martin population as far out as current projections are made, network/ Research Fellow, Future of Humanity Institute at the 2100. http://esa.un.org/wpp/ Looking further into the World Economic Forum University of Oxford together with Seth Baum, Executive far future, beyond 500 years, it is likely that humanity Director of the Global Catastrophic Risk Institute and http://www.weforum.org/reports/global-risks-2014- will have expanded into space and have a much larger affiliate researcher at the Center for Research on report population. Environmental Decisions, Columbia University. Tower Watson 52 Posner, Richard A. Catastrophe: Risk and Response. 71 The literature overview is included as Appendix 1. http://www.towerswatson.com/en/Insights/IC-Types/ Oxford University Press, 2004, Loc 2378 72 Survey-Research-Results/2013/10/Extreme-risks-2013 53 http://gcrinstitute.org/people/ http://www.esa.int/Our_Activities/Space_Science/ 84 73 http://cred.columbia.edu/about-cred/people/ http://www.towerswatson.com/en/Insights/IC-Types/ Herschel/How_many_stars_are_there_in_the_Universe? Survey-Research-Results/2013/10/Extreme-risks-2013 54 affiliated-researchers/sethbaum/ http://www.existential-risk.org/concept.pdf 85 74 http://wokinfo.com/ Page 50 http://www3.weforum.org/docs/WEF_ 55 McSharry, Patrick E., and David Orrell (2009). A GlobalRisks_Report_2014.pdf 75 http://scholar.google.com Systems Approach to Forecasting. FORESIGHT: 86 Page 50 http://www3.weforum.org/docs/WEF_ 76 See Appendix 1 for the full overview. The International Journal of Applied Forecasting, GlobalRisks_Report_2014.pdf 77 referring to Wolfram, Stephen (2002):A new kind http://highlycited.com/ 87 http://www.weforum.org/reports/global-risks-2014- of science, V Wolfram media. http://www.irgc. 78 Specifically, publications selected met one or more report org/wp-content/uploads/2013/06/McSharry- of the following criteria, and were more likely to be 88 http://www.iea.org/newsroomandevents/ OrrellMcSharry2009foresight.pdf selected if they met multiple criteria: (1) published in a pressreleases/2011/november/name,20318,en.html 56 From A Systems Approach to Forecasting” by peer-reviewed journal or academic press; (2) authored http://www.ipcc.ch/pdf/assessment-report/ar4/wg3/ar4- David Orrell and Patrick McSharry, http://www. by scholars of established distinction; (3) published in wg3-chapter12.pdf irgc.org/wp-content/uploads/2013/06/McSharry- an authoritative popular media outlet; (4) authored by 89 OrrellMcSharry2009foresight.pdf journalists or other non-academic writers of established Wolfe, Nathan. The viral storm: the dawn of a new pandemic age. Macmillan, 2011. referring to Wolfram, S. (2002). A New Kind of Science. distinction; (5) highly cited by other scholars; (6) thorough 90 https://www.wolframscience.com/ in presentation and analysis. This includes any research that indicates the 79 probabilities are much lower than first believed. 57 A possible infinite threshold could be compared with http://www.existential-risk.org/concept.pdf 91 the two events that are often cited as the world’s worst 80 Climate change is a good example. Very little is done For a list of other overviews please see Annex 1 92 wars and anthropogenic disasters: the second world on the infinite impacts where global warming would See Podsiadlowski, Ph, et al. “The rates of war, with 40-71 million dead (1.7-3.1% of the world’s result in an average temperature rise of 6°C or more. hypernovae and gamma-ray bursts: implications for their population) and the Mongol Conquests, 1206-1368, with 81 The list of organisations is included as Appendix 2. progenitors.” The Astrophysical Journal Letters 607.1 30 million dead (7.5% of the global total). See: http:// 82 http://gcrinstitute.org/organization-directory/ (2004). 93 en.wikipedia.org/wiki/List_of_wars_and_anthropogenic_ 83 The 19 organisations are: See Coleman, Sidney, and Frank De Luccia. disasters_by_death_toll “Gravitational effects on and of vacuum decay.” Physical Brookings 58 The number two billion was established during Review D 21.12 (1980): 3305-3315. http://www.brookings.edu/research#topics/ workshops arranged during the process and is not an 94 See Iliopoulos, John, et al. “Study of potentially exact number. Further research is needed to establish a Bulletin of the Atomic Scientists dangerous events during heavy-ion collisions at the LHC: better understanding of thresholds that can result in an http://thebulletin.org/ report of the LHC safety study group.” Vol. 1. CERN infinite impact. The number of people dead is only one CSER (2003). http://194.109.159.7/cern/20101116222315/ factor and different global risks are likley to have very http://cser.org/emerging-risks-from-technology/ cdsweb.cern.ch/record/613175/files/p1.pdf different thresholds. Center for International Security and Cooperation 95 See for instance “Solar storm Risk to the North 59 HSE 2001. Reducing Risks, Protecting People – http://cisac.stanford.edu/ American electric grid“, Lloyds (2013). HSE’s Decision-Making Process, 2001. 96 Club of Rome See Bennett, Oliver. “Cultural pessimism: Narratives http://www.hse.gov.uk/risk/theory/r2p2.pdf of decline in the postmodern world.” Edinburgh http://www.clubofrome.org/ 60 Bedford, Paul, Millard, Stephen and Jing Yang, University Press (2001). Council on Foreign Relations Assessing operational risk in CHAPS Sterling: a 97 See Klare, Michael T. “Resource wars: The new http://www.cfr.org/issue/ simulation approach (2004). landscape of global conflict.” Macmillan (2001). Federation of American Scientists 61 SO 16732-1, Fire Safety Engineering – Guidance 98 See for instance the genocides listed in White, on Fire Risk Assessment, International Organization for http://www.fas.org/programs Matthew “Atrocitology: Humanity’s 100 Deadliest Standardization, Geneva, Switzerland, 2012. Future of Humanity Institute Achievements.” Canongate Books (2011). 62 http://en.wikipedia.org/wiki/ALARP http://www.fhi.ox.ac.uk/research/research-areas/ 99 See for instance the exponents on different power 63 http://www.risktec.co.uk/media/43520/risk%20 Global Catastrophic Risk Institute laws of natural and other disasters in Hanson, Robin.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 185 Endnotes

“Catastrophe, social collapse, and human extinction.” Nick Bostrom & Milan Cirkovic Eds.(Oxford University event.” Trends in Ecology & Evolution 18.7 (2003): 358- Global Catastrophic Risks 1 (2008). Press, 2008) Global Catastrophic Risks. 365. 100 See the damage and impacts listed in Carr, Jeffrey. Martin Rees, “Our final century”, Random House, 2003. 130 See Shepherd, J. G.: “Geoengineering the climate: “Inside cyber warfare: Mapping the cyber underworld.” Martin, James, 2007. The Meaning of the 21st Century. science, governance and uncertainty.” Royal Society, O’Reilly Media, Inc. (2011). http://www.amazon. New York: Riverhead Penguin. (2009). co.uk/Inside-Cyber-Warfare-Mapping-Underworld/ Posner, Richard, 2004. Catastrophe: Risk and Response. 131 See Robock, Alan, et al: “Benefits, risks, and costs of dp/1449310044 Oxford: Oxford University Press. stratospheric geoengineering.” Geophysical Research 101 See the medium estimate in United Nations. Dept. 112 See appendix 2 for participants at workshops Letters 36.19 (2009). 132 of Economic and Social Affairs. “World Population 113 http://en.wikipedia.org/wiki/Climate_change See Brovkin, Victor, et al.: “Geoengineering climate by to 2300.” New York: United Nations, Department of 114 See the IPCC’s Fifth Assessment Report. stratospheric sulfur injections: Earth system vulnerability Economic and Social Affairs, (2004). https://www. to technological failure.” Climatic Change 92.3-4 (2009): 115 See Chateld, Chris. “Model uncertainty, data mining un.org/esa/population/publications/longrange2/ 243-259. and statistical inference.” JR Statist. Soc. A 158 (1995): WorldPop2300final.pdf 133 419-466, and for climate change models specifically, See Baum, Seth D., Timothy M. Maher Jr, and 102 See the article “Delayed biological recovery from see Pindyck, Robert S. “Climate Change Policy: What Jacob Haqq-Misra.: “Double catastrophe: intermittent extinctions throughout the fossil record” by James W. do the models tell us?“ No. w19244. National Bureau of stratospheric geoengineering induced by societal Kirchner & Anne Weil in Nature: http://seismo.berkeley. Economic Research (2013). collapse.” Environment Systems & Decisions (2013): edu/~kirchner/reprints/2000_34_delayed_recovery.pdf 1-13. 116 See Stern, Nicholas. “The structure of economic 103 Attractor: a set of physical properties towards which a 134 modeling of the potential impacts of climate change: http://www.nature.com/ncomms/2014/140225/ system tends to evolve. Strange attractor: a complicated grafting gross underestimation of risk onto already ncomms4304/full/ncomms4304.html set with a fractal structure. http://en.wikipedia.org/wiki/ narrow science models.” Journal of Economic Literature http://www.geomar.de/en/news/article/climate- Attractor 51.3 (2013): 838-859. engineering-geringes-potential-grosse-nebenwirkungen/ http://www.stsci.edu/~lbradley/seminar/attractors.html 135 117 See Weitzman, Martin L. “Tail-Hedge Discounting See Tetlock, Philip E.: “Expert political judgment: How 104 Folke, C., S. R. Carpenter, B. Walker, M. Scheffer, and the Social Cost of Carbon.” Journal of Economic good is it? How can we know?” Princeton University T. Chapin, and J. Rockström. 2010. Resilience Literature 51.3 (2013): 873-882. Press (2005). thinking: integrating resilience, adaptability and 136 118 See Bodman, Roger W., Peter J. Rayner, and David J. See the Carbon Tracker Initiative report: “Unburnable transformability. Ecology and Society 15(4): 20. [online] Karoly. “Uncertainty in temperature projections reduced carbon 2013: Wasted capital and stranded assets.” URL: http://www.ecologyandsociety.org/vol15/iss4/ using carbon cycle and climate observations.” Nature 137 These figures are broadly in line with those of art20/ Climate Change (2013). Meinshausen, Malte, et al.: “Greenhouse-gas emission 105 We define use Jared Diamond's definition of 119 Some of the severe consequences of this are targets for limiting global warming to 2°C.” Nature collapse for ”end of civilisation”, but focus only on explored in the World Bank report “Turn Down The Heat: 458.7242 (2009): 1158-1162. global impacts, so the definiton is ”A drastic decrease in why a 4°C warmer world must be avoided“ (2012). 138 See for instance the Scientific American report: “400 human population size and/or political/economic/social 120 See Lynas, Mark. “Six degrees: Our future on a hotter PPM: Carbon Dioxide in the Atmosphere Reaches complexity, globally for an extended time.” http://www. planet.” HarperCollins UK (2008), also summarised by Prehistoric Levels.” jareddiamond.org/Jared_Diamond/Collapse.html its author in a Guardian article “Six steps to hell,” and 139 See Lüthi, Dieter, et al.: “High-resolution carbon 106 See Korhonen, Jouni, and Thomas P. Seager. Schneider, Stephen. “The worst-case scenario.” Nature dioxide concentration record 650,000–800,000 years “Beyond eco‐efficiency: a resilience perspective.” 458.7242 (2009): 1104-1105. before present.” Nature 453.7193 (2008): 379-382. Business Strategy and the Environment 17.7 (2008): 411- 121 See Zimov, Sergey A., Edward AG Schuur, and F. 140 See the IPCC’s Fourth Assessment Report. 419. Stuart Chapin III. “Permafrost and the global carbon 141 See the Independent report “China agrees to impose 107 Maher, Timothy M., and Seth D. Baum. “Adaptation budget.” Science 312.5780 (2006): 1612-1613. carbon targets by 2016.” to and recovery from global catastrophe.” Sustainability 122 See Malhi, Yadvinder, et al. “Exploring the likelihood 142 5.4 (2013): 1461-1479. See the UN Millennium Development Goal database and mechanism of a climate-change-induced dieback “Carbon dioxide emissions (CO2), thousand metric tons 108 Such as the Svalbard Global Seed Vault. of the Amazon rainforest.” Proceedings of the National of CO2 (CDIAC).” 109 The fall of the Roman Empire, for instance, resulted Academy of Sciences 106.49 (2009): 20610-20615. 143 See the announcement by the Chinese Bureau of in the loss of many technologies and techniques: 123 There are arguments that climate change is Meteorology: “National Low Carbon Day logo and Aiyar, Shekhar, Carl-Johan Dalgaard, and Omer Moav. structurally unsuited to treaty-based emissions curbs; slogan officially announced.” “Technological progress and regress in pre-industrial see Barrett, Scott. “Climate treaties and approaching 144 times.” Journal of Economic Growth 13.2 (2008): 125- See the Warsaw Climate Change Conference Press catastrophes.” Journal of Environmental Economics and 144. Release “UN Climate Change Conference in Warsaw Management (2013). 110 keeps governments on a track towards 2015 climate See Ćirković, Milan M., Anders Sandberg, and Nick 124 The World Bank has estimated that the cost of agreement.” Bostrom. “Anthropic shadow: Observation selection adapting to 2°C climate change would be of the order of 145 effects and human extinction risks.” Risk analysis 30.10 See the UNFCCC’s “Private Sector Initiative - $70-100 billion a year (source: Economics of Adaptation (2010): 1495-1506. database of actions on adaptation.” to Climate Change, World Bank). 146 111 Organisations working with global challenges See Buchanan, James M.: “Externality.” Economica 125 See Lynas, Mark. “Six degrees: Our future on a hotter included in the process where challenges were selected. 29.116 (1962): 371-384. planet.” HarperCollins UK (2008), also summarised by its Below: example of organisations where information was 147 See the IPCC’s Fifth Assessment Report. author in a Guardian article “Six steps to hell.” gathered from, in alphabetical order: 148 See the Global Challenge Foundation’s explanation 126 See for instance the open letter from the Partnership http://cass.cssn.cn/ “About the Global Risk and Opportunity Indicator.” for a Secure America: “Thirty-Eight Leading U.S. National 149 See the Warsaw Climate Change Conference Press http://www.cfr.org Security Experts Urge Action on International Climate Release “UN Climate Change Conference in Warsaw http://cser.org Change Initiatives“ (2013). keeps governments on a track towards 2015 climate http://www.thebulletin.org 127 See Weiss, Harvey, and Raymond Bradley; “What agreement.” http://www.fhi.ox.ac.uk drives societal collapse?“ The Anthropology of Climate 150 http://en.wikipedia.org/wiki/Public_good Change: An Historical Reader (2013): 151-154. http://www.sipri.org 151 128 See Feldman, Allan M., and A. M. Feldman.: “Welfare http://www.skollglobalthreats.org See Patz, Jonathan A., et al.; “Global climate change economics and social choice theory.” Boston: Martinus and emerging infectious diseases.” JAMA: the journal http://www.stockholmresilience.org Nijhoff (1980). of the American Medical Association 275.3 (1996): 217- http://www.un.org/en/sc/ 152 223. See the Warsaw Climate Change Conference Press http://www.weforum.org/issues/global-risks Release “UN Climate Change Conference in Warsaw 129 See Benton, Michael J., and Richard J. Twitchett. Examples of relevant literature: keeps governments on a track towards 2015 climate “How to kill (almost) all life: the end-Permian extinction

186 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Endnotes

agreement.” 105.14 (2008): 5307-5312. Congress, 2010. 153 “The so-called “Warsaw International Mechanism 173 Though some have argued for a significant climate 191 See Bennett, Bruce W., and Jennifer Lind.: “The for Loss and Damage” will from next year commit effect of the nuclear explosions themselves, see Fujii, Collapse of North Korea: Military Missions and developed nations to provide expertise and potentially Yoshiaki.: “The role of atmospheric nuclear explosions Requirements.” International Security 36.2 (2011): 84- aid to countries hit by climate-related impacts. [...] on the stagnation of global warming in the mid 20th 119. However, the vague wording fell short of the kind of century.” Journal of Atmospheric and Solar-Terrestrial 192 Even the 1968 capture by North Korea of the USS detailed commitments on additional funding and the Physics 73.5 (2011): 643-652. Pueblo did not reignite conflict (see Armbrister, Trevor.: commitment to compensation that many developing 174 There is (fortunately) very little empirical evidence “A Matter of Accountability: The True Story of the Pueblo nations had been seeking.” (source: Business Green: on the impact of nuclear bombs on cities. Hiroshima Affair.” Globe Pequot, 2004). “COP 19: Warsaw climate deal finalised as deadlock suffered a firestorm, while Nagasaki did not – and both 193 See the Norwegian Ministry of Foreign Affairs: broken“). cities and nuclear weapons are very different now from “Conference: Humanitarian Impact of Nuclear weapons.” 154 See for instance Bulkeley, Harriet, and Peter Newell.: what they were in 1945. 194 This conference took place 13-14 February of 2014 175 “Governing climate change.” Routledge (2010). See Toon, Owen B., et al.: “Atmospheric effects and http://www.sre.gob.mx/en/index.php/humanimpact- 155 See the National Research Council: “Abrupt Impacts societal consequences of regional scale nuclear conflicts nayarit-2014 of Climate Change: Anticipating Surprises.” Washington, and acts of individual nuclear terrorism.” Atmospheric 195 See The 1983 War Scare: “The Last Paroxysm” of DC: The National Academies Press (2013). Chemistry and Physics 7.8 (2007): 1973-2002, and the Cold War Part I, The 1983 War Scare: “The Last 156 See NASA’s “GISS Surface Temperature Analysis.” Robock, Alan, Luke Oman, and Georgiy L. Stenchikov.: Paroxysm” of the Cold War Part II, and The 1983 War 157 See the IPCC’s Fourth Assessment Report. “Nuclear winter revisited with a modern climate Scare: “The Last Paroxysm” of the Cold War Part III. model and current nuclear arsenals: Still catastrophic 158 See the IPCC’s Fifth Assessment Report. 196 See The 1983 War Scare: “The Last Paroxysm” of the consequences.” Journal of Geophysical Research: 159 See the IUCN’s Red List of Threatened Species. Cold War Part III. Atmospheres (1984–2012) 112.D13 (2007). 197 160 See McManus, J. F., et al.: “Collapse and rapid An extensive but certainly not exhaustive list can be 176 Arsenals have been reduced, but there remain resumption of Atlantic meridional circulation linked to found at http://en.wikipedia.org/wiki/List_of_military_ over 17,000 nuclear warheads in the world’s arsenals deglacial climate changes.” Nature 428.6985 (2004): nuclear_accidents. (source: SIPRI yearbook 2013), down from a peak of 834-837. 198 See especially the role of Vasili Arkhipov in preventing some 68,000 in 1983, still more than enough to trigger a 161 the conflict from going nuclear – a role that was only See Schuur, Edward AG, et al.: “Vulnerability of nuclear winter. permafrost carbon to climate change: Implications for revealed in 2002 (source: Roberts, Priscilla, ed. “Cuban 177 See Toon, Owen B., et al.: “Atmospheric effects and the global carbon cycle.” BioScience 58.8 (2008): 701- Missile Crisis: The Essential Reference Guide.” ABC- societal consequences of regional scale nuclear conflicts 714. CLIO, 2012). and acts of individual nuclear terrorism.” Atmospheric 162 199 See Scott Shane, writing in the Baltimore Sun: “The See Archer, David.: “Methane hydrate stability Chemistry and Physics 7.8 (2007): 1973-2002, and Nuclear War that Almost Happened in 1983” (August 31, and anthropogenic climate change.” Biogeosciences Helfand, Ira.: “NUCLEAR FAMINE: TWO BILLION 2003). Discussions 4.2 (2007): 993-1057. PEOPLE AT RISK?.” International Physicians for the 200 163 For example, as climate warms, the destabilization of Prevention of Nuclear War (2013). See the PBS report on the incident. 201 the West Antarctic ice sheet could raise sea level rapidly, 178 Maher, Timothy M., and Seth D. Baum.: “Adaptation Errors of a different kind happened in the early days with serious consequences for coastal communities. to and recovery from global catastrophe.” Sustainability of the Cold War, when simple game theory models were 164 http://simple.wikipedia.org/wiki/Nuclear_war 5.4 (2013): 1461-1479. used to argue for a nuclear first strike (source: Kaku, 165 Michio, and Daniel Axelrod. “To win a nuclear war: the For an analysis of the possibility of accidental nuclear 179 See the later part of the scenario in Tonn, Bruce, and Pentagon’s secret war plans.” Black Rose Books Ltd., war between Russia and the USA, see Barrett, Anthony Donald MacGregor.: “A singular chain of events.” Futures 1987). M., Seth D. Baum, and Kelly Hostetler.: “Analyzing and 41.10 (2009): 706-714. 202 Some have modelled the nuclear standoff as a Reducing the Risks of Inadvertent Nuclear War Between 180 A synthesis blogpost by Carl Shulman of the Future “perceptual dilemma,” where misperception of the the United States and Russia.” Science & Global of Humanity Institute puts the risk of extinction, given adversary is the main cause of arms races (source: Security 21.2 (2013): 106-133. civilisation collapse, at no more than 10%. 166 Plous, Scott.: “The nuclear arms race: prisoner’s Hellman, Martin E.: “How risky is nuclear optimism?.” 181 See Tetlock, Philip E.: “Expert political judgment: How dilemma or perceptual dilemma?.” Journal of Peace Bulletin of the Atomic Scientists 67.2 (2011): 47-56. good is it? How can we know?” Princeton University Research 30.2 (1993): 163-179). 167 See Lundgren, Carl.: “What are the odds? Assessing Press (2005). 203 See Barrett, Anthony M., Seth D. Baum, and Kelly the probability of a nuclear war.” The Nonproliferation 182 See the official Remarks by Ambassador Susan Hostetler.: “Analyzing and Reducing the Risks of Review 20.2 (2013): 361-374. E. Rice, U.S. Permanent Representative to the United Inadvertent Nuclear War Between the United States and 168 A 1979 study by the United States Office of Nations, at the Security Council Stakeout. Russia.” Science & Global Security 21.2 (2013): 106-133. Technology Assessment, “The Effects of Nuclear War,” 183 “North Korean nuclear test draws anger, including 204 See Barrett, Anthony M., Seth D. Baum, and Kelly estimated 20-160 million immediate casualties from a from China” (Reuters). full-scale nuclear war. Hostetler.: “Analyzing and Reducing the Risks of 184 See the Security Council Resolution 2087 (2013). 169 Inadvertent Nuclear War Between the United States and Currently estimated at around 17,000 (source: SIPRI 185 See the description of the sanctions in the Security Russia.” Science & Global Security 21.2 (2013): 106-133. yearbook 2013). Council Committee established pursuant to resolution 205 See Podvig, Pavel.: “Reducing the risk of an 170 Though it has been argued that scientists, under 1718. Accidental Launch.” Science and Global Security 14.2-3 pressure from governments and industry, have 186 See the States Parties of the Nuclear Non- (2006): 75-115. systematically underestimated the deleterious global Proliferation Treaty (NPT). 206 impact of radiation. See Perrow, Charles.: “Nuclear See Intriligator, Michael D., and Dagobert L. Brito.: 187 See North Korean security challenges: a net denial: From Hiroshima to Fukushima.” Bulletin of the “Accidental nuclear war: a significant issue for arms assessment. International Institute for Strategic Studies, Atomic Scientists 69.5 (2013): 56-67. control.” Current Research on Peace and Violence 2011, Chapter 5. 11.1/2 (1988): 14-23. 171 The seminal American paper was Turco, Richard 188 See North Korean security challenges: a net 207 P., et al.: “Nuclear winter: global consequences of See Ayson, Robert.: “After a terrorist nuclear attack: assessment. International Institute for Strategic Studies, multiple nuclear explosions.” Science 222.4630 (1983): Envisaging catalytic effects.” Studies in Conflict & 2011, Chapter 6. 1283-1292. The Soviet Union was working on similar Terrorism 33.7 (2010): 571-593. 189 208 simulations; see for instance Peterson, Jeannie.: “The See North Korean security challenges: a net See Hellman, Martin E.: “How risky is nuclear aftermath: The human and ecological consequences of assessment. International Institute for Strategic Studies, optimism?.” Bulletin of the Atomic Scientists 67.2 (2011): nuclear war.” New York: Pantheon (1983). 2011, Chapter 8. 47-56. 190 209 172 See Mills, Michael J., et al.: “Massive global ozone See Chanlett-Avery, Emma, and Mi Ae Taylor.: “North See Shanteau, James.: “Competence in experts: The loss predicted following regional nuclear conflict.” Korea: US Relations, Nuclear Diplomacy, and Internal role of task characteristics.” Organizational behavior and Proceedings of the National Academy of Sciences Situation.” Congressional Research Service, Library of human decision processes 53.2 (1992): 252-266.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 187 Endnotes

210 See also Mosher, David E., et al.: Beyond the Nuclear (Nuclear Threat Initiative). 254 See Schlager, Edella, and Elinor Ostrom. “Property- Shadow. “A Phased Approach for Improving Nuclear 230 http://en.wikipedia.org/wiki/List_of_states_with_ rights regimes and natural resources: a conceptual Safety and US Russian Relations.” No. RAND/MR-1666- nuclear_weapons analysis.” Land economics (1992): 249-262. 255 NSRD. RAND CORP SANTA MONICA CA, 2003. 231 http://en.wikipedia.org/wiki/Ecological_collapse See Frank, Kenneth T., et al. “Trophic cascades in a 211 See the Report of the Open-ended Working Group 232 See Pimm, Stuart L., et al.: “The future of formerly cod-dominated ecosystem.” Science 308.5728 to develop proposals to take forward multilateral nuclear biodiversity.” Science 269 (1995): 347-347. (2005): 1621-1623. disarmament negotiations for the achievement and 256 233 See Rockström, Johan, et al.: “Planetary boundaries: See the IUCN’s Red List of Threatened Species. maintenance of a world without nuclear weapons. 257 exploring the safe operating space for humanity.” See Zalasiewicz, Jan, et al. “Are we now living in the 212 See the text of the Treaty on the Non-Proliferation of Ecology and society 14.2 (2009). Anthropocene?” GSA Today 18.2 (2008): 4. Nuclear Weapons. 258 234 See Barnosky, Anthony D., et al.: “Approaching a See Barnosky, Anthony D., et al.: “Approaching a 213 Treaty Between the United States of America and the state shift in Earth’s biosphere.” Nature 486.7401 (2012): state shift in Earth’s biosphere.” Nature 486.7401 (2012): Union of Soviet Socialist Republics on the Elimination of 52-58. 52-58. Their Intermediate-Range and Shorter-Range Missiles. 259 235 See Ehrlich, Paul R., and Anne H. Ehrlich.: “Can a See Dol, Marcel, ed.: “Recognising the intrinsic 214 Treaty Between the United States of America and the collapse of global civilization be avoided?.” Proceedings value of animals: beyond animal welfare.” Uitgeverij Van Union of Soviet Socialist Republics on the Reduction of the Royal Society B: Biological Sciences 280.1754 Gorcum (1999). and Limitation of Strategic Offensive Arms. (2013). 260 http://en.wikipedia.org/wiki/Pandemic 215 Treaty Between the United States of America and the 236 See Galaz, Victor, et al.: “Global environmental 261 See Hanson, Robin.: “Catastrophe, social collapse, Russian Federation On Strategic Offensive Reductions. governance and planetary boundaries: An introduction.” and human extinction.” Global Catastrophic Risks 1 216 The Treaty between the United States of America Ecological Economics 81 (2012): 1-3. (2008): 357. and the Russian Federation on Measures for the Further 237 See Dol, Marcel, ed.: “Recognizing the intrinsic 262 See Asmussen, Søren. “Steady-State Properties of Reduction and Limitation of Strategic Offensive Arms value of animals: beyond animal welfare.” Uitgeverij Van of GI/G/1.” Applied Probability and Queues (2003): 266- also known as the New START Treaty. Gorcum (1999). 301. 217 See Sovacool, Benjamin K. “Contesting the future of 238 See Dasgupta, Partha.: “Human well-being and the 263 Possessing neither variance nor mean. nuclear power: a critical global assessment of atomic natural environment.” OUP Catalogue (2001). 264 This explains why governments and international energy.” World Scientific, 2011. 239 See the author’s blog post: “Water, food or energy: organisations seem to over-react to pandemics (such 218 Helfand, Ira.: “Nuclear Famine: Two Billion People we won’t lack them.” summarising the current evidence. as the swine flu scare). Though any single pandemic is at Risk?” International Physicians for the Prevention of 240 See “2013 World Hunger and Poverty Facts and unlikely to cause a large number of casualties, there is Nuclear War (2013). Statistics.” World Hunger Education Service (2013). always the small risk that it could get out of hand and kill 219 See for instance Robock, Alan, Luke Oman, and 241 Maher, Timothy M., and Seth D. Baum.: “Adaptation millions. Georgiy L. Stenchikov.: “Nuclear winter revisited with a 265 to and recovery from global catastrophe.” Sustainability There is always a probability that the power law will modern climate model and current nuclear arsenals: Still 5.4 (2013): 1461-1479. not hold for future pandemics – but the majority of the catastrophic consequences.” Journal of Geophysical 242 See Dullinger, Stefan, et al.: “Europe’s other debt risk is concentrated in scenarios where it does. Research: Atmospheres (1984–2012) 112.D13 (2007), 266 crisis caused by the long legacy of future extinctions.” See Hanson, Robin.: “Catastrophe, social collapse, Toon, Owen B., et al. “Atmospheric effects and societal Proceedings of the National Academy of Sciences and human extinction.” Global Catastrophic Risks 1 consequences of regional scale nuclear conflicts and 110.18 (2013): 7342-7347. (2008): 357. acts of individual nuclear terrorism.” Atmospheric 243 267 Source: EM-DAT, The International Disaster Database. Chemistry and Physics 7.8 (2007): 1973-2002, and G8 2007 Environment Ministers Meeting. 268 Stenke, A., et al. “Climate and chemistry effects of a 244 See Pimm, Stuart L., et al. “The future of biodiversity.” See Jeffs, Benjamin.: “A clinical guide to viral regional scale nuclear conflict.” Atmospheric Chemistry Science 269 (1995): 347-347. haemorrhagic fevers: Ebola, Marburg and Lassa.” and Physics Discussions 13.5 (2013): 12089-12134. 245 See Jones-Walters, Lawrence, and Ivo Mulder: Tropical Doctor 36.1 (2006): 1-4. 269 220 Helfand, Ira.: “An assessment of the extent of “Valuing nature: The economics of biodiversity.” Journal See the WHO Factsheet 99 on Rabies. projected global famine resulting from limited, regional for Nature Conservation 17.4 (2009): 245-247. 270 See Arroll, Bruce.: “Common cold.” Clinical evidence nuclear war.” Royal Soc. Med, London (2007). 246 See the UN-mandated “Millennium Ecosystem 2011 (2011). 221 See Özdoğan, Mutlu, Alan Robock, and Christopher Assessment, Ecosystems and Human Well-being: 271 See “Stages of HIV.” U.S. Department of Health & J. Kucharik.: “Impacts of a nuclear war in South Asia on Synthesis“ (2005). Human Services. soybean and maize production in the Midwest United 247 See Bottrill, Madeleine C., et al.: “Is conservation 272 See Bouvier, Nicole M., and Peter Palese.: “The States.” Climatic Change 116.2 (2013): 373-387. triage just smart decision making?“ Trends in Ecology & biology of influenza viruses.” Vaccine 26 (2008): Xia, Lili, and Alan Robock.: “Impacts of a nuclear war Evolution 23.12 (2008): 649-654. D49-D53. in South Asia on rice production in Mainland China.” 248 See Fox, Jonathan A., and Lloyd David Brown, eds.: 273 See for instance Risk Management Solutions’ Climatic Change 116.2 (2013): 357-372. “The struggle for accountability: the World Bank, NGOs, pandemic model, overviewed in “The RMS® Influenza 223 See Stommel H, Stommel E.: “Volcano Weather: The and grassroots movements.” MIT Press (1998). Pandemic Risk Model.” Story of 1816, The Year without a Summer.” Seven Seas 249 http://www.esri.com/news/releases/11-4qtr/gis- 274 SARS (Severe), for instance, started in China and Press: Newport (1983). cloud-featured-at-eye-on-earth-summit.html spread to 17 other countries, including Canada (source: 224 See the European External Action Service Joint Plan 250 See Microsoft’s press release “Eye on Earth Enables Lung Disease & Respiratory Health Center, SARS: of Action (24 November 2013). Cloud-Based Environmental Data Sharing” (2011). Timeline of an Outbreak). 275 225 See the report of the Director General of the IAEA (8 251 “The main objective of the MyOcean2 project is to See Riley, Lee W., et al.: “Slum health: diseases of November 2011). deliver and operate a rigorous, robust and sustainable neglected populations.” BMC international health and 226 See Security Council Resolution 1737 (2006). Ocean Monitoring and Forecasting system of the GMES human rights 7.1 (2007): 2. 276 227 See the text of the Treaty on the Non-Proliferation of Marine Service (OMF/GMS) to users for all marine Brazil alone is estimated to have over 50 uncontacted Nuclear Weapons. applications: maritime safety, marine resources, marine tribes (source: Survival International). and coastal environment and climate, seasonal and 277 228 See for instance President Kennedy’s quote Maher, Timothy M., and Seth D. Baum.: “Adaptation weather forecasting.” (Source the MyOcean2 website). during the third Nixon-Kennedy presidential debate, to and recovery from global catastrophe.” Sustainability 252 October 13, 1960: “There are indications because of See the “Communication from the (European) 5.4 (2013): 1461-1479. new inventions, that 10, 15, or 20 nations will have a commission to the council, concerning a consultation on 278 See the WHO Interim Guidance “Pandemic Influenza nuclear capacity, including Red China, by the end of the Fishing Opportunities for 2014.” Risk Management.” 253 Presidential office in 1964.” The famous “tragedy of the commons.” See Garrett 279 See the WHO guidance document “Pandemic 229 “More Than 40 Countries Could Have Nuclear Hardin. “The Tragedy of the Commons” Science influenza preparedness and response.” Weapons Know-How, IAEA Chief ElBaradei Warns” 162.3859 (1968): 1243-1248. 280 See Trifonov, Vladimir, Hossein Khiabanian, and

188 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Endnotes

Raul Rabadan.: “Geographic dependence, surveillance, Release Reports in the United States—2004-2010. Appl Defect: How Globalization Creates Systemic Risks, and and origins of the 2009 influenza A (H1N1) virus.” New Biosaf 17 (2012):171-180.) What to Do about It. Princeton University Press (2014). England Journal of Medicine 361.2 (2009): 115-119. 301 It is suspected that the 1977-1978 Russian Flu 321 See Goldin, Ian, and Tiffany Vogel.: Global 281 See the WHO Sixty Fourth World Health Assembly epidemic was caused by a laboratory incident in governance and systemic risk in the 21st century: “Resolutions and Decisions,” resolution WHA64.5. Russia or northern China (source: NPR’s interview lessons from the financial crisis. Global Policy 1.1 (2010): 282 See the WHO International Health Regulations with Professor John Oxford at St. Bart’s and the Royal 4-15. (2005) “Alert, response, and capacity building under the London Hospital). 322 See Korhonen, Jouni, and Thomas P. Seager.: International Health Regulations (IHR).” 302 See Pennington, Hugh.: Smallpox and bioterrorism. Beyond eco‐efficiency: a resilience perspective. 283 See the “Report of the Review Committee on the Bulletin of the World Health Organization 81.10 (2003): Business Strategy and the Environment 17.7 (2008): 411- Functioning of the International Health Regulations 762-767. 419. (2005) in Relation to Pandemic (H1N1) 2009.” Geneva, 303 See Klotz, L. C., and E. J. Sylvester.: The 322 See J. Doyne Farmer.: Toward better quantitative World Health Organization (2011). unacceptable risks of a man-made pandemic. Bulletin of models of systemic risk. Presentation at Institute for New 284 See Kerkhove, Maria D., et al.: “Estimating age‐ the Atomic Scientists 7 (2012). Economic Thinking (2010). specific cumulative incidence for the 2009 influenza 304 The highest level of biosecurity. 323 See the Lloyds Insurance report “Solar storm risk to pandemic: a meta‐analysis of A (H1N1) pdm09 305 A Marburg virus laboratory-acquired infection at the the North American electric grid.” (2013). serological studies from 19 countries.” Influenza and Vector facility in the Soviet Union in 1990, a foot-and- 324 See Barro, Robert J., and Xavier Sala-i-Martin.: Other Respiratory Viruses (2013). mouth disease virus escape from the Pirbright facility in Economic growth and convergence across the United 285 Report in Nature by Maryn McKenna, “Antibiotic England, and a SARS virus laboratory-acquired infection States. No. w3419. National Bureau of Economic resistance: The last resort.” from a BSL-4-rated biosafety cabinet in a Taiwan Research (1990). 286 See Alchon, Suzanne Austin. “A Pest in the Land: laboratory. 325 See Hanson, Robin.: Catastrophe, social collapse, New World Epidemics in a Global Perspective.” 306 See the Towers Watson poll of global insurance and human extinction. Global Catastrophic Risks 1 University of New Mexico Press (2003). executives on the most worrying extreme risks facing the (2008): 357. 287 See Rothschild, Bruce M..: “History of syphilis.” insurance industry. 326 See for instance the temporary collapse of the Clinical infectious diseases: an official publication of the 307 See the CDC’s report Taubenberger, Jeffery K., and Albanian state in 1997 over poorly regulated investment Infectious Diseases Society of America 40.10 (2005): David M. Morens.: 1918 Influenza: the mother of all pyramid schemes: Jarvis, Christopher.: The rise 1454–63 pandemics. Rev Biomed 17 (2006): 69-79. and fall of Albania’s pyramid schemes. Finance and 288 See Lawn, Stephen D., and Zumla, Alimuddin I.: 308 See the World Health Organization Global Influenza Development 37.1 (2000): 46-49. “Tuberculosis,” The Lancet 378.9785 (2011): 57-72. Programme: Pandemic influenza preparedness and 327 See Acemoglu, Daron, James A. Robinson, and Dan 289 See the World Health Organization Fact Sheet No 104 response: a WHO guidance document. World Health Woren.: Why nations fail: the origins of power, prosperity, on Tuberculosis. Organisation (2009). and poverty. Vol. 4. New York: Crown Business (2012). 309 328 290 See Harrison, J. W., and T. A. Svec: “The beginning of See Merton, Robert K.: Bureaucratic structure and See the LSE press release Research centre to study the end of the antibiotic era? Part II. Proposed solutions personality. Social forces 18.4 (1940): 560-568, and risks to financial system launched at LSE (2013). to antibiotic abuse.” Quintessence international (Berlin, Turner, Marlene E., and Anthony R. Pratkanis: Twenty- 330 See Ang, Andrew, and Francis A. Longstaff.: Systemic Germany: 1985) 29.4 (1998): 223. five years of groupthink theory and research: Lessons sovereign credit risk: Lessons from the US and Europe. 291 See Mathew, Alan G., Robin Cissell, and S. from the evaluation of a theory. Organizational Behavior Journal of Monetary Economics (2013). Liamthong: “Antibiotic resistance in bacteria associated and Human Decision Processes 73.2 (1998): 105-115. 331 See for instance Brunnermeier, Markus K., and Lasse 310 with food animals: a United States perspective of http://en.wikipedia.org/wiki/Economic_collapse Heje Pedersen.: Market liquidity and funding liquidity. livestock production.” Foodborne pathogens and http://en.wikipedia.org/wiki/Societal_collapse Review of Financial studies 22.6 (2009): 2201-2238. disease 4.2 (2007): 115-133. 311 See Helbing, Dirk.: Globally networked risks and how 332 See for instance Calomiris, Charles W., and Joseph R. 292 See the final summary from the Skoll Global Threats to respond. Nature 497.7447 (2013): 51-59. Mason.: Fundamentals, panics, and bank distress during Fund “Wrap up from Epihack.” 312 See Anders Sandberg, Nick Beckstead, Stuart the depression. American Economic Review (2003): 293 See Sridhar, Saranya, et al.: “Cellular immune Armstrong.: “Defining systemic risk.” Report from 1615-1647. correlates of protection against symptomatic pandemic the FHI-Amlin Systemic Risk of Risk Modelling 333 See the IMF working paper Systemic Risk Monitoring influenza.” Nature Medicine 19.10 (2013): 1305-1312. Collaboration. Oxford University (2014). (“SysMo”) Toolkit—A User Guide (2013). 294 See Taubenberger, Jeffery K., and David M. Morens.: 313 See Gunderson, Lance H.: “Ecological resilience--in 334 See for instance Moorhouse, Andrew.: An “1918 Influenza: the mother of all pandemics.” Rev theory and application.” Annual review of ecology and introduction to financial soundness indicators. Monetary Biomed 17 (2006): 69-79. systematics (2000): 425-439, and Kambhu, John, Scott & Financial Statistics (2004). 295 See Valleron, Alain-Jacques, et al.: “Transmissibility Weidman, and Neel Krishnan. Part 3: Systemic risk in 335 See for instance Baldacci, E., McHugh, J. and and geographic spread of the 1889 influenza pandemic.” ecology and engineering. Economic Policy Review 13.2 Petrova, I.: Measuring Fiscal Vulnerabilities and Fiscal Proceedings of the National Academy of Sciences (2007). Stress: A Proposed Set of Indicators, IMF Working Paper 107.19 (2010): 8778-8781. 314 See De Bandt, Olivier, and Philipp Hartmann.: 11/94 (2011). 296 See the WHO report Ten things you need to know Systemic risk: A survey. ECB Working Paper No. 35 336 See for instance Hamilton, James D., and Raul about pandemic influenza. (2000). Susmel.: Autoregressive conditional heteroskedasticity 315 297 See Dawood, Fatimah S., et al.: “Estimated global See Rinaldi, Steven M., James P. Peerenboom, and changes in regime. Journal of Econometrics 64.1 mortality associated with the first 12 months of 2009 and Terrence K. Kelly.: Identifying, understanding, and (1994): 307-333. pandemic influenza A H1N1 virus circulation: a modelling analyzing critical infrastructure interdependencies. 337 See for instance Gray, Dale F., and Andreas A. Jobst.: study.” The Lancet infectious diseases 12.9 (2012): 687- Control Systems, IEEE 21.6 (2001): 11-25. Modelling systemic financial sector and sovereign risk. 695. 316 See Homer-Dixon, Thomas.: Complexity Science. Sveriges Riksbank Economic Review (2011): 68-97. 298 See Merler, Stefano, et al.: “Containing the accidental Oxford Leadership Journal 2.1 (2011): 15. 338 See Citigroup’s paper 2013 Outlook and Strategies laboratory escape of potential pandemic influenza 317 See Allen, Franklin, and Douglas Gale.: Financial (2013) and the Financial Post news report by David Pett: viruses.” BMC medicine 11.1 (2013): 252. contagion. Journal of political economy 108.1 (2000): Three geopolitical risks investors should keep an eye on 299 Report in Nature by Declan Butler, “Work resumes on 1-33. in 2014 (2013). lethal flu strains.” 318 See Buldyrev, Sergey V., et al.: Catastrophic cascade 339 Used by such diverse bodies as the US Department 300 From 2004 to 2010, 639 release reports were of failures in interdependent networks. Nature 464.7291 of Energy (source: the US Department of Energy’s Cost reported to the Centers for Disease Control (CDC), (2010): 1025-1028. Estimating Guide (2011)) and AACE (Association for the 11 of them reporting laboratory-acquired infections 319 See Vespignani, Alessandro.: Predicting the behavior Advancement of Cost Engineering) International (source: (source: Henkel, Richard D., Thomas Miller, and Robbin of techno-social systems. Science 325.5939 (2009): 425. AACE Recommended Practice No. 56R-08 Cost S. Weyant. Monitoring Select Agent Theft, Loss and 320 See Goldin, Ian, and Mike Mariathasan.: The Butterfly Estimate Classification System (2012)).

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 189 Endnotes

340 http://en.wikipedia.org/wiki/Impact_event (2010): 1495-1506. en.wikipedia.org/wiki/Timetable_of_major_worldwide_ 341 See Jablonski, David, and W. G. Chaloner.: 356 See NASA’s press release on the meteor, Additional volcanic_eruptions. Extinctions in the Fossil Record [and Discussion]. Details on the Large Fireball Event over Russia on Feb. 378 See Newhall, Christopher G., and Stephen Self.: Philosophical Transactions of the Royal Society of 15, 2013. The Volcanic Explosivity Index (VEI): an estimate of London. Series B: Biological Sciences 344.1307 (1994): 357 See Borovička, Jiří, et al.: The trajectory, structure explosive magnitude for historical volcanism. Journal 11-17. and origin of the Chelyabinsk asteroidal impactor. Nature of Geophysical Research: Oceans (1978–2012) 87.C2 342 See Collins, Gareth S., H. Jay Melosh, and Robert (2013). (1982): 1231-1238. 379 A. Marcus.: Earth Impact Effects Program: A Web‐ 358 See Chyba, Christopher F., Paul J. Thomas, and See Aspinall, W., et al.: Volcano hazard and based computer program for calculating the regional Kevin J. Zahnle.: The 1908 Tunguska explosion: exposure in GDRFF priority countries and risk mitigation environmental consequences of a meteoroid impact on atmospheric disruption of a stony asteroid. Nature measures-GFDRR Volcano Risk Study. Bristol: Bristol Earth. Meteoritics & planetary science 40.6 (2005): 817- 361.6407 (1993): 40-44. University Cabot Institute and NGI Norway for the World 840. Input data: Projectile Diameter: 5000m, Projectile 359 See for instance the Russian reaction: PM Medvedev Bank: NGI Report 20100806.2011 (2011). 380 Density: 2600 kg/m3, Impact Velocity: 17.00 km/s, Says Russian Meteorite KEF-2013 Shows “Entire Planet” See Mason, Ben G., David M. Pyle, and Clive Impact Angle: 45 degrees. Vulnerable. Oppenheimer.: The size and frequency of the largest 343 The “Tsar Bomba” at 2x1017 Joules (source: 360 See the Space Research Institute of Russian explosive eruptions on Earth. Bulletin of Volcanology Adamsky, Viktor, and Yuri Smirnov.: Moscow’s Biggest Academy of Science report Deflecting Hazardous 66.8 (2004): 735-748. 381 Bomb: The 50-Megaton Test of October 1961. Cold War Asteroids from Collision with the Earth by Using Small See Stephen Self and Stephen Blake, International History Project Bulletin 4 (1994): 19-21.). Asteroids. “Consequences of Explosive Super-eruptions” 344 The effects of various sizes of impacts have been 361 See for instance Collins, Gareth S., H. Jay Melosh, ELEMENTS , VOL . 4, 41–46: http://chuma.cas.usf. estimated in Atkinson, Harry, Crispin Tickell, and and Robert A. Marcus.: Earth Impact Effects Program: edu/~juster/GLY2030-F09/supervolcanoes-self%20 David Williams.: Report of the task force on potentially A Web‐based computer program for calculating the and%20blake.pdf 382 hazardous near-Earth objects. (2000). regional environmental consequences of a meteoroid See Zielinski, G. A., et al.: Potential atmospheric 345 See for instance Collins, Gareth S., H. Jay Melosh, impact on Earth. Meteoritics & planetary science 40.6 impact of the Toba Mega‐Eruption 71,000 years ago. and Robert A. Marcus.: Earth Impact Effects Program: (2005): 817-840, and Jeffers, S. V., et al.: Near‐Earth Geophysical Research Letters 23.8 (1996): 837-840. A Web‐based computer program for calculating the object velocity distributions and consequences for 383 It is estimated that the largest volcanic eruptions have regional environmental consequences of a meteoroid the Chicxulub impactor. Monthly Notices of the Royal an energy greater than 1021 Joules (source: Mason, Ben impact on Earth. Meteoritics & planetary science Astronomical Society 327.1 (2001): 126-132. G., David M. Pyle, and Clive Oppenheimer.: The size and 40.6 (2005): 817-840, Jeffers, S. V., et al.: Near‐Earth 362 See Hall, Christopher D., and I. Michael Ross.: frequency of the largest explosive eruptions on Earth. object velocity distributions and consequences for Dynamics and control problems in the deflection of Bulletin of Volcanology 66.8 (2004): 735-748.), while the Chicxulub impactor. Monthly Notices of the Royal Near-Earth Objects. Adv. Astronaut. Sci 67.640 (1997): the largest nuclear device ever exploded had a yield of Astronomical Society 327.1 (2001): 126-132, and Board, 1-18. around 2x1017 Joules (source: Adamsky, Viktor, and Yuri Space Studies.: Defending Planet Earth: Near-Earth 363 See NASA’s Asteroid Watch, Asteroid 2013 TV135 - A Smirnov.: Moscow’s Biggest Bomb: The 50-Megaton Object Surveys and Hazard Mitigation Strategies. Reality Check. Test of October 1961. Cold War International History National Academies Press (2010). A summary of the last Project Bulletin 4 (1994): 19-21.). 364 The effects of various sizes of impacts have been paper can be found at The Danger from Asteroid impact. 384 estimated in Atkinson, Harry, Crispin Tickell, and See Scarpa, Roberto.: Predicting volcanic eruptions. 346 See the NASA press release NASA Spacecraft David Williams.: Report of the task force on potentially Science 293.5530 (2001): 615-616. Reactivated to Hunt for Asteroids. 385 hazardous near-Earth objects. (2000). Sparks, Stephen. Super-eruptions: Global effects and 347 See NASA’s Near Earth Objects Discovery Statistics. 365 See Live Science, The Odds of Dying. future threats, report of a Geological Society of London 348 See Harris, Alan.: What Spaceguard did. Nature working group. Geological Society, 2005. 366 See Scientific American, United Nations to Adopt 453.7199 (2008): 1178-1179. 386 Asteroid Defense Plan. See 2013 World Hunger and Poverty Facts and 349 http://www.nature.com/nature/journal/v453/n7199/ Statistics. World Hunger Education Service (2013). 367 See Brown, P. G., et al.: A 500-kiloton airburst over fig_tab/4531178a_F2.html 387 Chelyabinsk and an enhanced hazard from small Maher, Timothy M., and Seth D. Baum.: Adaptation to 350 See Hall, Christopher D., and I. Michael Ross.: impactors. Nature (2013). and recovery from global catastrophe. Sustainability 5.4 Dynamics and control problems in the deflection of (2013): 1461-1479. 368 See Glasstone, Samuel.: The effects of nuclear Near-Earth Objects. Adv. Astronaut. Sci 67.640 (1997): 388 weapons. US Department of Defense (1964). See Sparks, S. & Self. S. et al., 2005: Super-eruptions: 1-18. global effects and future threats: Report of a Geological 369 See NBC’s report Third time’s the charm: SpaceX 351 Some are more optimistic than others: “An impact Society of London working group. http://www.geo.mtu. launches big commercial satellite. can be predicted in advance in ways that remain edu/~raman/VBigIdeas/Supereruptions_files/Super- 370 See Hall, Christopher D., and I. Michael Ross.: imperfect but are much more reliable than predictions eruptionsGeolSocLon.pdf Dynamics and control problems in the deflection of of earthquakes or even storms, and the components 389 http://en.wikipedia.org/wiki/Volcanic_Explosivity_ Near-Earth Objects. Adv. Astronaut. Sci 67.640 (1997): of technology exist - at affordable costs given the Index 1-18. consequences of an actual impact - to move any 371 http://volcanoes.usgs.gov/images/pglossary/vei.php threatening object away and avoid the disaster See NASA News, NASA Announces New Homes for 390 See Ćirković, Milan M., Anders Sandberg, and Nick altogether.” (source: Chapman, Clark R. “The hazard Space Shuttle Orbiters After Retirement. Bostrom.: Anthropic shadow: Observation selection of near-Earth asteroid impacts on earth.” Earth and 372 See NASA Report Launching into History. effects and human extinction risks. Risk analysis 30.10 Planetary Science Letters 222.1 (2004): 1-15.) See also 373 http://en.wikipedia.org/wiki/Supervolcano (2010): 1495-1506. NASA’s report to Congress “Near-Earth Object Survey 374 See Reichow, Marc K., et al.: The timing and extent of 391 See Lane, Christine S., Ben T. Chorn, and Thomas and Deflection Analysis of Alternatives“ (2007). the eruption of the Siberian Traps large igneous province: C. Johnson.: Ash from the Toba supereruption in Lake 352 See 2013 World Hunger and Poverty Facts and Implications for the end-Permian environmental crisis. Malawi shows no volcanic winter in East Africa at 75 Statistics. World Hunger Education Service (2013). Earth and Planetary Science Letters 277.1 (2009): 9-20, ka. Proceedings of the National Academy of Sciences 353 and Benton, Michael J., and M. J. Benton.: When life See Maher, Timothy M., and Seth D. Baum.: 110.20 (2013): 8025-8029. Adaptation to and recovery from global catastrophe. nearly died: the greatest mass extinction of all time. Vol. 392 See Zielinski, G. A., et al.: Potential atmospheric Sustainability 5.4 (2013): 1461-1479. 5. London: Thames & Hudson (2003). impact of the Toba Mega‐Eruption ∼ 71,000 years ago. 354 375 See Shen, Shu-zhong, et al.: Calibrating the end- Report of the Task Force on potentially hazardous Geophysical Research Letters 23.8 (1996): 837-840. Near-Earth Objects. http://spaceguardcentre.com/wp- Permian mass extinction. Science 334.6061 (2011): 393 See Yoichi Nakamura, Shigeo Aramaki and Eisuke content/uploads/2014/04/full_report.pdf 1367-1372. Fujita, Japan’s Volcanic Disaster Mitigation Initiatives: 355 376 Kerr, Richard A.: Mega-Eruptions Drove the Mother of See Ćirković, Milan M., Anders Sandberg, and Nick Activities of the Commission on Mitigation of Volcanic Mass Extinctions. Science 342.6165 (2013): 1424-1424. Bostrom.: Anthropic shadow: Observation selection Disasters, the Volcanological Society of Japan, Technical effects and human extinction risks. Risk analysis 30.10 377 A partial list can be found on Wikipedia: http://

190 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Endnotes

Note of the National Research Institute for Earth Science and existential risks from emerging technologies through strains. Nature 493.7433 (2013): 459-459. and Disaster Prevention 380 (2013). international law. Available at SSRN 2179094 (2012). 431 See Imai, Masaki, et al.: Experimental adaptation 394 See Barnes, Gina L.: Origins of the Japanese Islands: 411 See Kelle, Alexander.: Ensuring the security of of an influenza H5 HA confers respiratory droplet The New “Big Picture”. Nichibunken Japan Review synthetic biology—towards a 5P governance strategy. transmission to a reassortant H5 HA/H1N1 virus in (2003): 3-50. Systems and synthetic biology 3.1-4 (2009): 85-90, ferrets. Nature 486.7403 (2012): 420-428, and Herfst, 395 See Nakamura, Y., Fukushima, K., Jin, X., Ukawa, M., and Bernauer, Hubert, et al.: Technical solutions for Sander, et al.: Airborne transmission of influenza A/H5N1 Sato, T., and Hotta, Y.: Mitigation Systems by Hazard biosecurity in synthetic biology. Industry Association virus between ferrets. Science 336.6088 (2012): 1534- Maps, Mitigation Plans, and Risk Analyses Regarding Synthetic Biology, Munich, Germany (2008). 1541. Volcanic Disasters in Japan. J. Disaster Research, 3-4, 412 Though see some of the suggestions in Garfinkel, 432 See Butler, Declan.: Fears grow over lab-bred flu. (2007) 297-304. Michele S., et al.: Synthetic genomics: options for Nature 480 (2011): 421-422. 396 See Omer, Haim, and Nahman Alon.: The continuity governance. Industrial Biotechnology 3.4 (2007): 333- 433 See Fouchier, R. A., et al.: Pause on avian flu principle: A unified approach to disaster and trauma. 365. transmission research. Science 335.6067 (2012): 400- American Journal of Community Psychology 22.2 413 See Dennis, Carina.: The bugs of war. Nature 401. (1994): 273-287. 411.6835 (2001): 232-235. 434 See Sherrill, Michelle.: Self-Serving Bias. American 397 See Smith, Robert B. et al.: Discoveries and 414 See Smith, Bruce D., and Mark Nesbitt.: The Psychologist 41: 954-969 (2008). breakthroughs from integrative analysis of Yellowstone emergence of agriculture. New York: Scientific American 435 See for instance the London Independent: Leading science revealing new tectonic-volcanic models and Library (1995). scientists condemn decision to continue controversial a new assessment of earthquake-volcano hazards. 415 See Tucker, Jonathan B.: The Current Bioweapons research into deadly H5N1 bird-flu virus. Geological Society of America Abstracts with Programs. Threat. Biopreparedness and Public Health. Springer 436 See the Report of the WHO Informal Consultation on Vol. 45. No. 7. 2013. Netherlands (2013) 7-16. Dual Use Research of Concern. 398 416 See Earthquake-hazards Scenario for a M7 With corporations targeting each other’s biological 437 See Imai, Masaki, et al.: Experimental adaptation Earthquake on the Salt Lake City Segment of the products, for instance. of an influenza H5 HA confers respiratory droplet Wasatch Fault Zone, Utah. No. 111. Utah Geological 417 A risk the US Government is already taking seriously; transmission to a reassortant H5 HA/H1N1 virus in Survey (2004). see for instance President George W. Bush’s April 2004 ferrets. Nature 486.7403 (2012): 420-428, and Herfst, 399 See the Towers Watson poll of global insurance Biodefense for the 21st Century initiative. The threat Sander, et al.: Airborne transmission of influenza A/H5N1 executives on the most worrying extreme risks facing of bioterrorism is also prominent in the 2007 report virus between ferrets. Science 336.6088 (2012): 1534- the insurance industry. Mapping the Global Future: Report of the National 1541. 400 See Merton, Robert K.: Bureaucratic structure and Intelligence Council’s 2020 Project. 438 See the WHO report Responsible life sciences personality. Social forces 18.4 (1940): 560-568, and 418 The World Health Organization estimates that many research for global health security: a guidance Turner, Marlene E., and Anthony R. Pratkanis.: Twenty- countries and institutions lack oversight of such dual document. (2010). five years of groupthink theory and research: Lessons use technologies (source: Report of the WHO Informal 439 See Jackson, Ronald J., et al.: Expression of from the evaluation of a theory. Organizational Behavior Consultation on Dual Use Research of Concern, World mouse interleukin-4 by a recombinant ectromelia and Human Decision Processes 73.2 (1998): 105-115. Health Organization). virus suppresses cytolytic lymphocyte responses and 401 See Kerr, Richard A.: Mega-Eruptions Drove the 419 See Baum, Seth, and Grant Wilson.: The Ethics of overcomes genetic resistance to mousepox. Journal of Mother of Mass Extinctions. Science 342.6165 (2013): Global Catastrophic Risk from Dual-Use Bioengineering. virology 75.3 (2001): 1205-1210. 1424-1424. Ethics in Biology, Engineering and Medicine 4 (2013). 440 See Rosengard, Ariella M., et al.: Variola virus immune 402 A partial list can be found on Wikipedia: http:// 420 Attempts at containing such a leak may not be evasion design: expression of a highly efficient inhibitor en.wikipedia.org/wiki/Timetable_of_major_worldwide_ successful; see Merler, Stefano, et al.: Containing the of human complement. Proceedings of the National volcanic_eruptions. accidental laboratory escape of potential pandemic Academy of Sciences 99.13 (2002): 8808-8813. 403 See Reichow, Marc K., et al.: The timing and influenza viruses. BMC medicine 11.1 (2013): 252. 441 See Cello, Jeronimo, Aniko V. Paul, and Eckard extent of the eruption of the Siberian Traps large 421 See for instance Hwang, In Young, et al.: Wimmer.: Chemical synthesis of poliovirus cDNA: igneous province: Implications for the end-Permian Reprogramming Microbes to Be Pathogen-Seeking generation of infectious virus in the absence of natural environmental crisis. Earth and Planetary Science Killers. ACS synthetic biology (2013). template. Science 297.5583 (2002): 1016-1018. Letters 277.1 (2009): 9-20, and Benton, Michael J., and 422 Brazil alone is estimated to have over 50 uncontacted 442 See Tumpey, Terrence M., et al.: Characterization of M. J. Benton.: When life nearly died: the greatest mass tribes (source: Survival International). the reconstructed 1918 Spanish influenza pandemic extinction of all time. Vol. 5. London: Thames & Hudson 423 See Top U.S. Disease Fighters Warn of New virus. Science 310.5745 (2005): 77-80. (2003). Engineered Pathogens but Call Bioweapons Doomsday 443 See Gibson, Daniel G., et al.: Complete chemical 404 See Shen, Shu-zhong, et al.: Calibrating the end- Unlikely. Global Security Newswire (2005). synthesis, assembly, and cloning of a Mycoplasma Permian mass extinction. Science 334.6061 (2011): 424 Maher, Timothy M., and Seth D. Baum.: Adaptation to genitalium genome. Science 319.5867 (2008): 1215- 1367-1372. and recovery from global catastrophe. Sustainability 5.4 1220. 405 http://en.wikipedia.org/wiki/Synthetic_biology (2013): 1461-1479. 444 See Zhang, Ying, et al.: H5N1 hybrid viruses bearing 406 See Schmidt, Markus, ed.: Synthetic biology: 425 See the later part of the scenario in Tonn, Bruce, and 2009/H1N1 virus genes transmit in guinea pigs by industrial and environmental applications. Wiley- Donald MacGregor.: A singular chain of events. Futures respiratory droplet. Science 340.6139 (2013): 1459- Blackwell (2012). 41.10 (2009): 706-714. 1463. 407 445 Making Synthetic biology into an “epistemic” risk: the 426 See Guan, Zheng-Jun, et al.: Biosafety The Human Fatality and Economic Burden of a Man- risk might be minute or huge, it’s unclear at present. See Considerations of Synthetic Biology in the International made Influenza Pandemic. Sahlin, Nils-Eric, and Johannes Persson.: Epistemic risk: Genetically Engineered Machine (iGEM) Competition. 446 See Drexler, K. Eric.: Engines of Creation: the Coming the significance of knowing what one does not know. BioScience 63.1 (2013): 25-34. Era of Nanotechnology. Fourth Estate (1990). Future risks and risk management. Springer Netherlands 427 See Wired Magazine’s feature article Genome at 447 See Drexler, K. Eric.: Radical Abundance: How a (1994) 37-62. Home: Biohackers Build Their Own Labs. Revolution in Nanotechnology Will Change Civilization. 408 The decision to lift the moratorium on potentially 428 Such as Bernauer, Hubert, et al.: Technical solutions Public Affairs (2013). dangerous gain of function flu research was taken by for biosecurity in synthetic biology. Industry Association 448 See Freitas, Robert A., and Ralph Charles Merkle.: the researchers themselves (source: Declan Butler: Work Synthetic Biology, Munich, Germany (2008). Kinematic self-replicating machines. Georgetown, TX: resumes on lethal flu strains, Nature News). 429 See Lenox, Michael J., and Jennifer Nash.: Industry Landes Bioscience/Eurekah. com (2004). 409 See Schmidt, Markus.: Diffusion of synthetic biology: self‐regulation and adverse selection: a comparison 449 See Committee to Review the National a challenge to biosafety. Systems and synthetic biology across four trade association programs. Business Nanotechnology Initiative.: A matter of size: Triennial 2.1-2 (2008): 1-6. strategy and the environment 12.6 (2003): 343-356. review of the National Nanotechnology Initiative. (2006). 410 See Wilson, Grant.: Minimizing global catastrophic 430 See Butler, Declan.: Work resumes on lethal flu 450 According to the CIA World Factbook.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 191 Endnotes

451 See Roco, Mihail C., and William S. Bainbridge.: 473 See Drexler, K. Eric.: Radical Abundance: How a avoiding this. “Friendly AI” projects focus on the AI’s Societal implications of nanoscience and Revolution in Nanotechnology Will Change Civilization. goals directly, and attempt to make these compatible nanotechnology: Maximizing human benefit. Journal of PublicAffairs (2013). with human survival. The Oracle AI approach attempts Nanoparticle Research 7.1 (2005): 1-13. 474 See Drexler, K. Eric.: Engines of Creation: the Coming to “box” the AI (constrain its abilities to influence the 452 See the report of the Center for Responsible Era of Nanotechnology. Fourth Estate (1990). world) in order to prevent it from acquiring power. The Nanotechnology,: Dangers of Molecular Manufacturing. 475 See Nanostart’s presentation at the reduced impact AI is a novel approach which attempts 453 See Drexler, K. Eric.: Radical Abundance: How a EuroNanoForum 2013, Dublin: Venture Capital to construct AIs goals that are not friendly per se, but Revolution in Nanotechnology Will Change Civilization. Investments in Nanotech (2013). that motivate the AI to have a very limited impact on the world, thus constraining the potential damage it could PublicAffairs (2013). 476 See Nanostart’s newsletter, A space elevator would do. 454 See Barber, Kevin, et al.: Nanotechnology for Uranium be a tremendous benefit for humankind (2012). 494 There are many different designs for AIs, and Separations and Immobilization. Advances in Materials 477 See Levine, Michael E., and Jennifer L. Forrence.: disagreements about whether they could be considered Science for Environmental and Nuclear Technology: Regulatory capture, public interest, and the public moral agents or capable of suffering. At one end of Ceramic Transactions 107 (2010): 177. agenda: Toward a synthesis. Journal of Law, Economics, the scale is the theoretical AIXI(tl), which is essentially 455 See Gubrud, Mark Avrum.: Nanotechnology and & Organization 6 (1990): 167-198. nothing more than an equation with vast computing international security. Fifth Foresight Conference on 478 See the Congressional Research Service report, power; at the other lie whole brain emulations, copies of Molecular Nanotechnology (1997). Nanotechnology: a Policy Primer (2013). human minds implemented inside a computer. Suffering 456 See Altmann, Jürgen.: Military uses of 479 Nano-replicator: A system able to build copies of itself for the first design seems unlikely; suffering for the nanotechnology: perspectives and concerns. Security when provided with raw materials and energy second seems very possible. Determining whether a Dialogue 35.1 (2004): 61-79. 480 http://en.wikipedia.org/wiki/Artificial_intelligence specific AI design can suffer will be an important part of 457 See the Oxford Martin School publication by Drexler, 481 See Armstrong, Stuart, and Kaj Sotala. 2012.: How figuring out what to do with it; it seems unlikely that we’ll K. Eric and Pamlin, Dennis: Nano-solutions for the 21st We’re Predicting AI—or Failing To. In Beyond AI: Artificial have hard and fast rules in advance. Century, (2013). Dreams, edited by Jan Romportl, Pavel Ircing, Eva 495 The idea behind uploads, also called whole brain 458 See Wilson, Grant.: Minimizing global catastrophic Zackova, Michal Polak, and Radek Schuster, 52–75 emulations, is to take a human brain, copy the position and existential risks from emerging technologies through 482 See Chalmers, David, The singularity: A philosophical of its neurones and connections to sufficient precision, international law. Available at SSRN 2179094 (2012). analysis. Journal of Consciousness Studies 17.9-10 and then run the emulation forwards according to the 459 See Drexler, K. Eric.: Engines of Creation: the Coming (2010): 9-10. laws of physics and chemistry. It should then behave Era of Nanotechnology. Fourth Estate (1990). 483 See Muehlhauser, Luke, and Anna Salamon.: exactly as a human brain in the real world. See http:// 460 See Phoenix, Chris, and Eric Drexler.: Safe Intelligence explosion: Evidence and import. Singularity www.fhi.ox.ac.uk/brain-emulation-roadmap-report.pdf exponential manufacturing. Nanotechnology 15.8 (2004): Hypotheses. Springer Berlin Heidelberg (2012) 15-42. for more details, which also goes into the requirements 869. 484 See Yudkowsky, Eliezer.: Intelligence Explosion and tries to estimate the likelihood off success of such 461 See Freitas Jr, Robert A.: Some limits to global Microeconomics. Technical report 2013-1 Berkeley, CA: an approach (it depends on certain likely but uncertain ecophagy by biovorous nanoreplicators, with public Machine Intelligence Research Institute (2013). assumptions about how the brain works at the small scale; ie that most things below a certain scale can policy recommendations. Foresight Institute (2000). 485 See Wilson, Grant.: Minimizing global catastrophic 462 be handled statistically rather than through detailed Such as computer viruses today, which endure and existential risks from emerging technologies through modelling). despite efforts at eradication, but don’t threaten the international law. Available at SSRN 2179094 (2012). 496 There’s two ways there could be diminishing returns system as a whole. 486 See Omohundro, Stephen M.: The basic AI drives. 463 to intelligence: first, it might not be possible for an AI See Leary, Scott P., Charles Y. Liu, and Michael LJ Frontiers in Artificial Intelligence and applications 171 to improve its own intelligence strongly. Maybe it could Apuzzo.: Toward the emergence of nanoneurosurgery: (2008): 483. find some ways of better executing its algorithms, and Part III-Nanomedicine: Targeted nanotherapy, 487 See Muehlhauser, Luke, and Louie Helm.: nanosurgery, and progress toward the realization of a few other low-hanging fruits, but it’s very possible that Intelligence Explosion and Machine Ethics. In Singularity after doing this, it will find that the remaining intelligence- nanoneurosurgery. Neurosurgery 58.6 (2006): 1009- Hypotheses: A Scientific and Philosophical Assessment, 1026.. increase problems are hard, and that it hasn’t improved edited by Amnon Eden, Johnny Søraker, James H. Moor, itself sufficiently to make them easy (you could analogise 464 See the later part of the scenario in Tonn, Bruce, and and Eric Steinhart. Berlin: Springer (2012) this to dampened resonance - initially the AI can improve Donald MacGregor.: A singular chain of events. Futures 488 Dealing with most risks comes under the category its intelligence quite easily, and this gives it more 41.10 (2009): 706-714. of decision theory: finding the right approaches to intelligence with which to work, allowing it to improve its 465 See Tetlock, Philip E.: Expert political judgment: How maximise the probability of the most preferred options. intelligence still further; but each time, the effect is less, good is it? How can we know? Princeton University But an intelligent agent can react to decisions in a way and it soon plateaus). Press (2005). the environment cannot, meaning that interactions The other way there could be diminishing returns to 466 See Lewandowski, Bartosz, et al.: Sequence- with AIs are better modelled by the more complicated intelligence is if intelligence doesn’t translate into power specific peptide synthesis by an artificial small-molecule discipline of game theory. effectively. This is more likely with things like social 489 machine. Science 339.6116 (2013): 189-193. See Yudkowsky, Eliezer.: Artificial intelligence as intelligence: it seems plausible that no entity could 467 See Freitas, Robert A., and Ralph Charles Merkle.: a positive and negative factor in global risk. Global convince, say, 99% of Britain to vote for their party, Kinematic self-replicating machines. Georgetown, TX: catastrophic risks 1 (2008): 303. no matter how socially intelligent it was. Scientific and Landes Bioscience/Eurekah. com (2004). 490 See Muehlhauser, Luke, and Nick Bostrom.: Why we technological intelligence could conceivably be limited 468 Defined by Eric Drexler as a “device able to guide need friendly AI. Think: Philosophy for Everyone (2014). by creativity (unlikely), constraints on the speed of chemical reactions by positioning reactive molecules 491 The balance is currently very uneven, with only three making experiments, or just the fact that humans have with atomic precision.” small organisations – the Future of Humanity Institute, solved the easy problems already. 469 See Vugt, Gert Van.: The Killer Idea: How Some the Machine Intelligence Research Institute and the 497 See DARPA’s briefing, DARPA-SN-13-30: Gunslinging Anarchists Held Freedom of Speech at Cambridge Centre for Existential Risk – and focused Probabilistic Programming for Advancing Machine Gunpoint. Proceedings of the eLaw Conference 3D mainly on the risks of AI, along with some private Learning (PPAML) (2013). Printing: Destiny, Doom or Dream (2013). individuals. 498 See for instance Curtis, Jon, Gavin Matthews, and 470 See the report of the Center for Responsible 492 See Sandberg, Anders, and Nick Bostrom.: Whole David Baxter.: On the effective use of Cyc in a question Nanotechnology, Dangers of Molecular Manufacturing. brain emulation: A roadmap. Future of Humanity Institute answering system. Proc Workshop on Knowledge and 471 See Excell, John, and Stuart Nathan.: The rise of Technical Report #2008-3, Oxford University (2008). Reasoning for Answering Questions. (2005). additive manufacturing. The Engineer (2010). 493 A dangerous AI would be a very intelligent that one 499 See The Rise of Industrial Big Data, GE Intelligent 472 The gun designer, Cody Wilson, who describes himself could acquire great power in the world, and would Platforms (2013). as a crypto-anarchist, said his plans to make the design have goals that were not compatible with full human 500 Such as Watson’s triumph on “Jeopardy!” (source: available were “about liberty” (source: BBC News). survival and flourishing. There are a variety of ways of Ferrucci, David, et al.: Building Watson: An overview of

192 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Endnotes

the DeepQA project. AI magazine 31.3 (2010): 59-79). Steve.: Rational artificial intelligence for the greater good. life and sharpening the Fermi paradox. Acta Astronautica 501 See the Wired report by Steven Levy, How Ray Singularity Hypotheses. Springer Berlin Heidelberg. (2013). Kurzweil Will Help Google Make the Ultimate AI Brain (2012) 161-179. 540 See Bostrom, Nick.: Anthropic bias: Observation (2013). 519 See James Barrat.: Our Final Invention: Artificial selection effects in science and philosophy. Psychology 502 “Soon after [1988] came a period called the ‘AI Intelligence and the End of the Human Era. (2013). Press (2002). Winter,’ in which many companies fell by the wayside as 520 See Omohundro, Stephen M.: The basic AI drives. 541 Maher, Timothy M., and Seth D. Baum.: Adaptation to they failed to deliver on extravagant promises.” (Source: Frontiers in Artificial Intelligence and applications 171 and recovery from global catastrophe. Sustainability 5.4 Stuart, Russell, and Norvig Peter. “Artificial intelligence: (2008): 483. (2013): 1461-1479. a modern approach.” Upper Saddle River, New Jersey: 521 Directly coding safe behaviour is seen to be an 542 See Wilson, Grant.: Minimizing global catastrophic Prentice Hall (2003) pp24). extremely difficult challenge. and existential risks from emerging technologies through 503 “At its low point, some computer scientists and 522 See Armstrong, Stuart, Bostrom, Nick, and Shulman, international law. Available at SSRN 2179094 (2012). software engineers avoided the term artificial intelligence Carl.: Racing to the precipice: a model of artificial 543 See Maher, Timothy M., and Seth D. Baum: for fear of being viewed as wild-eyed dreamers.” intelligence development. Technical Report #2013-1, Adaptation to and recovery from global catastrophe. (Source: New York Times report by John Markov: Behind Future of Humanity Institute, Oxford University (2013).: Sustainability 5.4 (2013): 1461-1479. Artificial Intelligence, a Squadron of Bright Real People http://www.fhi.ox.ac.uk/wp-content/uploads/Racing- 544 One possible exception could be James Lovelock: (2005)). to-the-precipice-a-model-of-artificial-intelligence- The Revenge of Gaia. Why the Earth is fighting back— 504 See Ferrucci, David, et al.: Building Watson: An development.pdf and how we can still save humanity. (2006). 523 overview of the DeepQA project. AI magazine 31.3 See Smith, Theresa Clair.: Arms race instability and 545 See Tonn, Bruce, and Dorian Stiefel.: Evaluating (2010): 59-79. war. Journal of Conflict Resolution 24.2 (1980): 253-284. methods for estimating existential risks. Risk Analysis 505 See Guizzo, Erico.: How Google’s self-driving car http://jcr.sagepub.com/content/24/2/253.full.pdf+html (2013). 524 works. IEEE Spectrum Online (2011). See Rapoport, Anatol.: Prisoner’s dilemma: A study 546 For more information about the different methods: 506 See Kurzweil, Ray.: The Age of Spiritual Machines: in conflict and cooperation. University of Michigan Press Tonn, Bruce, and Dorian Stiefel. “Evaluating methods How we will live, work and think in the new age of (1965). for estimating existential risks.” Risk Analysis 33.10 intelligent machines. Orion (1999). 525 See the special events on the program of the 23rd (2013): 1772-1787. http://www.ncbi.nlm.nih.gov/ 507 See Sotala, Kaj and Roman V. Yampolskiy.: International Joint Conference on Artificial Intelligence. pubmed/23551083 Responses to Catastrophic AGI Risk: A Survey. http://ijcai13.org/program/special_events_the_impact_ 547 See Tonn, Bruce E.: A methodology for organizing Technical report 2013-2. Berkeley, CA: Machine of_ai_on_society and quantifying the results of environmental scanning Intelligence Research Institute (2013) 526 See Bostrom, Nick.: Existential Risk Prevention as exercises. Technological Forecasting and Social Change 508 The term AI is now ambiguous, with various “narrow Global Priority. Global Policy 4.1 (2013): 15-31. 75.5 (2008): 595-609. AIs” being produced (such as self-driving cars or 527 See Sherrill, Michelle.: Self-Serving Bias. American 548 See Armstrong, Stuart, and Anders Sandberg.: stock-trading programmes). An “AGI” returns to the Psychologist 41: 954-969 (2008). https://facultystaff. Eternity in six hours: Intergalactic spreading of intelligent original meaning of AI, that of an artificial being with richmond.edu/~dforsyth/pubs/forsyth2008selfserving. life and sharpening the Fermi paradox. Acta Astronautica general intelligence skills. This report will use the term AI pdf (2013). interchangeably with AGI, however. 528 See Yudkowsky, Eliezer.: Artificial intelligence as 549 See Webb, Stephen.: If the Universe is teeming with 509 See Armstrong, Stuart, and Kaj Sotala. 2012.: How a positive and negative factor in global risk. Global aliens... Where is everybody?: Fifty solutions to the Fermi We’re Predicting AI—or Failing To. In Beyond AI: Artificial catastrophic risks 1 (2008): 303. http://intelligence.org/ paradox and the problem of extraterrestrial life. Springer (2002). Dreams, edited by Jan Romportl, Pavel Ircing, Eva files/AIPosNegFactor.pdf 550 See Hanson, Robin.: The great filter - are we almost Zackova, Michal Polak, Radek Schuster, 52–75 529 One of the authors of Artificial Intelligence: a past it? (1998). 510 See Yudkowsky, Eliezer.: Levels of Organization in modern approach. http://www.amazon.co.uk/ 551 See Lineweaver, Charles H., Yeshe Fenner, and Brad General Intelligence. In Artificial General Intelligence, Artificial-Intelligence-Approach-International-Edition/ K. Gibson.: The galactic habitable zone and the age edited by Ben Goertzel and Cassio Pennachin 389–501 dp/0130803022 distribution of complex life in the Milky Way. Science (2007). 530 See CSER’s list of external advisors on their website. 303.5654 (2004): 59-62. 511 See Bostrom, Nick.: The superintelligent will: http://cser.org/ 552 http://www.fhi.ox.ac.uk/intergalactic-spreading.pdf 531 Motivation and instrumental rationality in advanced See for instance the “X factors” section of Global 553 See The Philippine Star news report Top artificial agents. Minds and Machines 22.2 (2012): 71-85, Risks 2013 - Eighth Edition. World Economic Forum conglomerates spearhead creation of private sector and Armstrong, Stuart.:General purpose intelligence: (2013). disaster response body (2013). arguing the Orthogonality thesis.” Analysis and 532 See Ellis, John, et al.: Review of the safety of LHC 554 See the UNISDR report Private Sector Strengths Metaphysics (2013). collisions. Journal of Physics G: Nuclear and Particle Applied (2013). http://www.philstar.com:8080/ 512 See the Less Wrong summary post Complexity of Physics 35.11 (2008): 115004. business/2013/12/10/1266103/top-conglomerates- value for a discussion on this issue. 533 See S Korean dies after games session, BBC News. spearhead-creation-private-sector-disaster-response 513 See Yampolskiy, Roman V.: Leakproofing the 534 See Alex Knapp, Is It Ethical to Make Animals As http://www.unisdr.org/files/33594_ Singularity. Journal of Consciousness Studies 19.1-2 Smart As People? Forbes. privatesectorstrengthsapplied2013di.pdf (2012): 194-214. 535 See George Dvorsky, New Project to Message Aliens 555 http://www.who.int/trade/glossary/story038/en/ 514 See Armstrong, Stuart, Anders Sandberg, and Nick is Both Useless and Potentially Reckless, io9. 556 There are known effective strategies for poverty Bostrom.: Thinking inside the box: Using and controlling 536 See Shanteau, James.: Competence in experts: The reduction. See for instance Ravallion, Martin.: A an Oracle AI. Minds and Machines (2012). role of task characteristics. Organisational behavior and comparative perspective on poverty reduction in Brazil, 515 See Armstrong, Stuart, Anders Sandberg, and Nick human decision processes 53.2 (1992): 252-266. China and India. World Bank Policy Research Working Bostrom.: Thinking inside the box: Using and controlling 537 See Ord, Toby, Rafaela Hillerbrand, and Anders Paper Series (2009). an Oracle AI. Minds and Machines (2012). Sandberg.: Probing the improbable: Methodological 557 See Bostrom, Nick.: What is a singleton? Linguistic 516 See Dewey, Daniel. “Learning what to value.” Artificial challenges for risks with low probabilities and high and Philosophical Investigations 5.2 (2006): 48-54, and General Intelligence. Springer Berlin Heidelberg, (2011). stakes. Journal of Risk Research 13.2 (2010): 191-205. Caplan, Bryan.: The totalitarian threat. (2008). 309-314. 538 See Ćirković, Milan M.: Small Theories and Large 558 See Tetlock, Philip E.: Expert political judgment: How 517 See Thomas, Michael SC, and James L. McClelland. Risks—Is Risk Analysis Relevant for Epistemology?. Risk good is it? How can we know? Princeton University “Connectionist models of cognition.” The Cambridge Analysis 32.11 (2012): 1994-2004. Press (2005). handbook of computational psychology (2008): 23-58. 539 This paper demonstrates the relative ease of moving 559 See Tetlock, Philip E.: Thinking the unthinkable: 518 See Yudkowsky, Eliezer.: Artificial intelligence as between galaxies, making the Fermi paradox more Sacred values and taboo cognitions. Trends in cognitive a positive and negative factor in global risk. Global problematic: Armstrong, Stuart, and Anders Sandberg.: sciences 7.7 (2003): 320-324. catastrophic risks 1 (2008): 303, and Omohundro, Eternity in six hours: Intergalactic spreading of intelligent 560 See for instance the multiplicity of values in the World

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 193 Endnotes

Values Survey. Those countries with stronger survival, 11-17. Cameron Heath (Chair), John Campbell, Daniel Clarke, and weaker self expression, values would be presumably 579 See Zielinski, G. A., et al.: Potential atmospheric Darren Farr, Gladys Hosken, Gillian James, Andrew more likely to tolerate dictatorships in exchange for impact of the Toba Mega‐Eruption 71,000 years ago. Newman, David Simmons, and Hannes Van Rensburg reduced casualties. Geophysical Research Letters 23.8 (1996): 837-840. www.actuaries.org.uk/system/files/documents/pdf/1- 561 These papers provide an overview of the potential 580 See Jablonski, David, and W. G. Chaloner.: 200-final-paper.pdf 600 resources for future human generations, hinting at their Extinctions in the Fossil Record [and Discussion]. http://www.itm.su.se/page.php?pid=1081 potential size: Bostrom, Nick.: Astronomical waste: The Philosophical Transactions of the Royal Society of 601 http://www.ipcc.ch opportunity cost of delayed technological development. London. Series B: Biological Sciences 344.1307 (1994): 602 Schaeffer, Michiel; Hare, William; Rahmstorf, Stefan; Utilitas 15.03 (2003): 308-314, and Armstrong, Stuart, 11-17. Vermeer, Martin (2009): Long-term sea-level rise implied and Anders Sandberg.: Eternity in six hours: Intergalactic 581 See Hall, Christopher D., and I. Michael Ross.: by 1.5 C and 2 C warming levels. Nature http://www. spreading of intelligent life and sharpening the Fermi Dynamics and control problems in the deflection of nature.com/nclimate/journal/v2/n12/full/nclimate1584. paradox. Acta Astronautica (2013). Near-Earth Objects. Adv. Astronaut. Sci 67.640 (1997): html 562 See Bostrom, Nick.: Existential Risk Prevention as 1-18. 603 Sources for the estimates include: Global Priority. Global Policy 4.1 (2013): 15-31. http:// 582 See Parmesan, Camille.: Ecological and evolutionary Fekete, Hanna; Vieweg, Marion; Rocha, Marcia; www.existential-risk.org/concept.pdf responses to recent climate change. Annu. Rev. Ecol. Braun, Nadine; Lindberg, Marie; Gütschow, Johannes; 563 See Oxford Poverty and Human Development Evol. Syst. 37 (2006): 637-669. Jefferey, Louise; Höhne, Niklas ; Hare, Bill; Schaeffer, Initiative (2013), Multidimensional Poverty Index (MPI) 583 See Epstein, Paul R.: Climate and health. Science Michiel; Macey, Kirsten and Larkin, Julia (2013): Data Bank. OPHI, University of Oxford. 285.5426 (1999): 347-348. Analysis of current greenhouse gas emission trends 564 See Ravallion, Martin.: A comparative perspective on 584 See the World Health Organization Global Influenza http://climateactiontracker.org/assets/publications/ poverty reduction in Brazil, China and India. World Bank Programme.: Pandemic influenza preparedness and publications/CAT_Trend_Report.pdf Policy Research Working Paper Series (2009). response: a WHO guidance document. World Health New, Mark G.; Liverman, Diana M.; Betts, Richard 565 See for instance Jeanneney, Sylviane Guillaumont, Organization (2009). A.; Anderson, Kevin L. and West, Chris C. (2011): and Kangni Kpodar.: Financial development and poverty 585 See Yudkowsky, Eliezer.:Artificial intelligence as Four degrees and beyond: the potential for a global reduction: Can there be a benefit without a cost? The a positive and negative factor in global risk. Global temperature increase of four degrees and its implications Journal of Development Studies 47.1 (2011): 143-163. catastrophic risks 1 (2008): 303. http://rsta.royalsocietypublishing.org/content/369/1934. 566 See the Guardian’s report, The NSA Files. 586 See Freitas Jr, Robert A.: Some limits to global toc 567 The reaction was both within the USA and without; ecophagy by biovorous nanoreplicators, with public Rogelj, Joeri (2013): Risk shifts under changing climate see for instance Der Spiegel’s report, Out of Hand: policy recommendations. Foresight Institute (2000). sensitivity estimates, http://global-risk-indicator.net/data/ Europe Furious Over US Spying Allegations. 587 See the report of the Center for Responsible pdf_01.pdf 568 Many news stories raised the issue; see for instance Nanotechnology, Dangers of Molecular Manufacturing. Schneider, Stephen H. (2005): What is the Probability of the PBS report NSA repeatedly failed to obey court 588 See for instance Imai, Masaki, et al.: Experimental ‘Dangerous’ Climate Change? http://stephenschneider. orders on how to conduct surveillance on Americans. adaptation of an influenza H5 HA confers respiratory stanford.edu/Climate/Climate_Impacts/ 569 See the UNEP press release UNDP-UNEP Poverty- droplet transmission to a reassortant H5 HA/H1N1 virus WhatIsTheProbability.html Environment Initiative Launches New Five-Year Phase to in ferrets. Nature 486.7403 (2012): 420-428. Sokolov, A. P., and Co-authors, 2009: Probabilistic Meet Growing Demand from Member States (2013). 589 See Drexler, K. Eric.: Radical Abundance: How a Forecast for Twenty-First-Century Climate Based on 570 See the UNEP news report Sharing the waste, Revolution in Nanotechnology Will Change Civilization. Uncertainties in Emissions (Without Policy) and Climate sharing the wealth: Uruguay uses the law to catalyse the Public Affairs (2013). Parameters. J. Climate, 22, 5175–5204. transition to an inclusive green economy. 590 See Drexler, K. Eric.: Radical Abundance: How a http://journals.ametsoc.org/doi/ 571 See the UNEP news report Malawi changes course Revolution in Nanotechnology Will Change Civilization. abs/10.1175/2009JCLI2863.1 after analysing the real costs and benefits of policy Public Affairs (2013). World Bank (2013): Turn Down the Heat: Climate choices. 591 See Sailor, William C., et al.: A nuclear solution to Extremes, Regional Impacts, and the Case for Resilience 572 See Ravallion, Martin.: A comparative perspective on climate change? Science (Washington) 288.5469 (2000): http://www.worldbank.org/content/dam/Worldbank/ poverty reduction in Brazil, China and India. World Bank 1177-1178. http://faculty.publicpolicy.umd.edu/sites/ document/Full_Report_Vol_2_Turn_Down_The_ Policy Research Working Paper Series (2009). default/files/fetter/files/2000-Science-NE.pdf Heat_%20Climate_Extremes_Regional_Impacts_Case_ 573 See Dollar, David, and Aart Kraay.: Trade, growth, 592 See Harding, Jim.: Economics of nuclear power for_Resilience_Print%20version_FINAL.pdf 604 and poverty World Bank, Development Research Group, and proliferation risks in a carbon-constrained http://news.stanford.edu/news/2003/september24/ Macroeconomics and Growth (2001). world. The Electricity Journal 20.10 (2007): 65-76. tellerobit-924.html 574 World Bank address to a ministerial conference on http://www.sciencedirect.com/science/article/pii/ 605 Robock, A., L. Oman, and G. L. Stenchikov responses to pandemic threat, 2008, in Background S1040619007001285 (2007), Nuclear winter revisited with a modern climate Paper, Pandemic risk, by Olga B. Jonas, The World 593 Morse, Stephen S., et al.: Prediction and prevention model and current nuclear arsenals: Still catastrophic Bank: http://un-influenza.org/sites/default/files/WDR14_ of the next pandemic zoonosis. The Lancet 380.9857 consequences, J. Geophys. Res., 112, D13107, bp_Pandemic_Risk_Jonas.pdf (2012): 1956-1965. http://www.thelancet.com/journals/ doi:10.1029/2006JD008235. http://www.envsci.rutgers. 575 See Esser, James K.: Alive and well after 25 years: A lancet/article/PIIS0140-6736(12)61684-5/abstract edu/~gera/nwinter/nw6accepted.pdf 606 review of groupthink research. Organizational behavior 594 See this text by Nick Bostrom discussing difficulties Sources for the estimates include: and human decision processes 73.2 (1998): 116-141 estimating existential risks.: http://www.nickbostrom. Brams, Steven J.;Kilgour, D. Marc; Fichtner, John; & and Kerr, Norbert L., and R. Scott Tindale.: Group com/existential/risks.html Avenhaus, Rudolf (1989): The Probability of Nuclear War, performance and decision making. Annu. Rev. Psychol. 595 For an explanation of how to calculate probabilities Journal of Peace Research 26(1): 91–99. http://econ. 55 (2004): 623-655. see: http://en.wikipedia.org/wiki/100-year_flood as.nyu.edu/docs/IO/9397/RR86-24.pdf 576 See Homer-Dixon, Thomas.: The upside of down: 596 http://www.lloyds.com/the-market/business- Helfand, Ira. “NUCLEAR FAMINE: TWO BILLION catastrophe, creativity, and the renewal of civilization. timetable/solvency/ica PEOPLE AT RISK?” International Physicians for the Island Press (2008). 597 http://en.wikipedia.org/wiki/Solvency_II Prevention of Nuclear War (2013). http://www.psr.org/ 577 See Turco, Richard P., et al.: Nuclear winter: global assets/pdfs/two-billion-at-risk.pdf 598 What is a 1-in-200? Presented at GIRO 2009 by consequences of multiple nuclear explosions. Science Cameron Heath (Chair), John Campbell, Daniel Clarke, Hellman, Martin E. (2008); ”Risk Analysis of Nuclear 222.4630 (1983): 1283-1292. Darren Farr, Gladys Hosken, Gillian James, Andrew Deterrence” in THE BENT OF TAU BETA PI http:// 578 See Jablonski, David, and W. G. Chaloner.: Newman, David Simmons, and Hannes Van Rensburg nuclearrisk.org/paper.pdf Extinctions in the Fossil Record [and Discussion]. www.actuaries.org.uk/system/files/documents/pdf/1- Lugar, Richard G (2005): ”The Lugar Survey On Philosophical Transactions of the Royal Society of 200-final-paper.pdf Proliferation Threats and Responses” https://www.fas. London. Series B: Biological Sciences 344.1307 (1994): 599 What is a 1-in-200? Presented at GIRO 2009 by org/irp/threat/lugar_survey.pdf

194 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Endnotes

Robock, A., L. Oman, and G. L. Stenchikov 612 http://en.wikipedia.org/wiki/Bullwhip_effect and the future of biosecurity.” Politics and the Life (2007), Nuclear winter revisited with a modern 613 See for example Gao, Qin and Ma, Junhai (2009): Sciences 28.2 (2009): 2-26. climate model and current nuclear arsenals: Still Chaos and Hopf bifurcation of a finance system and Russ, Zachary N. “Synthetic biology: enormous catastrophic consequences, J. Geophys. Res., 112, Hommes, Cars, and Florian Wagener (2009): Complex possibility, exaggerated perils.” Journal of biological D13107, doi:10.1029/2006JD008235. http://web. evolutionary systems in behavioral finance. Handbook of engineering 2.7 (2008). kaust.edu.sa/faculty/GeorgiyStenchikov/nwinter/ financial markets: Dynamics and evolution Radosavljevic, Vladan, and Goran Belojevic. “A new RobockJgr2006JD008235.pdf 614 Sleep, Norman H., et al. (1989): Annihilation of model of bioterrorism risk assessment.” Biosecurity and Shulman, Carl (2012): Nuclear winter and human ecosystems by large asteroid impacts on the early Earth. bioterrorism: biodefense strategy, practice, and science extinction: Q&A with Luke Oman http://www. Nature 342.6246 7.4 (2009): 443-451. overcomingbias.com/2012/11/nuclear-winter-and- 615 http://neo.jpl.nasa.gov/risks/doc/sentry.html Tucker, Jonathan B., and Raymond A. Zilinskas. “The human-extinction-qa-with-luke-oman.html 616 https://b612foundation.org/ promise and perils of synthetic biology.” New Atlantis 607 See for example: http://metabiota.com/ and http:// 617 http://neo.jpl.nasa.gov/risks/ 12.1 (2006): 25-45. instedd.org/ 623 618 Altmann, Jürgen, and Mark A. Gubrud.: Military, 608 Sources for the estimates include: Sources for the estimates include: arms control, and security aspects of nanotechnology. Jablonski, David, and W. G. Chaloner.: Extinctions Bagus, Ghalid (2008): Pandemic Risk Modeling Discovering the Nanoscale (2004): 269 http:// in the Fossil Record [and Discussion]. Philosophical nanoinvesting.my-place.us/altmann-gubrud.pdf http://www.chicagoactuarialassociation.org/CAA_ Transactions of the Royal Society of London. Series B: 624 PandemicRiskModelingBagus_Jun08.pdf Biological Sciences 344.1307 (1994): 11-17. Quoted in Kaku, Michio: Visions: how science will revolutionize the 21st century. Oxford University Press, Broekhoven, Henk van, Hellman, Anni (2006): Actuarial Collins, Gareth S., H. Jay Melosh, and Robert A. 1999 reflections on pandemic risk and its consequences Marcus.: Earth Impact Effects Program: A Web‐ 625 http://actuary.eu/documents/pandemics_web.pdf based computer program for calculating the regional Altmann, Jürgen, and Mark A. Gubrud.: Military, Brockmann, Dirk and Helbing, Dirk (2013): The Hidden environmental consequences of a meteoroid impact arms control, and security aspects of nanotechnology. Geometry of Complex, Network-Driven Contagion on Earth. Meteoritics & planetary science 40.6 (2005): Discovering the Nanoscale (2004): 269 http:// Phenomena SCIENCE VOL 342 http://rocs.hu-berlin.de/ 817-840. nanoinvesting.my-place.us/altmann-gubrud.pdf resources/HiddenGeometryPaper.pdf Monthly Notices of the Royal Astronomical Society 327.1 Berube, David M., et al. “Communicating Risk in the W. Bruine de Bruin, B. Fischhoff; L. Brilliant and D. (2001): 126-132, and Board, Space Studies.: Defending 21st Century: The case of nanotechnology.” National Caruso (2006): Planet Earth: Near-Earth Object Surveys and Hazard Nanotechnology Coordination Office, Arlington (2010). Expert judgments of pandemic influenza risks, Global Mitigation Strategies. National Academies Press (2010). Bostrom, Nick and Sandberg, Anders (2008): Global Public Health, June 2006; 1(2): 178193 http://www.cmu. http://neo.jpl.nasa.gov/risks/ catastrophic risks survey civil wars 98.3 http://www.fhi. ox.ac.uk/gcr-report.pdf edu/dietrich/sds/docs/fischhoff/AF-GPH.pdf Neukum, G., and B. A. Ivanov. (1994): Crater size Khan K, Sears J, Hu VW, Brownstein JS, Hay S, distributions and impact probabilities on Earth from lunar, Turchin, Alexey.: Structure of the global catastrophe. Kossowsky D, Eckhardt R, Chim T, Berry I, Bogoch terrestrial-planet, and asteroid cratering data. Hazards Risks of human extinction in the XXI century. Lulu. com, I, Cetron M.: Potential for the International Spread due to Comets and Asteroids 359 2008. of Middle East Respiratory Syndrome in Association Chodas, P., and D. Yeomans. (1999): Orbit determination Williams, Richard A., et al. “Risk characterization for with Mass Gatherings in Saudi Arabia. PLOS Currents and estimation of impact probability for near-Earth nanotechnology.” Risk Analysis 30.11 (2010): 1671-1679. 626 Outbreaks. 2013 Jul 17. Edition 1. doi: 10.1371/currents. objects. http://www.fhi.ox.ac.uk/wp-content/uploads/Racing- outbreaks.a7b70897ac2fa4f79b59f90d24c860b8. Chapman, Clark R., and David Morrison. (1994): Impacts to-the-precipice-a-model-of-artificial-intelligence- http://currents.plos.org/outbreaks/article/assessing-risk- on the Earth by asteroids and comets: assessing the development.pdf 627 for-the-international-spread-of-middle-east-respiratory- hazard. Nature 367.6458 Bostrom, Nick and Sandberg, Anders (2008): Global syndrome-in-association-with-mass-gatherings-in- Chapman, Clark R., Daniel D. Durda, and Robert E. catastrophic risks survey civil wars 98.3 http://www.fhi. saudi-arabia/ Gold. (2001): The comet/asteroid impact hazard: a ox.ac.uk/gcr-report.pdf Murray, Christopher JL, et al.: Estimation of potential systems approach. Office of Space Studies, Southwest Kruel Alexander (2013): Probability of unfriendly and global pandemic influenza mortality on the basis of vital Research Institute, Boulder CO 80302 friendly AI, including comments to the blog post registry data from the 1918–20 pandemic: a quantitative 619 Sources for the estimations include: http://kruel.co/2013/09/23/probability-of-unfriendly-and- analysis. The Lancet 368.9554 (2007): 2211-2218. http:// 620 E.g. US Government laboratories had 395 incidents friendly-ai/#sthash.xRhFOGHW.dpbs www.thelancet.com/journals/lancet/article/PIIS0140- 628 that involved the potential release of selected agents Bostrom, Nick and Sandberg, Anders (2008): Global 6736(06)69895-4/fulltext between 2003 and 2009, though only seven related catastrophic risks survey civil wars 98.3 http://www.fhi. Sandman , Peter M. (2007): Talking about a flu pandemic infections were reported. The accidents, including ox.ac.uk/gcr-report.pdf worst-case scenario, http://www.cidrap.umn.edu/news- animal bites, needle sticks, and other mishaps, are 629 http://www.who.int/trade/glossary/story038/en/ perspective/2007/03/talking-about-flu-pandemic-worst- mentioned briefly in an NRC report on the plans for a 630 One of those who led the development of how case-scenario risk assessment for an Army biodefense lab to be built at organisations can become destructive is Philip 609 Sources for the estimates include: Peterson, G. D., S. Ft. Detrick in Frederick, Md. National Research Council. Zimbardo, Stanford University http://www.lucifereffect. R. Carpenter, and William A. Brock.: Uncertainty and the Review of Risk Assessment Work Plan for the Medical com/ management of multistate ecosystems: an apparently Countermeasures Test and Evaluation Facility at Fort 631 http://www.economist.com/news/briefing/21578643- rational route to collapse. Ecology 84.6 (2003): 1403- Detrick: A Letter Report. Washington, DC: The National world-has-astonishing-chance-take-billion-people- 1411. Academies Press, 2011: http://www.nap.edu/catalog. out-extreme-poverty-2030-not?fsrc=scn/tw/te/pe/im/ Butler, Colin D., Carlos F. Corvalan, and Hillel S. Koren.: php?record_id=13265 notalwayswithus Human health, well-being, and global ecological 621 Clauset, Aaron, and Ryan Woodard. (2013): 632 http://www.worldbank.org/en/news/press- scenarios. Ecosystems 8.2 (2005): 153-162.Brook, Barry Estimating the historical and future probabilities of large release/2013/10/09/world-bank-sets-interim-poverty- W., Navjot S. Sodhi, and Corey J A Bradshaw.: Synergies terrorist events. The Annals of Applied Statistics 7.4: target-at-9-percent-in-2020 among extinction drivers under global change. Trends in 1838-1865. http://arxiv.org/pdf/1209.0089v3.pdf 633 http://www.worldbank.org/en/news/press- ecology & evolution 23.8 (2008): 453-460. 622 Bennett, Gaymon, et al. “From synthetic biology to release/2013/10/09/world-bank-sets-interim-poverty- Conversations with Johan Rockstrom, Executive director biohacking: are we prepared?.” Nature Biotechnology target-at-9-percent-in-2020 of the Stockholm Resilience Centre. 27.12 (2009): 1109-1111. 634 http://econ.worldbank.org/external/default/ 610 This is true for all complex systems. E.g. Sage, A.P., Bostrom, Nick and Sandberg, Anders (2008): Global main?pagePK=64165259&theSitePK=469372&piPK=64 Systems engineering: Fundamental limits and future catastrophic risks survey: civil wars 98.3 http://www.fhi. 165421&menuPK=64166093&entityID=000094946_030 prospects, Proceedings of the IEEE , vol.69, no.2, ox.ac.uk/gcr-report.pdf pp.158,166, Feb. 1981 11004010512 Mukunda, Gautam, Kenneth A. Oye, and Scott C. Mohr. 611 635 http://en.wikipedia.org/wiki/History_of_Russia_ Sources for the estimates include: “What rough beast? Synthetic biology, uncertainty,

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 195 Endnotes

(1992%E2%80%93present) 656 Nagy, Béla; Doyne Farmer, J.; M. Bui, Quan; E. 636 http://www.brookings.edu/research/interactives/2013/ Trancik, Jessika (2013): A projection of future PV ending-extreme-poverty electricity costs from the Photovoltaics2 historical data 637 http://www.brookings.edu/research/interactives/2013/ set (1977–2009) using Moore’s exponential functional ending-extreme-poverty form. Figure_6.tif. PLOS ONE. 10.1371/journal. pone.0052669.g006. 638 http://www.economist.com/news/briefing/21578643- 657 world-has-astonishing-chance-take-billion-people- http://www.itu.int/en/ITU-D/Statistics/Pages/intlcoop/ out-extreme-poverty-2030-not?fsrc=scn/tw/te/pe/im/ mdg/default.aspx 658 notalwayswithus http://www.un.org/esa/population/publications/ 639 http://en.wikipedia.org/wiki/Malthusian_catastrophe worldageing19502050/pdf/90chapteriv.pdf 659 640 http://www.worldbank.org/depweb/english/beyond/ http://www.un.org/esa/population/publications/ global/chapter3.html worldageing19502050/pdf/90chapteriv.pdf 660 641 http://www.worldbank.org/depweb/english/beyond/ http://www.un.org/esa/population/publications/ global/chapter3.html worldageing19502050/pdf/90chapteriv.pdf 661 642 http://www.worldbank.org/depweb/english/beyond/ http://www.un.org/esa/population/publications/ global/chapter3.html worldageing19502050/pdf/90chapteriv.pdf 662 643 http://esa.un.org/wpp/Documentation/pdf/WPP2012_ http://en.wikipedia.org/wiki/Big_data 663 HIGHLIGHTS.pdf For some examples of visualisation tools: http://www. 644 http://esa.un.org/wpp/Documentation/pdf/WPP2012_ creativebloq.com/design-tools/data-visualization-712402 664 HIGHLIGHTS.pdf For a discussion about risk and the importance 645 http://esa.un.org/wpp/Documentation/pdf/WPP2012_ of probability curves of risk, see Kaplan, Stanley and HIGHLIGHTS.pdf Garrick, John B., On the Quantitative Definition of Risk, Risk Analysis, Vol. I, No. 1, 1981, in which the 646 http://www.worldometers.info/world-population/ authors state (pp. 11–27): “We prefer to say that ‘risk is 647 http://esa.un.org/unpd/wpp/Documentation/pdf/ probability and consequence.’ In the case of a single WPP2010_Wallchart_Text.pdf scenario the probability times consequence viewpoint 648 http://wwf.panda.org/about_our_earth/all_ would equate a low-probability high-damage scenario publications/living_planet_report/ with a high-probability low-damage scenario – clearly not 649 http://www.worldbank.org/depweb/english/beyond/ the same thing at all. In the case of multiple scenarios the global/chapter8.html probability times consequence view would correspond 650 http://en.wikipedia.org/wiki/List_of_wars_and_ to saying the risk is the expected value of damage, i.e., anthropogenic_disasters_by_death_toll the mean of the risk curve. We say it is not the mean of 651 Van Evera, Stephen (2013): Causes of war: Power the curve, but the curve itself which is the risk. A single and the roots of conflict. Cornell University Press, 2013. number is not a big enough concept to communicate Xu Jin (2006): The Strategic Implications of Changes the idea of risk. It takes a whole curve. Now the truth is in Military Technology, Chinese Journal of International that a curve is not a big enough concept either. It takes a Politics 2006 1: 163-193. http://cjip.oxfordjournals.org/ whole family of curves to fully communicate the idea of content/1/2/163.full risk.” 665 652 http://www.plosone.org/article/ http://www.ipcc.ch/pdf/supporting-material/ info%3Adoi%2F10.1371%2Fjournal.pone.0052669 uncertainty-guidance-note.pdf 666 653 http://www.kurzweilai.net/the-law-of-accelerating- SNRA covers risks such as school shootings, returns heatwaves and collapses of hydroelectric dams 667 654 Lloyd, Seth (2000): Ultimate physical limits to https://www.msb.se/RibData/Filer/pdf/26561.pdf 668 computation, http://arxiv.org/pdf/quant-ph/9908043. http://www.ipcc.ch/pdf/supporting-material/ pdf?origin=publication_detail uncertainty-guidance-note.pdf 669 http://www.nature.com/nature/journal/v406/n6799/ Richard Possner has described key elements of such full/4061047a0.html a centre in the book Catastrophe: risk and response. 655 E.g. Claw E. (1998) The natural limits of technological Posner, Richard A. Catastrophe: risk and response. innovation: http://www.sciencedirect.com/science/ Oxford University Press, 2004. article/pii/S0160791X98000050

196 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Endnotes

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 197 Appendix 1 – Global Challenges Bibliography

Appendix 1 – Global Challenges Bibliography

The Global Catastrophic Risk Insti- More publications were identified Further, the term is used by re- tute (GCRI) published a GCR bibliog- by searching scholarly databases searchers from a variety of different raphy compiled in July 2011 by Seth (mainly Web of Science and Google backgrounds. This makes it a par- Baum, available at http://gcrinstitute. Scholar) and databases of popular ticularly fruitful term for discovering org/bibliography. This contains 115 literature (mainly Amazon and the new global challenges research. entries, emphasising publications New York Public Library) for relevant surveying the breadth of the risks or keywords and for citations of the One hallmark of the global chal- discussing other topics of general publications already identified. lenges topic is that it is studied by interest to the study of GCR, with distinct research communities that less emphasis on analysis of spe- Several keywords and phrases have limited interaction with each cific global challenges. It has been were searched for in the databases: other. As research communities often updated for this Global Challenges “existential catastrophe”; “exis- develop their own terminology, it can Foundation report and now contains tential risk”; “global catastrophe”; be difficult to discover one commu- 178 entries. “global catastrophic risk”; “greatest nity by searching for another’s terms. global challenges”; “human extinc- For example, “existential risk” is The reason for focusing on general tion”; “xrisk”; and “infinite risk”. The used heavily by researchers studying interest publications is because the results of these searches were then risk from artificial intelligence and literature on specific global challeng- screened for relevant publications. other emerging technologies, but it is es is far too voluminous to cata- Many of the results were not rele- rarely used by researchers studying logue. It would include, for example, vant, because these terms are used environmental risks. Discovering and a significant portion of the literatures in other ways. For example, “existen- connecting the disparate corners on climate change, energy, nuclear tial risk” is sometimes used to refer of the GCR research is an ongoing weapons, infectious diseases and to risks to the existence of business- challenge for the GCR community. biodiversity, all topics that receive es, countries or other entities; “hu- Bibliography searches such as these extensive research attention. Thus man extinction” is used in the study are an important way to meet this the full bibliography compiled by of memory. The publications that challenge. GCRI is only a small portion of the use these terms in the same sense total global challenges literature. as the bibliography were then further screened for publications of general Publications for the full bibliography global challenges interest, not for were identified in several ways. The specific global challenges. bibliography began with publications already known to fit the selection The most productive search term for criteria. Additional publications were the database searches turned out to identified by examining the reference be “global catastrophe”. This term lists of the initial publications. produced a relatively large number of hits and relatively few publications on unrelated topics.

198 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Appendix 1 – Global Challenges Bibliography

– Andersen, Ross, 2012. We’re under- – Baum, Seth D., 2009. Global Cat- – Bloom, Barry, 2003. Bioterrorism and estimating the risk of human extinc- astrophic Risks (book review). Risk the university: The threats to security – tion. The Atlantic, March 6. http:// Analysis, vol. 29, no. 1 (January), and to openness. Harvard Magazine, www.theatlantic.com/technology/ pages 155-156. http://sethbaum.com/ November-December, pages 48-52. archive/2012/03/were-underesti- ac/2009_Rev-GCR.pdf mating-the-risk-of-human-extinc- – Bommier, Antoine and Stéphane tion/253821 – Baum, Seth D., 2009. Cost-ben- Zuber, 2008. Can preferences for efit analysis of space exploration: catastrophe avoidance reconcile social – Andersen, Ross, 2013. Omens. Some ethical considerations. Space discounting with intergenerational eq- Aeon, February 25. http://aeon.co/ Policy, vol. 25, no. 2 (May), pages uity?. Social Choice and Welfare, vol. magazine/world-views/ross-anders- 75-80. http://sethbaum.com/ac/2009_ 31, no. 3 (October), pages 415-434. en-human-extinction CBA-SpaceExploration.html – Bostrom, Nick 1999. The doomsday – Asimov, Isaac, 1981. A Choice of Ca- – Baum, Seth D., 2010. Is humanity argument is alive and kicking. Mind, tastrophes: The Disasters That Threat- doomed? Insights from astrobiology. vol. 108, no. 431, pages 539-550. en Our World. New York: Ballantine Sustainability, vol. 2, no. 2 (February), http://www.anthropic-principle.com/ Books. pages 591-603. preprints/ali/alive.html

– Barrett, Scott, 2006. The problem of – Baum, Seth D., 2013. Teaching – Bostrom, Nick, 2001. The doomsday averting global catastrophe. Chicago astrobiology in a sustainability course. argument, Adam & Eve, UN++, and Journal of International Law, vol. 6, no. Journal of Sustainability Education, Quantum Joe. Synthese, vol. 127, 2, pages 527-552. February (online): http://www.jsed- no. 3, pages 359-387. http://www. imensions.org/wordpress/content/ anthropic-principle.com/preprints/cau/ – Barrett, Scott, 2007. Why Cooper- teaching-astrobiology-in-a-sustainabil- paradoxes.html ate? The Incentive to Supply Global ity-course_2013_02. Public Goods. Oxford: Oxford Univer- – Bostrom, Nick, 2002. Existential sity Press. – Baum, Seth D., Timothy M. Maher risks: Analyzing human extinction Jr., and Jacob Haqq-Misra, 2013. scenarios and related hazards. Journal – Barrett, Anthony M., Seth D. Baum, Double catastrophe: Intermittent of Evolution and Technology, 9. http:// and Kelly R. Hostetler, 2013. Analyzing stratospheric geoengineering induced www.jetpress.org/volume9/risks.pdf and reducing the risks of inadvertent by societal collapse. Environment, nuclear war between the United States Systems, and Decisions, vol. 33, no. 1 – Bostrom, Nick, 2003. Astronomical and Russia. Science and Global Secu- (March), pages 168-180. waste: The opportunity cost of delayed rity, vol. 21, no. 2, pages 106-133. technological development. Utilitas, – Baum, Seth D. and Grant S. Wilson, Vol. 15, No. 3, pp. 308-314. http:// – Baudiment, Fabienne Goux, 2009. 2013. The ethics of global catastrophic www.nickbostrom.com/astronomical/ Tomorrow will die. Futures, vol. 41, no. risk from dual-use bioengineering. Eth- waste.pdf 10 (December), pages 746-753. ics in Biology, Engineering and Medi- cine, vol. 4, no. 1, pages 59-72. – Bostrom, Nick, 2006. Dinosaurs, – Baum, Seth, 2008. Reducing cat- dodos, humans?. Global Agenda, astrophic risk through integrative – Beckstead, Nicholas, 2013. On The January, pages 230-231. http://www. assessment. Hawaii Reporter, August Overwhelming Importance Of Shaping nickbostrom.com/papers/globalagen- 25; Daily Camera (Boulder, CO), Sep- The Far Future. Doctoral Dissertation, da.pdf tember 2 http://www.amergeog.org/ Department of Philosophy, Rutgers newsrelease/baum-hawaii08.pdf University. – Bostrom, Nick, 2011. Existential risk FAQ. http://www.existentialrisk.com/ – Baum, Seth D., 2008. Better to exist: – Benatar, David, 2006. Better Never to faq.html A reply to Benatar. Journal of Medical Have Been: The Harm of Coming Into Ethics, vol. 34, no. 12 (December), Existence. Oxford: Clarendon Press. – Bostrom, Nick (2013) Existential risk pages 875-876. http://sethbaum.com/ prevention as a global priority. Global ac/2008_BetterToExist.pdf Policy 4(1) 15-31.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 199 Appendix 1 – Global Challenges Bibliography

– Bostrom, Nick and Milan M. Ćirković, – Charles, Daniel, 2006. A ‘forever’ – Ćirković, Milan, Anders Sandberg, 2003. The doomsday argument and seed bank takes root in the Arctic. and Nick Bostrom, 2010. Anthropic the self-indication assumption: Reply Science, vol. 312, no. 5781 (23 June), shadow: Observation selection effects to Olum. Philosophical Quarterly, vol. pages 1730-1731. and human extinction risks. Risk Anal- 53, no. 210 (January), pages 83-91. ysis, vol. 30, no. 10 (October), pages http://www.anthropic-principle.com/ – Chichilnisky, Graciela, 2009. Avoid- 1495-1506. http://www.nickbostrom. preprints/olum/sia.pdf ing extinction: Equal treatment of the com/papers/anthropicshadow.pdf present and the future. Economics, – Bostrom, Nick and Milan Ćirković, vol. 3, article 2009-32. http://www. – Cloninger, C. Robert, 2013. What 2008. Global Catastrophic Risks. economics-ejournal.org/economics/ Makes People Healthy, Happy, and Oxford: Oxford University Press. http:// journalarticles/2009-32 Fulfilled In The Face Of Current World www.global-catastrophic-risks.com/ Challenges? Mens Sana Monographs book.html – Chichilnisky, Graciela, 2009. Avoiding 11(1): 16–24. http://www.ncbi.nlm.nih. extinction: The future of economics. gov/pmc/articles/PMC3653221 – Bostrom, Nick and Milan Ćirković, International Journal of Green Eco- 2008. Introduction. In Nick Bostrom nomics, vol. 3, no. 1, pages 1-18. – Coates, Joseph F. et al., 1979. Risks and Milan Ćirković (eds), Global Cata- to Humankind. Washington, DC: U.S. strophic Risks. Oxford: Oxford Uni- – Chichilnisky, Graciela, 2009. Cata- Congress, Office of Technology As- versity Press http://www.global-catastrophic risks. International Journal of sessment, March. strophic-risks.com/docs/Chap01.pdf Green Economics, vol. 3, no. 2, pages 130-141. – Coates, Joseph F., 2009. Risks and – Buchholz, Wolfgang, Michael Schy- threats to civilization, humankind, and mura, 2012. Expected utility theory – Chichilnisky, Graciela and Peter Ei- the earth. Futures, vol. 41, no. 10 (De- and the tyranny of catastrophic risks. senberger, 2010. Asteroids: Assessing cember), pages 694-705. Ecological Economics, vol. 77, pages catastrophic risks. Journal of Proba- 234–239. bility and Statistics, vol. 2010, article – Colebrook, Claire, 2013. Framing the 954750. http://www.hindawi.com/jour- End of the Species: Images without – Butzer, Karl W., Georgina H. Endfield, nals/jps/2010/954750 Bodies. symploke, Volume 21, Num- editors, 2012. Critical Perspectives bers 1-2, pages 51-63. on Historical Collapse. Proceedings – Ćirković, Milan M., 2002. Cosmo- of the National Academy of Scienc- logical Forecast and its Practical – Cooke, Roger M. and Carolyn es (Special Feature), vol. 109, pages Significance. Journal of Evolution and Kousky, 2009. Climate change and risk 3628-3681. Technology, xii. http://www.jetpress. management: challenges for insur- org/volume12/CosmologicalForecast. ance, adaptation, and loss estimation. – Cairns, John Jr., 2010. Global Crisis pdf Resources for the Future Discussion Collaboration: The Key to the Surviv- Paper 09-03. al of Civilization in the 21st Century. – Ćirković, Milan M., 2007. Evolu- Asian Journal of Experimental Scienc- tionary catastrophes and the Goldi- – Coughlan, Sean, 2013. How are hu- es, Vol. 24, No. 1, 33-38. locks problem. International Journal mans going to become extinct? BBC of Astrobiology, vol. 6, no. 4, pages News, April 24. http://www.bbc.co.uk/ – Carpenter, P. A. and P. C. Bishop, 325-329. http://arxiv.org/ftp/arxiv/pa- news/business-22002530 2009. A review of previous mass pers/0709/0709.2309.pdf extinctions and historic catastrophic – Curry, Nathan, 2013. Some Cred- events. Futures, vol. 41, no. 10 (De- –Ćirković, Milan M., 2012. Small Theo- ible Scientists Believe Humanity is cember), pages 676-682. ries and Large Risks—Is Risk Analysis Irreparably Close to Destruction. Vice Relevant for Epistemology? Risk Anal- Canada. http://www.vice.com/en_ca/ – Carpenter, P. A. and P. C. Bishop, ysis, Vol. 32, No. 11, p. 1994-2004. read/near-term-extinctionists-believe- 2009. The seventh mass extinction: the-world-is-going-to-end-very-soon Human-caused events contribute to a fatal consequence. Futures, vol. 41, no. 10 (December), pages 715-722.

200 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Appendix 1 – Global Challenges Bibliography

– de Neufville, Robert, 2013. Can the – Eckhardt, William, 1993. Probability – Haller, Stephen F., 2002. Apoca- human race survive? Anthropocene, theory and the doomsday argument. lypse Soon? Wagering on Warnings of January 31. http://anthropoceneblog. Mind, vol. 102, no. 407 (July), pages Global Catastrophe. Montreal: McGill- wordpress.com/2013/01/31/can-the- 483-488. http://williameckhardt.com. Queens University Press. human-race-surviv p2.hostingprod.com/yahoo_site_ad- min/assets/docs/William_Eckhardt_-_ – Hanson, Robin, 1998. Critiquing the – Dieks, Dennis, 1992. Doomsday - Or: Probability_Theory_and_the_Dooms- doomsday argument. Unpublished The dangers of statistics. Philosophical day_Argument.131122037.pdf manuscript, 27 August. http://hanson. Quarterly, vol. 42, no. 166 (January), gmu.edu/nodoom.html pages 78-84. – Epstein, Richard J. and Y. Zhao, 2009. The threat that dare not speak – Hayne, David M. (ed), 1999. Human – Draper, John William, 2013. Ex- its name: Human extinction. Perspec- Survivability in the 21st Century. Toron- tinction and Risk. U of Penn Law tives in Biology and Medicine, vol. 52, to: University of Toronto Press. School, Public Law Research Paper no. 1 (Winter), pages 116-125. Hempsell, C. M., 2004. The investi- No. 12-24; U of Penn, Inst for Law gation of natural global catastrophes. & Econ Research Paper No. 12-14. – Faber, M.H., 2010. Critical Issues in Journal of the British Interplanetary Available at SSRN: http://ssrn.com/ the Management of Catastrophic and Society, vol. 57, pages 2-13. abstract=2030241 Global Risks, ISRERM 2010. http:// www.adaptationlearning.net/system/ – Hempsell, C. M., 2004. The poten- – Dvorsky, George, 2012. Does a files/GlobalCatastrophicRisks.pdf tial for space intervention in global galaxy filled with habitable planets catastrophes. Journal of the British mean humanity is doomed? io9, June – Faber, M.H., 2011. On the govern- Interplanetary Society, vol. 57, pages 18. http://io9.com/5919110/does-a- ance of global and catastrophic risks. 14-21. galaxy-filled-with-habitable-planets- International Journal of Risk Assess- mean-humanity-is-doomed ment and Management, vol. 15, no. – Herz, John H., 2003. On human sur- 5-6, pages 400-416. vival: Reflections on survival research – Dvorsky, George, 2012. 9 ways and survival policies. World Futures, humanity could bring about its own – Franck, Thomas M., 2006. Collective vol. 59, no. 3-4 (April-June), pages destruction. io9, December 12. http:// security and UN reform: Between the 135-143. io9.com/5967660/9-ways-humanity- necessary and the possible. Chicago could-bring-about-our-own-destruc- Journal of International Law, vol. 6, no. – Horton, Richard, 2005. Threats to tion 2, pages 597-612. human survival: A WIRE to warn the world. Lancet, vol. 365 (15 January), – Dvorsky, George, 2013. Can one – German Advisory Council on Global 191-193. single person destroy the entire Change, 1998. World in Transition: world? io9, http://io9.com/could-a-sin- Strategies for Managing Global Envi- – Iklé, Fred Charles, 2006. Annihilation gle-individual-really-destroy-the-wor ronmental Risks. Berlin: Springer from Within: The Ultimate Threat to ld-1471212186/1471327744 Nations. New York: Columbia Univer- – Gott, J. Richard III, 1993. Implica- sity Press. – Economist, The, 1998. Human ex- tions of the Copernican Principle for tinction: Sui genocide. The Economist, our future prospects. Nature, vol. 363, – Jablonowski, Mark, 2012. Surviving vol. 349, no. 8099 (December 19), pages 315-319. http://www-psych. progress: Managing the collective risks pages 130-131. http://www.econo- stanford.edu/~jbt/224/Gott_93.pdf of civilization. real-world economics mist.com/node/179963 – Grausam, Daniel, 2011. On Endings: review, 62, p. 121-131. http://www. American Postmodern Fiction and the paecon.net/PAEReview/issue62/Jablo- Cold War. Charlottesville: University of nowski62.pdf Virginia Press. – Jha, Alok, 2011. The Doomsday – Guterl, Fred, 2012. The Fate of the Handbook - 50 Ways to the End of the Species: Why the Human Race May World. London: Quercus. Cause Its Own Extinction and How We Can Stop It. New York: Bloomsbury USA.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 201 Appendix 1 – Global Challenges Bibliography

– Jones, Andrew R., 2009. The next – Koopmans, Tjalling C., 1974. Proof – Leslie, John, 1997. Observer-relative mass extinction: Human evolution or for a case where discounting advanc- chances and the doomsday argument. human eradication. Journal of Cos- es the doomsday. Review of Economic Inquiry, vol. 40, no. 4, pages 427-436. mology, vol. 2, pages 316-333. http:// Studies, vol. 41, pages 117-120. journalofcosmology.com/Extinc- – Leslie, John, 2010. The Risk that tion108.html – Kuznick, Peter J., 2007. Prophets of Humans Will Soon Be Extinct. Philoso- doom or voices of sanity? The evolv- phy, Volume 85, Issue 04, p. 447-463. – Jones, Charles I., 2011. Life and ing discourse of annihilation in the first growth. Working paper, Stanford decade and a half of the nuclear age. – Levin, Josh, 2009. How is America University Graduate School of Busi- Journal of Genocide Research, vol. 9, going to end? Slate’s ‘Choose your ness. http://www.stanford.edu/~chadj/ no. 3 (September), pages 411–441. own apocalypse’ lets you map out life200.pdf the death of the United States. Slate, – Leggett, Mark, 2006. An indicative 7 August. http://www.slate.com/ – Jones, Christopher B., 2009. Gaia costed plan for the mitigation of global id/2223285 bites back: Accelerated warming. risks. Futures, 38, 778-809. Futures, vol. 41, no. 10 (December), Lenman, James 2002. On becoming – Levin, Josh, 2009. How is America pages 723-730. extinct. Pacific Philosophical Quarter- going to end? The world’s leading ly, vol. 83, no. 3 (September), pages futurologists have four theories. Slate, – Joy, Bill, 2000. Why the future 253-269. 7 August. http://www.slate.com/ doesn’t need us. Wired, April, pages id/2223962 238-262. http://www.wired.com/wired/ – Leslie, John, 1989. Risking the archive/8.04/joy.html world’s end. Canadian Nuclear Society - Levin, Josh, 2009. How is America Bulletin, May, pages 10-15. going to end? The apocalypse you – Kaplan, Ralph and Harvey Silver- chose. Plus, the end-of-America social glate, 2006. An urgent cause for – Leslie, John, 1990. Is the end of the network. Slate, 7 August. http://www. philanthropy. Boston Globe, July world nigh?. Philosophical Quarterly, slate.com/id/2224425 23. http://www.boston.com/news/ vol. 40, no. 158 (January), pages 65- globe/editorial_opinion/oped/arti- 72. - Lisboa, Maria Manuel, 2011. The End cles/2006/07/23/an_urgent_cause_for_ of the World: Apocalypse and Its After- philanthropy – Leslie, John, 1990. Risking the math in Western Culture. Cambridge, world’s end. Interchange, vol. 21, no. 1 UK: Open Book Publishers – Kaplan, Ralph and Harvey Silver- (Spring), pages 49-58. glate, 2007. The next generation of - Lopes, Tobin, Thomas J. Chermack, threats. The Boston Globe, February – Leslie, John, 1992. Time and the Deb Demers, Madhavi Kari, Berna- 24. http://www.boston.com/news/ anthropic principle. Mind, vol. 101, no. dette Kasshanna, Tiffani Payne, 2009. globe/editorial_opinion/oped/arti- 403 (July), pages 521-540. Human Extinction Scenario Frame- cles/2007/02/24/the_next_genera- works. Futures, vol. 41, no. 10 (De- tion_of_threats – Leslie, John, 1992. Doomsday revis- cember), pages 731-737. ited. Philosophical Quarterly, vol. 42, – Kent, Adrian, 2003. A critical look at no. 166 (January), pages 85-89. – Maher, Timothy M., Jr. and Seth risk assessments for global catastro- D. Baum, 2013. Adaptation to and phes. Risk Analysis, vol. 24, no. 1 – Leslie, John, 1993. Doom and prob- recovery from global catastrophe. Sus- (February), pages 157-168. abilities. Mind, vol. 102, no. 407 (July) tainability, vol. 5, no. 4 (April), pages Mind, vol. 102, no. 407 (July), pages 1461-1479. – Kock, R. A., 2013. Will the damage 489-491. be done before we feel the heat? – Mann, Charles C., 2012. State of the Infectious disease emergence and – Leslie, John, 1996. The End of the species. Orion Magazine, November/ human response. Animal Health Re- World: The Science and Ethics of Hu- December, pages 16-27. http://www. search Reviews, Volume 14, Issue 2, man Extinction. London: Routledge. orionmagazine.org/index.php/articles/ pp 127-132 article/7146

202 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Appendix 1 – Global Challenges Bibliography

– Martin, James, 2007. The Meaning of – Nordhaus, William D., 2009. An anal- – Posner, Richard A., 2006. Efficient the 21st Century. New York: Riverhead ysis of the dismal theorem. Cowles responses to catastrophic risk. Chica- Penguin. Foundation Discussion Paper, No. go Journal of International Law, vol. 6, 1686. http://cowles.econ.yale.edu/P/ no. 2, pages 511-526. – Mason, Colin, 2003. The 2030 Spike: cd/d16b/d1686.pdf Countdown to Global Catastrophe. – Powell, Corey S., 2000. 20 ways the London: Earthscan. – Norgaard, Richard B. and Paul Baer, world could end swept away. Discover, 2003. Seeing the whole picture. World vol. 21, no. 10 (October). – Matheny, Jason G., 2007. Reducing Futures, vol. 59, no. 3-4 (April-June), the risk of human extinction. Risk Anal- pages 225-239. – Randers, Jorgen, 2008. Global ysis vol. 27, no. 5, pages 1335-1344. collapse — Fact or fiction?. Futures, http://jgmatheny.org/matheny_extinc- – Ord, Toby, Rafaela Hillerbrand, and vol. 40, no. 10 (December), pages tion_risk.htm Anders Sandberg, 2010. Probing the 853-864. improbable: Methodological challeng- – Matheny, Jason G., undated. Ought es for risks with low probabilities and – Rees, Martin, 2003. Our Final Cen- we worry about human extinction?. high stakes. Journal of Risk Research, tury: Will the Human Race Survive the Unpublished working paper, http://jg- vol. 13, no. 2 (March), pages 191-205. Twenty-first Century? Oxford: William matheny.org/extinctionethics.htm http://www.fhi.ox.ac.uk/__data/assets/ Heinemann. pdf_file/0006/4020/probing-the-im- – Matson, John and John Pavlus, probable.pdf – Rees, Martin, 2013. Denial of Cat- 2010. Laying odds on the apocalypse. astrophic Risks. Science, vol. 339 (8 Scientific American, vol. 303, no. 3 – Parson, Edward A, 2007. The big March), page 1123 (September), pages 82-83. one: A review of Richard Posner’s Ca- tastrophe: Risk and Response. Journal – Rees, Martin, 2013. We are in denial – McGuire, Bill, 2005. Global Ca- of Economic Literature, XLV (March), about catastrophic risks. Edge. http:// tastrophes: A Very Short Introduction. pp. 147-164 edge.org/response-detail/23864 Oxford: Oxford University Press. – Pindyck, Robert S., 2011. Fat Tails, – Rifkin, Lawrence, 2013. The survival – Morgan, Dennis Ray, 2009. World on Thin Tails, and Climate Change Policy. of humanity. Scientific American, Sep- fire: two scenarios of the destruction Review of Environmental Economics tember 13. http://blogs.scientificamer- of human civilization and possible and Policy, 5 (2) 258-274. ican.com/guest-blog/2013/09/13/ extinction of the human race. Futures, the-survival-of-humanity vol. 41, no. 10 (December), pages – Pindyck, Robert S. and Neng Wang, 683-693. 2011. The economic and policy con- – Roberts, Patrick S., 2008. Catastro- sequences of catastrophes. NBER phe: Risk and Response, by Richard – Ng, Yew-Kwang, 1991. Should we Working Papers, no. 15373. http:// A. Posner. Homeland Security Affairs, be very cautious or extremely cautious www.nber.org/papers/w15373. http:// vol. 4, no. 1 (January). on measures that may involve our de- web.mit.edu/rpindyck/www/Papers/ struction?. Social Choice and Welfare, EconDisastersJanuary2011.pdf vol. 8, pages 79-88. – Posner, Richard, 2004. Catastrophe: – Ng, Yew-Kwang, 2011. Consumption Risk and Response. Oxford: Oxford tradeoff vs. catastrophes avoidance: University Press. Implications of some recent results in happiness studies on the economics – Posner, Richard A., 2005. Cata- of climate change. Climatic change, strophic risks, resource allocation, and Volume 105, Issue 1-2, pp 109-127. homeland security. Journal of Home- land Security, October. http://www. homelandsecurity.org/journal/Default. aspx?oid=133&ocat=1

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 203 Appendix 1 – Global Challenges Bibliography

– Rockström, Johan, Will Steffen, – Sandberg, Anders and Nick Bo- – Shapovalov, Viktor I., Nickolay V. Kevin Noone, Åsa Persson, F. Stuart III strom, 2008. Global catastrophic Kazakov, 2013. The fundamental rea- Chapin, Eric Lambin, Timothy M. Len- risks survey. Technical Report 2008/1, sons for global catastrophes. Natural ton, Marten Scheffer, Carl Folke, Hans Future of Humanity Institute, Ox- Science Vol.5, No.6, 673-677. Joachim Schellnhuber, Björn Nykvist, ford University. http://www.fhi.ox.ac. Cynthia A. de Wit, Terry Hughes, uk/Reports/2008-1.pdf – Shapiro, Robert, 2009. A new Sander van der Leeuw, Henning Rod- rationale for returning to the Moon? he, Sverker Sörlin, Peter K. Snyder, – Sandberg, Anders, Jason G. Mathe- Protecting civilization with a sanctuary. Robert Costanza, Uno Svedin, Malin ny, and Milan M. Ćirković, 2008. How Space Policy, vol. 25, no. 1 (February), Falkenmark, Louise Karlberg, Robert can we reduce the risk of human pages 1-5. W. Corell, Victoria J. Fabry, James extinction?. Bulletin of the Atomic Hansen, Brian Walker, Diana Liverman, Scientists, September. http://www. – Shuckburgh, Emily, 2008, editor. Sur- Katherine Richardson, Paul Crutzen, thebulletin.org/web-edition/features/ vival: The Survival of the Human Race. and Jonathan Foley, 2009. Planetary how-can-we-reduce-the-risk-of-hu- Cambridge, UK: Cambridge University boundaries: Exploring the safe oper- man-extinction Press. ating space for humanity. Ecology and Society, vol. 14, no. 2, article 32, http:// – Scheffler, Samuel, 2013. The im- – Singer, Peter, 2005. ‘Catastrophe’: www.ecologyandsociety.org/vol14/ portance of the afterlife. Seriously. Apocalypse when? (book review iss2/art32 The New York Times, September of R. Posner). New York Times, 2 21. http://opinionator.blogs.ny- January 2005 http://www.nytimes. – Rockström, Johan, Will Steffen, times.com/2013/09/21/the-impor- com/2005/01/02/books/review/02SIN- Kevin Noone, Åsa Persson, F. Stuart tance-of-the-afterlife-seriously GERL.html III Chapin, Eric Lambin, Timothy M. Lenton, Marten Scheffer, Carl Folke, – Schild, Rudy, 2009. Asteroids, – Smil, Vaclav, 2005. The next 50 Hans Joachim Schellnhuber, Björn Meteors, Comets, Climate and Mass years: Fatal discontinuities. Population Nykvist, Cynthia A. de Wit, Terry Extinctions. Journal of Cosmology & Development Review, 31:201-236 Hughes, Sander van der Leeuw, Hen- (special issue), November. http://jour- http://home.cc.umanitoba.ca/~vsmil/ ning Rodhe, Sverker Sörlin, Peter K. nalofcosmology.com/Contents2.html publications_pdf.html Snyder, Robert Costanza, Uno Svedin, Malin Falkenmark, Louise Karlberg, – Seidel, Peter, 2003. Introduction. – Smil, Vaclav, 2008. Global Catastro- Robert W. Corell, Victoria J. Fabry, World Futures, vol. 59, no. 3-4 (April- phes and Trends: The Next Fifty Years. James Hansen, Brian Walker, Diana June), pages 127-128. Cambridge, MA: MIT Press. Liverman, Katherine Richardson, Paul Crutzen, and Jonathan Foley, 2009. A – Seidel, Peter, 2003. ‘Survival re- – Sunstein, Cass R., 2005-2006. safe operating space for humanity. Na- search’: A new discipline needed now. Irreversible and catastrophic: Glob- ture, vol. 461 (24 September), pages World Futures, vol. 59, no. 3-4 (April- al warming, terrorism, and other 472-475. June), pages 129-133. problems. Pace Environmental Law Review, vol. 23, no. 1 (Winter), pages – Russell, Bertrand, Albert Einstein, – Seidel, Peter, 2009. Is it inevitable 3-19. Max Born, Percy W. Bridgman, that evolution self-destruct?. Futures, Leopold Infeld, Frederic Joliot Cu- vol. 41, no. 10 (December), pages – Sunstein, Cass R., 2006. Irreversible rie, Herman J. Muller, Linus Pauling, 754-759. and catastrophic. Cornell Law Review, Cecil F. Powell, Joseph Rotblat, Hideki vol. 91, no. 4, pages 841-897. Yukawa, 1955. The Russell-Einstein – Sevilla, J. P., 2012. On the Underlying manifesto. July 9. http://www.pug- Ontology of the View that Extinction – Sunstein, Cass R, 2007. Worst-Case wash.org/about/manifesto.htm is Bad. Available at SSRN: http://ssrn. Scenarios. Cambridge, MA: Harvard com/abstract=1755298 or http://dx. University Press. doi.org/10.2139/ssrn.1755298 – Tännsjö, Torbjörn, 1997. Doom soon?. Inquiry, vol. 40, no. 2, pages 243-252.

204 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Appendix 1 – Global Challenges Bibliography

– Taylor, Kenneth B., 2013. Demise – Tonn, Bruce and Donald MacGregor, – Weitzman, Martin L., 2009. Structural of the Modern Human. World Future 2009. Are we doomed?. Futures, vol. uncertainty and the value of statistical Review, vol. 5, no. 2, p. 85-93. 41, no. 10 (December), pages 673- life in the economics of catastrophic 675. climate change. Review of Economics – Taylor, Matthew A., 2012. Whither and Statistics, vol. 91, no. 1, pages life? Subjectivity (2012) 5, 276–289. – Tonn, Bruce and Donald MacGre- 1-19. gor, 2009. A singular chain of events. – Tegmark, Max and Nick Bostrom, Futures, vol. 41, no. 10 (December), – Wiener, Jonathan B., 2005. Review 2005. Is a doomsday catastrophe pages 706-714. of: Catastrophe: Risk and Response likely?. Nature, vol. 438 (8 December), and Collapse: How Societies Choose page 754. – Tonn, Bruce and Jenna Tonn, 2009. to Fail or Succeed. Journal of Policy A literary human extinction scenario. Analysis and Management, vol. 24, no. – Thacker, Eugene, 2012. Notes on Ex- Futures, vol. 41, no. 10 (December), 4, pages 885-890. tinction and Existence. Configurations, pages 760-765. Volume 20, Number 1-2, 137-148 – Wilson, E.O., 1993. Is humanity sui- – Tonn, Bruce, Dorian Stiefel, 2013. cidal? The New York Times Magazine, – Tonn, Bruce E., 1999. Transcending Evaluating Methods for Estimating May 30, pages 24-28. oblivion. Futures, vol. 31, pages 351- Existential Risks. Risk Analysis, Vol. 359. 33, No. 10, pages 1772-1787. – Wilson, Grant S., 2013. Minimizing global catastrophic and existential – Tonn, Bruce E., 2000. Ensuring the – Trisel, Brooke Allen, 2004. Human risks from emerging technologies future. Futures, vol. 32, pages 17-26. extinction and the value of our efforts. through international law. Virginia Envi- The Philosophical Forum, vol. 35, no. 3 ronmental Law Journal, vol. 31, no. 2. – Tonn, Bruce E., 2002. Distant futures (September), pages 371-391. and the environment. Futures, vol. 34, – Wuthnow, Robert, 2010. Be Very pages 117-132. – Tudge, C., 1989. The rise and fall of Afraid: The Cultural Response to – Tonn, Bruce E., 2004. Research so- Homo sapiens sapiens. Philosophical Terror, Pandemics, Environmental ciety: science and technology for the Transactions of the Royal Society of Devastation, Nuclear Annihilation, and ages. Futures, 36, 335–346 London, Series B, vol. 325, no. 1228 (6 Other Threats. Oxford: Oxford Univer- November), pages 479-488. sity Press. – Tonn, Bruce E., 2007. Futures sustainability. Futures 39, pages 1097- – Union of Concerned Scientists, 1116. 1993. World Scientists’ Warning To Humanity. Cambridge, MA: Union of – Tonn, Bruce E., 2009. Obligations Concerned Scientists. to future generations and acceptable risks of human extinction. Futures, vol. – Vikulin, A. V., N. V. Semenets, M. 41, no. 7, pages 427-435. A. Vikulina, 2013. Global disasters: – Tonn, Bruce E., 2009. Beliefs about Geodynamics and society. Izvestiya, human extinction. Futures, vol. 41, no. Atmospheric and Oceanic Physics, 10 (December), pages 766-773. Volume 49, Issue 7, pp 691-714

– Tonn, Bruce, 2009. Preventing the – Waldman, Ronald, 2006. Respond- next mass extinction: Ethical obliga- ing to catastrophes: A public health tions. Journal of Cosmology, vol. 2, perspective. Chicago Journal of pages 334-343. International Law, vol. 6, no. 2, pages 553-572. – Tonn, Bruce, Don MacGregor, editors, 2009. Human Extinction. Futures - Wells, Willard, 2009. Apocalypse (special issue), Volume 41, Issue 10, When? Calculating How Long the Pages 673-774. Human Race Will Survive. Chichester, UK: Springer-Praxis.

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 205 Appendix 2 – Workshops

Appendix 2 – Workshops

Workshop 1 14 January 2014, at the Future of Humanity Institute (FHI), University of Oxford:

Participants Stuart Armstrong, James Martin Re- Patrick McSharry, head of Smith Jules Peck, Founding Partner, Jericho search Fellow, FHI, Oxford School’s Catastrophe Risk Financing Chambers; Trustee, New Economics Seth Baum, Executive Director of the research area. Foundation Global Catastrophic Risk Institute Vincent Müller, James Martin Research Anders Sandberg, James Martin Re- Nick Beckstead, Research Fellow, Fellow, FHI, Oxford search Fellow, FHI, Oxford FHI, Oxford Robert de Neufville, Professional As- Andrew Simms, Author, Fellow at the Eric Drexler, Academic Visitor, James sociate, Global Catastrophic New Economics Foundation and Chief Martin Research Fellow, FHI, Oxford Risk Institute Analyst at Global Witness Madeleine Enarsson, Transformative Toby Ord, James Martin Research Andrew Snyder-Beattie, Academic Catalyst, 21st Century Frontiers Fellow, FHI, Oxford Project Manager, FHI, Oxford Seán Ó hÉigeartaigh, James Martin Aca- Dennis Pamlin, Executive Project James Taplan, Principal Sustainability demic Project Manager, FHI, Oxford and Manager, Global Challenges Advisor, Forum for the Future project manager, Cambridge’s Centre for Foundation Raj Thamotheram, CEO, Preventable Study of Existential Risk, Cambridge Surprises

Agenda 10.40-11.40 14.30-15.30 Major events during 2013 and links to 2014 Emerging challenges 09.30 Welcome Discussing the draft list and structure. Probabilities and impacts, discussions What is missing, especially with regards to regarding the challenges. Who can 09.35-09.50: Round of introductions emerging countries? What could happen provide an estimation of the risk and who during 2014 that we think should be can continue to work on these? Are there 09.50-10.10 reflected in the report? synergies between them and between Introducing the Global Challenges Foun- them and today’s major challenges? dation and the work with the “Global 11.40-12.30 Challenges Report” Infinite impacts 15.30-16.00 What is the background for this work, How can infinite impacts be defined and Natural challenges and policy challenges draft structure of the report and agenda presented in ways that make policy mak- How to deal with these challenges that are for the day? ers take them seriously? What models different, especially policy challenges? and narratives exist, what coalitions can 10.10-10.40 work with such questions? 16.00-16.30 Global Challenges in society today and Next steps and need for funding global risks 12.30-13.30 Lunch Based on the discussion during the day, Who cares about and works with the what are possible next steps? Are there major global challenges that pose an 13.30-14.30 initiatives/work in need of funding? What existential threat and who wants to cancel Major challenges role could a quarterly Global Challenges/ the apocalypse? In short: In what context Probabilities and impacts, discussions risk report have? will the report be received? regarding the challenges. Who can provide an estimation of the risk and who can continue to work on these? Are there synergies between them?

206 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks Appendix 2 – Workshops

Workshop 2 15 January 2014 at the Munich RE office in London:

Participants Oliver Bettis, pricing actuary Mu- Nick Silver, director of Callund Con- nich RE and fellow of the Chartered sulting and founder and director of the Insurance Institute and the Institute & Climate Bonds Initiative (CBI) Faculty of Actuaries. Liang Yin, Investment Consultant at Madeleine Enarsson, Transformative Towers Watson Catalyst, 21st Century Frontiers Jennifer Morgan, Founder & Co-Con- vener, the finance lab Dennis Pamlin, Executive Project Man- ager, Global Challenges Foundation

Agenda 12.45-13.00 How can infinite impacts be included 11.00-11.05 in mainstream actuary work? How can Welcome thought leaders in the actuarial profession begin to include infinite impacts? 11.05-11.15 Round of introductions 13.00-13.30 What should the report include to be 11.15-11.30 relevant for actuaries/financial sector? The Global Challenges Foundation and a risk report 13.30-14.00 Possible ways forward 11.30-12.00 Infinite impacts and global challenges.

12.00-12.15 What risks with a possible infinite impact are on the radar screen of actuaries today (comparing the list from Oxford with current actuary work) and how can “Infinite impacts” be addressed?

12.15-12.45 Relations and overview

Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 207 Notes

Notes

208 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks design by wearebwa.co.uk Published February 2015 by Global Challenges Foundation

globalchallenges.org

Global Challenges Foundation Stureplan 4C 114 35 Stockholm

For comments and/or queries on this report, please contact the co-authors:

Dennis Pamlin Dr Stuart Armstrong Executive Project Manager James Martin Research Fellow Global Risks Future of Humanity Institute Global Challenges Foundation Oxford Martin School & Faculty of Philosophy [email protected] University of Oxford globalchallenges.org [email protected] fhi.ox.ac.uk