Global Challenges Foundation — Twelve Risks That Threaten Human

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial Extreme Future Bad Global Global System Major Asteroid Global Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Challenges Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Ecological NanotechnologyIntelligence NuclearClimate WarChange Super-volcanoGlobal Governance PandemicSynthetic UnknownCollapse Impact Risks that threaten Catastrophe Biology Consequences humanArtificial civilisationExtreme Future Bad Global Global System Major Asteroid 12 Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial Extreme Future Bad Global Global System Major Asteroid IntelligenceEcological ClimateNanotechnology Change NuclearGlobal Governance War Super-volcanoPandemic CollapseSynthetic UnknownImpact Catastrophe Biology Consequences

Executive SummaryEcological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Ecological Nanotechnology Nuclear War Super-volcano SyntheticIntelligence ClimateUnknown Change Global Governance Pandemic Collapse Impact Catastrophe Biology Consequences

The case for Ecologicala new riskNanotechnology categoryNuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences The Global Challenges Foundation works to raise awareness of the greatest threats facing humanity. In particular climate change, other environmental damage and political violence, and how these threats are linked to poverty and the rapid growth in global population. These problems appear insurmountable without an international body with decision-making mandate. The Foundation is therefore working to identify possible solutions and models as to how the United Nations can develop, and initiate new ideas on working global governance.

Contents page

- Introduction 1 - 12 Global risks 2 - The goals of the report 4 - Report structure 5 - A new category of global risk 6 - Infinite impacts and thresholds 7 - Methodology 8 - Quick overview of each risk 9 - Relations between global risks 16 - The technical difficulty of reducing the risk and the difficulty of collaboration 17 - Uncertainties 18 - Probability 19 - Possible ways forward 20 Introduction

This is the executive summary of a report about a limited Introduction number of global risks that pose a threat to human civilisation, or even possibly to all human life.

With such a focus it may surprise some readers to find that the report’s essential aim is to inspire action and dialogue as well as an increased use of the methodologies used for risk assessment. The real focus is not on the almost unimaginable impacts of the risks the report outlines. Its fundamental purpose is to encourage global collaboration and to use this new category of risk as a driver for innovation.

The idea that we face a number of global challenges threatening the very basis of our civilisation at the beginning of the 21st century is well accepted in the scientific community, and is studied at a number of leading universities.1 However, there is still no coordinated approach to address this group of challenges and turn them into opportunities.

Full report can be downloaded at globalchallenges.org/globalrisks

It is only 70 years ago that Robert Oppenheimer, who History: Edward Teller, one of the led the Manhattan Project to greatest physicists of his time, develop the nuclear bomb, the LA-602 with his back-of-the-envelope halted the project to see calculations, produced results whether Teller’s calculations document that differed drastically from were correct. The resulting all that had gone before. His document, LA-602: Ignition of calculations showed that the the Atmosphere with Nuclear explosion of a nuclear bomb – a Bombs, concluded that Teller creation of some of the brightest was wrong. But the sheer minds on the planet, including complexity drove the assessors Teller himself – could result in to end their study by writing a chain reaction so powerful that “further work on the that it would ignite the world’s subject [is] highly desirable”. atmosphere, thereby ending The LA-602 document can human life on Earth. be seen as the first global challenge report addressing a category of risks where the worst possible impact in all practical senses is infinite.

1 http://en.wikipedia.org/wiki/Global_catastrophic_risk

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 1 12 Global risks

This report has, to the best of the authors’ knowledge, 12 Global created the first list of global risks with impacts that for all practical purposes can be called infinite. It is risks also the first structured overview of key events related to such challenges and has tried to provide initial rough quantifications for the probabilities of these impacts. In the next phase of the project, these placeholder estimates will be improved and refined by a variety of methods (expert elicitation, fault trees, simulations, etc.) appropriate to each specific risk.

The report conducts its exploration It is worth noting that complex within carefully defined bounds, systems are often stable only within resulting in a list of twelve risks with certain boundaries outside which potentially infinite outcomes. the system can collapse and rapidly change to a new stable state. Such a There were many challenges which collapse can trigger a process where might have been included on the change continues for a long time until list because of their ability to pose a new stable state is found. None of severe damage to humanity. They the risks in this report are likely to were excluded for one or more of result directly in an infinite impact, three reasons: and some cannot do so physically. All the risks however are big enough to 1. Limited impact – tsunamis, for reach a threshold where the social and example, and chemical pollution. ecological systems become so unstable that an infinite impact could ensue. 2. No effective countermeasures – the report focuses on promoting This is a report about two extremes, effective interventions and so not one. It is about how a better ignores challenges where nothing understanding of the magnitude of useful can be done to prevent or the challenges can help the world mitigate the impact, as with to address the risks it faces, and nearby gamma-ray bursts. can help to create a path towards more sustainable development. It 3. Included in other challenges – is a scientific assessment about many challenges are already the possibility of oblivion, certainly, covered by others, or are very but more than that it is a call for similar to them. Population growth, action based on the assumption that for one, is significant for climate humanity is able to rise to challenges change and ecosystem and turn them into opportunities. catastrophe, but without direct We are confronted with possibly large-scale impacts of its own. the greatest challenge ever and our response needs to match this through global collaboration in new and innovative ways.

2 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic 12 Global risks Collapse Impact

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact ArtificialEcologicalArtificial ExtremeNanotechnologyExtreme FutureNuclearFuture Bad BadWar GlobalSuper-volcanoGlobal GlobalSyntheticGlobal System System MajorUnknownMajor Asteroid Asteroid IntelligenceCatastropheIntelligence ClimateClimate Change Change GlobalGlobal Governance Governance PandemicPandemic CollapseBiologyCollapse ImpactConsequencesImpact

ArtificialArtificial ExtremeExtreme FutureFuture Bad Bad GlobalGlobal GlobalGlobalArtificial System System MajorMajorExtreme Asteroid Asteroid Future Bad Global Global System Major Asteroid IntelligenceIntelligence ClimateClimate Change Change GlobalGlobal Governance Governance PandemicPandemic CollapseCollapseIntelligence ImpactImpactClimate Change Global Governance Pandemic Collapse Impact ArtificialArtificial ExtremeExtreme FutureFuture Bad Bad EcologicalGlobalGlobal NanotechnologyGlobalGlobal System System NuclearMajorMajor Asteroid AsteroidWar Super-volcano Synthetic Unknown IntelligenceIntelligence ClimateClimate Change Change GlobalGlobal Governance Governance CatastrophePandemicPandemic CollapseCollapse ImpactImpact Biology Consequences EcologicalEcological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSynthetic UnknownUnknown CatastropheCatastrophe BiologyBiology ConsequencesConsequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact EcologicalEcological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSyntheticEcological UnknownUnknownNanotechnology Nuclear War Super-volcano Synthetic Unknown CatastropheCatastrophe BiologyBiologyCatastrophe ConsequencesConsequences Biology Consequences EcologicalEcological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSynthetic UnknownUnknown CatastropheCatastrophe BiologyBiology ConsequencesConsequences

Ecological Nanotechnology Nuclear War Super-volcano ArtificialSyntheticArtificial ExtremeUnknownExtreme FutureFuture Bad Bad GlobalGlobal GlobalGlobal System System MajorMajor Asteroid Asteroid Catastrophe IntelligenceBiologyIntelligence ClimateConsequencesClimate Change Change GlobalGlobal Governance Governance PandemicPandemic CollapseCollapse ImpactImpact

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 3

EcologicalEcological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSynthetic UnknownUnknown CatastropheCatastrophe BiologyBiology ConsequencesConsequences The goals of the report

The four main goals of this report are to acknowledge, The goals inspire, connect and deliver. of the report The first of the report’s goals – The third goal is to connect acknowledging the existence of different groups at every level, risks with potentially infinite impact – so that leaders in different sectors seeks to help key stakeholders to connect with each other to acknowledge the existence of the encourage collaboration. This will category of risks that could result need a specific focus on financial in infinite impact, and to show them and security policy, where significant that we can reduce or even eliminate risks combine to demand action most of them. beyond the incremental.

The second goal is to inspire by The fourth goal is to deliver actual showing the practical action that strategies and initiatives that is taking place today. This report produce actual results. The report seeks to show that helping to meet is a first step and its success will these global challenges is perhaps ultimately be measured only on how the most important contribution it contributes to concrete results. anyone can make today, and highlights concrete examples to The report will have achieved its inspire a new generation of leaders. goals when key decision-makers recognise the magnitude of the possible risks and our ability to reduce or even eliminate most of them.

The goals

1. to acknowledge the existence of risks with potentially infinite impact. 2. to inspire by showing the practical action that is taking place today. 3. to connect different groups at every level. 4. to deliver actual strategies and initiatives that produce actual results.

4 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Report structure

The first part of the report introduces and defines the Report global challenges and includes the methodology for structure selecting them. The third part of the report discusses The second part is an overview of the relationship between the different the twelve challenges and key challenges, as action to address events that illustrate strategic work one can increase the risk of to address them. It also lists for each another. Many solutions can also challenge five important factors that address multiple challenges, so influence its probability or impact. there are significant benefits from The challenges are divided into four understanding how they are linked. different categories: The fourth part is an overview, the first – current challenges includes those ever to the authors’ knowledge, of the which currently threaten humanity probabilities of global challenges with because of its economic and potentially infinite impacts. technological development; The fifth part presents some of the exogenic challenges – are those most important underlying trends where the basic probability of an that influence the challenges, which event is beyond human control, but often build up slowly to a threshold where the probability and magnitude where very rapid changes can of the impact can be influenced; ensue. – emerging challenges could both help reduce the risks associated The sixth part presents an overview with current challenges and also of possible ways forward. result in infinite impacts;

– the last of the twelve challenges are global policy challenges, threats arising from future global governance as it resorts to destructive policies in response to the categories of challenge listed above.

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 5 A new category of global risk

The idea that there may be risks where the impact can A new category be described as infinite, defined as the end of human civilisation or even human life, is not new. However, of global risk it excites relatively little political or academic interest, and the way it is treated in popular culture makes a serious discussion more difficult.

For several reasons the potentially The review of literature indicates that, infinite impacts of the challenges in under a business as usual scenario, this report are not as well known as new risks with potential infinite impact they should be. One reason is the are probably inseparable from the way that extreme impacts are often rapid technological development masked by most of the theories and in areas like synthetic biology, models used by governments and nanotechnology and AI. business today. Most risks are linked to increased Climate change is a good example, knowledge, economic and technical where almost all of the focus is on development that has brought many the most likely scenarios, and there benefits. E.g. climate change is a are few public studies that include result from the industrial revolution the low-probability high-impact and fossil fuel based development. scenarios. In most reports about The increased potential for global climate impacts, those caused by pandemics is one consequence of warming beyond five or six degrees an integrated global economy where Celsius are omitted from tables goods and services move quickly and graphs. Other aspects that internationally. Similar challenges can contribute to this relative invisibility be expected for synthetic biology, include the fact that extreme nanotechnology and AI. impacts are difficult to translate into monetary terms, as they have There are remedies, including a global scope and often require a technological and institutional, time-horizon of a century or more. for all risks. But they will require They cannot be understood simply collaboration of a sort humanity has by linear extrapolation of current not achieved before, and the creation trends, and they lack historical of systems which can deal with precedents. There is also the fact that problems pre-emptively. It is important the measures required to significantly to understand that much of the reduce the probability of infinite knowledge and many tools that we impacts will be radical compared to a have, and will develop, can be business-as-usual scenario. both a risk and a solution to risks depending on context. A scientific approach requires us to base our decisions on the whole probability distribution.

Risk = Probability x Impact

6 Global Challenges – Twelve risks that threaten human civilisation – Executive summary

Normal Risks New Category Traditional measures Requires new measures and tools and tools applicable

Threshold probability

0 impact Infinite impacts and thresholds

There is a clear ethical dimension to the concept of Infinite impacts infinite impact, because a very small group alive today can take decisions that will fundamentally affect all and thresholds future generations.

Using traditional economic tools events that could result first in a Risk= Probabilityx Impactis problematic and can generate civilisation collapse, and then later disagreement over issues such result in an infinite impact. Such as discounting, which the report thresholds are especially important examines in some detail, considering to recognise in a complex and for example the role of tipping points. interconnected society where resilience is decreasing. The report distinguishes between the concepts of infinite impact – where A collapse of civilisation is defined civilisation collapses to a state of as a drastic decrease in human great suffering and does not recover, population size and political/ or a situation where all human life economic/social complexity, globally ends – and infinite impact threshold – and for an extended time. an impact that can trigger a chain of

Normal Risks New Category Traditional measures Requires new measures and tools and tools applicable

Threshold probability

0 impact

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 7 Methodology

In order to establish a list of global challenges Methodology with potentially infinite impact, a methodological triangulation was used, consisting of:

– A quantitative assessment of Two workshops were arranged relevant literature. where the selection of challenges was discussed, one with risk experts – A strategic selection of relevant in Oxford at the Future of Humanity organisations and their priorities. Institute and the other in London – A qualitative assessment with the with experts from the financial sector. help of expert workshops. No challenge was excluded at the workshops, but one was added: the participants agreed to include Global System Collapse as a category.

Relevant literature Identification of credible sources: search relevant literature in academic literature included in World of Knowledge and Google Scholar. 1 Estimations of impact Only literature where there is some estimation of impact that indicates the possibility of an infinite impact is included.

Leading organisations’ priorities 2 In order to increase the probability of covering all relevant risks an overview of leading organisations' work was conducted. This list was then compared with the initial list and subjected to the same filter regarding the possibility to affect the probability or impact.

3 Possibility of addressing the risk From the risks gathered from literature and organisations, only those where the probability or impact can be affected by human actions are included.

Expert review 4 Qualitative assessment: Expert review in order to increase the probability of covering all relevant global risks. 5 List of risks Result: List of risks with potentially infinite impacts. 6

8 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Quick overview of each risk

Quick overview of each risk

Current Current Current risk risk risk

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial Extreme EcologicalFuture Bad NanotechnologyGlobal ArtificialNuclearGlobal War System ExtremeSuper-volcanoMajor Asteroid FutureSynthetic Bad GlobalUnknown Global System Major Asteroid Intelligence Climate Change CatastropheGlobal Governance Pandemic IntelligenceCollapse ClimateImpact Change GlobalBiology Governance PandemicConsequences Collapse Impact

Current Current Exogenic risk risk risk

Ecological Nanotechnology Nuclear War Super-volcano EcologicalSynthetic NanotechnologyUnknown Nuclear War Super-volcano Synthetic Unknown Catastrophe Artificial Extreme Future Bad CatastropheGlobalBiology ConsequencesGlobal System Major Asteroid Biology Consequences Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial ExtremeEcological ArtificialFutureNanotechnology Bad ExtremeGlobalNuclear War FutureGlobal BadSuper-volcano System GlobalMajorSynthetic Asteroid Global SystemUnknown Major Asteroid Intelligence ClimateCatastrophe Change IntelligenceGlobal Governance ClimatePandemic Change GlobalCollapse Governance PandemicImpactBiology CollapseConsequences Impact

Exogenic Emerging Emerging risk risk risk

Ecological Nanotechnology EcologicalNuclear War NanotechnologySuper-volcano NuclearSynthetic War Super-volcanoUnknown Synthetic Unknown Catastrophe Catastrophe Biology Consequences Biology Consequences Artificial Extreme ArtificialFuture Bad ExtremeGlobal FutureGlobal Bad System GlobalMajor Asteroid Global System Major Asteroid Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change IntelligenceGlobal Governance ClimatePandemic Change GlobalCollapse Governance PandemicImpact Collapse Impact Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology EcologicalNuclear War NanotechnologySuper-volcano NuclearSynthetic War Super-volcanoUnknown Synthetic Unknown Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Catastrophe Biology Consequences Biology Consequences Catastrophe Biology Consequences

Emerging Emerging Global Policy risk risk risk

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology NuclearArtificial War Super-volcanoExtreme SyntheticFuture Bad UnknownGlobal ArtificialGlobal System ExtremeMajor Asteroid Future Bad Global Global System Major Asteroid Catastrophe Intelligence Climate Change BiologyGlobal Governance ConsequencesPandemic IntelligenceCollapse Climate ImpactChange Global Governance Pandemic Collapse Impact

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 9

Ecological Nanotechnology Nuclear War Super-volcano EcologicalSynthetic NanotechnologyUnknown Nuclear War Super-volcano Synthetic Unknown Catastrophe CatastropheBiology Consequences Biology Consequences Quick overview of each risk - Extreme Climat Change / Nuclear War

Current risk Extreme 5 key Climate Change factors: Extreme1 The uncertainties in As for all risks there are uncertainties Mass deaths and famines, social Climate Change climate sensitivity models, in the estimates, and warming could collapse and mass migration are including the tail. be much more extreme than the certainly possible in this scenario. 2 The likelihood - or not - of middle estimates suggest. Feedback Combined with shocks to the global coordination on loops could mean global average agriculture and biosphere-dependent controlling emissions. temperatures increase by 4°C or industries of the more developed even 6°C over pre-industrial levels. countries, this could lead to global 3 The future uptake of Feedbacks could be the release conflict and possibly civilisation low carbon economies, of methane from permafrost or the collapse. Further evidence of the including energy, mobility dieback of the Amazon rainforest. risk comes from signs that past and food systems. The impact of global warming would civilisation collapses have been 4 Whether technological be strongest in poorer countries, driven by climate change. innovations will improve which could become completely or worsen the situation, uninhabitable for the highest range and by how much. of warming. 5 The long-term climate impact caused by global warming.

Current risk

5 key Nuclear War factors:

Nuclear1 How War relations between The likelihood of a full-scale nuclear plunge temperatures below freezing current and future war between the USA and Russia around the globe and possibly also nuclear powers develop. has probably decreased. Still, the destroy most of the ozone layer. 2 The probability of potential for deliberate or accidental The detonations would need to accidental war. nuclear conflict has not been start firestorms in the targeted removed, with some estimates cities, which could lift the soot up 3 Whether disarmament putting the risk in the next century into the stratosphere. The risks are efforts will succeed in or so at around 10%. A larger impact severe and recent models have reducing the number of would depend on whether or not the confirmed the earlier analysis. The nuclear warheads. war triggered what is often called a disintegration of the global food 4 The likelihood of a nuclear winter or something similar – supply would make mass starvation nuclear winter. the creation of a pall of smoke high and state collapse likely. 5 The long-term effects in the stratosphere that would of a nuclear war on climate, infrastructure and technology. A new category of global risk.

10 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Quick overview of each risk - Global Pandemic / Ecological Collapse

Current risk Global 5 key Pandemic factors:

Global1 What the true probability An epidemic of infectious disease (and influenza has demonstrated Pandemic distribution for pandemics that has spread through human antigenic shift, the ability to combine is, especially at the tail. populations across a large region or features from different viruses), its 2 The capacity of international even worldwide. There are grounds death toll would be extreme. health systems to deal for suspecting that such a high- The world has changed considerably, with an extreme pandemic. impact epidemic is more probable making comparisons with the past than usually assumed. All the problematic.Today it has better 3 How fast medical research features of an extremely devastating sanitation and medical research, as can proceed in an disease already exist in nature: well as national and supra-national emergency. essentially incurable (Ebola), nearly institutions dedicated to combating 4 How mobility of goods and always fatal (rabies), extremely diseases. But modern transport people, as well as population infectious (common cold), and and dense human population allow density, will affect pandemic long incubation periods (HIV). If infections to spread much more transmission. a pathogen were to emerge that rapidly, and slums can be breeding 5 Whether humans can somehow combined these features grounds for disease. develop novel and effective anti-pandemic solutions.

Current risk Ecological 5 key Collapse factors: Ecological Collapse1 The extent to which This is where an ecosystem suffers the damage and (unlike previous, humans are dependent on a drastic, possibly permanent, localised collapses) the whole world the ecosystem. reduction in carrying capacity for is potentially at risk. 2 Whether there will be all organisms, often resulting in It seems plausible that some human effective political measures mass extinction. Humans are part lifestyles could be sustained in a taken to protect the of the global ecosystem and so relatively ecosystem independent ecosystem on a large scale. fundamentally depend on it. Species way, at relatively low costs. Whether extinction is now far faster than the this can be achieved on a large 3 The likelihood of the historic rate, and attempts to scale in practice, especially during emergence of sustainable quantify a safe ecological operating a collapse, will be a technological economies. space place humanity well outside it. challenge and whether it is something 4 The positive and negative Many of the problems of ecological we want is an ethical question. impacts on the ecosystems degradation interact to multiply of both wealth and poverty. 5 The long-term effects of an ecological collapse on ecosystems.

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 11 Quick overview of each risk - Global System Collapse / Major Asteroid Impact

Current risk Global System 5 key Collapse factors: Global1 WhetherSystem global system An economic or societal collapse structure of the network – even if Collapse collapse will trigger on the global scale. The term has each component of the network is subsequent collapses or been used to describe a broad reliable. This gives rise to systemic fragility in other areas. range of conditions. Often economic risk, when parts that individually may 2 What the true trade-off is collapse is accompanied by social function well become vulnerable between efficiency chaos, civil unrest and sometimes a when connected as a system to a and resilience. breakdown of law and order. Societal self-reinforcing joint risk that can collapse usually refers to the fall or spread from part to part, potentially 3 Whether effective disintegration of human societies, affecting the entire system and regulation and resilience often along with their life support possibly spilling over to related can be developed. systems. The world economic and outside systems. Such effects have 4 Whether an external political system is made up of been observed in ecology, finance disruption will trigger many actors with many objectives and critical infrastructure such a collapse. and many links between them. as power grids. The possibility of 5 Whether an internal event Such intricate, interconnected collapse becomes more acute when will trigger a collapse. systems are subject to unexpected several independent networks system-wide failures caused by the depend on each other.

Exogenic risk Major Asteroid 5 key Impact factors: Major Asteroid Impact1 Whether detection and Large asteroid collisions – with There has been some discussion tracking of asteroids and objects 5 km or more in size – about possible methods for other dangerous space happen about once every twenty deflecting asteroids found on a objects is sufficiently million years and would have an collision course with the planet. exhaustive. energy a hundred thousand times Should an impact occur the main 2 How feasible it is to deflect greater than the largest bomb ever destruction will not be from the an asteroid. detonated. A land impact would initial impact, but from the clouds destroy an area the size of a nation of dust projected into the upper 3 Whether measures such as like Holland. Larger asteroids could atmosphere. The damage from such evacuation could reduce be extinction-level events. Asteroid an “impact winter” could affect the damage of an impact. impacts are probably one of the best the climate, damage the biosphere, 4 The short- and long-term understood of all risks in this report. affect food supplies, and create climate consequences of a political instability. collision. 5 Whether our current civilisation could adapt to a post-impact world.

12 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Quick overview of each risk - Super-volcano / Synthetic Biology

Exogenic risk

5 key Super-volcano factors: Super-volcano1 Whether countries will Any volcano capable of producing while the Toba eruption around coordinate globally an eruption with an ejecta volume 70,000 years ago is thought by some against super-volcano risk greater than 1,000 km3. This is to have cooled global temperatures and damage. thousands of times larger than for over two centuries. 2 The predictability of super- normal eruptions. The danger from The effect of these eruptions could volcanic eruptions. super-volcanoes is the amount of be best compared with that of a aerosols and dust projected into the nuclear war. The eruption would 3 How directly destructive upper atmosphere. This dust would be more violent than the nuclear an eruption would be. absorb the Sun’s rays and cause explosions, but would be less likely 4 The effectiveness of a global volcanic winter. The Mt to ignite firestorms and other general mitigation efforts. Pinatubo eruption of 1991 caused secondary effects. 5 How severe the long-term an average global cooling of surface climate effects would be. temperatures by 0.5°C over three years,

Emerging risk Synthetic 5 key Biology factors: Synthetic Biology1 The true destructive potential The design and construction of This could emerge through military of synthetic biology, especially biological devices and systems or commercial bio-warfare, bio- the tail risk. for useful purposes, but adding terrorism (possibly using dual-use 2 Whether the field will be human intentionality to traditional products developed by legitimate successfully regulated, or pandemic risks. Attempts at researchers, and currently successfully manage to regulation or self-regulation are unprotected by international legal regulate itself. currently in their infancy, and may regimes), or dangerous pathogens not develop as fast as research leaked from a lab. Of relevance is 3 Whether the field will usher does. One of the most damaging whether synthetic biology products in a new era of bio-warfare. impacts from synthetic biology become integrated into the global 4 Whether the tools of synthetic would come from an engineered economy or biosphere. This could biology can be used pathogen targeting humans or a lead to additional vulnerabilities (a defensively to create effective crucial component of the ecosystem. benign but widespread synthetic counter measures. biology product could be specifically 5 The dangers of relying targeted as an entry point through on synthetic biologists to which to cause damage). estimate the danger of synthetic biology.

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 13 Quick overview of each risk - Nanotechnology / Artificial Intelligence

Emerging risk

5 key Nanotechnology factors: Nanotechnology1 The timeline for nanotech Atomically precise manufacturing, Of particular relevance is whether development. the creation of effective, high- nanotechnology allows the 2 Which aspects of throughput manufacturing processes construction of nuclear bombs. nanotech research will that operate at the atomic or But many of the world’s current progress in what order. molecular level. It could create problems may be solvable with the new products – such as smart or manufacturing possibilities that 3 Whether small groups can extremely resilient materials – and nanotechnology would offer, such assemble a weapons would allow many different groups as depletion of natural resources, arsenal quickly. or even individuals to manufacture a pollution, climate change, clean 4 Whether nanotech tools wide range of things. This could lead water and even poverty. Some have can be used defensively to the easy construction of large conjectured special self-replicating or for surveillance. arsenals of conventional or more nanomachines which would be 5 Whether nanotech tools or novel weapons made possible by engineered to consume the entire weaponry are made to be atomically precise manufacturing. environment. The misuse of medical outside human control. nanotechnology is another risk scenario.

Emerging risk

Artificial And if these motivations do not detail the survival and value of 5 key Intelligence humanity, the intelligence will be factors: driven to construct a world without humans. This makes extremely Artificial Intelligence1 The reliability of AI AI is the intelligence exhibited by intelligent AIs a unique risk, in that predictions. machines or software, and the extinction is more likely than lesser 2 Whether there will be a branch of computer science that impacts. On a more positive note, single dominant AI or a develops machines and software an intelligence of such power could plethora of entities. with human-level intelligence. easily combat most other risks 3 How intelligent AIs will The field is often defined as “the in this report, making extremely become. study and design of intelligent intelligent AI into a tool of great 4 Whether extremely agents”, systems that perceive their potential. There is also the possibility intelligent AIs can be environment and act to maximise of AI-enabled warfare and all the controlled, and if so, how. their chances of success. Such risks of the technologies that AIs 5 Whether whole brain extreme intelligences could not would make possible. An interesting emulations (human minds easily be controlled (either by the version of this scenario is the in computer form) will groups creating them, or by some possible creation of “whole brain arrive before true AIs. international regulatory regime), emulations”: human brains scanned and would probably act to boost and physically represented in a their own intelligence and acquire machine. This would make the AIs maximal resources for almost all into properly human minds, possibly initial AI motivations. alleviating a lot of problems.

14 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Quick overview of each risk - Unknown Consequences / Future Bad Global Governance

Emerging risk Unknown 5 key Consequences factors: Unknown1 Whether there will be These represent the unknown generic probability of intelligent life Consequences extensive research into unknowns in the family of global (self-)destruction, which includes unknown risks and their catastrophic challenges. They uncertain risks. Anthropic reasoning probabilities. constitute an amalgamation of all the can also bound the total risk of 2 The capacity to develop risks that can appear extremely human extinction, and hence methods for limiting unlikely in isolation, but can combine estimate the unknown component. the combined probability to represent a not insignificant Non risk-specific resilience and post- of all uncertain risks. proportion of the risk exposure. One disaster rebuilding efforts will also 3 The capacity for estimating resolution to the Fermi paradox – reduce the damage from uncertain “out of-model” risks. the apparent absence of alien life risks, as would appropriate national 4 The culture of risk in the galaxy – is that intelligent life and international regulatory regimes. assessment in potentially destroys itself before beginning to Most of these methods would also risky areas. expand into the galaxy. Results that help with the more conventional, 5 Whether general, non- increase or decrease the probability known risks, which badly need risk-specific mitigation or of this explanation modify the more investment. resilience measures are implemented.

Global Policy risk Future Bad 5 key Global Governance factors: Future Bad Global1 HowGovernance the severity of non- There are two main divisions in Two issues with governance deadly policy failures governance disasters: failing to disasters are first, the difficulty can be compared with solve major solvable problems, and of estimating their probability, potential casualties. actively causing worse outcomes. and second, the dependence of 2 Whether poor governance An example of the first would be the impact of these disasters on will result in a collapse failing to alleviate absolute poverty; subjective comparative evaluations: of the world system. of the second, constructing a it is not impartially obvious how to global totalitarian state. Technology, rank continued poverty and global 3 How mass surveillance political and social change may totalitarianism against billions of and other technological enable the construction of new casualties or civilisation collapse. innovations will affect forms of governance, which may be governance. either much better or much worse. 4 Whether there will be new systems of governance in the future. 5 Whether a world dictatorship may end up being constructed.

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 15 Relations between global risks

Two things make the understanding of the relation Relations between between the global risks particularly important. global risks 1. Impacts: The risks are 2. Specific measures to address a interconnected in different ways. risk: Global risks often require Often the situation resembles a significant changes, which will set of dominoes: if one falls, many result in situations where measures to follow. Even small impacts can reduce the risk in one area affect start a process where different the probability and/or the impact in risks interact. other areas, for better or worse.

ALL RISKS

first risk worsens second risk

solving first risk improves second risk

both of the above

16 Global Challenges – Twelve risks that threaten human civilisation – Executive summary The technical difficulty of reducing the risk and the difficulty of collaboration

In order to better understand the relations between The technical different global risks, work could start to analyse difficulty of similarities and differences. reducing the risk Below is an example of an overview of how different global risks can be and the difficulty plotted depending on the technical difficulty of reducing the risk and the of collaboration difficulty of collaborating to reduce it. technical difficulty of reducing risk of reducing technical difficulty

collaboration difficulty of reducing risk

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 17 Uncertainties

As the different challenges are very different and the Uncertainties status of probability estimates varies significantly, the initial probability numbers are provided together with estimates regarding:

1. Understanding1. Understanding 2. Data2. Data 3. Existing probability 1. Understanding of 1. sequence ofUnderstanding sequence 2. availabilityData availability2. Data 3. estimationExisting probability of sequence of sequence availability availability estimation all parts all data all parts all data all parts all data all parts all parts all data all data calculations with small uncertainty calculations with small uncertainty calculations with small uncertainty most parts most data most parts most data most parts most data most parts most parts most data most data calculations with uncertainty large calculations with uncertainty large calculations with uncertainty large some parts some data some parts some data best guesses by experts some parts some data best guesses by experts some parts some parts some data some data best guesses by experts no estimates none at all no data none at all no data

degreedegreenone at all of events of events from today’sfrom today’s actions actions amountno data amount of data of todata make to make probability probability kindno estimates of estimation and uncertainty to infiniteto infinite impact impact assessmentassessment on all onrelevant all relevant steps steps no estimates none at all none at all ofno data theof sequence the sequenceno data

degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps degree of eventsdegree from of events today’s from actions today’s actions amountof the sequence of dataamount to make of data probability to make probabilitykind of estimation and uncertainty to infinite impactto infinite impact assessmentassessment on all relevant on allsteps relevant steps of the sequenceof the sequence

18 Global Challenges – Twelve risks that threaten human civilisation – Executive summary

n / a

0.00003% n / a 0.0001% 0.00013% 0.005% 0.01% 0.1% 0-10%

infinite impact %

n / a

n / a 0.002% 0.01% 0.5% 0.8% 1% 5% 0-10%

infinite treshold %

0.0000001% 0.00001% 0.0001% 0.001% 0.01% 0.1% 1% 10% 100% one in one one one in one in one one one one hundred in ten in a hundred ten in a in a in ten in one million million million thousand thousand thousand hundred Probability

These estimates are an attempt to assemble existing Probability estimates in order to encourage efforts to improve the numbers. They express estimates of probabilities over 100 years, except in the case of extreme climate change, where the time frame is 200 years.

Global challenges need to be seen in Population growth – the UN’s estimates the light of trends which help to shape range from 6.8 billion people by 2100 the wider society. These include: to a high-variant projection of 16.6 bn (which would require the resources of Poverty – although it has fallen, 10 Earth-like planets to provide everyone it could increase again. This is with a modern Western lifestyle). especially relevant to climate change Other trends include technological and pandemics. development and demographic changes.

n / a

0.00003% n / a 0.0001% 0.00013% 0.005% 0.01% 0.1% 0-10%

Probability of infinite impact (%)

n / a

n / a 0.002% 0.01% 0.5% 0.8% 1% 5% 0-10%

Probability of reaching or exceeding the infinite threshold (%)

0.0000001% 0.00001% 0.0001% 0.001% 0.01% 0.1% 1% 10% 100% one in a one one one in a one in one one one one hundred in ten in a hundred ten in a in a in ten in one million million million thousand thousand thousand hundred

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 19

0.01% 0.01% 0.001% one in ten 0.1% one in hundred thousand one in thousand thousand 0.0001% 1% one in a one in n/a n/a 0.8% million hundred

0.00001% 10% one in ten one in million infinite impact % ten

0.0000001% infinite threshold % 100% one in hundred one in one 0.01% million 0.01% 0.01%

0.00013%

0.01% 0.0001% 0.002% 0.1%

5% 5% 0.00003% 5%

0.005% 0.5% 0.01%

5% n/a 1% n/a n/a Possible ways forward

There are ten areas that could help mitigate immediate Possible threats while also contributing to a future global governance system capable of addressing global ways forward risks with a potential infinite impact:

Global challenges 1. leadership networks Better quality risk 2. assessment for global challenges

Development of 3. early warning systems Encouraging 4. visualisation of complex systems

Highlighting 5. early movers Including the whole 6. probability distribution Increasing the focus on 7. the probability of extreme events

Encouraging 8. appropriate language to describe extreme risks

Establishing a Global Risk and 9. Opportunity Indicator to guide governance

Explore the possibility of establishing a 10. Global Risk Organisation (GRO)

20 Global Challenges – Twelve risks that threaten human civilisation – Executive summary The main authors of this report are Dennis Pamlin, Executive Project Manager, Global Challenges Foundation and Dr Stuart Armstrong, James Martin Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford. Stuart Armstrong wrote the chapter covering the twelve global challenges, under the direction of Dennis Pamlin who served as project manager and himself wrote and edited the rest of the report. Seth Baum, Executive Director of the Global Catastrophic Risk Institute and affiliate researcher at the Center for Research on Environmental Decisions, Columbia University, also played an important role as he helped develop the methodology chapter regarding the selection of the global risks with a potentially infinite impact as well as providing helpful input throughout the process. The report is the result of a collaborative approach where many people have provided invaluable contributions.

The authors would therefore like to thank a few people in particular. First and foremost László Szombatfalvy, Chairman of the Global Challenges Foundation, whose work is the basis for this report and whose guidance on all levels has been invaluable. The rest of the board of the Global Challenges Foundation have also contributed in many different ways, in particular, Johan Rockström has provided important input regarding the structure and methodology. Outside the foundation Prof Nick Bostrom, Professor & Director of the Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford, who initiated the possibility of working with the Future of Humanity Institute at the University of Oxford, played a particularly important role. Patrick McSharry, Head of Smith School’s Catastrophe Risk Financing research area, provided invaluable input regarding complex systems and ways that the economic system can respond to risks with potentially infinite impacts. Alex Kirby also played a key part as he did so much more than proofread the text; the report would hardly be possible to read without his help. Various additional edits and changes were made by Peter Brietbart.

Others that must be mentioned, including those who participated in the workshop on 14 January 2014, at the Future of Humanity Institute (FHI), University of Oxford and the workshop at the Munich RE office in London on 15 January 2014, and helped provide input regarding the economic and finance aspects, include (in alphabetical order):

Dr Nick Beckstead, Research Fellow, Future Aled Jones, Director of the Global Andrew Simms, Author, Fellow at the New of Humanity Institute, Oxford Martin School & Sustainability Institute (GSI) at Anglia Ruskin Economics Foundation and Chief Analyst at Faculty of Philosophy, University of Oxford University Global Witness Kennette Benedict, Executive Director Nick Mabey, Chief Executive and Andrew Snyder-Beattie, Academic Project and Publisher of the Bulletin of the Atomic Founding Director of E3G (Third Generation Manager, Future of Humanity Institute, Oxford Scientists Environmentalism) Martin School & Faculty of Philosophy, Oliver Bettis, Pricing Actuary, Munich RE and Jennifer Morgan, Founder & Co-Convener, University of Oxford Fellow of the Chartered Insurance Institute The Finance Lab James Taplan, Principal Sustainability Advisor, and the Institute & Faculty of Actuaries Prof Vincent Müller, James Martin Research Forum for the Future Dr Eric Drexler, Academic Visitor, Future of Fellow, Future of Humanity Institute, Oxford Raj Thamotheram, CEO, Preventable Surprises Humanity Institute, Oxford Martin School & Martin School & Faculty of Philosophy, Nathan Wolfe, Director of Global Viral and Faculty of Philosophy, University of Oxford University of Oxford the Lorry I. Lokey Visiting Professor in Human Madeleine Enarsson, Transformative Robert de Neufville, Professional Associate, Biology at Stanford University Catalyst, 21st Century Frontiers Global Catastrophic Risk Institute Liang Yin, Investment Consultant at Towers Dr Seán Ó hÉigeartaigh, Senior Academic Dr Toby Ord, James Martin Research Fellow, Watson Manager, Future of Humanity Institute, Future of Humanity Institute, Oxford Martin University of Oxford and Executive Director, School & Faculty of Philosophy, University of Centre for the Study of Existential Risk, Oxford University of Cambridge Jules Peck, Founding Partner, Jericho Martin Hellman, Professor Emeritus of Chambers; Trustee, New Economics Electrical Engineering, Stanford University Foundation Pan Jiahua, Director of the Institute for Dr Anders Sandberg, James Martin Urban and Environmental Studies, Chinese Research Fellow, Future of Humanity Academy of Social Sciences (CASS); Institute, Oxford Martin School & Faculty of Professor of economics at CASS; Vice- Philosophy, University of Oxford President Chinese Society for Ecological Nick Silver, Director of Callund Consulting Economics; Member of the National Expert and founder and director of the Climate Panel on Climate Change and National Bonds Initiative (CBI) Foreign Policy Advisory Committee, China

design by wearebwa.co.uk Published February 2015 by Global Challenges Foundation

globalchallenges.org

Global Challenges Foundation Stureplan 4C 114 35 Stockholm

The opinions expressed in this report are those of the authors and do not necessarily reflect those of Global Challenges Foundation.

Any inaccuracies in the report remain the responsibility of the authors.

The material and the geographical designations in this report do not imply the expression of any opinion whatsoever on the part of Global Challenges Foundation, concerning the legal status of any country, territory, or area, or concerning the delimitation of its frontiers or boundaries. For comments and/or queries on this report, please contact the co-authors:

Dennis Pamlin Dr Stuart Armstrong Executive Project Manager James Martin Research Fellow Global Risks Future of Humanity Institute Global Challenges Foundation Oxford Martin School & Faculty of Philosophy [email protected] University of Oxford globalchallenges.org [email protected] fhi.ox.ac.uk