Global Challenges Foundation Works to Raise Awareness of the Greatest Threats the Idea That We Face a Number of Global Challenges Facing Humanity

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial Extreme Future Bad Global Global System Major Asteroid Global Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Challenges Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Ecological NanotechnologyIntelligence NuclearClimate WarChange Super-volcanoGlobal Governance PandemicSynthetic UnknownCollapse Impact Risks that threaten Catastrophe Biology Consequences humanArtificial civilisationExtreme Future Bad Global Global System Major Asteroid 12 Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Artificial Extreme Future Bad Global Global System Major Asteroid IntelligenceEcological ClimateNanotechnology Change NuclearGlobal Governance War Super-volcanoPandemic CollapseSynthetic UnknownImpact Catastrophe Biology Consequences

Executive SummaryEcological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Ecological Nanotechnology Nuclear War Super-volcano SyntheticIntelligence ClimateUnknown Change Global Governance Pandemic Collapse Impact Catastrophe Biology Consequences

The case for Ecologicala new riskNanotechnology categoryNuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences

Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact

Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Catastrophe Biology Consequences Introduction

This is the executive summary of a report about a limited Introduction number of global risks that pose a threat to human civilisation, or even possibly to all human life.

With such a focus it may surprise some readers to find that the report’s essential aim is to inspire action and dialogue as well as an increased use of the methodologies used for risk assessment. The real focus is not on the almost unimaginable impacts of the risks the report outlines. Its fundamental purpose is to encourage global collaboration and to use this new category of risk as a driver for innovation.

The Global Challenges Foundation works to raise awareness of the greatest threats The idea that we face a number of global challenges facing humanity. In particular climate change, other environmental damage and political threatening the very basis of our civilisation at the beginning violence, and how these threats are linked to poverty and the rapid growth in global of the 21st century is well accepted in the scientific community, population. These problems appear insurmountable without an international body and is studied at a number of leading universities.1 But there with decision-making mandate. The Foundation is therefore working to identify is still no coordinated approach to address this group of possible solutions and models as to how the United Nations can develop, and initiate new ideas on working global governance. challenges and turn them into opportunities.

Full report can be downloaded at globalchallenges.org/globalrisks Contents page It is only 70 years ago that Robert Oppenheimer, who History: Edward Teller, one of the led the Manhattan Project to - Introduction 1 greatest physicists of his time, develop the nuclear bomb, - 12 Global risks 2 the LA-602 with his back-of-the-envelope halted the project to see calculations, produced results whether Teller’s calculations - The goals of the report 4 document that differed drastically from were correct. The resulting all that had gone before.His document, LA-602: Ignition of - Report structure 5 calculations showed that the the Atmosphere with Nuclear - A new category of global risk 6 explosion of a nuclear bomb – a Bombs, concluded that Teller creation of some of the brightest was wrong. But the sheer - Infinite impacts and thresholds 7 minds on the planet, including complexity drove the assessors - Methodology 8 Teller himself – could result in to end their study by writing a chain reaction so powerful that “further work on the - Quick overview of each risk 9 that it would ignite the world’s subject [is] highly desirable”. atmosphere, thereby ending The LA-602 document can - Relations between global risks 16 human life on Earth. be seen as the first global - The technical difficulty of reducing the challenge report addressing a category of risks where the risk and the difficulty of collaboration 17 worst possible impact in all - Uncertainties 18 practical senses is infinite. - Probability 19

- Possible ways forward 20 1 http://en.wikipedia.org/wiki/Global_catastrophic_risk

Global Challenges – Twelve risks that threaten human civilisation – Executive summary 1 Artificial Extreme Future Bad Global Global System Major Asteroid 12 Global risks Intelligence Climate Change Global Governance Pandemic 12 Global risks Collapse Impact

This report has, to the best of the authors’ knowledge, 12 Global created the first list of global risks with impacts that for all practical purposes can be called infinite. It is risks also the first structured overview of key events related to such challenges and has tried to provide initial rough quantifications for the probabilities of these impacts. Artificial Extreme Future Bad Global Global System Major Asteroid

The report conducts its exploration It is worth noting that complex Intelligence Climate Change Global Governance Pandemic Collapse Impact within carefully defined bounds, systems are often stable ArtificialonlyEcologicalArtificial within ExtremeNanotechnologyExtreme FutureNuclearFuture Bad BadWar GlobalSuper-volcanoGlobal GlobalSyntheticGlobal System System MajorUnknownMajor Asteroid Asteroid resulting in a list of twelve risks with certain boundaries outsideIntelligenceCatastrophe Intelligencewhich ClimateClimate Change Change GlobalGlobal Governance Governance PandemicPandemic CollapseBiologyCollapse ImpactConsequencesImpact a possible infinite outcome the system can collapse and rapidly change to a new stable state. Such a There were many challenges which collapse can trigger a process where might have been included on the change continues for a long time until list because of their ability to pose a new stable state is found. None severe damage to humanity. They of the risks in this report is likely to were excluded for one or more of result directly in an infinite impact, three reasons: and some cannot do so physically. All the risks however are big enough to ArtificialArtificial 1. Limited impact – tsunamis,ExtremeExtreme for reach a threshold where theFutureFuture social and Bad Bad GlobalGlobal GlobalGlobalArtificial System System MajorMajorExtreme Asteroid Asteroid Future Bad Global Global System Major Asteroid IntelligenceIntelligenceexample, and chemical Climatepollution.Climate Change Changeecological systems becomeGlobalGlobal so unstable Governance Governance PandemicPandemic CollapseCollapseIntelligence ImpactImpactClimate Change Global Governance Pandemic Collapse Impact ArtificialArtificial ExtremeExtreme that an infinite impact couldFutureFuture ensue. Bad Bad EcologicalGlobalGlobal NanotechnologyGlobalGlobal System System NuclearMajorMajor Asteroid AsteroidWar Super-volcano Synthetic Unknown IntelligenceIntelligence2. No effective countermeasuresClimateClimate – Change Change GlobalGlobal Governance Governance CatastrophePandemicPandemic CollapseCollapse ImpactImpact Biology Consequences the report focuses on promoting This is a report about two Ecologicalextremes,Ecological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSynthetic UnknownUnknown effective interventions and so not one. It is about how a better ignores challenges where nothing understanding of the magnitudeCatastropheCatastrophe of BiologyBiology ConsequencesConsequences useful can be done to prevent or the challenges can help the world mitigate the impact, as with to address the risks it faces, and nearby gamma-ray bursts. can help to create a path towards more sustainable development. It 3. Included in other challenges. is a scientific assessment about Many challenges are already the possibility of oblivion, certainly, Artificial Extreme covered by others, or areFuture very Badbut even more it is a call forGlobal action Global System Major Asteroid Intelligence Climate Changesimilar to them. PopulationGlobal growth, Governancebased on the assumption thatPandemic Collapse Impact EcologicalEcological for one, is significant forNanotechnologyNanotechnology climate humanity is able to rise toNuclear challengesNuclear War War Super-volcanoSuper-volcano SyntheticSyntheticEcological UnknownUnknownNanotechnology Nuclear War Super-volcano Synthetic Unknown change and ecosystem and turn them into opportunities. CatastropheCatastrophecatastrophe, but without direct We are confronted with possibly BiologyBiologyCatastrophe ConsequencesConsequences Biology Consequences EcologicalEcological large-scale impacts of itsNanotechnology Nanotechnologyown. the greatest challenge everNuclearNuclear and War War Super-volcanoSuper-volcano SyntheticSynthetic UnknownUnknown CatastropheCatastrophe our response need to match this BiologyBiology ConsequencesConsequences thorough global collaboration in new and innovative ways.

Ecological Nanotechnology Nuclear War Super-volcano ArtificialSyntheticArtificial ExtremeUnknownExtreme FutureFuture Bad Bad GlobalGlobal GlobalGlobal System System MajorMajor Asteroid Asteroid Catastrophe IntelligenceBiologyIntelligence ClimateConsequencesClimate Change Change GlobalGlobal Governance Governance PandemicPandemic CollapseCollapse ImpactImpact

2 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Global Challenges – Twelve risks that threaten human civilisation – Executive summary 3

EcologicalEcological NanotechnologyNanotechnology NuclearNuclear War War Super-volcanoSuper-volcano SyntheticSynthetic UnknownUnknown CatastropheCatastrophe BiologyBiology ConsequencesConsequences The goals of the report Report structure

The four main goals of this report are to acknowledge, The first part of the report introduces and defines the The goals inspire, connect and deliver. Report global challenges and includes the methodology for of the report structure selecting them. The first of the report’s goals – The third goal is to connect The third part of the report discusses acknowledging the existence of different groups at every level, The second part is an overview of the relationship between the different risks with potential infinite impact – so that leaders in different sectors the twelve challenges and key challenges, because action to seeks to help key stakeholders to connect with each other to events that illustrate strategic work address one can increase the risk acknowledge the existence of the encourage collaboration. This will to address them. It also lists for each of another. Many solutions can also category of risks that could result need a specific focus on financial challenge five important factors that address multiple challenges, so in infinite impact and to show them and security policy where significant influence its probability or impact. there are significant benefits from that we can reduce or even eliminate risks combine to demand action The challengesare divided into four understanding how they are linked. most of them. beyond the incremental. different categories: The fourth part is an overview, the first The second inspires by showing The fourth goal is to deliver actual – current challenges includes those ever to the authors’ knowledge, of the the practical action that is taking strategies and initiatives that which currently threaten humanity probabilities of global challenges with place today. This report seeks to produce actual results. The report because of its economic and potential infinite impacts. show that helping to meet these is a first step and its success will technological development; global challenges is perhaps the ultimately be measured only on how The fifth presents some of the most exogenic challenges most important contribution anyone it contributes to concrete results. – are those important underlying trends that can make today, and highlights where the basic probability of an influence the challenges, which often concrete examples to inspire a new The report will have achieved its event is beyond human control, but build up slowly to a threshold where generation of leaders. goals when key decision-makers where the probability and magnitude very rapid changes can ensue. recognise the magnitude of the of the impact can be influenced; possible risks and our ability to – emerging challenges could both The sixth part presents an overview reduce or even eliminate most help to reduce the risks associated of possible ways forward. of them. with current challenges and also result in infinite impacts;

– the last of the twelve is defined as a global policy challenge, a threat arising from future global governance as it resorts to destructive policies in response to the categories of challenge listed above.

The goals

1. to acknowledge the existence of risks with potential infinite impact 2. to inspire by showing the practical action that is taking place today. 3. to connect different groups at every level. 4. to deliver actual strategies and initiatives that produce actual results.

4 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Global Challenges – Twelve risks that threaten human civilisation – Executive summary 5 A new category of global risk Infinite impacts and thresholds

The idea that there may be risks where the impact can There is a clear ethical dimension to the concept of A new category be described as infinite, defined as the end of human Infinite impacts infinite impact, because a very small group alive today civilisation or even human life, is not new. However, can take decisions that will fundamentally affect all of global risk it excites relatively little political or academic interest and thresholds future generations. and the way it is treated in popular culture makes a serious discussion harder. Its equally clear economic component events that could result first in a Risk = Probability x Impactcan generate disagreement over issues civilisation collapse, and then later For several reasons the potentially The review of literature indicate that, such as discounting, which the report result in an infinite impact. Such infinite impacts of the challenges in under a business as usual scenario, examines in some detail,considering thresholds are especially important this report are not as well known as new risks with potential infinite impact for example the role of tipping points. to recognise in a complex and they should be. One reason is the is probably inseparable from the rapid interconnected society where way that extreme impacts are often technological development in areas The report distinguishes between the resilience is decreeing. masked by most of the theories and like synthetic biology, nanotechnology concepts of infinite impact – where models used by governments and and AI. civilisation collapses to a state of A collapse of civilisation is defined business today. great suffering and do not recover, as a drastic decrease in human Most risks are linked to increased or a situation where all human life population size and political/ Climate change is a good example, knowledge, economic and technical end – and infinite impact threshold – economic/social complexity, globally where almost all of the focus is on development that has brought many an impact that can trigger a chain of for an extended time. the most likely scenarios and there benefits. E.g. climate change is a are few public studies that include result from the industrial revolution the low-probability high-impact and fossil fuelbased development. scenarios. In most reports about The increased potential for global climate impacts, those caused by pandemics is one consequence of warming beyond five or six degrees an integrated global economy where Celsius are omitted from tables goods and services move quickly and graphs. Other aspects that internationally. Similar challenges can Normal Risks New Category contribute to this relative invisibility be expected for synthetic biology, include the fact that extreme nanotechnology and AI. Traditional measures Requires new measures and tools and tools applicable impacts are difficult to translate into monetary terms, they have a global There are remedies, including scope, and they often require a time- technological and institutional, horizon of a century or more. They for all risks. But they will require cannot be understood simply by collaboration of a sort humanity has linear extrapolation of current trends, not achieved before, and the creation and they lack historical precedents. of systems which can deal with There is also the fact that the problems pre-emptively. It is important measures required to significantly to understand that much of the reduce the probability of infinite knowledge and many tools that we impacts will be radical compared to have, and will develop, can be a business-as-usual scenario. both a risk and a solution to risks Threshold depending on context. A scientific approach requires us to base our decisions on the whole probability distribution. probability = x Risk Probability Impact 0 impact

6 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Global Challenges – Twelve risks that threaten human civilisation – Executive summary 7

Normal Risks New Category Traditional measures Requires new measures and tools and tools applicable

Threshold probability

0 impact Methodology Quick overview of each risk

In order to establish a list of global challenges Methodology with potential infinite impact a methodological Quick overview triangulation was used, consisting of: of each risk

– A quantitative assessment of Two workshops were arranged relevant literature. where the selection of challenges was discussed, one with risk experts Current Current Current – A strategic selection of relevant risk risk risk in Oxford at the Future of Humanity organisations and their priorities. Institute and the other in London with experts from the financial sector. – A qualitative assessment with the Artificial Extreme Future Bad Global Global System Major Asteroid help of expert workshops. No challenge was excluded at the Intelligence Climate Change Global Governance Pandemic Collapse Impact workshops, but one was added: the participants agreed to include Global System Collapse as a category.

Artificial Extreme EcologicalFuture Bad NanotechnologyGlobal ArtificialNuclearGlobal War System ExtremeSuper-volcanoMajor Asteroid FutureSynthetic Bad GlobalUnknown Global System Major Asteroid Intelligence Climate Change CatastropheGlobal Governance Pandemic IntelligenceCollapse ClimateImpact Change GlobalBiology Governance PandemicConsequences Collapse Impact

Relevant literature Current Current Exogenic Identification of credible sources: search relevant literature risk risk risk in academic literature included in World of Knowledge and Google Scholar.

Ecological Nanotechnology Nuclear War Super-volcano EcologicalSynthetic NanotechnologyUnknown Nuclear War Super-volcano Synthetic Unknown Catastrophe Artificial Extreme Future Bad CatastropheGlobalBiology ConsequencesGlobal System Major Asteroid Biology Consequences 1 Intelligence Climate Change Global Governance Pandemic Collapse Impact Estimations of impact Only literature where there is some estimation of impact that indicates the possibility of an infinite impact is included.

Artificial ExtremeEcological ArtificialFutureNanotechnology Bad ExtremeGlobalNuclear War FutureGlobal BadSuper-volcano System GlobalMajorSynthetic Asteroid Global SystemUnknown Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact Collapse Leading organisations’ priorities 2 Catastrophe Intelligence Climate Change Global Governance PandemicBiology Consequences Impact In order to increase the probability of covering all relevant risks an overview Exogenic Emerging Emerging of leading organisations' work was conducted. This list was then compared with the initial list risk risk risk and subjected to the same filter regarding the possibility to affect the probability or impact.

Ecological Nanotechnology EcologicalNuclear War NanotechnologySuper-volcano NuclearSynthetic War Super-volcanoUnknown Synthetic Unknown Catastrophe Catastrophe Biology Consequences Biology Consequences Artificial Extreme ArtificialFuture Bad ExtremeGlobal FutureGlobal Bad System GlobalMajor Asteroid Global System Major Asteroid Artificial Extreme Future Bad Global Global System Major Asteroid 3 Possibility of addressing the risk Intelligence Climate Change IntelligenceGlobal Governance ClimatePandemic Change GlobalCollapse Governance PandemicImpact Collapse Impact Intelligence Climate Change Global Governance Pandemic Collapse Impact Possibility of addressing the risk: From the risks gathered from literature and organisations only those where the probability or impact can be affected by human actions are included.

Ecological Nanotechnology EcologicalNuclear War NanotechnologySuper-volcano NuclearSynthetic War Super-volcanoUnknown Synthetic Unknown Ecological Nanotechnology Nuclear War Super-volcano Synthetic Unknown Expert review Catastrophe 4 Catastrophe Biology Consequences Biology Consequences Catastrophe Biology Consequences Qualitative assessment: Expert review in order to increase the Emerging Emerging Global Policy probability of covering all relevant global risks. risk risk risk

5 Artificial Extreme Future Bad Global Global System Major Asteroid Intelligence Climate Change Global Governance Pandemic Collapse Impact List of risks Result: List of risks with potentially infinite impacts.

Ecological6 Nanotechnology NuclearArtificial War Super-volcanoExtreme SyntheticFuture Bad UnknownGlobal Global System Major Asteroid Global System Artificial Extreme Future Bad Global Major Asteroid Catastrophe Intelligence Climate Change BiologyGlobal Governance ConsequencesPandemic IntelligenceCollapse Climate ImpactChange Global Governance Pandemic Collapse Impact

8 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Global Challenges – Twelve risks that threaten human civilisation – Executive summary 9

Ecological Nanotechnology Nuclear War Super-volcano EcologicalSynthetic NanotechnologyUnknown Nuclear War Super-volcano Synthetic Unknown Catastrophe CatastropheBiology Consequences Biology Consequences Quick overview of each risk - Extreme Climat Change / Nuclear War Quick overview of each risk - Global Pandemic / Ecological Collapse

Current risk Current risk Extreme Global 5 key Climate Change 5 key Pandemic factors: factors: Extreme1 The uncertainties in As for all risks there are uncertainties Mass deaths and famines, social Global1 What the true probability An epidemic of infectious disease (and influenza has demonstrated Climate Change Pandemic climate sensitivity models, in the estimates, and warming could collapse and mass migration are distribution for pandemics that has spread through human antigenic shift, the ability to combine including the tail. be much more extreme than the certainly possible in this scenario. is, especially at the tail. populations across a large region or features from different viruses), its 2 The likelihood - or not - of middle estimates suggest. Feedback Combined with shocks to the 2 The capacity of international even worldwide. There are grounds death toll would be extreme. global coordination on loops could mean global average agriculture and biosphere-dependent health systems to deal for suspecting that such a high- The world has changed considerably, controlling emissions. temperatures increase by 4°C or industries of the more developed with an extreme pandemic. impact epidemic is more probable making comparisons with the past even 6°C over pre-industrial levels. countries, this could lead to global than usually assumed. All the problematic.Today it has better 3 The future uptake of 3 How fast medical research Feedbacks could be the release conflict and possibly civilisation features of an extremely devastating sanitation and medical research, as low carbon economies, can proceed in an of methane from permafrost or the collapse. Further evidence of the disease already exist in nature: well as national and supra-national including energy, mobility emergency. dieback of the Amazon rainforest. risk comes from signs that past essentially incurable (Ebola), nearly institutions dedicated to combating and food systems. The impact of global warming would civilisation collapses have been 4 How mobility of goods and always fatal (rabies), extremely diseases. But modern transport 4 Whether technological be strongest in poorer countries, driven by climate change. people, as well as population infectious (common cold), and and dense human population allow innovations will improve which could become completely density, will affect pandemic long incubation periods (HIV). If infections to spread much more or worsen the situation, uninhabitable for the highest range transmission. a pathogen were to emerge that rapidly, and slums can be breeding and by how much. of warming. 5 Whether humans can somehow combined these features grounds for disease. 5 The long-term climate develop novel and effective impact caused by anti-pandemic solutions. global warming.

Current risk Current risk Ecological 5 key Nuclear War 5 key Collapse factors: factors: Nuclear War Ecological 1 How relations between The likelihood of a full-scale nuclear plunge temperatures below freezing Collapse1 The extent to which This is where an ecosystem suffers the damage and (unlike previous, current and future war between the USA and Russia around the globe and possibly also humans are dependent on a drastic, possibly permanent, localised collapses) the whole world nuclear powers develop. has probably decreased. Still, the destroy most of the ozone layer. the ecosystem. reduction in carrying capacity for is potentially at risk. 2 The probability of potential for deliberate or accidental The detonations would need to 2 Whether there will be all organisms, often resulting in It seems plausible that some human accidental war. nuclear conflict has not been start firestorms in the targeted effective political measures mass extinction. Humans are part lifestyles could be sustained in a removed, with some estimates cities, which could lift the soot up of the global ecosystem and so relatively ecosystem independent 3 Whether disarmament taken to protect the putting the risk in the next century into the stratosphere. The risks are fundamentally depend on it. Species way, at relatively low costs. Whether efforts will succeed in ecosystem on a large scale. or so at around 10%. A larger impact severe and recent models have extinction is now far faster than the this can be achieved on a large reducing the number of 3 The likelihood of the would depend on whether or not the confirmed the earlier analysis. The historic rate, and attempts to scale in practice, especially during nuclear warheads. emergence of sustainable war triggered what is often called a disintegration of the global food quantify a safe ecological operating a collapse, will be a technological economies. 4 The likelihood of a nuclear winter or something similar – supply would make mass starvation space place humanity well outside it. challenge and whether it is something nuclear winter. the creation of a pall of smoke high and state collapse likely. 4 The positive and negative Many of the problems of ecological we want is an ethical question. 5 The long-term effects in the stratosphere that would impacts on the ecosystems degradation interact to multiply of a nuclear war on of both wealth and poverty. climate, infrastructure 5 The long-term effects of and technology. A new an ecological collapse category of global risk. on ecosystems.

10 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Global Challenges – Twelve risks that threaten human civilisation – Executive summary 11 Quick overview of each risk - Global System Collapse / Major Asteroid Impact Quick overview of each risk - Super-volcano / Synthetic Biology

Current risk Exogenic risk Global System 5 key Collapse 5 key Super-volcano factors: factors: Global1 WhetherSystem global system An economic or societal collapse structure of the network – even if Super-volcano1 Whether countries will Any volcano capable of producing while the Toba eruption around Collapse collapse will trigger on the global scale. The term has each component of the network is coordinate globally an eruption with an ejecta volume 70,000 years ago is thought by some subsequent collapses or been used to describe a broad reliable. This gives rise to systemic against super-volcano risk greater than 1,000 km3. This is to have cooled global temperatures fragility in other areas. range of conditions. Often economic risk, when parts that individually may and damage. thousands of times larger than for over two centuries. 2 What the true trade-off is collapse is accompanied by social function well become vulnerable 2 The predictability of super- normal eruptions. The danger from The effect of these eruptions could between efficiency chaos, civil unrest and sometimes a when connected as a system to a volcanic eruptions. super-volcanoes is the amount of be best compared with that of a breakdown of law and order. Societal self-reinforcing joint risk that can aerosols and dust projected into the nuclear war. The eruption would and resilience. 3 How directly destructive collapse usually refers to the fall or spread from part to part, potentially upper atmosphere. This dust would be more violent than the nuclear 3 Whether effective an eruption would be. disintegration of human societies, affecting the entire system and absorb the Sun’s rays and cause explosions, but would be less likely regulation and resilience often along with their life support possibly spilling over to related 4 The effectiveness of a global volcanic winter. The Mt to ignite firestorms and other can be developed. systems. The world economic and outside systems. Such effects have general mitigation efforts. Pinatubo eruption of 1991 caused secondary effects. 4 Whether an external political system is made up of been observed in ecology, finance 5 How severe the long-term an average global cooling of surface disruption will trigger many actors with many objectives and critical infrastructure such climate effects would be. temperatures by 0.5°C over three years, a collapse. and many links between them. as power grids. The possibility of 5 Whether an internal event Such intricate, interconnected collapse becomes more acute when will trigger a collapse. systems are subject to unexpected several independent networks system-wide failures caused by the depend on each other.

Exogenic risk Emerging risk Major Asteroid Synthetic 5 key Impact 5 key Biology factors: factors: Major Asteroid Synthetic Impact1 Whether detection and Large asteroid collisions – with There has been some discussion Biology1 The true destructive potential The design and construction of This could emerge through military tracking of asteroids and objects 5 km or more in size – about possible methods for of synthetic biology, especially biological devices and systems or commercial bio-warfare, bio- other dangerous space happen about once every twenty deflecting asteroids found on a the tail risk. for useful purposes, but adding terrorism (possibly using dual-use objects is sufficiently million years and would have an collision course with the planet. 2 Whether the field will be human intentionality to traditional products developed by legitimate exhaustive. energy a hundred thousand times Should an impact occur the main successfully regulated, or pandemic risks. Attempts at researchers, and currently 2 How feasible it is to deflect greater than the largest bomb ever destruction will not be from the successfully manage to regulation or self-regulation are unprotected by international legal an asteroid. detonated. A land impact would initial impact, but from the clouds regulate itself. currently in their infancy, and may regimes), or dangerous pathogens destroy an area the size of a nation of dust projected into the upper not develop as fast as research leaked from a lab. Of relevance is 3 Whether measures such as 3 Whether the field will usher like Holland. Larger asteroids could atmosphere. The damage from such does. One of the most damaging whether synthetic biology products evacuation could reduce in a new era of bio-warfare. be extinction-level events. Asteroid an “impact winter” could affect impacts from synthetic biology become integrated into the global the damage of an impact. impacts are probably one of the best the climate, damage the biosphere, 4 Whether the tools of synthetic would come from an engineered economy or biosphere. This could 4 The short- and long-term understood of all risks in this report. affect food supplies, and create biology can be used pathogen targeting humans or a lead to additional vulnerabilities (a climate consequences of a political instability. defensively to create effective crucial component of the ecosystem. benign but widespread synthetic collision. counter measures. biology product could be specifically 5 Whether our current 5 The dangers of relying targeted as an entry point through civilisation could adapt to a on synthetic biologists to which to cause damage). post-impact world. estimate the danger of synthetic biology.

12 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Global Challenges – Twelve risks that threaten human civilisation – Executive summary 13 Quick overview of each risk - Nanotechnology / Artificial Intelligence Quick overview of each risk - Unknown Consequences / Future Bad Global Governance

Emerging risk Emerging risk Unknown 5 key Nanotechnology 5 key Consequences factors: factors: Nanotechnology1 The timeline for nanotech Atomically precise manufacturing, Of particular relevance is whether Unknown1 Whether there will be These represent the unknown generic probability of intelligent life Consequences development. the creation of effective, high- nanotechnology allows the extensive research into unknowns in the family of global (self-)destruction, which includes 2 Which aspects of throughput manufacturing processes construction of nuclear bombs. unknown risks and their catastrophic challenges. They uncertain risks. Anthropic reasoning nanotech research will that operate at the atomic or But many of the world’s current probabilities. constitute an amalgamation of all the can also bound the total risk of human progress in what order. molecular level. It could create problems may be solvable with the 2 The capacity to develop risks that can appear extremely extinction, and hence estimate the new products – such as smart or manufacturing possibilities that methods for limiting unlikely in isolation, but can combine unknown component. Nonrisk- 3 Whether small groups can extremely resilient materials – and nanotechnology would offer, such the combined probability to represent a not insignificant specific resilience and post-disaster assemble a weapons would allow many different groups as depletion of natural resources, of all uncertain risks proportion of the risk exposure. One rebuilding efforts will also reduce arsenal quickly. or even individuals to manufacture a pollution, climate change, clean 3 The capacity for estimating resolution to the Fermi paradox – the damage from uncertain risks, 4 Whether nanotech tools wide range of things. This could lead water and even poverty. Some have “out of-model” risks. the apparent absence of alien life as would appropriate national and can be used defensively to the easy construction of large conjectured special self-replicating 4 The culture of risk in the galaxy – is that intelligent life international regulatory regimes. or for surveillance. arsenals of conventional or more nanomachines which would be assessment in potentially destroys itself before beginning to Most of these methods would also help 5 Whether nanotech tools or novel weapons made possible by engineered to consume the entire risky areas expand into the galaxy. Results that with the more conventional, known weaponry are made to be atomically precise manufacturing. environment. The misuse of medical 5 Whether general, non- increase or decrease the probability risks, and badly need more investment. outside human control. nanotechnology is another risk scenario. risk-specific mitigation or of this explanation modify the resilience measures are implemented.

Emerging risk Global Policy risk

Artificial And if these motivations do not Future Bad detail the survival and value of 5 key Intelligence humanity, the intelligence will be 5 key Global Governance factors: driven to construct a world without factors: humans. This makes extremely Artificial Future Bad Intelligence1 The reliability of AI AI is the intelligence exhibited by intelligent AIs a unique risk, in that Global1 HowGovernance the severity of non- There are two main divisions in Two issues with governance predictions. machines or software, and the extinction is more likely than lesser deadly policy failures governance disasters: failing to disasters are first, the difficulty 2 Whether there will be a branch of computer science that impacts. On a more positive note, can be compared with solve major solvable problems, and of estimating their probability, single dominant AI or a develops machines and software an intelligence of such power could potential casualties. actively causing worse outcomes. and second, the dependence of plethora of entities. with human-level intelligence. easily combat most other risks 2 Whether poor governance An example of the first would be the impact of these disasters on 3 How intelligent AIs will The field is often defined as “the in this report, making extremely will result in a collapse failing to alleviate absolute poverty; subjective comparative evaluations: become. study and design of intelligent intelligent AI into a tool of great of the world system. of the second, constructing a it is not impartially obvious how to agents”, systems that perceive their potential. There is also the possibility global totalitarian state. Technology, rank continued poverty and global 4 Whether extremely 3 How mass surveillance environment and act to maximise of AI-enabled warfare and all the political and social change may totalitarianism against billions of intelligent AIs can be and other technological controlled, and how. their chances of success. Such risks of the technologies that AIs enable the construction of new casualties or civilisation collapse. innovations will affect extreme intelligences could not would make possible. An interesting forms of governance, which may be 5 Whether whole brain governance. emulations (human minds easily be controlled (either by the version of this scenario is the either much better or much worse. in computer form) will groups creating them, or by some possible creation of “whole brain 4 Whether there will be new arrive before true AIs. international regulatory regime), emulations”, human brains scanned systems of governance in and would probably act to boost and physically represented in a the future. their own intelligence and acquire machine. This would make the AIs 5 Whether a world maximal resources for almost all into properly human minds, possibly dictatorship may end up initial AI motivations. alleviating a lot of problems. being constructed.

14 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Global Challenges – Twelve risks that threaten human civilisation – Executive summary 15 Relations between global risks The technical difficulty of reducing the risk and the difficulty of collaboration

Two things make the understanding of the relation In order to better understand the relations between Relations between between the global risks particularly important. The technical different global risks work could start to analyse global risks difficulty of similarities and differences. 1. Impacts: The risks are 2. Specific measures to address a interconnected in different ways. risk: Global risks often require reducing the risk Below is an example of an overview Often the situation resembles a significant changes, which will of how different global risks can be set of dominoes: if one falls, many result in situations where measures to and the difficulty plotted depending on the technical follow. Even small impacts can reduce the risk in one area affect difficulty of reducing the risk and the start a process where different the probability and/or the impact in of collaboration difficulty of collaborating to reduce it. risks interact. other areas, for better or worse.

ALL RISKS

first risk worsens second risk

solving first risk improves second risk risk of reducing technical difficulty

both of the above collaboration difficulty of reducing risk

16 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Global Challenges – Twelve risks that threaten human civilisation – Executive summary 17 Uncertainties Probability

As the different challenges are very different and the These estimates are an attempt to assemble existing Uncertainties status of probability estimates varies significantly, the Probability estimates in order to encourage efforts to improve initial probability numbers are provided together with the numbers. They express estimates of probabilities estimates regarding: over 100 years, except in the case of extreme climate change, where the time frame is 200 years.

Global challenges need to be seen in Population growth – the UN’s estimates 1. Understanding1. Understanding 2. Data2. Data 3. Existing probability the light of trends which help to shape range from 6.8 billion people by 2100 1. Understanding of 1. sequence ofUnderstanding sequence 2. availabilityData availability2. Data 3. estimationExisting probability the wider society. These include: to a high-variant projection of 16.6 bn of sequence of sequence availability availability estimation (which would require the resources of Poverty – although it has fallen, 10 Earth-like planets to provide everyone it could increase again. This is with a modern Western lifestyle). especially relevant to climate change Other trends include technological and pandemics. development and demographic changes. all parts all data all parts all data all parts all data all parts all parts all data all data calculations with small uncertainty calculations with small uncertainty calculations with small uncertainty

n / a most parts most data most parts most data most parts most data n / a most parts most parts most data most data calculations with uncertainty large

calculations with uncertainty large 0.00003% n / a 0.0001% 0.00013% 0.005% 0.01% 0.1% 0-10% calculations with uncertainty large Probability of infinite impact (%) some parts some data some parts some data best guesses by experts some parts some data best guesses by experts some parts some parts some data some data best guesses by experts no estimates none at all no data none at all no data n / a

degreedegreenone at all of events of events from today’sfrom today’s actions actions amountno data amount of data of todata make to make probability probability kindno estimates of estimation and uncertainty to infiniteto infinite impact impact assessmentassessment on all onrelevant all relevant steps steps no estimates none at all none at all ofno data theof sequence the sequenceno data n / a 0.002% 0.01% 0.5% 0.8% 1% 5% 0-10%

Probability of reaching or exceeding the infinite threshold (%) degree of events from today’s actions amount of data to make probability kind of estimation and uncertainty to infinite impact assessment on all relevant steps 0.0000001% 0.00001% 0.0001% 0.001% 0.01% 0.1% 1% 10% 100% amountof the sequence of dataamount to make of data probability to make probability degree of eventsdegree from of events today’s from actions today’s actions kind of estimation and uncertainty one in one one one in one in one one one one to infinite impactto infinite impact assessmentassessment on all relevant on allsteps relevant steps hundred in ten in a hundred ten in a in a in ten in one of the sequenceof the sequence million million million thousand thousand thousand hundred

18 Global Challenges – Twelve risks that threaten human civilisation – Executive summary Global Challenges – Twelve risks that threaten human civilisation – Executive summary 19

n / a

0.01% 0.01% 0.001% one in ten 0.1% one in hundred thousand one in n / a thousand thousand 0.0001% 1% one in a one in n/a n/a 0.8% million hundred 0.00003% n / a 0.0001% 0.00013% 0.005% 0.01% 0.1% 0-10% 0.00001% 10% one in ten one in million infinite impact % ten infinite impact % 0.0000001% infinite threshold % 100% one in hundred one in one 0.01% million 0.01% 0.01%

0.00013%

0.01% 0.0001% 0.002% 0.1%

n / a

5% 5% 0.00003% 5%

n / a 0.002% 0.01% 0.5% 0.8% 1% 5% 0-10%

infinite treshold %

0.0000001% 0.00001% 0.0001% 0.001% 0.01% 0.1% 1% 10% 100% one in one one one in one in one one one one 0.005% 0.5% 0.01% hundred in ten in a hundred ten in a in a in ten in one million million million thousand thousand thousand hundred 5% n/a 1% n/a n/a Possible ways forward

There are ten areas that could help mitigate immediate The main authors of this report are Dennis Pamlin, Executive Project Manager, Global Challenges Foundation and Dr Stuart Armstrong, James Martin Research Fellow, Future of Humanity Institute, Possible threats while also contributing to a future global Oxford Martin School & Faculty of Philosophy, University of Oxford. Stuart Armstrong wrote the chapter governance system capable of addressing global covering the twelve global challenges, under the direction of Dennis Pamlin who served as project ways forward manager and himself wrote and edited the rest of the report. Seth Baum, Executive Director of the risks with a potential infinite impact: Global Catastrophic Risk Institute and affiliate researcher at the Center for Research on Environmental Decisions, Columbia University, also played an important role as he helped develop the methodology chapter regarding the selection of the global risks with a potentially infinite impact as well as providing helpful input throughout the process. But the report is the result of a collaborative approach where many people have provided invaluable contributions. Global challenges leadership networks The authors would therefore like to thank a few people in particular. First and foremost László 1. Szombatfalvy, chairman of Global Challenges Foundation, whose work is the basis for this report and whose guidance on all levels has been invaluable. The rest of the board of the Global Challenges Better quality risk Foundation have also contributed in many different ways, in particular Johan Rockström has provided assessment for important input regarding the structure and methodology. Outside the foundation Prof Nick Bostrom, 2. Professor & Director of the Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, global challenges University of Oxford, who initiated the possibility of working with the Future of Humanity Institute at the University of Oxford, played a particularly important role. Patrick McSharry, head of Smith School’s Catastrophe Risk Financing research area, provided invaluable input regarding complex systems and Development of ways that the economic system can respond to risks with potentially infinite impacts. Alex Kirby also early warning systems played a key part as he did so much more than proofread the text; the report would hardly be possible 3. to read without his help.

Encouraging Others that must be mentioned, including those who participated in the workshop on January 14th 2014, visualisation of at the Future of Humanity Institute (FHI), University of Oxford and the workshop at the Munich RE office 4. in London, January 15th 2014, and helped provide input regarding the economic and finance aspects,

complex systems include (in alphabetical order):

Highlighting Dr Nick Beckstead, Research Fellow, Future Aled Jones, Director of the Global Andrew Simms, Author, Fellow at the New 5. early movers of Humanity Institute, Oxford Martin School & Sustainability Institute (GSI) at Anglia Ruskin Economics Foundation and Chief Analyst at Faculty of Philosophy, University of Oxford University Global Witness Including the whole Kennette Benedict, Executive Director Nick Mabey, Chief Executive and a Andrew Snyder-Beattie, Academic Project and Publisher of the Bulletin of the Atomic founder director of E3G (Third Generation Manager, Future of Humanity Institute, Oxford 6. probability distribution Scientists Environmentalism) Martin School & Faculty of Philosophy, Oliver Bettis, pricing actuary Munich RE and Jennifer Morgan, Founder & Co-Convener, University of Oxford Increasing fellow of the Chartered Insurance Institute the finance lab James Taplan, Principal Sustainability Advisor, the focus on and the Institute & Faculty of Actuaries. Prof Vincent Müller, James Martin Research Forum for the Future 7. the probability Dr Eric Drexler, Academic Visitor, Future of Fellow, Future of Humanity Institute, Oxford Raj Thamotheram, CEO, Preventable Surprises Humanity Institute, Oxford Martin School & Martin School & Faculty of Philosophy, Nathan Wolfe, Director of Global Viral and of extreme events Faculty of Philosophy, University of Oxford University of Oxford the Lorry I. Lokey Visiting Professor in Human Madeleine Enarsson, Transformative Robert de Neufville, Professional Associate, Biology at Stanford University. Encouraging Catalyst, 21st Century Frontiers Global Catastrophic Risk Institute Liang Yin, Investment Consultant at Towers appropriate language Dr Seán Ó hÉigeartaigh, Senior Academic Dr Toby Ord, James Martin Research Fellow, Watson 8. Manager, Future of Humanity Institute, Future of Humanity Institute, Oxford Martin to describe extreme risks University of Oxford and Executive Director, School & Faculty of Philosophy, University of Centre for the Study of Existential Risk, Oxford University of Cambridge Jules Peck, Founding Partner, Jericho Establishing Martin Hellman, Professor Emeritus of Chambers; Trustee, New Economics 9. a Global Risk and Electrical Engineering, Stanford University Foundation Opportunity Indicator Pan Jiahua, Director of the Institute for Dr Anders Sandberg, James Martin to guide governance Urban and Environmental Studies, Chinese Research Fellow, Future of Humanity Academy of Social Sciences (CASS); Institute, Oxford Martin School & Faculty of professor of economics at CASS; vice Philosophy, University of Oxford Explore the possibility president Chinese Society for Ecological Nick Silver, director of Callund Consulting Economics; member National Expert Panel and founder and director of the Climate of establishing a on Climate Change and National Foreign Bonds Initiative (CBI) 10. Global Risk Organisation (GRO) Policy Advisory Committee, China

20 Global Challenges – Twelve risks that threaten human civilisation – Executive summary design by wearebwa.co.uk Published February 2015 by Global Challenges Foundation

globalchallenges.org

Global Challenges Foundation Stureplan 4C 114 35 Stockholm

The opinions expressed in this report are those of the authors and do not necessarily reflect those of Global Challenges Foundation.

Any inaccuracies in the report remain the responsibility of the authors.

The material and the geographical designations in this report do not imply the expression of any opinion whatsoever on the part of Global Challenges Foundation, concerning the legal status of any country, territory, or area, or concerning the delimitation of its frontiers or boundaries. For comments and/or queries on this report, please contact the co-authors:

Dennis Pamlin Dr Stuart Armstrong Executive Project Manager James Martin Research Fellow, Global Risks Future of Humanity Institute, Global Challenges Foundation Oxford Martin School & Faculty of Philosophy, [email protected] University of Oxford globalchallenges.org [email protected] fhi.ox.ac.uk