The AIGO: A Framework for Planning, Developing, and Deploying Artificial Intelligence in Intergovernmental Organizations

Jannik Seger Nicolas Miailhe Simon Mueller

JANUARY 2019 The AIGO: A Framework for Planning, Developing, and Deploying Artificial Intelligence in Intergovernmental Organizations

JANUARY 2019

Jannik Seger | Nicolas Miailhe | Simon Mueller

Artificial Intelligence (AI) has created significant value in prediction technology and illustrates the evolution from crucial areas such as healthcare, finance, energy, retrospective to AI-based, forward looking data analytics transportation, and environmental protection. The • Section IV details the value-creation potential of AI that productivity gains caused by AI hold the potential to bolster IGOs may realize by firmly integrating AI into their core growth and development over the next decades by making processes: improved insight generation, faster and decision-making processes and resource management of larger-scale evidence-based decision making, and complex systems more efficient. optimization of output delivery processes Similar to the steam engine or electricity, AI is a versatile • Section V provides a detailed, multi-step roadmap for an enabling technology with a diverse range of applications. effective implementation of AI in IGOs. It shows how to While the previous two paved the way for the creation of develop a strategic intent for AI, ensure senior specialized machines that could replace human physical executives’ support, lay the organizational foundations, labor for defined tasks, AI will eventually allow for the and ensure adequate data quality. IGO executives find development of self-optimizing information systems capable practical advice on how to prepare the cultural transition, of replacing human cognitive labor in specific settings and identify and prioritize use cases, and ensure a successful applications. This will, as the history of previous eras of rollout of AI solutions accelerated technological progress show, fundamentally • Section VII summarizes the results and contains change the nature of global relationships, sources of concluding remarks on the wider the role IGOs in the competitive advantage and opportunities for economic and global AI revolution social development. • Lastly, the appendix provides IGO executives with an AI So far, intergovernmental organizations (IGOs) and national maturity framework and a comprehensive database of AI governments have not always been adequately positioned use cases to harness the positive effects of these technologies, as well as protect their citizens from the negative ones. Although the present state of technology is already sufficient to About The Future Society substantially increase processing speed and prediction The Future Society (TFS) is an independent think-and-do tank accuracy, experts anticipate that rapid progression in the incubated at Harvard Kennedy School of Government that domain of specialized AI will continue. aims at addressing the profound consequences of the current technological explosion. Its mission is to help build a future that This paper details the value proposition of AI for IGOs and preserves humanity while harnessing the upsides of proposes a comprehensive framework to plan, develop, and technology. TFS is a charitable non-profit 501(c)(3) deploy AI solutions across their organizations. Moreover, a organization based in Cambridge, Massachusetts. brief prospect is given on how IGOs may act as multipliers: About this report building on strong internal capabilities and systematic, This report aims at analyzing trends in adoption, potential use system-wide utilization of AI, their global presence cases, and disruptive potential of advanced analytics and AI in predestines them to assist member countries in developing IGOs in general. It has been the result of an extensive desk research, literature review as well as an expert panel and 15 in- national AI capacities. depth semi-structured interviews with executives, researchers The report proceeds as follows: and thought leaders from IGOs, the private sector, and academia. This report has been fully funded by TFS's • Section I provides an overview of recent developments institutional budget, and not by any individual donors. and trends in AI adoption in private and public sectors, Acknowledgements and IGOs We would like to extend our thanks to all interview partners and • Section II details the IGO landscape, global issues research associates who generously shared their expertise and addressed, functions served, and the value contribution provided input. This report is based on independent research of these entities within the international arena and represents TFS’s findings. To provide comments or get in touch with the authors of this report, please write • Section III defines and introduces AI, AI-enabled technologies, and corresponding applications with a [email protected] focus on data analytics capabilities. It depicts AI as a

1 I. Trends in AI adoption across sectors Intergovernmental organizations Across all domains, decision-making is increasingly IGOs, too, have recognized the potential benefits from supported by algorithms. Private and public sector agents implementing AI. An increasing number of organizations as well as IGOs are progressively adopting AI technologies have started to put AI on their agendas, be it to initiate for predictive capabilities and to make processes more ideation on how the organizations themselves may benefit efficient. from AI or to evaluate the technology’s potential impact on key stakeholders, e.g. the effects of AI on employment in This paper examines the various types of applications and developing economies. In the words of UN Secretary- value creation use cases of AI in IGOs and provides General António Guterres, speaking at the AI for Good practical and actionable steps for decision makers to Global Summit in June 2017: “The time has arrived for all of capitalize on AI systematically. us – governments, industry and civil society – to consider how AI will affect our future.”12 Private sector International Data Corporation, a market intelligence II. Intergovernmental organization as enablers of global provider, reports that worldwide spending on AI systems solutions has totaled $12.0 billion in 2017 (an increase of 59.1% over 2016) and forecasts this figure to reach $57.6 billion in Intergovernmental organizations (IGO) are entities created 2021, translating to a compound annual growth rate by treaty between nation states to work in good faith on (CAGR) of 50.1% over the 2016-2021 period which is driven issues of common interest. They are vehicles to solve 13,14 by aggressive investments across industries.1 Moreover, coordination problems on a global level. the Economist estimates that companies spent around $22 IGOs possess adequate organizational structure and billion on AI-related mergers and acquisitions in 2017, about autonomy to administer multilateral mechanisms of 2 26 times more than in 2015. Patents in the field of Machine decision-making between states, and the capacity to learning (ML), a core subset of AI, grew at a 34% CAGR execute the collective will of their member states.15 They 3 between 2013 and 2017. have their own international legal personalities and may enter into enforceable agreements among themselves or 16 Public sector with states. This differentiates them form simple groupings or coalitions of states, such as the G8, which lack a Likewise, federal, state and local governments are constituent document or exist only as task groups. increasingly applying AI to public sector service provision. For example, the Singaporean government utilizes AI-based The United Nations (UN), the Universal Postal Union, and chatbot services to resolve queries from the public.4 The the North Atlantic Treaty Organization (NATO) are among Mexican government has piloted a similar initiative to the oldest IGOs. Other well-known IGOs include the categorize citizen petitions and route them to the European Union (EU), the Organisation for Economic Co- appropriate office.5 In the UK, the Department for Work and operation and Development (OECD), the Association of Pensions detects large-scale benefits fraud by means of AI Southeast Asian Nations (ASEAN), the African that identifies patterns in writing style,6 and the technology Development Bank (ADB), and Related Organizations of the was used at the 2018 PyeongChang Winter Olympics to UN, i.e. the International Atomic Energy Agency (IAEA), the provide real-time translation services.7 International Organization for Migration (IOM), and the World Trade Organization (WTO). A study revealed that documentation and administration- related tasks consume 37.5% of Colorado Department of Since the founding of the UN and NATO, IGOs have Human Services workers’ hours, compared to just 9% spent become indispensable actors on the international stage. on contact with children and families.8 In this example, AI Since many IGOs are able to establish binding rules and automation of administrative procedures could free up time exert power in member states, their global impact is for great value add face-to-face service provision for more significant. citizens. It is estimated that the automation of thousands of IGOs significantly vary regarding the global issues they routine federal government tasks could save at least 96.7 address, which include: million federal hours (out of 4.3 billion yearly hours worked) in the US alone, resulting in potential annual savings of • International relations, such as promoting peace, $3.3 billion.9 The US Federal Government’s investment in protecting transnational security, and preventing armed unclassified R&D for AI has increased by more than 40% conflict (e.g. following the devastation of World War II, since 2015, in addition to considerable classified the UN was established in 1945 with the central mission 17 investments across the defense and intelligence to maintain international peace and security) communities.10 The President’s Management Agenda calls • Society, such as increasing human development, for applying automation software to enhance government maintaining public health, and protecting justice (e.g. the services efficiency and expanding federal data sharing with World Bank Group pursues to end extreme poverty the public to bolster non-federal AI research.11 within a generation and boost shared prosperity) 18

2 • World economy, such as fostering global monetary can help to promote cooperative behavior. IGOs also serve cooperation, facilitating international trade, and valuable purposes for individual states, which frequently preventing and mitigating financial crisis (e.g. through employ them as instruments of foreign policy to legitimate surveillance, technical assistance, and lending, the their actions and interest and to constrain or sanction the International Monetary Fund aims at strengthening the behavior of other states. international monetary and financial system)19 While the daily operations of most IGOs are conducted by • Environment, such as combatting climate change, staff, ultimate authority resides with member states. preventing pollution, and improving of production of Moreover, IGOs have to account to member states and to agricultural production (e.g. the IPCC provides an increasing number of private actors as financial policymakers with objective, scientific assessments of contributors about their spending of funds. These donors climate change, its political and economic impacts, and have a substantial interest in the best possible allocation of 20 options for mitigation) resources provided. They expect high degrees of Moreover, they serve many diverse functions, including organizational effectiveness and efficiency (i.e. strategic • Collecting and preparing information and monitoring of and operational procedures, relationship and knowledge 21 trends (e.g. the World Meteorological Organization) management), which makes leveraging AI imperative for IGOs. • Delivering services, aid and humanitarian relief (e.g. the World Health Organization) Even though IGOs have very different missions and goals, • Providing forums and platforms for bargaining (e.g. the many of the core processes are similar – irrespective of European Union) and settling disputes (e.g. the World their specific institutional design choices and scopes. This Trade Organization) allows to identify potential use cases across the entire IGO value creation process systematically, i.e. from strategy As they constitute political institutions through which states formulation via project delivery through to evaluation. A can work together to accomplish common objectives, IGOs schematic illustration is provided in Figure 1 22,23.

Figure 1: The IGO value chain: Core and supporting / cross-divisional activities

III. A short introduction to AI There are numerous of ways to simulate human AI, as a broad discipline that contains many diverse intelligence, and some methods are more sophisticated subfields, is not a new concept. Modern AI found its origin in than others. While rule-based expert systems as precedents a conference held at Dartmouth College in 195624, where for modern-day AI have been around for decades, much of John McCarthy coined the term "Artificial Intelligence". the recent progress is a result of improved algorithm design, Widely regarded as the godfather of AI, McCarthy defines the ability to capture and store massive datasets, and the it as the "science and engineering of making intelligent sufficiently powerful computing hardware to process them. 25 machines, especially intelligent computer programs“ . One aspect that separates machine learning (ML), as a However, the definitional boundaries of AI remained subset or method of AI, is its ability to improve when contested,26 and the confusion around the term is exposed to more data. ML is traditionally focused on compounded by the fact that media and vendors tend to predicting and creating structure out of unstructured or label many technologies as AI and portray it with features structured (labelled) data by processing data and including sentience while researchers commonly take a experiences, rather than by obtaining explicit programming purer view of the definition. instruction

3 To simplify, ML algorithms are commonly divided into three Reinforcement learning broad categories: supervised learning, unsupervised Reinforcement learning refers to goal-oriented algorithms learning, and reinforcement learning.27 which learn to perform a task or attain a complex objective (goal) over many steps by maximizing rewards it receives for its actions, e.g. the goal might be to make a robot travel Supervised learning from point A to point B, and every step towards point B Supervised learning refers to algorithms learning to classify could be assigned a certain reward. data based on other examples of labelled pairs. There are numerous models which enable predictions, such as A reinforcement learning algorithm assesses actions based classifying images or individuals into groups. Supervised on the results they produce. It is goal oriented and aims to learning generally builds on three components: optimize for the best series of actions to achieve its goal, • A set of data to learn from. “Supervised” refers to the fact i.e. to maximize its objective function, by correcting itself 30 that this data is labeled with the correct categories over time. regarding the characteristic of interest (e.g. how the given inputs “time of year” and “number of droughts” relate to Deep learning the given output “grain prices”) Deep Learning (DL) is a subset of ML that is able to process • A ML model that determines how the data is assessed, a wider range of data sources, requires less data with parameters to fine-tune the model to render preprocessing by humans, and may yield more accurate predictions as accurate as possible results than conventional ML approaches. While • The cost function to calculate the error, or how far the supervised, unsupervised, and reinforcement learning are algorithm deviates from optimal performance overarching methods, deep learning constitutes an architecture that can implement those methods. DL employs The goal of supervised learning is to adjust the parameters layers of nodes that in some sense mirror the construct of to find the configuration that minimizes overall error, which human neural networks. Each layer of neurons ingests data allows the model to reveal the patterns in the data that are from the layer below it, performs a calculation, and provides essential for prediction. Once the model has determined the its output to the layer above it. It is especially useful for best parameter configuration, it can use this configuration to capturing hierarchical meaning as it can combine an make predictions on new data sets.28 unsupervised process to extract the features of the underlying data (e.g. the edge of a face) and then pass on Unsupervised Learning that information to a supervised learning algorithm to Unsupervised learning algorithms explore input data without identify features as well as the final result (correctly being given an explicit output variable of interest about identifying the person in the picture). As perhaps the most which to make a prediction or assessment. While promising area of ML today, DL is applied in a wide supervised learning algorithms benefit from the guidance spectrum of use cases from understanding molecular provided by the training data and real-time feedback, interactions inside human cells to enhancing computer unsupervised learning algorithms are fed large amounts of vison and natural language processing.31 data to identify key structures, or patterns, within them. These methods allow to design special-purpose agents to One of the most common applications of unsupervised perform useful cognitive tasks, in some cases even learning methods, e.g. k-means clustering or hierarchical surpassing the capabilities of their human counterparts. On clustering, is to develop a better understanding of data a meta-level, their capabilities can be grouped around structures to design improved supervised learning sensing and recognition, knowledge processing, and algorithms. For example, unsupervised learning may be decision making and support. These increasingly intelligent utilized to aggregate the pixels of a picture into a small technologies can be employed on a wide range of purposes number of distinctive recognizable features, such as the from the recognition of images over pattern detection and structures of eyes, which then serve as an input for a trend prediction, to automation of tasks and provision of 29 supervised learning facial recognition algorithm. intelligence to autonomous robotic systems, as shown in Figure 2.

Figure 2: The AI technology cluster The basis to understand how IGOs should get started with Predictive Analytics AI is to frame it within the context of analytics, as shown in Standard data analytics consequently leads to predictive Figure 3.32 analytics that harnesses pre-collected, historical data to determine predictions on what might happen. Human This report generally references AI to enclose advanced oversight is required to query data, validate correlations, analytics techniques, which refer to the process of and create and subsequently test assumptions. interpretation of data that involves classical statistical Assumptions derived from past experiences and events methods, as well as more sophisticated, machine-driven imply that the future will follow the same patterns. In techniques (usually beyond the capabilities of traditional addition, predictive capability is restricted by the number, business intelligence), to identify patterns, correlations and processing speed, time and cost constraints of human data groupings in data sets, make predictions, or generate analysts. recommendations.33 IGO decision makers are well advised “to focus on what is most important for the[ir] organizations: predictions and better decision support”34, as stated during AI-based prescriptive and cognitive analytics the 29th Meeting of the UN’s Chief Executives Board for Coordination ICT Network in Fall 2017. AI is a logical continuation of the concepts centered around predictive analytics, however, with one crucial difference: AI This transition from data analytics to AI-based analysis systems are capable of making assumptions, testing and regularly occurs through the application of ML to build reassessing the model, re-evaluating the data, and learning predictive models from large sets of structured data. As independently. This implies that human engineers are not organizations get increasingly acquainted with this required to program for every possible situation. Complex, approach, they start creating higher levels of abstraction multi-tiered analyses with various variables and interactions and exploit further, unstructured data sources that open the involved can be performed instantaneously, allowing the door for DL. The next progression after deploying predictive system to learn expeditiously. models is to build prescriptive models that merge the prediction with an IGO process to recommend a single Organizations that possess sophisticated business course or set of actions.35 intelligence systems that allow data storage, analytics, and governance, combined with the ability to manage structured and unstructured data, and visualization techniques, have Conventional descriptive and diagnostic data analytics the tools at hand to devise a AI strategy to implement AI. Standard data analytics aims at aggregating and Investment in big data architecture (with big data defined as summarizing data to report a result and find relationships complex sets of data that traditional data-processing between variables. Assumptions are defined by humans, applications are inadequate to deal with due to the extreme and data is retrieved to affirm these hypothesized volume, wide variety of data types, and required velocity to relationships. If valid, testing may be extended on additional process data, i.e. the 3 ‘V’s36) is crucial, particularly when data. Since it is based on past events, conventional data combining unstructured data with legacy structured data analytics is descriptive and retrospective. It does not predict sources. the effect of a change in one or more variables.

Figure 3: The evolution from retrospective (what happened) data analytics to AI-based, forward looking analytics (what will happen)

5 IV. Value-creation potential of AI in IGOs Press, are already using AI to mine data, generate text IGOs have started to explore the use of AI across numerous blocks for datasets, and draft documents at a pace of 40 domains, with most pilots focusing on the support of 2,000 stories per second ) operations and project delivery (e.g. UN Global Pulse, the flagship innovation initiative of the UN Secretary-General on big data, provides a repository of its collaborative research Higher prediction accuracy and enhanced project projects, prototypes and other experiments online.37) effectiveness With ability to process vast amounts of data, AI systems One way of classifying the spectrum of available AI may reach more accurate predictions and help IGOs track solutions for IGOs is to think of them as automating and the efficacy of policy interventions and advocate course augmenting. The consequent automation of routine, corrections when needed. predictable and rule-based tasks can enhance overall • A Kansas (USA) municipality employs AI to protect its productivity and performance. Automated systems are able most vulnerable populations. By consolidating and to come to decisions instantaneously, consistently, and with analyzing data from different county departments, the AI negligible error rates. Contrarily, other activities are application concludes who is in jeopardy of being currently not (and perhaps will never be) eligible for arrested and incarcerated. From this data pool, AI automation, e.g. highly ad hoc or unforeseeable situations identifies risk-prone residents who are not using where real-time human analysis and judgment are available assistance services, so that the county can essential. These tasks should be considered as suitable for support them in ways that could save them from a prison augmentation, i.e. division of labor between AI and human sentence. On average, the system’s prediction is 25% experts, or AI assistance in data analysis. more accurate than if human case workers in the field The following exemplary case studies of AI in governmental had used the standard procedure of assessing one or agencies demonstrate how IGOs may employ the two features to identify at-risk people41 technology to solve practical issues in delivering on their . mission and realize substantial value across four main Figure 4: Two dimensions of value creation: categories of value creation. A non-exhaustive list of further Automation and Augmentation AI based use cases in IGOs from a functional point of view is provided in the appendix. The figure below illustrates the value creation potential of AI through augmentation (y-axis) and automation (x-axis). Well-designed AI systems (such Higher efficiency and lower cost as the case study high lighted in yellow) leverage AI systems will allow IGOs to replicate human actions and both. judgment to address common pain points like resource constraints, paperwork burdens, and backlogs. • US Citizenship and Immigration Services (USCIS) launched a chatbot named “Emma” in 2015 to administer a large number of customer queries (USCIS receives approx. 14 million calls on immigration issues per year). Emma has been devised to assist visitors in navigating the USCIS website and finding relevant information quickly. The agent is able to provide short answers and links to contents by sifting through the USCIS website. It can provide written answers in English and Spanish and communicate in English38

Increased process speed AI applications may permit IGOs to decrease processing time of transactions by shifting larger fractions of a task to machines. • Japan’s Ministry for Economy, Trade, and Industry is piloting an application to help offices of members of parliament respond to citizen inquiries faster by formulating answers using AI39 (dozens of news agencies worldwide, including Bloomberg and the Associated

6 Figure 5: Case study – Preventing foodborne illness by mining social media data42 The Southern Nevada Health District (SNHD) conducts more than 35,000 food inspections on nearly 16,000 facilities each year. To improve its effectiveness, SNHD is leveraging AI applications. In an experimental trial in Las Vegas, half of the city’s food inspections were assigned randomly; the other half relies on the AI application. • Data collection: AI system automatically scans a daily average of 16,000 tweets from 3,600 users in Las Vegas in real time; human involvement: 0% • Geo-tagging: Each tweet within 50 meters of a food venue is automatically assigned to the closest restaurant based on the Google Places API. This procedure narrows down to 1,000 tweets from 600 users; human involvement: 0% • Analyzing tweets: These 600 users are tracked for the next five days. Their tweets are analyzed and scored using machine learning (with about 12 a day mentioning food poisoning). During the training phase, 8,000 tweets scanned by human observers were fed into the model; human involvement: 10–15% • Ranking of restaurants: A ranking is determined based on the number of tweets with sickness scores exceeding a certain threshold. High-priority locations are identified and adaptive inspections are assigned; human involvement: 25– 30%

Results: Adaptive inspections significantly outperformed random inspections: they uncovered significantly more demerits (nine versus six per random inspection, on avg.) and resulted in citations in 15% of inspections (compared with 9% in the randomized selection). If every inspection was adaptive, it would result in 9,000 fewer food poisoning incidents and 557 fewer hospitalizations in the city each year

V. Implementation framework for AI in IGOs Like any institutional change process, this affects various AI allows for improved insight generation, faster and larger- aspects of organizational behavior. Interviews conducted scale evidence-based decision making, and optimization of revealed that most IGOs are not adequately prepared for output delivery processes. The majority of IGOs studied as the substantial investments in capacity building and cultural part of this report has already deployed some form of AI change that are required to achieve sustained success with pilot. However, without organization-wide implementation, AI, which include re- and upskilling of the workforce, IGOs will only capture a fraction of the realizable value. expanding the nature of decisions influenced by data, and cultivating judgment that blends AI insights with intuition. Achieving true advantage with AI requires determination Therefore, a multi-step roadmap for an effective AI and a sustained commitment to advancing the role of data. transformation is proposed (see Figure 5).

Figure 6: The AI implementation framework

5. Implement the AI solution from prototype to full rollout

4. Identify and prioritize use cases: Impact and feasibility assessment / Return on AI (ROAI)

3. Prepare the cultural transition

2. Lay the foundations: Data / Team, skills, and tools

Gradual learning learning Gradual and and improvement 1. Develop a strategic intent for AI, ensure senior executives’ support, and appoint a sponsor

1. Develop a strategic intent for AI, ensure senior Moreover, a clear allocation of responsibility and executives’ support, and appoint a sponsor accountability is recommended. To lead an organization Without a comprehensive and coherent strategy for from experimenting with AI to systematically leveraging it to advancing the use of AI, the anticipated results from data- achieve strategic ends, IGOs may appoint or externally hire driven insights are likely to be elusive. For example, this a designated person in charge for the AI transformation. An could take the form of a strategic vision and three-year ‘IGO translator’, who combines a data science background roadmap. with a solid understanding of IGO procedures and processes, should promote cultural awareness and Ideally, this roadmap should include timing for investments adoption. As an internal champion, the executive is also in in data infrastructure for AI, i.e. data storage and charge of achieving and communicating early wins to gain governance, big data technology, business intelligence and organizational buy-in. visualization tools.43

7 2.1 Lay the foundations – Data Data inventory as prerequisite for robust analytics One of the biggest impediments to the successful The Bank of England and the City of Amsterdam both started off to strengthen their analytics foundations by mapping their realization of AI projects is the fragmented nature of data in respective organizations’ data sets. Through this admittedly 44 ‘data silos’ which render it prohibitively costly to extract arduous task, almost a thousand data sets were identified at the data. Silos can arise for various reasons: UK’s central bank and 12,000 among city departments in the • Structural: Software applications are regularly Dutch capital, where executives remarked that “inventory sounds developed and customized for a particular sub- quite boring, but it’s fundamental. We need to know what we’ve 48 organization or division of the IGO at a designated point got to know how to manage it.” in time. Applications are optimized for their main function and data sharing is not a primary requirement 2.2 Lay the foundations – Org, skills, and tools • Legal or political: Groups within an organization might not be permitted by law or due to political reasons to Establishing AI systems requires a clear sense of how the share data with other entities. Information barriers may initiative is going to be introduced, sustained, and scaled up have been erected deliberately to prevent exchanges or in terms of team structure. Two organizational structures communication that could lead to conflicts of interest are proposed, and IGOs may gradually evolve from one into the other as they embark on the AI journey. • Legacy: Most IGOs have undergone various generations of leadership, philosophies, and expansions or changes Center of excellence approach of scope, which might have resulted in semi- or Many organizations of both public and private sectors, as incompatible systems. Data is also not a neutral entity, well as the UN with its innovation initiative Global Pulse, but has to be interpreted in light of its historical origin start with a centralized center of excellence for AI (and data • Vendor lock-in: Software vendors’ ‘customer retention’ science) to build the core technical infrastructure.49 strategies may hamper attempts to export data contained This approach is especially viable for IGOs that see in applications. themselves confronted with resource constraints that make hiring data scientists for each sub-organization prohibitively Importance of data-driven decision making in IGOs expensive. Essential tasks, i.e. data preparation (pre- Speaking at the launch of the Centre for Humanitarian Data in processing raw data from one or more (disparate data) The Hague in December 2017, UN Secretary-General António sources into refined, higher-quality information assets), Guterres highlighted the increasing interest placed on data- model training, creation of user interfaces, and model driven systems for disaster response. The meeting showcased deployment within the IGO’s IT infrastructure are managed the role data-informed decision-making plays in approaching by one central unit50 in coordination with a designated social development issues and crisis response. “Accurate data is person in charge in the sub-unit. The center may also act as the lifeblood of good policy and decision-making,” Mr. Guterres explained. “Obtaining it, and sharing it across hundreds of an incubator or education platform for data science organizations, in the middle of a humanitarian emergency, is professionals to be deployed in sub-organizations later. complicated and time-consuming, but it is absolutely crucial.”45 Central coordination of AI activities at US DoD UN Assistant Secretary-General and Chief Information The United States Department of Defense (DoD) launched a new Technology Officer, Atefeh Riazi stated: “Effective use of data Joint Artificial Intelligence Center (JAIC) in June 2018 that will can help us expand our knowledge of fundamental issues, such oversee and coordinate nearly all service and defense agency AI as social and political instability, natural disasters and climate efforts. Aiming at establishing a common set of AI “standard […] change, to improve decision-making and enhance our ability to tools, shared data, reusable technology, processes, and predict, and even pre-empt, the next crisis.”46 expertise”, it is responsible for an initial list of National Mission Initiatives that exceed a certain annual project volume. It will report to Chief Information Officer and cooperate with other To enable higher value use cases, IGOs need to overcome government agencies, “industry, academics, and U.S. allies.”51 existing data silos – and identify ways to transfer latent "Hub and spoke" approach information into digitally useful data. Data management Under this structure, embedded groups of data scientists needs to be accompanied by effective data governance are located next to each major sub-organization to identify policies and practices. These define the type of data the specific data science problem, develop, optimize and available as well as access and utilization rights. Data retain specialized models, and maximize adoption of policies, standards and definitions must be designed in a solutions. A data science platform facilitates technical way to support an effective analytics and AI strategy while knowledge distribution, enforces best practice sharing, and complying with regulatory or legal requirements, privacy, provides transparency regarding the various organization- security and other considerations. wide initiatives. This approach benefits from the proximity to While some IGOs have already created the preconditions sub-organizations in both ideation, delivery and monitoring for an AI transformation (e.g. the UN by rolling out an phases of AI solutions.52 organization-wide ERP system47), for others it is less feasible to establish a suitable infrastructure from scratch. These approaches mark the end points of a continuum of These IGOs must tread an incremental path towards an possible organizational design choices, as indicated in integrated data platform. Figure 6.

8 Figure 7: Organizational design choices to deploy AI in IGOs

With both approaches, IGOs may choose between interfaces. Complex data science tasks that encompass machine learning as a service (MLaaS) or in-house research, application of multiple ML models tailored to development of AI capabilities. various facets of decision-making, or various ML-backed services are adequately addressed by this specialized Machine learning as a service (MLaaS) structure model. MLaaS is an umbrella term for platforms that provide ML tools and various infrastructure services such as pre- Pros of in-house solution: processing of data, model training, and model evaluation • Centralized data science management and oversight as part of cloud computing services. The respective • Establishing a full-featured data science environment provider's data centers handle the actual computation; with greater scalability potential results obtained can be can transferred to internal IGO IT • Expanded problem-solving and prediction competences infrastructure through APIs.53 AutoML, the newest innovation in the MLaaS space, simplifies the application of Cons of in-house solution: ML to real-world problems in IGOs even further by • Erect and maintain a complex computational automating the end-to-end process from manual data pre- infrastructure processing via feature engineering, extraction, to algorithm • Heavy time and resource investments into data science 54 selection and hyperparameter optimization for algorithms. talent acquisition required; data science talent Pros of MLaaS approach: engagement and retention challenges • Provider takes responsibility for management and Talent acquisition and retention storage of data, no on-site computing infrastructure In exceedingly competitive fields like analytics and data required science, IGOs need to provide competitive salaries and • Shortened implementation time for comparatively simple benefits. ML engineers, data scientists, and big data ML tasks requiring one or a few models engineers rank among the top emerging jobs on LinkedIn;55 Cons of MLaaS approach: average salaries vary considerably across different publications, a comprehensive 04/2018 study reports • Provides a limited number of ML methods and data median base salaries for data scientists ranging from cleaning procedures only $95,000 for young professionals with 0-3 years of • Data isn't stored locally, which might result in hesitations experience to $165,000 for experienced hires with 9+ years around data safety of experience (non-managers) and base salaries up to 36% • While most MLaaS platforms allow developers to higher than other predictive analytics professionals).56 evaluate before committing to a platform on a limited trial Moreover, they should offer a well-defined, structured basis, model training, testing, and prediction are often career path and engage data scientists with creative, billed separately later. This results in uncertainty of explorative projects and cross-training. eventual cost per prediction as the number of necessary iterations can’t be accurately estimated in advance Even if experienced data scientists cannot be hired at the very beginning, IGOs may circumvent this barrier by In-house solution establishing partnerships with universities and other To reduce management effort and create a (online) educational institutions (massive open online comprehensive, full scale ML framework, IGOs may courses, sometimes offered by renowned institutions, allow perform the entire ML workflow within in-house AI / data for efficient, affordable acquisition of knowledge). IGOs science departments. They may deploy more sophisticated should, however, not overlook the significance of process ML tools and libraries designed for researchers and engineers, IT infrastructure developers, and project experts with data science backgrounds, e.g. Tensor Flow managers. These functions and skills are essential to build or Theano. This approach is the costliest, as a dedicated robust, sustainable AI solutions. If these skills are lacking data science team undertakes all operations, from data or disregarded, it will be even more challenging to quickly preparation to model training and creating front-end deploy AI solutions across the organization.

9 3. Prepare the cultural transition have revealed that AI can reflect the very prejudices Besides investment in financial, human, and technological humans aim to overcome as a result of the way AI is 59 resources, sustained success with AI necessitates a trained, a phenomenon known as algorithmic bias. profound cultural change. The ultimate success of the There are several ways to approach "de-biasing“60 AI which transition from pilots to organization-wide deployment include the utilization of interpretable ML models, the right depends on the IGO’s ability to reorganize core processes to explanation as found in the EU's General Data around the new technology. Moreover, cultivating an AI Protection Regulation GDPR, and further education about strategy and linking it with the IGO’s overall strategy the issue. These measures may lead to improved selection requires resolve to fundamentally elevate the role of data in of training data and model tuning to overcome imbalance decision making. This type of transformation is more problems, where the bias results from under- or feasible in cultures that promote experimentation, learning overrepresentation of one class in the training data.61 agility and design thinking. Another concern stems from the lack of transparency about IGO leaders (through their designated internal champion) how some of these systems reach their end result. In are responsible for educating both their management teams general, a so-called "black-box"62 AI is a ML system in and workforce at large that AI is not only a business reality which the input and output data are classifiable and and increasingly prevalent across governments worldwide, understood, but the manner in which it processes the data but that both strategic and operational day-to-day work of is unclear. The suitability of black-box AI algorithms hinges IGOs will intertwine with these technologies at ever heavily on the use case to which they are applied, and IGO increasing pace. executives may decide not to use this type of AI until it is Leaders should acknowledge and proactively communicate subject to certain accountability standards. that AI will imply a reduction of repetitive and transactional tasks, be honest and open about the substitution potential of AI for the workforce, and simultaneously help employees 4. Identify and prioritize use cases across the comprehend the multiple ways in which AI will amplify their organization roles and make many jobs easier to carry out. As a first step, it is essential to get an overview of all current and previous AI initiatives, pilots and experiences Responsible IGO executives also demonstrate that they are throughout the organization. This should lead to a solid willing to commit the necessary resources to re- and upskill baseline that includes use cases, proof of concepts, studies, their workforce in new technologies. Recent studies external cooperation partners, data bases, etc. Then, a demonstrate that current AI technologies are able to thorough evaluation should be conducted. Instead execute literacy and numeracy tasks at or above the anecdotes or pure 'intuitions’, these assessments should 57 proficiency of 89% of adults in OECD countries , which include hard numbers, such as cost savings and improved suggests that IGO executives should consider preparing outcomes, (e.g. lower error rates or decreased processing their workforce in more sophisticated areas such as times) in absolute and relative terms. problem solving, data and statistical literacy, abstract and computational thinking, and digital technology. Based on this internal stock take, IGOs should systematically evaluate needs and capabilities to develop a Education to re- and upskill the population at large prioritized portfolio of AI initiatives. In doing so, decision In January 2016, the Government of Singapore launched the makers should evaluate two dimensions: impact, i.e. the SkillsFuture initiative, an integrated system of education and expected benefit from implementing AI, and feasibility, training to provide citizens with opportunities to acquire greater meaning the expected likelihood of success of these skills proficiency and expertise in areas of expected employment measures. growth. To allow Singaporeans to take time out of full employment, a credit for workforce retraining may be used by everyone aged over 25, with additional training subsidies available 4.1 Impact and feasibility assessment to those aged over 40.58 Decomposition/systematic breakdown of processes Mitigating new types of risks that algorithms pose Frequently, there are areas for which ‘knowledge’ (i.e. crucial insights derived from data analysis) is highly coveted IGOs should train their staff how to deal with algorithms but not available under the status quo due to one or a responsibly, i.e. understanding the difference between combination of the following reasons: correlation and causation, false positives and false negatives etc. This also includes a basic training in what • Sub-optimal distribution: Bottlenecks in the flow of data to collect, how to store it, estimating the value of data information cause scarcity of data-driven insights, e.g. in itself, and how to assess data set completeness, reliability departments, practices, or regional subsidiaries of the and accuracy. IGO • Problematic knowledge extraction: Knowledge exists, IGO executives have to ensure that users of AI are aware of but the exploitation process is inefficient, time-consuming potential biases associated with the technology. Studies or disproportionately expensive to scale • Inadequate processing capabilities: collection of data IGO leaders must determine the value potential from an is greater than existing human or computer resources organization-wide implementation . While a sophisticated can process and evaluate. IGOs often produce statistical demand forecasting model deployed in one specialized data and time series data for indicators for numerous agency may deliver substantial value on its own, it may countries and country groups. However, some IGOs even positively affect strategic planning accuracy in a sister seem unaware about how these ‘data treasure chests’ organization. may be analyzed and strategically applied Integrating different datasets into one consolidated solution, recommendation, or series of actions will have a more Benefits of a common, defined data structure profound impact than if data remains in siloes. Based on The United Nations‘ Enterprise Resource Planning (ERP) these criteria, use cases may be prioritized according to system ‘Umoja’ provides an integrated suite of applications their short- and long-term value. that support core functions such as finance and budget Feasibility management, HR management, supply chain management, and central support services. It represents a joint effort of the The feasibility and speed of implementation of AI solutions UN’s Procurement Division, Office of Legal Affairs, the Umoja depend on the following factors: team, the Office of Information and Communications Technology, and others. External partners include the world’s • Model complexity and algorithm’s performance leading provider of ERP software and two professional (performance requirements sufficiency varies according services firms that assisted in initial design and rollout. to the pursued objective and cost of wrong prediction) Umoja aims at enabling “high-quality, cost-effective service • Quality and quantity of the training data delivery to support the execution of evolving mandates […] through re-designed and common processes for the • Organizational buy-in from leadership at the top of the management of all financial, human and physical organization and the department resources.”63 As a centerpiece of then Secretary-General Ban Ki-moon’s endeavors to “transform the United Nations • Costs of designing, setting up, running and maintaining into a more modern, flexible, and global organization”64 it “will the AI solution replace hundreds of disconnected legacy software • Speed and timelines of implementation applications on which the organization currently relies.”65 Despite receiving strongly positive feedback on the system the UN acknowledges that its implementation “creates disruption, and it takes time to stabilize the system.”66 Also, 4.2 Return on AI (ROAI) there has been reported a “shortfall in the number of IGO decision makers can map each alternative according computer troubleshooters and trainers”67 required to onboard approx. 32,000 UN staff members in more than 400 locations. to its assigned score on an impact/feasibility matrix and start with “low hanging fruits” i.e. high feasibility-high impact, then consider others. Value-add through AI In addition, while most organizations have rigorous ‘Return Next, IGO decision makers should identify the use cases in on Investment’ (ROI) criteria to evaluate financing which AI applications are expected to yield substantial decisions, interviews with IGO decision makers revealed value and contribute to increased effectiveness and that no such metric yet exists for dealing with AI projects. efficiency in the achievement of IGO objectives. This Therefore, a ‘Return on AI‘ (ROAI) is proposed: involves assessing the impact on the high-level strategic objective vs. subordinate program or agency-specific goals.

Figure 8: Return on Artificial Intelligence (ROAI)

(Benefits of AI – Cost of AI) ROAI = > 0 Cost of AI

Benefits of AI Cost of AI: All project costs incurred by multi-level AI • Full value proposition as portrayed earlier: Shorter infrastructure delivery times, lower FTE requirements, lower error • Time & Expenses for internal or external team rate, higher forecasting/prediction accuracy, etc. • Potential data acquisition or digitalization costs • Should reflect a conservative estimate of how much • Computing resources and licensing costs for HW/SW improvement IGO may expect through AI • Training/adjustment time for stakeholders 5. Implementation: From prototype to full rollout to develop detailed plans for scaling up. This process Analysis phase depends on close collaboration of technology experts and The actual AI project delivery starts with analyzing the current owners of the processes being automated. current processes, data assets and goals/targets. The project team defines success factors (expected metrics improvements), a suitable technology portfolio, timeline as VI. Concluding remarks well as budget. All these parameters are recorded in a This report aims at demonstrating how IGOs may leverage corresponding documentation. At the end of this phase, the AI strategically to deliver high-quality, efficient, and cost- team defines the scope of work required for the next step: effective services to support their comprehensive agendas. prototype development. It establishes a link between ongoing efforts to transform IGOs into data-driven organizations and nascent Prototype development, implementation and evaluation endeavors to build up comprehensive AI platforms in Next, an AI-based prototype is developed and tested, IGOs. This allows IGOs to adequately respond to the fact which can take the form of a simple mock-up or a more that major challenges have shifted from collecting massive 68 elaborate code-based prototype. Simple prototypes range amounts of data to understanding and actively leveraging from Excel spreadsheets to R or python code. Prototypes it, i.e. turning data into knowledge, conclusions, and should be produced quickly and inexpensively. They are tangible actions. AI represents an organizational need that flexible to adjust and their degree of sophistication depends should no longer be considered as facultative, but rather a on the project complexity and spectrum of tools available. means to implement effective organizational responses. Prototyping permits to validate requirements and design Interview partners in IGOs seemed well aware of the choices, which reduces the risks and costs associated with challenges to overcome when mainstreaming AI AI implementations dramatically, as specifications and technologies throughout the organization, i.e. adequate functionality can be examined early on (an example of the executive support, necessary willingness and budget to one and a half year process from prototyping to a fully scale promising pilots and compliance of AI innovations operational ML based radio content analysis tool with organizational and donor policies. developed by UN Global Pulse Lab Kampala and partners can be found here69). In addition, by building up internal AI capabilities, IGOs lay the groundwork to spark the debate on AI and related new The need to fundamentally redesign IGO processes and technologies in all member states. Increasing AI literacy account for inevitable changes in the division of labor across the organization allows them to assist member between humans and AI agents is frequently overlooked in states in addressing the most critical threats posed by the the development and implementation of such projects. malicious use of AI to international peace and security. Systematic remodeling (and potentially even first-time formalization) of workflows is required to when automating As advanced, disruptive technologies are usually not existing de-facto practices. designed or developed by nation states, it is mainly the tech sector and research communities that set the pace in establishing the basis for formal or self-regulation. This Minimum viable product (MVP) trend is particularly prone with regards to AI technologies, The MVP is a product incorporating a set of basic witness the Asilomar principles, IEEE standards, or ethical functional features developed in consideration of the AI principles recently published by numerous multinational findings from the prototype. A MVP builds on the IGO’s technology companies, among other voluntary guidelines.70 actual data and is exposed to a group of selected internal IGOs (and governments) should not relinquish their (or external) test users as the 'minimal' initial version of the decisional power on key ethical decisions with far-reaching 'ultimate' later product solution. Their feedback is highly implications for humanity to a small number of private relevant, as it is significantly cheaper to modify and entities. Instead, they should cultivate a more augment the system at this level, than when it is completely sophisticated, deepened engagement with the private developed. For larger projects, IGOs may want to test sector. The variety of existing and upcoming normative, extended MVPs with full user experience (UX) as an regulatory and cooperation processes might even prove as intermediate step before the final rollout. an advantage, as they provide established platforms to foster public understanding and formulate best practices on AI (e.g. the UNDP and UNICEF joined a consortium of Product release and scale-up companies, academics, and NGOs focused on ensuring At a last step, the solution with its complete set of that AI is developed in a safe, ethical, and transparent predefined features is developed and subsequently manner71). launched. As the prior steps emphasized requirements These measures allow IGO executives to make informed elicitation and validation, the end product is finished with decisions about where and how to deploy AI in their minimal risk, and the cost of this phase can usually be organizations, realize substantial benefits, deepen the assessed accurately during previous stages. public understanding of the effects of AI on the broader To achieve adoption within the respective sub-organization society, and assist their stakeholders in preparing for a or branch, transfer to sister organizations, and eventually, future of AI. adoption across the whole parent organization, IGOs need

12 Appendix

AI in IGOs maturity framework e.g. cloud computing (defined as the provision of on- The maturity framework for AI in IGOs outlines a series of demand computing resources – ranging from servers, evolutionary levels that pass through during their AI storage, and databases to software, analytics, and more – 2 transformation (Figure 1).1 over the internet on a pay-for-use basis ) are well underway. An IGO-wide strategic plan is developed to direct Level 1: Unaware or risk averse investment efforts as work processes are re-thought and re- Due to a lack of awareness about AI systems and/or risk shaped. Staff is called upon to share their expertise; key aversion against the introduction of emerging technologies, lessons learned from pilots and exploratory projects are IGOs on this stage have not yet begun to explore the evaluated centrally and will be used to shape initial system potential of AI. Most organizations on this stage spend a deployments. External partners and academia are substantial amount of their IT budget in an attempt to keep consulted regarding system design, deployment, and their present systems relevant. However, the obsolescence evaluation procedures. rate of legacy information and communications technology (ICT) systems presents them with substantial challenges. A Level 4: AI harnessed at scale whole series of structured tasks is still carried out manually Motivated by first successes from the initial system by their workforce, often resulting in protracted processing deployment efforts, IGOs on this stage are eager to deploy times and significant backlogs. A potential lack of strategic AI systems to scale. Senior IGO executives and a foresight and direction may hinder the proliferation of designated internal champion take responsibility that the AI technology enabled innovation. transformation is given strategic attention and progress on AI projects is tracked regularly. AI is increasingly embedded Level 2: Aware and resourceful into work practices and administrative processes to IGOs on this level appreciate the potential of AI and senior unburden staff and provide them with unprecedented (IT-) leaders have initiated conversations on the role that AI insights. An established collaborative network of internal may resume within the overall organization. A first experts, industry, and academia empowers the IGO to audit assessment of key strategic areas and core processes and update an organization-wide portfolio of AI systems. where AI can be most beneficial has been conducted. Initial pilots and initiatives are deployed to gain first-hand Level 5: The AIGO experiences with AI systems. IGOs actively reach out to IGOs on this level command a comprehensive AI platform external partners to collaborate with on further exploratory which is considered as a strategic asset. AI features are initiatives and start to overhaul the IT infrastructure and embedded in almost all facets to achieve mission revise their data governance mechanisms to gear up for objectives. Both leaders and staff acknowledge the value further AI adoption. provided by AI, actively question existing processes and deliberately involve AI in potential solution strategies. The Level 3: Fully developed strategic plan performance of systems is continuously tracked to identify Executives on this level have committed to employ AI within areas for refinement. As the IGO proactively engages in their IGO to advance service delivery, improve work promoting innovation and works collaboratively with external practices, and foster innovation. IT modernization efforts, partners, it is widely recognized as a leader in the AI field re-organization of data and transition to modern solutions, and lends its expertise to peer agencies and stakeholders, thus facilitating external capacity building.

Figure 1: The AI in IGOs maturity framework Appendix Table Table 1: Core activities Evaluation implementation Project and development Project formulation Programme forecasting planning and Budgeting, formulation Strategy Activity selection Selected Selected AI • • • • • • • • Exemplary AIuse cases / applications • • • • • • • Noteworthy projects include: mission outcomes and improve the efficiency, quality, and cost data. AI allowsto discover insights that werepreviously hidden or overlooked and can nowhelp accelerate Translate information into strategic insights: IGOs hold unprecedented and exponentially growing amounts of Noteworthy examples include: delivering on time (upper and lower bounds of task duration) of the project and back log estimates, AI may learn the completion rate and predict the likelihood of Predict project outcomes: Basedon existent project data, e.g. planned start and end date of various phases Noteworthy examples include: mostly on proof of concept level In general, the outcomes basedon their affiliation (citizenship, political party etc.), thus providing neural network models may associate opinions to members of decision making bodies on defined issues Anticipate voting outcomes: Supporting vector machines (SVM) and back propagation learning algorithm mostly task Even modern smartphone assistants exhibit limited AI capabilities and emotional intelligence and provide basedsoftware engines that provide scripted answerswhenspotting certain trigger wordsor sentences. Deploy chatbots aspersonal assistantsto decision makers: Majority of today’schatbots are simple rule insights on sentiment volumes of written data allowto recognize the most salient features, category, syntactic structure and offer information and anomalies without extensive prior instruction. Statistical analysis techniques applied to large areas, as isit able to read andunderstand (legal) documents in multiple languages and can find significant Automatize labor data from the same domain real text entire sentence into account, thus enabling for much more accurate results. Potential applications include Leverage machine translation: While earlier systemswork by translating phrases, AI takes the context of an insights in natural language automatically transforms data into reports, detailing the drivers of proposed resolutions and communicating actively seeking clarifications during meetings). Advanced natural language generation platforms may even Facilitate document creation: AI systems are able to automatically generate meeting minutes (while inter government sources,internet) considering the archives. AI may provide atool to link internal data to external data (e.g. from newssources, Make data more accessibleand meaningful: IGOs are repository for a vast amount of data, esp. when assessments including recommendations health with financial reports; AI enhanced dashboardsare able to provide dynamic (risks) scenario Enhance dashboarding functionalities: Simple dashboardsenable IGO executives to assessorganizational reasoning inconsistencies and abnormal activities that may require further human judgement andcircumstantial (sub Supplement human processes: Even specific if processescannot beentirely trusted to an AI engine, certain Framework) reach a settlement on final documents asprincipal policy directive of the organization (e.g. UN Strategic releasing capacities for drafting strategic planning documents (e.g. UNProposed Strategic Framework) and procure budgeting, planning and forecasting (BP&F) departments spenton strategic activities. Processessuch as Automate routine decisions andreports: Mainly due to deficits of automation, only a fraction of time in - - − − − − − − − − − − − − − - to time translations and machine translation system may be adapted to a specific domain by using training )tasks may be, e.g. pattern recognition within complex data setscan highlight potential discrepancies, - text, text - e.g. challenges byproviding tools, data and solutions; utilized to solve11 inquiries since recent launch, UN: Collaborative platform (Unite Ideas, service opento anyUN entity) applied to solve complex the CEBICT Network in fall 2017, wouldallow for improved decision support collaborative project and development platform (Bluemix), aspresented during the 29th Meeting of IBM’s which might whichmight prove helpful for evaluating impacts of transport infrastructure interventions buildings going upand changes in building heights pinpoint urban growthcenters in Dares Salaam, World Bank: U reconnaissance (ISR) activities operational and tactical levels, with substantial potential for NATO’s intelligence, surveillance, and NATO: AI projects initiated by NATO’s Allied Command Transformation (ACT) pursued mainly on requests, tools for conducting virtual autopsies, and autonomous patrol vehicles techniques to autonomously research, analyze and respond to international mutual legal assistance discussedincluded facial recognition software, crime prediction tools; behavior detection systems; enforcement organized byInterpol’s Innovation Centre and the UNICRI (July 2018); usecases Interpol: First global meeting to examine opportunities and potential risks of AI androbotics for law part of the International Comparison Program (ICP) photos andprice World Bank: Applied to AI create image discussionsaired by local radio stations in Ugandato support the government's refugee policy UNGlobal Pulse / UN DOCO: Project an automated speech research processand providing fast and accurate fact checking (e.g. events, people, location and dates) they can make information more accessible, simplifying the By searching through data in real time and extracting information basedon requested categories and act aspersonal assistantsduring negotiations and voting processes Future chatbots may acquire apersonality, customizing the language or interface to aparticular user statements; this capability might also beextended to video content by overlaying facts and figures It may also augment IGO executives’ decision sources,assign semantic tagsand organize it to pre Data extraction tool could aggregate and extract information from both IGO’s databasesand outside complexities manual tasksthat might be unable to account for future dynamic operational and financial directional “step change” In the longer term, it enables to move towardsperpetual, scenario based BP&F, which would mark a agenda the right direction, asthis encouragingly showsthat automation and standardization are onthe Establishment of Tableau asaglobal visual analytics standard acrossthe UN(03/2018) is astep in to 14 - pay, record - based advice and responses use ▪ ▪ ▪ Data Science Experience ( programme 6 - to weaponsystems; could also assist in tracking down missile sites Leverage AI to delve into and exploit massive amounts of data generated by advanced instability index) public sources, real correlation betweenoutbreak of violence and country’s commodity prices (all data from betweena selection of variables asinput to build a linear regression model to measure current newsreports andsocial media posts: time seriesforecast model and the correlation Application designed to predict violence at arecent election basedon historical data, NewYork based universities) Machine - andtime - speech, speech cases sing satellite data to evaluate the impact of regional or macro policies, e.g. track new - to - - related data for avariety of household goodsand services in 162 in 15countries as - report, even close based text analysis of national cybersecurity strategies (in partnerships withtwo - delivery delivery stepdisplays highest level of AI maturity in NGOs, albeit these are intensive review processes: AI may perform contract analysis for various (legal) 7 in 4 for for IGOs; replacing current planning cycle processesdependent on many - to - time time analysisused to monitor events and to determine a political - core text, text, speech DSx - to ), ), Watson Machine Learning (WML), and - disclose can be turned over in wholeor part to AI, - IGO recognition models for more than 1 million crowdsourced - to - speech, image (of words) - making capability bycreating pop activities 17 - defined categories) - effectiveness of the delivered services - to - an AI anticipation text text technology to analyze public - to 19 - text text with near 3 U 18 - N centric Ncentric up facts for certain system of voting 21 - 5 accurate, thus 16 - - • • • Private / public sector example • • • • • • • • • scenariosbefore they occur trends. Moreover, allowsit to calculate the probability of financial abnormal patterns or unique behaviors, sets of structured and unstructured data and transactions to identify Financial institutions andinsurers use AI’s ability to churn through large tools lag behind forefront visualization techniques and experimental AI accuracy, speed andforecast horizon in BP&F. Even advanced analytic Corporates and public sector steadily realize that AI may increase their offerings leveraging real exploitation), leading internet companies excel at automatically retooling through trial and error), and ambidexterity (balancing exploration and Building onconcepts of agility (rapid adjustment), adaptation (learning growth inflection constraints, thereby achieving early course of the introduction of a newtechnology under capital and resources Electronics corporation leveraged AI to ensure performance accessto theseareas infrastructure (Wi It expects to employ this tool to identify the right wirelessnetwork settlements in developing regions and link that to population density data. deep learning AI techniques on satellite imagery to pinpoint human American social media service company deployed computer vision and optimizing loads basedoptimizations like least cost routing for minimizing fuel costsor AI has proven particularly useful for transportation and logistics, e.g. ML outcomes in the US Successfully applied in scientific contexts to predict congressional voting participants speaking at the same time conversations, even capable of full duplex communication, i.e. both user views, and offer independent opinions through text display emotional intelligence, demonstrate care and affection, oppose software agent primarily targeted at the Chinesecommunity; able to Unique in the current chatbot landscape: advanced natural language organization, and other areas in commercial compliance, lease abstraction, real estate, corporate whytop devote anestimated 30 Private sample of newsstories similarly to a professional human translator translation of a text from Chineseto English, translating sentences from a Technology company’s AI hasasachieved so published across different platforms it to relevant phrases in a story template to create a narrative that could be bot during the 2016 Olympic Games; collected it information andmatched Large US publishing houseexperimented with an automated newswriting newsoutlets’ RSSfeeds National broadcasting organization’s extraction tool tracks 850 global − - than a decade) housecontent creation, while expanding to 190 countries, in less evolving from DVDrental businessto streaming services to in and models, resource allocation processes,and structures (e.g. These organizations frequently adjust their businessstrategies - sector legal firms or companies’ in tier tier lawfirms are increasingly applying AI software to documents - Fi, cellular, microwave backhaul) to provide internet 15 20 4 - 50% of their time on contract analysis, whichis 9 - 12 time data on customer behavior - 10 majority customer adoption and 13 8 suggesting actions basedon - house legal departments - called called "human parity" in - or voice - to - plan in the - based 11 - Appendix Table Table 2: Supporting and cross-divisional activities Compliance / PR Communication IT management resources Human management Supply management & Organizational treasury Finance & Activity member chain Selected Selected use AIcases supporting cross in and • • • • • • • • • • • • • • • Exemplary AIuse cases / applications • • • • • • potential potential impacts from black swan events and other unlikely event scenarios shifts allows for better preparation (e.g. dynamic hedging to mitigate possibleoutcomes); may also calculate Enable better predictions: AI policies (read receipts, audit expensesand alert humans about possibleinfractions) Automatize expense management: Reviewing and approving of expenses to ensurecompliance with IGO and voluntary contributions from member states(able to learn accounting codesetc.) Facilitate accounts payable/receivable processing: AI poweredsystems streamline processing of assessed through the UN’sprocurement system monitored database for weapons of member statescontributing peacekeeping forces andarms purchased of weaponsand millions of rounds of ammunition lost over the past towdecades Ensure internal compliance: Prevent arms and ammunition losseson UNpeacekeeping missions (thousands clustered together to monitor behavior patterns in relation to other of the same cluster at any given time features byidentifying relationships betweendata points and reducing them to core components, whichcan be Enhance fraud detection: Implement deep learning to the creation of user third services industry to screen enormous amounts of daily transactions for market manipulation, insider trading, statesto detect money Improve outcomes of designated regulations controlling and enforcing compliance of member states, e.g. withUN Security Council’s and ICJ’srules and Promote member state compliance: Support general fact tool content and filter downto target audiences (e.g. basedon age, gender, location, interests) constitutes powerful and A/B testing before right target audience or social media channel is identified. Ability to specifically target Content distribution on social media, however, isgenerally characterized by high degrees of experimentation statistics; sendout alarm if shifts from normal behavior are notices, helping securean IGO’s weaktargets Scan crucial documents and files: Determine value of eachfile for IGO operations, track usageand access vectors (real enables cybersecurity systemsthat automatically update software databasesand algorithms with newthreat time needed to analyze cyberattack data and formulate a response(commonly handled by human experts). AI Prevent against cybersecurity threats: Traditional cybersecurity systems: counteraction often constrained by well expressions) may beused as early detection system against physical traumata, thus optimizing the health and Detect illness symptoms early: Emotion recognition through advanced computer vision (reading of facial concerns to HRteam members withthe right expertise andleadership role required to resolve them quickly Decrease intention to quit: AI technologies embedded in employee feedback tools automatically channel Personalize trainings: Automatically tailor trainings towards specific role, situation in the country etc. questionsand requests; thusincreasing the focus on more strategic HRwork employees (allocating space,provisioning a laptop etc.), benefits management, and triaging common Automate repetitive tasks: presents AI opportunity to automate standardized tasks,e.g. onboarding of new improve hiring communications and ensurebias Remove biases: AI algorithms may be designed to help IGOs identify and remove biaspatterns in language to factor, esp.when competing withprivate sector digital expectations and speedup the process(currently takesup to several months) might be a differentiating Enhance candidate experience: IGOs participate in global war for talent; technologies that meet candidate’s suppliers, place purchasing requests,receive/file/document of invoices for order requestsand payment Enhance operational procurement efficiency: Utilize chatbots for routine tasks,e.g. to communicate with suppliers and service providers to the WFP language data to harnessuntapped information and streamline auditing and compliance actions, e.g. for Remove existing language barriers betweenbuyer correlations between external weather, demand and other influences) in historical demand data responseby using predictive demand modeling software to identify hidden patterns (e.g. seasonaldemand or Optimize demand forecasting: Improve predictability, timeliness and efficiency of humanitarian emergency member communication and appeal for funds by accounting for previous donation and sponsorshiphabits finance most of UN’shumanitarian relief and development activities Personalize member communication: In light of the pivotal importance of voluntary contributions (e.g. they member/stakeholder satisfaction service interfaces, and to anticipate and resolve issuesbefore they arrive, hence increasing transparency and and the wider IGO stakeholder community. This allows to offer real inquiries towardsdecision Redesign member and stakeholder management: AI is able to learn from previously gathered information (e.g. - - being of field staff deployed in troubled regions party risk etc.) − − - time, dynamic threat protection though adaptive ML algorithms) engine that automates content distribution optimize click (regarding howwella particular article woulddo); system can then create a schedule order to Algorithm could scanIGO’s newsarticles/campaign posts in real time and assigna score to each media AI - based solutions may help IGOs predict how welltheir content will do onceshared on social - laundering andfinancing of terrorism (AI technologies already used in the financial - making bodies), thus determining behavior patterns and preferences of members - throughs onsocial media or even be expanded to a centralized repository or - based t programmes reasury forecasting : : Assistthe UNODC in strengthening the ability of member - free free candidate screenings - supplier bodies: UseNLPto rapidly decipher foreign regarding weather, natural disasters,and geopolitical - finding missions and reporting duties as way of - 23 time support, potentially even through self ), ), AI hasthe potential to enable tailored - andtransaction - 30 divisional divisional activities IGO ) by establishing acentral AI - specific distinctive - - • • • • • • Private / public sector example • • • • Software for screening job candidates isincreasingly usedacross machine sensorsand internal enterprise softwareSAP inventory and distribution efficiency drawing ondata from factory sensorscoupled withML predictive analytics software to enhance German pharmaceutical and life sciencescompany deployed asuite of deliver parts faster and reduce inventory storage levels records at distribution centers) that provide newinsights enabling to supply chain data (e.g. historical shipping records or stock keeping USautomobile manufacturer: ML based platform to detect patterns in customers that allowfor personalized client approach data from acrossthe web,thus providing actionable insights ontarget on individual and account levels by filtering through massive amounts of Modern salesplatforms leverage AI to sourcenews and social updates usecases, e.g. All Big Four tax and accounting companies employ AI tools for various automate data entry, reconciliations etc. Major accounting software vendors currently offering capabilities to purchase profiling, merchant, device data, time of day, location, andtype of leveraging account information, e.g. customer value segmentation, risk to disclose normal and abnormal shopping spendingbehavior by Credit card providers examine howspecific accounts are used over time media page views over Facebook and +97% from Twitter through AI for social The three biggest French newssites realized, on average, +51% in to prevent, rather than reactively detect, viruses and malware following different approaches, e.g. applying AI to assignthreat scores Numerous startups have entered the AI cybersecurity solutionsspace more than 3 million job applications in 2016) supermarket chain, the country’s largest private employer, received UK2017 study: on average 24applicants per low posture, facial expression and vocal tone of a camera with AI tools like evaluating hardly perceptible changes in positions. Candidates are askedto answerinterview questions in front industries (e.g. consumer goods, investment banking) for higher − − − − 29 accounts members maintain bestaccess to decision makers at target Additionally, they are able to highlight to salesteams which benchmarks) AI/ML processesto attain better quality and costs in lessthan a year; company benchmarked both human and previous humans has proven three times more consistent andtwice asefficient as extract information and human leases to comply with the newlaw. Deployment of NLPto normally entailed manual re Review of newly announced lease accounting standards members to complete the same task in lessthan a month) half a year; now,an AI system is usedbya team of 6 of a business unit (usually occupied dozens of employees for Detection of changes in control provisions during a client’s sale include quizzes, psychometric tests,games and chatbots that scanned text CVsfor key wordshas now expanded to i.e. whatstarted more than 10 yearsago with simple programs Led to automatization of large parts of the recruitment process, 31 24 22 - only teams (system reached break - examination of tens of thousands of - in - the - loop validation of result - wagejob (e.g. 26 25 - - even even ROI 28 8 - skilled skilled 27 References

1 International Data Corporation (2017). IDC Spending Guide Forecasts Worldwide Spending on Cognitive and Artificial Intelligence Systems to Reach $57.6 Billion in 2021. [online] Available at: https://www.idc.com/getdoc.jsp?containerId=prUS430954177 2 The Economist (2018). AI-spy: The workplace of the future. [online] Available at: https://www.economist.com/leaders/2018/03/28/the- workplace-of-the-future 3 Columbus, L. (2018). Roundup Of Machine Learning Forecasts And Market Estimates, 2018. [online] Available at: https://www.forbes.com/sites/louiscolumbus/2018/02/18/roundup-of-machine-learning-forecasts-and-market-estimates- 2018/#62ff40f12225 4,5,8,40 Mehr, H. (2017). Artificial Intelligence for Citizen Services and Government. 6 Evenstad, L. (2018). Department for Work and Pensions uses artificial intelligence to crack down on benefits fraud. [online] Available at: https://www.computerweekly.com/news/252444049/Department-for-Work-and-Pensions-uses-artificial-intelligence-to-crack-down-on- benefits-fraud 7 Kim, Y. (2018). What it felt like to visit the most tech-centric Olympics ever. [online] Available at: https://www.technologyreview.com/s/610181/what-it-felt-like-to-visit-the-most-tech-centric-olympics-ever/ 9 Goldstein, P. (2017) 7 Ways AI Could Save the Government Money and Boost Productivity. [online] Available at: https://fedtechmagazine.com/article/2017/05/7-ways-ai-could-save-government-money-and-boost-productivity 10,11 The White House Office of Science and Technology Policy (2018). Summary of the 2018 White House Summit on Artificial Intelligence for American Industry. 12 Guterres, A. (2017). Video message to the AI for Good Global Summit. [online] Available at: https://news.un.org/en/story/2017/06/558962-un-artificial-intelligence-summit-aims-tackle-poverty-humanitys-grand 13 Bernard Koteen Office of Public Interest Advising (2018). Intergovernmental Organizations (IGOs) [online] Available at: https://hls.harvard.edu/dept/opia/what-is-public-interest-law/public-service-practice-settings/public-international-law/intergovernmental- organizations-igos/ 14 Union of International Associations (2018). What is an intergovernmental organization (IGO)?. [online] Available at: https://uia.org/faq/yb3 15 Volgy, T. J., Fausett, E., Grant, K. A., & Rodgers, S. (2008). Identifying Formal Intergovernmental Organizations. Journal of Peace Research, 45(6), 837-850. 16 Union of International Associations (2018). How do international organizations get registered/created?. [online] Available at: https://uia.org/faq/intorgs6 17 United Nations (2018). What we do. [online] Available at: http://www.un.org/en/sections/what-we-do/ 18 World Bank Group (2018). What we do. [online] Available at: http://www.worldbank.org/en/about/what-we-do 19 International Monetary Fund (2018). How we do it. [online] Available at: https://www.imf.org/external/about/howwedo.htm 20 Intergovernmental Panel on Climate Change (2018). IPCC Factsheet. [online] Available at: http://www.ipcc.ch/news_and_events/docs/factsheets/FS_what_ipcc.pdf 21 Multilateral Organisation Performance Assessment Network (2017). MOPAN 2015-16 Assessments: Institutional Assessment Report - United Nations Children’s Fund (UNICEF). 22 Federo, R., & Saz‐Carranza, A. (2017). Devising strategic plans to improve organizational performance of intergovernmental organizations. Global Policy, 8(2), 202-212. 23 Snow, G., Smith, M. and Stecher, K. (2018). NGO Reference Model. [online] Available at: https://www.ngoreferencemodel.org/ 24 Corea, F. (2017). A Brief History of AI. [online] Available at: https://medium.com/@Francesco_AI/a-brief-history-of-ai-baf0f362f5d6 25 The Society for the Study of Artificial Intelligence and Simulation of Behaviour (2018). What is Artificial Intelligence. [online] Available at: https://www.aisb.org.uk/public-engagement/what-is-ai 26 Miailhe, N. and Hodes, C. (2017). Making the AI revolution work for everyone. 27,28,29,20,31 Buchanan, B. and Miller, T. (2017). Machine Learning for Policymakers – What It Is and Why It Matters. 32 Gutierez, D. (2015). Adopting Big Data for Manufacturing. [online] Available at: https://insidebigdata.com/2015/04/22/adopting-big- data-for-manufacturing/ 33 Gartner (2018). Gartner IT Glossary – Advanced Analytics. [online] Available at: https://www.gartner.com/it-glossary/advanced- analytics/ 34 United Nations Chief Executives Board for Coordination (2017). 29th Meeting of the CEB ICT Network. [online] Available at: https://www.unsceb.org/system/files/20171205%20ICT%20Network%2029th%20Meeting%20Summary.pdf 35 Bartley, A. (2018). Predictive Analytics In Healthcare. 36 SAS Institute (2018). Big Data – What it is and why it matters. [online] Available at: https://www.sas.com/en_us/insights/big-data/what- is-big-data.html 37 UN Global Pulse (2018). Projects overview. [online] Available at: https://www.unglobalpulse.org/projects References

38 U.S. Citizenship and Immigration Services (2016). Emma: Friendly Presence and Innovative USCIS Resource - Available 24/7. [online] Available at: https://www.uscis.gov/archive/blog/2016/09/emma-friendly-presence-and-innovative 39 Chin, C. (2016). Japan trials AI for parliament use. [online] Available at: https://govinsider.asia/innovation/japan-trials-ai-for-parliament- use/ 41 Partnership for Public Service and IBM Center for The Business of Government (2018). The Future has Begun: Using Artificial Intelligence to Transform Government. 42 University of Rochester (2016). App helps fight food poisoning in Las Vegas with machine learning. [online] Available at: http://www.hajim.rochester.edu/news/2016-03-09_app_tracks_food_poisoning.html 43 Hillary, D. (2018) Strategies to Accelerate AI in the Enterprise. 44 Wilder-James, E. (2016). Breaking Down Data Silos. [online] Available at: https://hbr.org/2016/12/breaking-down-data-silos 45 Guterres, A. (2017). Remarks at the opening of the Centre for Humanitarian Data. [online] Available at: https://www.un.org/sg/en/content/sg/speeches/2017-12-22/opening-centre-humanitarian-data-remarks 46 Riazi, A. (2018). United Nations Establishes Tableau as Visual Analytics Standard, Streamlines Global Adoption of Platform. [online] Available at: https://www.un.org/press/en/2018/pi2226.doc.htm 47 United Nations (2018). About Umoja. [online] Available at: https://umoja.un.org/about 48 Ransbotham, S., Kiron, D., & Prentice, P. K. (2016). Beyond the hype: the hard work behind analytics success. MIT Sloan Management Review, 57(3). 49,52 Domino Data Lab (2017). The Practical Guide to Managing Data Science at Scale. 50 Altexsoft (2018). How to Structure a Data Science Team: Key Models and Roles to Consider. [online] Available at: https://www.altexsoft.com/blog/datascience/how-to-structure-data-science-team-key-models-and-roles/ 51 Freedberg Jr., S. J. (2018). Joint Artificial Intelligence Center Created Under DoD CIO. [online] Available at: https://breakingdefense.com/2018/06/joint-artificial-intelligence-center-created-under-dod-cio/ 53 Altexsoft (2018). Comparing Machine Learning as a Service: Amazon, Microsoft Azure, Google Cloud AI, IBM Watson. [online] Available at: https://www.altexsoft.com/blog/datascience/comparing-machine-learning-as-a-service-amazon-microsoft-azure-google- cloud-ai-ibm-watson/ 54 Thomas, R. (2018). Google's AutoML: Cutting Through the Hype. [online] Available at: http://www.fast.ai/2018/07/23/auto-ml-3/ 55 Columbus, L. (2017). LinkedIn's Fastest-Growing Jobs Today Are In Data Science And Machine Learning. [online] Available at: https://www.forbes.com/sites/louiscolumbus/2017/12/11/linkedins-fastest-growing-jobs-today-are-in-data-science-machine- learning/#5ef0297251bd 56 Burtch, L. (2018). The Burtch Works Study: Salaries of Data Scientists. 57,58 International Telecommunication Union (2018). Artificial Intelligence (AI) for Development Series: Introductory module. 59,60,61,62 Sanders IV, Lewis (2018). Can AI be free of bias?. [online] Available at: https://www.dw.com/en/can-ai-be-free-of-bias/a- 43910804 63,65 Uruburo, L. (2010). UN Umoja reaches key milestone by signing agreement with SAP. [online] Available at: https://www.unumoja.net/display/public/2010/06/30/UN+Umoja+reaches+key+milestone+by+signing+agreement+with+SAP 64,66,67 Lynch, T. (2016). At the United Nations, Umoja Translates as Bureaucratic Chaos. [online] Available at: https://foreignpolicy.com/2016/05/06/at-the-united-nations-umoja-translates-as-bureaucratic-chaos/ 68 Azati Software (2017). How much does artificial intelligence (AI) cost in 2018?. [online] Available at: https://azati.com/how-much-does- it-cost-to-utilize-machine-learning-artificial-intelligence/ 69 United Nations Global Pulse (2017). Using machine learning to accelerate sustainable development solutions in Uganda. [online] Available at: https://www.unglobalpulse.org/news/using-machine-learning-accelerate-sustainable-development-solutions-uganda-0 70 Muggah, R. and Kavanagh, C. (2018). 6 ways to ensure AI and new tech works for – not against – humanity. [online] Available at: https://www.weforum.org/agenda/2018/07/united-nations-artificial-intelligence-social-good/ 71 United Nations Development Programme (2018). UNDP joins Tech Giants in Partnership on AI. [online] Available at: http://www.undp.org/content/undp/en/home/news-centre/news/2018/undp-joins-tech-giants-in-partnership-on-ai.html References of the appendix

1 Desouza, K. C. (2018). Delivering Artificial Intelligence in Government: Challenges and Opportunities. 2 International Business Machines Corporation (2018). What is cloud computing? [online] Available at: https://www.ibm.com/cloud/learn/what-is-cloud-computing 3 United Nations Chief Executives Board for Coordination (2017). 29th Meeting of the CEB ICT Network. [online] Available at: https://www.unsceb.org/system/files/20171205%20ICT%20Network%2029th%20Meeting%20Summary.pdf 4 Reeves, M. and Whitaker K. (2018). Competing on the Rate of Learning. [online] Available at: https://www.bcg.com/de- de/publications/2018/competing-rate-learning.aspx 5,7 FirstAlign (2017). Whitepaper: Artificial Intelligence for Finance. 6 United Nations (2018). United Nations Establishes Tableau as Visual Analytics Standard, Streamlines Global Adoption of Platform. [online] Available at: https://www.un.org/press/en/2018/pi2226.doc.htm 8,12, 28,29 Kaul, A. and Wheelock C. (2016). Top 15 Use Cases for Artificial Intelligence. 9,10 Underwood, C. (2018). Automated Journalism – AI Applications at New York Times, Reuters, and Other Media Giants. [online] Available at: https://www.techemergence.com/automated-journalism-applications/ 11 Del Bello, L. (2018). AI Translates News Just as Well as a Human Would. [online] Available at: https://futurism.com/ai-translator- microsoft/ 13 Warren,T. (2018). Microsoft also has an AI bot that makes phone calls to humans. [online] Available at: https://www.theverge.com/2018/5/22/17379508/microsoft-xiaoice-chat-bot-phone-call-demo 14,15 Khashman, Z., & Khashman, A. (2016). Anticipation of political party voting using artificial intelligence. Procedia Computer Science, 102, 611-616. 16 United Nations Global Pulse (2017). Pilot studies: Using machine learning to analyse radio content in Uganda. [online] Available at: https://www.unglobalpulse.org/projects/pilot-studies-using-machine-learning-analyse-radio-content-uganda-2017 17 Wittich, J. (2018). Deep Learning and Artificial Intelligence Help World Bank Team Create Image-Recognition Models for Crowdsourced Photos. [online] Available at: https://itpeernetwork.intel.com/artificial-intelligence-world-bank-image-recognition/ 18 Interpol (2018). Tackling future policing challenges with AI, robotics. [online] Available at: https://www.interpol.int/News-and- media/News/2018/N2018-071 19 Jans, K. (2018). NATO Needs to Get Smarter About AI. [online] Available at: http://www.atlanticcouncil.org/blogs/new-atlanticist/nato- needs-to-get-smarter-about-ai 20 Dhande, M. (2017). How Facebook uses AI, machine learning & satellite imagery to locate unconnected people. [online] Available at: https://www.geospatialworld.net/article/facebook-uses-satellite-imagery-machine-learning-and-ai/ 21 McKenzie, D. (2018). How can machine learning and artificial intelligence be used in development interventions and impact evaluations? [online] Available at: https://blogs.worldbank.org/impactevaluations/how-can-machine-learning-and-artificial-intelligence-be- used-development-interventions-and-impact 22 Zhou, A. (2017). EY, Deloitte And PwC Embrace Artificial Intelligence For Tax And Accounting. [online] Available at: https://www.forbes.com/sites/adelynzhou/2017/11/14/ey-deloitte-and-pwc-embrace-artificial-intelligence-for-tax-and- accounting/#3ac1614f3498 23 Triolo, P. (2012). Simplifying the UN budget. [online] Available at: https://www.devex.com/news/simplifying-the-un-budget-77709 24 Hurley, K. (2017). 11 Artificial Intelligence Tools Transforming the B2B Sales World. [online] Available at: https://nudge.ai/artificial- intelligence-sales-tools/ 25,26 Bharadwaj, R. (2018). Artificial Intelligence in Supply Chain Management – Current Possibilities and Applications. [online] Available at: https://www.techemergence.com/artificial-intelligence-in-supply-chain-management-current-possibilities-and-applications/ 27 Buranyi, S. (2018). How to persuade a robot that you should get the job. [online] Available at: https://www.theguardian.com/technology/2018/mar/04/robots-screen-candidates-for-jobs-artificial-intelligence 30 Reinl, J. (2017). UN peacekeeping arms losses could equip an army: report. [online] Available at: https://www.aljazeera.com/news/2017/11/peacekeeping-arms-losses-equip-army-small-arms-survey-171113075518696.html 31 Chinner, V. (2018). Artificial Intelligence And The Future Of Financial Fraud Detection. [online] Available at: https://www.forbes.com/sites/theyec/2018/06/04/artificial-intelligence-and-the-future-of-financial-fraud-detection/#253978e4127a