For enterprise leaders and their legal & compliance advisors

AI Governance and Ethics for Boards

10 Things Boards Need to Know About AI

1 10 Things Boards Need to Know About AI

What you will learn

In the coming decades, will bring many changes for companies, government agencies, and non-profit organizations. Business processes will be transformed by automation. Entirely new products, services, and business models will be invented. Jobs will undoubtedly be lost, but many more will be created. Education will be more important than ever and will need to become more relevant and more effective than it is today.

AI will mean new competition for some and decline or even extinction for others. AI will be a key driver of economic growth and will create tremendous new wealth. Sharing that wealth equitably will be a crucial focus for policymakers and society as a whole, as will the imperative to devise sensible regulations for this powerful new technology.

Board members, CEOs, entrepreneurs, together with their legal and governance advisors need to educate themselves about how to choose among AI’s strategic options, how to pursue them effectively, and how to cope with the important ethical and social challenges that AI will bring. This AI Insight is a guide to these issues.

Contents

01 06 AI will be a crucial growth driver, embrace it Build AI on a foundation of trust

02 07 Learn the basics of what AI can do Learn how to make AI fair 03 08 AI’s tactical and strategic flavors, you need both AI regulation is coming—plan for it 04 09 Understand the value of your data for AI Manage AI’s impact on jobs 05 10 Leverage AI partners, build internal AI know-how Plan for creative destruction, do the right thing

AI Governance and Ethics for Boards 01 AI will be a crucial growth driver, embrace it

The most important technologies are those that can solve many kinds of problems.

AI is the defining business technology of the 21st century

Some innovations—the automobile, the jet airplane, Artificial intelligence promises to be one of the the smartphone—are solutions to practical needs most important enabling technologies of the 21st of great economic value, such as transportation century. Over the coming decades, it will reshape and social interaction. Others—the steam engine, organizations of every size and sector, from giant the electrical grid, the PC—do not solve specific corporations to corner stores, from single-issue problems. Rather, they are enabling technologies non-profits to sprawling government agencies, from that allow the creation of new and more productive individual households to our global economic system. solutions to many different kinds of problems.

Because of the endless opportunities for innovation “Microsoft has always seen itself as a builder of and business process tinkering that enabling enabling technologies.” technologies unleash, they have always been the secular engines of economic growth and social progress.

At Microsoft, we have always seen ourselves as a platform company—that is, as a builder of enabling technologies for others to create value with.

3 AI can deliver $13 trillion in additional economic output by 2030

An exhaustive recent study by the McKinsey Global Institute forecasts that AI could add 16% to annual world economic output by 2030, or about $13 trillion in net additional global GDP in that year.

The future is now

AI is still in its infancy, just taking its first halting steps For make no mistake. Regardless of your own AI on the world economic stage. It is far from mature, strategy, you will face rivals who bet big on AI— and no one really knows what it will look like when it both existing competitors in your industry and reaches maturity, if indeed it ever does. new entrants. The more you put off your own AI acceleration, the more advantage you will cede to But for enterprise leaders weighing their options these rivals. in the present, AI strategy must be shaped by foreseeable near and medium term benefits rather The race to rebuild existing business processes and than an unknowable distant future. Betting your invent rewarding new ones with AI is not a zero-sum business on a single massive AI investment will not game. Whatever the industry, whatever the scale, AI be the right strategy for most companies (though can benefit all who invest in it. But the winnings can it will be for some). But moving without delay to only go to those who enter the race in the first place. place multiple thoughtful bets on AI with a range of possible payouts is the minimum prudent strategy that every CEO and Board should embrace.

Action Item Institute an education program about AI strategy and opportunities for your Board and senior leaders. You might consider for example Microsoft’s new AI Business School, launched in partnership with Europe’s INSEAD graduate business school. The AI Business School is an online series of non-technical master classes that use written case studies and guides along with video lectures, perspectives, and talks to teach executives the essentials of AI business strategy.

You may also wish to download our broader book on digital business: DIgital Transformation in the Cloud: What enterprise leaders and their legal and compliance advisors need to know.

4 © 2019 Microsoft Corporation. All rights reserved. 02 Learn the basics of what AI can do

This report focuses on AI business strategy, not on AI’s technical foundations such as deep neural networks. Readers seeking a basic AI technical tutorial have a vast array of excellent and free online options to choose from. One of the best, which assumes no special background, is the series of brief online videos Neural Networks and Deep Learning by prominent Stanford AI researcher Andrew Ng. Below we provide a non-technical summary of the most important families of AI algorithms that business leaders need to know about.

Artificial intelligence is not new, but its recent progress is dramatic

AI is not new. At Microsoft our researchers have been working on the subject since the 1990s. But the term “artificial intelligence” is actually much older. It was coined by tanfordS computer scientist John McCarthy at a famous conference held at Dartmouth College in the summer of 1956. Bringing together a small group of brilliant pioneers that included Claude Shannon, Herbert Simon, and Marvin Minksy, the new discipline was to be based on:

“…the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.”

McCarthy’s forecast that significant progress on these questions could be made in the space of a summer proved overly optimistic. The early decades of the field saw much theoretical progress but few practical breakthroughs. AI remained largely confined to academia. In the past ten years, however, the pace of progress has dramatically accelerated. AI has suddenly erupted into the business world and everyday life, with stunning new capabilities that for the first time make it genuinely useful. And more is to come.

5 Artificial intelligence still falls short of human intelligence

Today’s AI surpasses human abilities in an increasing AI is also much slower to learn than humans—it number of narrow but important domains. It uses needs to see thousands of dog and cat photos to ingenious math and the awesome power of modern learn the difference between them, while a human computers to detect and exploit subtle statistical child may recognize “doggy” or “kitty” after just one patterns in great masses of digitized data, patterns example. Moreover, AI can make mistakes that are that pre-AI algorithms could not capture. But AI still obvious to us or be tricked by deceptions that would falls woefully short of our more general human ability never fool a human. to apply contextual knowledge and common sense to solve a broad range of problems that are hard to The world’s leading AI researchers say we are state in formal terms or have never been seen before. nowhere close to being able to create an Artificial AI can play the ancient games of chess and Go better General Intelligence (AGI) that can match us humans than any human who ever lived, but it still struggles in open-ended real-world intelligence. As Harvard to cope with unpredictable city traffic that the most cognitive scientist Steven Pinker puts it, it may be ordinary of human drivers master without conscious better to think of today’s AI (based largely on so- effort. called neural nets) as a kind of “idiot savant” that can perform certain specialized tasks with superhuman “AI still falls woefully short of our human skill, but does not yet understand the world in the ability to apply contextual knowledge and way that we humans do. common sense.”

However restricted the scope of AI’s superhuman skills, do not underestimate what it can achieve, now and in the future. The remarkable progress in just the past five years of face and speech recognition, machine translation, and chatbots is ample proof that AI already has the power to transform business, work, and society. And this is just the beginning.

Supervised learning

Almost all AI applications deployed today in non- apply these labels correctly to new unlabeled examples academic settings use what’s known as supervised it has never seen before. learning. Here an AI algorithm teaches itself to make human-like judgments by ingesting training examples Making supervised learning work reliably in a real-world previously labeled by humans with the correct answers. setting is more complicated than this simple description A “label” might be the name of the kind of animal implies. Many tricky implementation details must be shown in a photo (“cat”) or a caption for an image mastered, often by trial-and-error. But the most difficult describing what is in the scene (“A girl throwing a part of supervised learning is getting enough good frisbee”). Or it might be the French translation of an quality labeled training data in an area relevant to your English sentence (“Une fille lance un frisbee”). By being business. If you have such data or can create it, you exposed to thousands or even millions of such labeled have taken a fundamental step toward building an AI training examples, the algorithm progressively learns to application that creates real value.

6 © 2019 Microsoft Corporation. All rights reserved. Unsupervised learning

Unsupervised learning refers to algorithms that learning based on older statistical and machine sift through unlabeled data looking for patterns or learning methods has important practical applications clusters of interest. The challenge lies in the fact in many industries such as anomaly detection and that the number and nature of the most interesting data mining (e.g., spotting customer transactions that clusters are not known in advance. Modern AI in stand out as being unusual, perhaps because they are the sense of deep neural nets has not yet produced fraudulent or, on the contrary, represent your best major breakthroughs in unsupervised learning of customers). more than academic interest. But unsupervised

Self-supervised learning

Even when labeled training data is not available, Language models are algorithms that are fed massive useful AI algorithms can sometimes be developed amounts of text (for example, the entire corpus of by training them to recreate data identical or Wikipedia) and trained to predict the next word in a similar to the original data fed into them. Two sequence. Such models can then be used for other of the most famous examples of self-supervised tasks, such as guiding translation between languages learning are language models and generative or generating plausible “fake” text in a given style. adversarial networks (GANs).

GANs example

GANs have attracted media attention due to their ability to create entirely artificial yet strikingly realistic images of real-world entities, including human faces. They don’t yet have serious practical applications (other perhaps than generating so-called “deep fake” images or videos), but they are an influential research tool for understanding and manipulating complex real-world shapes in terms of a small number of underlying factors.

For example, images of human faces may be Images of entirely non-existent celebrities created by a GAN understood as being generated from factors AI trained on images of real celebrities. such as gender, age, skin tone, hair color, pose, emotional expression, presence or absence of glasses, and so on.

7 Transfer learning Reinforcement learning

Transfer learning is a variant of supervised and self- Reinforcement learning is the branch of AI that trains supervised learning, where an AI algorithm trained in robots to walk and game-playing programs to play one domain is applied to data in a different domain chess or Go. Unlike supervised learning, where an that nevertheless shares underlying features with the algorithm is fed labeled data that tells it whether original. For example, a language model trained on a given decision is right or wrong, a reinforcement all of Wikipedia might be used by a chatbot to create learning algorithm receives only a weaker “reward” answers that display common sense. signal that says “your last action didn’t kill you” or, perhaps after many steps, “you won (or lost) the game.”

Reinforcement learning example

The field of reinforcement learning has recently been reinvigorated by the infusion of modern AI methods based on deep neural nets. Its relevance to robotics is obvious. But the practical import of game-playing AI should also not be underestimated, because any process or phenomenon of practical interest that can be modeled as a game can now be subjected to powerful reinforcement learning methods. A striking example is the recent application of the same methods used to beat the world’s best human players of Go to the problem of predicting how proteins fold into specific three dimensional shapes. Here an AI developed to play games may one day play a role in the discovery of new life-saving drugs. Protein folding modeled by an AI originally created to play games such as chess and Go.

Case Study

In the future, customers will expect to interact with companies by just telling an AI what they want

” Customers interact with large corporations and Today, AI-powered chatbots are already beginning to government agencies every day, and the process is improve this experience. Spanish and Latin American already heavily automated, but not always smooth. telecom giant Telefónica provides a good example. Most of us are all too familiar with call center phone Its AI-powered digital assistant, Aura, launched in trees, limited-function mobile apps, and complicated February 2018, is now used by 2 million customers web sites, which are the dominant technologies of per month to talk to the company over multiple the pre-AI era for org-to-consumer interaction. channels, including the company’s own mobile app

8 © 2019 Microsoft Corporation. All rights reserved. and third-party apps such as WhatsApp, Facebook when you go to the web site or mobile app of your Messenger, and Assistant. Aura’s front-end bank, airline, phone company, or government tax uses AI to interpret customer questions about authority, you typically have to tap through multiple the products and services, create support claims, screens to get to what you want: a credit card manage devices such as WiFi routers, and track data statement, a boarding pass, a data usage update, or a consumption. On the backend, Aura searches for tax form. Soon these cumbersome interactions will be answers using software interfaces to many of the made obsolete by AI. You will just speak or text to the company’s core IT systems and databases. organization directly, tell it what you want (“get me my statement from last April”, “register one bag for In the future, AI assistants will evolve into articulate me and send me my boarding pass”, “tell me when and intelligent avatars of organizations that can my tax refund will arrive”) and the AI will take care of smoothly handle complex inquiries and service the rest. requests of nearly any kind from customers. Today

Action Item Your organization’s AI strategy will be more likely to focus on the right options if at least a few members of your board and senior executive team learn the technical basics of modern AI, for example by following an online course such as Andrew Ng’s introduction to the subject. But it is even more important for all members of your leadership team to understand (even in non-technical terms) the broad families of AI described above and what they can do in real-world settings.

9 03 AI comes in tactical and strategic flavors, you need both

You need AI that helps you keep pace with rivals and AI that helps you get ahead.

But how to build AI?

The promise of AI is evident. But how should you go about building AI in your enterprise? Where to start? AI comes in tactical and strategic flavors. You need both. This distinction is not necessarily a matter of scale. Tactical AI applications can be vast, while strategic ones can be quite small. The difference lies in how much competitive advantage and potential new growth the two kinds of AI bring.

Tactical AI is about doing ordinary things better and cheaper

There are many standard tasks embedded in your While these tasks may be components of larger business processes that will be performed more business processes that have strategic value, their efficiently by the right kind of AI. Applying AI to these automation by AI is usually tactical in nature. This tasks will save labor, avoid mistakes and waste, speed is because AI can be applied to them on a stand- outcomes, improve quality, and capture revenue that alone basis and does not necessarily require new might otherwise be lost. Here are some examples: organizational know-how. The data required to train the AI for these applications may be specific to your • Detecting defects in manufactured parts or systems enterprise, but it is generally simple in format and easy to gather. The AI algorithms used in these apps • Verifying the identity of customers or are not proprietary, but rely on standard methods for employees with face recognition processing images or natural language with neural • Allowing customers to tell you what they want nets. using natural language Tactical AI will increasingly become a built-in feature • Providing answers or guidance to customers of standard business applications such as Microsoft’s using natural language Dynamics 365 or be offered in the form of cloud • Compliance monitoring in legal contracts or services such as Microsoft Azure’s Face API. In most financial trades cases it will be quicker, cheaper, and better to deploy tactical AI using off-the-shelf offerings like these • Detecting fraudulent behavior or anomalous incidents rather than building it yourself.

10 © 2019 Microsoft Corporation. All rights reserved. A demonstration of customer analytics AI built into Dynamics 365

Tactical AI and competitive advantage

Because the tasks automated by tactical AI are similar tactical AI are more likely to be successful with in nature in many organizations and can be handled strategic AI. by standard solutions available to anyone, they can’t offer lasting competitive advantage. But they do While ridesharing service Uber uses a cloud-based offer short-term advantages to early adopters, and— face recognition service from Microsoft to verify just as important—they carry the danger of lasting the identity of its drivers with smartphone selfies, competitive disadvantage for technology laggards thus enhancing security for both drivers and riders, who fail to keep up with their peers. In this sense, this tactical use of AI is available in principle to any tactical AI applications are minimum table stakes organization, including Uber rivals like Lyft. that are indispensable for staying competitive in your industry.

Tactical AI need not be small. By dramatically reducing the cost of otherwise ordinary outputs or procedures, it may make it possible to scale up a process far beyond what was previously possible. In this way tactical AI can evolve into strategic AI.

Tactical AI is a good way to build the AI knowledge and skills of your teams. Teams that have mastered Tactical AI at work: Uber uses Microsoft’s cloud-based Face API to verify driver identities

Strategic AI is about new value creation and differentiation from peers

Strategic AI involves the transformation of one or multiple connected business processes so as to create large amounts of new value. Strategic AI does not necessarily use different technical approaches than tactical AI— both typically rely on the same kinds of neural nets and other machine learning methods. Strategic AI is often applied on a large scale, but can sometimes be narrow in scope.

The distinguishing characteristic of strategic AI is not simply that it creates great value, but that it augments the capabilities of the organization in ways that cannot readily be duplicated by others. Strategic AI produces lasting competitive advantage, breakthrough performance, or other transformational outcomes. For example,

11 Uber also uses AI to adjust ride pricing dynamically based on its unique real-time insights into customer demand. The latter application is strategic and cannot readily be duplicated.

Here are some other examples:

Rolls-Royce

Rolls-Royce streams real-time sensor data from thousands of jet engines to the cloud, then uses machine learning to optimize fuel consumption and predict maintenance needs. This combination of the cloud, big data, and AI not only delivers safer flying and millions in annual savings to the airlines that operate the engines, but also transforms Rolls- Royce’s business model from making engines to Rolls-Royce jet engine maintenance managing their entire operational lifecycle.

Ochsner Health

Ochsner Health, Louisiana’s largest non-profit hospital network, worked with Microsoft and Electronic Health Record software provider Epic to craft an AI system that predicts which hospital patients are at imminent risk of cardiac or respiratory arrest, allowing staff to intervene in time to prevent an emergency.

Ochsner Health System

M-KOPA Solar

Sometimes a radically new business model emerges that would not have been possible without new technology. Such is the case of Kenya’s M-KOPA Solar, which has sold solar panels to 600,000 rural households in Kenya, Tanzania, and Uganda. The panels provide 8 to 20 watts of power depending on the model, and are used to charge mobile phones, replace dangerous and unhealthy kerosene lamps with LEDs, and—for the 20-watt model—power small LED TVs. All this for payments averaging 50 cents per day made by an app on the customer’s mobile phone. Managing such a M-KOPA Solar vast infrastructure would not be possible without using machine learning to optimize system performance and predict customer behavior. 12 © 2019 Microsoft Corporation. All rights reserved. Benefits of strategic AI

Strategic AI has two distinguishing characteristics that the optimal window of opportunity to respond— we can illustrate with the example of Ochsner Health: too few alerts miss high-risk patients, too many produce alert fatigue, alerts sent too soon don’t 1. Strategic AI requires a large and continuing convey urgency, while those sent too late didn’t stream of data that only you have. While other allow for timely interventions. organizations have their own data, only you can exploit the unique characteristics of your data. While the benefit of strategic AI is usually expressed Ochsner knows that its patient mix is not typical in terms of competitive advantage or accelerated of all hospitals, and has therefore trained its AI growth, the examples of Ochsner Health and Kenya’s models to recognize that certain patients whose M-KOPA show that it can also take the form of vitals might seem alarming (such as a patient dramatically better human outcomes. with a flatline for blood pressure due to a heart- assistive device) are not in fact in imminent Strategic AI is developed with standard AI software danger. tools (many of which are open source) and will often be built on top of commercially available 2. Strategic AI depends not only on powerful AI cloud platforms such as Microsoft Azure or Google algorithms and unique data, but on context- Cloud Platform. What makes it strategic is not the specific human know-how embedded in your technology itself, but the hard-to-duplicate ways in organization, that is, the ability of your teams to which it leverages your organization’s proprietary continually fine-tune the business processes that information assets: your data and the know-how of AI supports. Based on their first-hand knowledge your employees. Strategic AI is not something that of the hospital’s procedures and clinical staff, can be bought off the shelf. It is something only your Ochsner’s data science team adjusted the timing organization can build. and frequency of AI alerts to give clinical staff

Action Item Ask your business unit leaders and their teams to identify the best tactical and strategic opportunities they see for AI in their operations. Then encourage them to place a balanced portfolio of AI bets, including both tactical and strategic options.

13 04 Understand the value of your data for AI

AI’s power to differentiate you from rivals comes from two things: having the right data and knowing how to use it.

AI success is all about adding value to data

Every organization has data: about customers, products and production processes, suppliers and supply chains, employees, prices, market conditions, and so on. As the digitization of business processes advances, the size of your data estate increases exponentially. Today even small businesses can accumulate vast quantities of data—if they are truly digital.

AI is a learning technology that requires data as raw Your organization is certainly creating all kinds of material to learn. Without data, there can be no AI. new data in many formats, some highly structured, But it must be the right data, available in the right others not. It can be very difficult to assess the value format at the right time. of raw, uncurated data or even keep an up-to-date inventory of it. How and where you store your data Most AI applications today rely on supervised will determine how much value you can extract from learning that requires training examples with human- it. You must ensure that your analytics and AI can get applied labels. An AI that detects skin cancer must at it usefully. The best place to put it is in the cloud. be trained with images that have been classified and labeled by human experts (a dermatologist must Consider data about customers. All businesses— say “this is cancer” or “this is not cancer”). Gathering from consumer goods companies to banks to non- large quantities of high-quality labeled data can be profits and government agencies—want to connect time-consuming and very expensive. The possession with customers. These days that means connecting of such AI-ready data is a competitive barrier and over different channels according to customer strategic asset. In medical imagery, for example, preference—web, phone, mobile, or standing in front while some labeled data sets are publicly available in of a service counter. To excel at customer connection anonymized forms, many clinical institutions jealously across all channels, you need to bring together guard such data and hope to exploit it to their own everything from website log data to mobile analytics advantage.

14 © 2019 Microsoft Corporation. All rights reserved. to point-of-sale transactions to CRM records and all can reason on top of it and create transformative the other data streams your organization generates. outcomes. Organizing your enterprise data estate for AI in a coherent way will not only help you connect Your AI-powered decision process about how to with customers, but will bring operational efficiencies connect with customers and what to say to them and even reveal new options for growth. needs access to all this data, both historically for training purposes and in real-time for each fresh “Without data, there can be no AI. But it must decision. This is both an AI problem and a data be the right data, available in the right format problem. The first step to solving it is to move your at the right time.” data estate into a unified repository in the cloud (the technical term for this is a data lake). Then you

Case Study

Good AI training data is so valuable, Walmart is paying humans to create it

One day in the not too distant future, retail giant Open by invitation only to a select set of customers Walmart hopes to offer you an AI personal shopper. in Manhattan who send in their orders mostly by This online bot will learn about your preferences, text messages, the service deploys dozens of agents listen to your instructions, converse intelligently with to go out and buy what the customers want, after you to figure out what you really want or need, then first asking follow-up questions to clarify intentions have it delivered to your door. But where to find the and offering suggestions where needed.Called training data needed to train such a bot? Today that Jetblack, this service is not intended to make money data doesn’t exist. So Walmart has taken the bold for Walmart, but only to create unique training data approach of setting up an experimental personal that can be used to guide the development of the shopping service powered by actual humans to retailer’s planned future shopping bot. create it.

Action Item Ask your CIO and your data science team to build a unified data repository in the cloud (a data lake) to feed the analytics needs and AI initiatives of your business units and departments. Measure the latter by the amount of new value they create from this unified data estate. If you determine that you don’t have the training data your AI needs, explore creative ways to get it from somewhere else or have humans create it for you.

15 05 Leverage AI partners, but also build internal AI know-how

Gaining competitive advantage with AI means learning how to do things that create new value and that others can’t do.

Combine public and private knowledge to create AI advantage

Even the largest companies don’t build their own Competitive advantage is based on something of microprocessors, design their own operating systems, value you have that others lack. Technology itself or create their own messaging and office productivity cannot bring competitive advantage—it only offers a suites. For these core technology components platform for you to build on. To create differentiating companies rely on specialized partners. In the traditional value with public technology, you must add private model for enterprise IT, companies acquired these know-how. Advantage comes when the people in an components, then built and operated their own organization know how to use a technology to build IT systems with them. In the cloud era companies a value-creating business process that few others are replacing more and more of their internal IT can match. It comes when your people know how to infrastructure with external computing and software do things others do not know how to do. Not even services such as Microsoft Azure and Web Microsoft or Google can know your business as well Services. as you do.

Basic economic logic tells us that when good standard Research from Boston University economist James solutions exist for common horizontal or vertical Bessen suggests that organizations with high levels of business problems, these solutions will usually be technology intensity become leaders in their industry. crystallized in off-the-shelf commercial offerings. It Further research from McKinsey and others confirms doesn’t make sense to reinvent the wheel. The same is that this rule is especially true of AI. Lasting business true of AI. Today the world’s leading technology firms, impact comes when you use technology to build both large and small, are creating the fundamental tools radically new and more effective business processes. for building and operating AI business applications. And getting maximum long-term value from these These firms compete to hire the best PhDs in machine new processes can only happen if they are deeply learning, sponsor the most influential open source AI embedded in your organization’s culture. That means software frameworks, and invest in the massive cloud you can’t rely solely on the skills of outsiders—you infrastructure needed to train and deploy AI. You should have to find ways to bring those value-creating not try to compete with Microsoft or Google or cutting- intellectual assets in-house and make sure they edge AI startups. It’s much more productive to partner. continue to grow.

16 © 2019 Microsoft Corporation. All rights reserved. “To exponentially increase their impact by building their own tech capabilities, companies need to invest in their human capital, so that they have a workplace culture that encourages capability-building and collaboration to spawn new, breakthrough concepts.”

“Our customers will increasingly need to build their own AI to extract insights from the ever-increasing amount of data they collect.” —Satya Nadella, Microsoft CEO

In the era of AI, you must become a learning organization

Where can your teams get the AI know-how that will allow them to create unique value? Part of it will come from the kinds of knowledge that good employee training programs provide: awareness of the latest AI research results, skill in using AI development tools, and mastery of AI production techniques (from ingesting raw data to deploying and operating robust applications). business processes to create value. Economists call this kind of knowledge “complementary knowledge.” It is We have the good fortune to live in a golden age of human knowledge that only has value in association knowledge diffusion, wherethe latest cutting-edge with a technical or business process. It’s not the AI research papers are posted online the minute machinery itself or even knowing how to design the they are finished, the most powerful AI development machinery, but knowing how to use and adjust the frameworks (such as Google’s TensorFlow or machinery to achieve maximum sustained value. Facebook’s PyTorch) are free and open source, and the world’s leading AI researchers offer in-depth Consider again Ochsner Health. The hospital relies online courses for free or, for those seeking a formal on Microsoft’s Azure cloud for its AI platform and certification, a modest fee (typically a few hundred on Epic’s EHR system to capture patient data. But dollars). Among the sites offering these courses are the hospital’s own technology leaders developed Coursera, edX, MIT Open Courseware, Udacity, fast. the vision for their new AI application over a period ai, and LinkedIn Learning. The technical knowledge of of years, and its own data scientists crafted the how to build AI is available to anyone who seeks it. machine learning model with tools provided by their Make sure your employees get it. technology partners. Finally, its medical staff devised and implemented a new working culture to maximize But the most important part of AI know-how is the system’s impact on patient care. knowing how AI works inside your organization’s

17 Action Item Build a formal AI skills training program for your internal teams on both the technology and business sides. Numerous high-quality courses in AI, machine learning, and data science are available online at little or no cost. Often taught by the world’s leading AI researchers, and sometimes offering useful certifications, these online courses are an effective method for rapidly jump-starting your organization’s internal AI culture.

Microsoft’s Shared Innovation Program

Launched in 2018, Microsoft’s Shared Innovation proposes a new strategy for the joint creation and sharing of intellectual property in the era of Digital Transformation. Under this program, when you work with our consultants and engineers to create new solutions, you own any new patents and design rights that result from our collective innovation. And the license back to Microsoft is limited to use in our platform technologies. If we develop source code together and you want to contribute that to an open source project, we’ll work with you to do that.

An example of Shared Innovation at work is our partnership with American retailer Kroger to develop transformative digital applications for retail stores based on the cloud and AI. Known as Retail as a Service, the solution will be jointly marketed by Kroger and Microsoft.

18 © 2019 Microsoft Corporation. All rights reserved. 06 Build AI on a foundation of trust

Your customers need to trust your AI and you need to trust your AI technology partners.

AI is so powerful it requires a new kind of trust

With great power comes great responsibility. We’ve concerned with corporate governance should ponder: seen examples of what AI can do. On the basis of those examples, we’ve argued that AI will be the ●● Safety. An AI that makes wrong predictions defining business technology of the 21st century. can cause accidents. Self-driving cars are an But a tool this powerful can do harm as well as good. obvious case. But remember: as Harvard Law’s Wielding it safely requires a new kind of trust, a trust Cass Sunstein argues, the right standard by that faces in two different directions at the same time: which to judge AI is not perfection (zero errors), but whether it actually does a better job than ●● On the one hand, you must earn the trust of humans in the same situation. When self-driving your stakeholders that you will use AI for cars are mature enough to emerge from their their benefit and will protect them from possible current experimental stage, they will certainly not collateral harms; be accident free. But if they nevertheless prove significantly safer than human drivers, they will ●● On the other hand, your AI technology partners save many lives. must also earn your trust and justify your confidence that AI will not harm your interests. ●● Bias and unfairness, especially toward groups who have historically suffered discrimination. The two sides of AI trust are as indissociable as An AI algorithm cannot “intend” to discriminate, the two sides of a mathematical equation. You because it is simply an algorithm. But when AI cannot have one without the other. If you cannot is trained on data that reflects historical biases trust the partners who help you build AI, your own in society, it can produce unfair results. For stakeholders will not be able to trust your AI. example, it is now widely understood that when face recognition systems are trained mostly on “A tool this powerful can do harm as well photos of white men, they have high error rates as good. Wielding it safely requires a new for women and people of color. These errors kind of trust.” can be corrected by paying proper attention to the quality of training data, but this requires management vigilance and most likely some extra What are some of the harms that poorly controlled cost. AI might cause? Here are two examples that Boards

19 Managing AI risks

When managing AI risks, enterprise leaders must Finding the right balance between no adjustment understand that they have skin in the game. Often for past discrimination and too much adjustment is there is no zero-risk option. For example, a bank that a delicate question that cannot be entrusted to AI fails to use AI to evaluate loan opportunities will lose alone. Ultimately the choice must be made by leaders business to rivals or make bad bets that compromise who listen to the data, continually reassess both the bank’s viability. But because the AI loan system algorithms and outcomes, and accept responsibility must be trained on past data, its decisions may before stakeholders for the decisions taken. unintentionally perpetuate historical discrimination against certain classes of applicants. Fairness An AI application that fails on safety or fairness can suggests adjusting the algorithm to counteract such do great damage to your organization’s reputation, discrimination. But recent work by AI researchers at as well as leading to unpleasant legal consequences. Berkeley shows that this may result in loans being Minimizing these risks requires proactive made to applicants who can’t repay them, thus doing management of your AI by people who understand the recipients more harm than good. what causes AI systems to fail and who have the authority to intervene.

No enterprise can carry the burden of responsible AI alone. Just as you owe an accounting of AI’s safety, ethical, and social effects to your stakeholders, the technology partners who help you build AI owe you a similar accounting.

“At a time when digital technology is transforming every industry and every part of our daily life and work, our customers are increasingly looking for a partner whose business interests are fundamentally aligned with their own. At Microsoft, our customers’ interests are core to our success. That is what engenders trust.” Satya Nadella, Microsoft CEO

Action Item Evaluate your technology partners carefully and insist that they meet the same high standards of AI managed for the benefit of all.

20 © 2019 Microsoft Corporation. All rights reserved. Principles for ethical AI

Microsoft’s leaders have committed the company to a systematic and sustained effort to ensure that our AI technology is worthy of your trust. To that end, we have adopted six core principles for ethical AI:

● Fairness: AI systems should treat all people fairly ● Inclusiveness: AI systems should empower everyone and engage people ● Reliability and Safety: AI systems should perform reliably and safely ● Transparency: AI systems should be understandable ● Privacy and Security: AI systems should be secure and respect privacy ● Accountability: AI systems should have algorithmic accountability

For more information, download a copy of the Microsoft book The Future Computed by two of our most respected senior executives, Brad Smith, President and Chief Legal Officer, and Harry Shum, Executive Vice President of Microsoft AI and Research Group.

21 07 Learn how to make AI fair

AI by itself doesn’t create injustice but it can sometimes reproduce human bias.

Why AI can sometimes be biased

AI is powerful, but it is not perfect. Sometimes its remarkable ability to detect patterns in data leads it to reflect or even amplify past instances of discrimination in human society. Enterprises deploying AI need to understand why AI sometimes discriminates and take proactive steps to detect and mitigate such behavior. Here we consider two kinds of AI discrimination and how to respond to them.

Discrimination due to bias in training were able to fix the problem by boosting the gender data and skin tone balance of faces in their training data.

In 2018 researchers from MIT and Microsoft published The moral of this story is that AI requires careful a troubling finding about AI software that analyzes human supervision. When a technology as powerful as human faces. They discovered that the online facial face recognition spreads into every corner of society analysis services provided by some of the biggest and the economy, as is happening today, we need to names in the industry—including Microsoft and think carefully about how it will be controlled. This is IBM—had higher error rates on dark-skinned women why Microsoft and others are calling for government than on light-skinned men.The good news is that regulation of face recognition AI. both Microsoft and IBM reacted quickly. Within a few months, both had rolled out improved services that scored dramatically better on minorities, especially on darker-skinned women.

Thanks to deep learning, accuracy in face recognition has improved dramatically in recent years, rising from around 92% on standard benchmarks in 2012 to better than 98.8% today. The problem with the biased versions of the IBM and Microsoft facial analysis services is that they were trained on data sets that did not contain a sufficiently representative sample of the Face recognition systems trained on samples that don’t reflect the diversity of human appearance have unacceptably high full diversity of human faces. Once the developers of error rates. Fortunately, such errors can be corrected with these services learned of their skewed results, they properly balanced training data.

22 © 2019 Microsoft Corporation. All rights reserved. Discrimination due to bias in reality

But not all forms of AI discrimination can be fixed What can we do to correct the injustices that arise when simply by gathering more representative training data. AI algorithms unwittingly update the old computing In another recent example that received wide media paradigm of “garbage in, garbage out” to “bias in, bias coverage, a large Internet firm sought to develop an out”? It turns out that research on how we can fix AI algorithm to scan resumes posted on the web and bias it is a burgeoning academic field, with important pick the top 5% of candidates. The goal was to speed new papers appearing frequently. Researchers have up hiring in a job market where talent is at a premium. identified policies that humans can layer on top of AI to But the firm found the algorithm was systematically compensate for historical bias. Cornell’s Jon Kleinberg rejecting female candidates. It tweaked the algorithm gives the example of the NFL’s so-called Rooney rule, to compensate for this bias, but realized it couldn’t be named for the former Pittsburg Steeler’s owner who certain no other forms of bias would arise in the system, proposed it. Adopted in 2002 to address the League’s and so decided to shelve the project. lack of African-American head coaches, the rule does not establish quotas, but merely stipulates that in every The contrast between these two examples of apparent job search the final short list of candidates must contain AI bias is instructive. In the face recognition case, at least one African-American. Kleinberg develops a algorithms made more mistakes when looking at groups mathematical model to show that, regardless of the they hadn’t seen during training. The fix was self-evident outcome in any individual hiring decision, this simple and straightforward to implement. rule can have the long-term effect of increasing the But in the case of the biased job candidate algorithm, a quality of the total talent pool. It does so by teaching more insidious kind of bias is at work, one that is harder talented members of groups who have suffered to correct. Simply put, the problem here is not bias in discrimination in the past that there is value in persisting. the algorithm’s training data, but bias in reality itself. The Individuals who might previously have been hesitant to hiring algorithm learned that in the past most candidates invest the effort needed to perfect their skills and pursue who were hired were men and concluded that successful their ambitions now see that they have a chance and candidates in the future were also likely to be men. This invest accordingly. problem is harder to fix than in the face recognition case. Once we realize that fair and ethical AI requires close It isn’t possible to increase the number of successful human supervision, it becomes apparent that we are female candidates in historical data used for training going to need a whole new set of regulations and AI if they don’t exist in reality. The algorithm here has ethical principles to govern this evolving relationship faithfully modeled a world where women interested in between people and our intelligent machines. Building technical careers have been discouraged by decades of this framework cannot be the task of legislators and discrimination, overt or implicit, and are consequently the tech sector alone. That is why Microsoft has made less numerous than they could be and should be. a strong commitment to work with stakeholders across the economy and around the world to ensure that we “The problem here is not bias in the algorithm’s develop the right policies for the brave new world of AI- training data, but bias in reality itself.” assisted work and living we are entering.

Action Item Require your AI teams to develop formal procedures for recognizing and correcting bias in every AI application you deploy that makes judgements affecting people.

23 08 AI regulation is coming—plan for it

Regulating powerful new technologies like AI is normal and nothing to be afraid of.

AI regulation will be a good thing

In just a few short years—and with very little public debate or regulation—AI has crept into many corners of modern society. But the era of unregulated AI is probably coming to an end, and that will be a good thing. Enterprises deploying AI and their many diverse stakeholders can only benefit from sensible government regulations that reduce the risk of AI being misused or abused.

Consider the example of face recognition. It is arguably AI’s most visible success story to date, and it Police in the U.S. and elsewhere are using face is most likely where AI regulation will begin. recognition to identify suspects caught on surveillance Many of face recognition’s applications are quite cameras during crimes. Some firms that make police wonderful. In India, New Delhi police have located body cameras are even looking at bringing real-time thousands of missing children with it. In the U.S. face recognition capabilities to cops on the beat. medical researchers are using it to diagnose rare Disturbingly, some countries governments are using diseases in children. National Australia Bank is testing face recognition to implement non-stop surveillance ATMs that recognize your face instead of asking of their citizens. you for a bank card. Dulles Airport near Washington DC is experimenting with a system that uses face recognition instead of checking passports for “We should welcome sensible government passengers boarding international flights. regulations that reduce the risk of AI being misused.” But there are also more controversial applications. Some retail stores are starting to use face recognition on customers, seeking to identify not just potential big spenders, but also suspected shoplifters.

24 © 2019 Microsoft Corporation. All rights reserved. What AI regulation should do But what kinds of regulation of face recognition make sense? Smith has some suggestions: Face recognition AI is a tremendously powerful ● Make the technology understandable to the technology that can be used for many purposes. And public and open to independent review. Firms that is why we at Microsoft are calling for the United that provide face recognition technology should States and other countries to consider reasonable explain how it works and what its limits are regulation. As Microsoft President Brad Smith puts it: (these may change over time as the technology improves). They should also let neutral third “The facial recognition genie is just emerging parties test it to check for bias or inaccuracy. from the bottle. Unless we act, we risk waking ● Make sure people know when they are being up five years from now to find that facial identified by face recognition—for example in recognition services have spread in ways that public places like retail stores—and consider exacerbate societal issues. By that time, these how to give them some choice in the matter. challenges will be much more difficult to bottle back up.” ● Make sure humans can intervene when an automated face recognition system makes a mistaken identification or even a correct but high-stakes decision for an individual. For example, if a retail face recognition system decides that someone is a known shoplifter, that person should be able to challenge the identification before being dragged off to jail.

● Don’t use the technology to discriminate against people or as a tool of political repression. Technology has come close to making feasible the dystopian world imagined by George Orwell in his novel 1984. But such a world is far from inevitable, and we should do all we can to make sure it never comes to pass.

The call for sensible regulation

AI regulation will eventually cover much more than just face recognition. Support is growing around the world for the idea that privacy is a fundamental human right. Europe has already passed its sweeping General Data Protection Regulation that provides many privacy guarantees against unwanted surveillance by automated technology, including strong requirements for consent, explanations, and human intervention. The U.S. lacks a comparable privacy law, but there appears to be an emerging consensus in Congress and the public that such a law would be desirable.

Enterprises banking their future on AI should welcome the call for sensible regulation of this powerful technology. Better still, they should work with providers of this technology like Microsoft to shape appropriate rules and legislation.

25 Action Item Enterprise leaders should educate themselves and their teams about proposals for AI regulation currently under discussion in the U.S., Europe, and elsewhere. Where appropriate, they should make their own views known to regulators and legislators.

26 © 2019 Microsoft Corporation. All rights reserved. 09 Manage AI’s impact on jobs for the benefit of all

We must help the workforce acquire the new skills needed in the AI era.

AI will make and unmake jobs

In the era of AI and advanced robotics, many fear these innovations will create a world where only an educated elite remains employable while the rest of humanity is condemned to idleness and poverty. Technology optimists respond by pointing to the many past instances where a new technology displaced older kinds of work but without devastating effects—for example, therise of the automobile in the early 20th century that ended the world of horse-drawn transportation. But pessimists insist that this time will be different.

Impact on employment

This debate is not going to end any time soon. But another lesson of history is that the acquisition of those who hope to benefit from AI must address new skills by large masses of individuals at risk of legitimate fears about its impact on employment and unemployment is not something that happens by take thoughtful measures that favor more positive itself. On the contrary, significant changes in public outcomes. policy and substantial new investments in human capital are essential. The heart of the debate about AI’s impact on employment is the question of human skills and “The acquisition of new skills by large masses their evolution in response to automation. History of individuals at risk of unemployment is not shows that every major wave of automation in the something that happens by itself.” past has ended up creating new kinds of jobs far outnumbering the old kinds it made obsolete. But

27 MIT economist David Autor cites a compelling historical What happened to all the farm workers whose labor example of how innovative public policy once facilitated became redundant in the early decades of the 20th a smooth transition from one era of work to another for century? Most of them moved to cities and got jobs millions of Americans. At the start of the 20th century, as factory workers or office clerks. These jobs paid nearly 40% of Americans worked on farms. In the next substantially more than the subsistence wages earned few decades farm automation caused this number to on farms and led to an entirely new lifestyle of mass plunge precipitously and today it has fallen to less than consumption that in turn powered more innovations, 2%. Yet in the 21st century this much smaller share of the more economic growth, and more jobs. workforce devoted to agriculture is more than enough to feed America’s population of 325 million people and still export large quantities of food to other countries.

Public and private stakeholders must invest in education

And how did those millions of farm girls and farm Just as we did during the high school era, to meet the boys acquire the skills needed to succeed in these challenge to employment that automation raises today new kinds of work? Quite simply, they went to high we must now invest in new forms of education that school. offer the skills needed to thrive in the AI economy.

The United States was the first country in the world to make high school education mandatory for all citizens through age 16. During the first half of the 20th century, local school districts all over the country competed to build public high schools that were free and accessible to all children. Here it must be said that in those times “all children” in reality often only meant “all white children,” a grievous historical wrong whose baleful consequences we still live with today. But for the majority, the change was dramatic. In 1910 only 18% of Americans aged 15 to 18 were enrolled in high school. By 1940 this number had reached 73%. By 1955, 80% of American youth were graduating from high school.

Automation’s potential to increase the value of human work

If your enterprise is embarking on—or is already be tempted to understate their impact on jobs. It’s well engaged in—digital transformation, you wiser to acknowledge that the goal of automation have probably already considered its impact on is to increase productivity by using less human the current and future shape of your workforce. work to perform certain tasks. When you undertake automation projects, don’t

28 © 2019 Microsoft Corporation. All rights reserved. Starting from the earliest planning stages of the project, any foreseeable future. But it’s clear that we are going explore the automation’s potential to increase the to need something like a modern version of the high business value of human work complementary to the school movement of the early 20th century if people tasks being automated. Rank proposed automation are to take advantage of the new kinds of work that projects not by how much labor cost they can save, but the AI economy makes possible. That means we will by how much new value they can add in your business need massive new investment in education, both the processes. Think of bank tellers displaced by ATMs traditional kind that takes place in formal institutions becoming relationship bankers who sell new services that inhabit physical buildings, and the new kind that instead of counting dollar bills. takes place online or on the job. This investment will have to come from both governments and employers. The question is not whether AI and robots will bring As Microsoft CEO Satya Nadella so often says, “The about the end of work. They will not, at least not in future we invent is a choice we will make.”

Action Item Undertake a careful inventory of the current skills of your employees affected by automation and compare them to the skills that will add the most complementary value to the automated tasks. If there is a gap between the two skillsets, take proactive measures to fill the gap, for example by establishing new training and reskilling programs for current employees and future hires. At the same time, think beyond the boundaries of your own enterprise by sponsoring educational programs and policies that build the new skills needed in this era of digital transformation.

Closing the computer science gap

Learning computer science empowers young people to compete in the global economy and pursue careers across all sectors because it teaches students computational thinking and problem-solving skills applicable in any industry. Students want to learn computer science, yet most high schools are unable to offer rigorous CS courses.

Microsoft Philanthropies TEALS (Technology Education and Literacy in Schools) helps high schools throughout the US and British Columbia, Canada build and grow sustainable computer science programs. In its proven High school students describe “computer science” in three words program, TEALS pairs trained computer science professionals from across the technology industry with classroom teachers to team-teach computer science. Industry volunteers and partner teachers create a ripple effect, impacting the students they teach, and the many students who will study CS in the future.

29 10 Plan for creative destruction and do the right thing

“We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten. Don’t let yourself be lulled into inaction.” —Bill Gates

Some turbulence ahead

Digital transformation powered by AI and the AI and digital transformation are going to drive cloud is only just getting started. Over the next tremendous new economic growth and create few decades it will utterly transform the global much new wealth. We will need to make sure that economy and human society. Every aspect of our this new wealth from AI benefits all—especially the lives, our homes, our cars, our workplaces, our poorest and least advantaged among us. The debate factories, and cities—every sector and industry will is only just beginning about the new policies we be transformed by digital technology, whether it’s must build to ensure that the fruits of AI wealth are precision agriculture, precision medicine, precision distributed fairly. Some argue for a universal basic manufacturing, new drugs and medical devices, income or a negative income tax for low-wage work. scientific discoveries made possible by the power of Others emphasize education and investment in the cloud, autonomous cars, personalized education, infrastructure, both physical and social. And much personalized banking, or—last but not least— work remains to be done to correct injustices of the amazing new forms of entertainment and leisure. past.

The ride won’t always be a smooth one. We should As the gales of creative destruction driven by expect some bumps. The development of AI solutions technology innovation blow across the landscape won’t always be consistent across sectors—some of global business, it is our duty and within our will move faster than others, for both technical and power to ensure that these powerful forces are institutional reasons. corralled and harnessed for the good of the world’s population. Some firms and perhaps even entire As AI solutions advance, we will need to take care to industries may not survive the storm. But the total build the right legal and regulatory frameworks that wealth of human society will increase, and there protect the fundamental rights of all humans. This is no reason why this wealth cannot be equitably work is just beginning, and it is time for policymakers shared by all. to pick up the pace.

30 © 2019 Microsoft Corporation. All rights reserved. Technology innovation is shortening the life span of large firms

The average lifespan of even large and successful corporations is being shortened by the waves of creative destruction that technology innovation is provoking. The 33-year average tenure of companies on the S&P 500 in 1964 narrowed to 24 years by 2016 and is forecast to shrink to just 12 years by 2027.

“The opportunity ahead in a world powered by an intelligent cloud and edge is unprecedented. Imagine a future where all of your apps and experiences revolve around you and transcend any single device; where data in any form is analyzed in real time so that computers can anticipate and even act on your behalf and augment what you would otherwise be able to accomplish on your own. And where computing is more distributed and embedded in the world, from intelligent digital assistants at work, on the go and in your home that you can communicate with in a myriad of ways—voice, eyes or gestures—to factories that adjust production in real time as demand fluctuates in global markets.” —Satya Nadella, Microsoft CEO

Action Item Business organizations stand to be conspicuous beneficiaries of the AI revolution. Start planning now the actions you and your organization will take to share the benefits you reap from AI with the world’s least fortunate citizens. Many opportunities exist to invest in charitable actions that can make a real difference in peoples’ lives. Take advantage of them.

31 AI for Good

Microsoft AI for Good is a corporate philanthropic program designed to address critical societal issues in three areas. Together these programs represent an investment of $115 million over 5 years spread across hundreds of grants to local groups around the world:

AI for Earth: The cloud and AI can help us understand how the environment is changing and develop steps to better manage natural resources. Through AI for Earth, we support work that focuses on issues related to climate, water, agriculture, and biodiversity.

AI for Accessibility: For the 1 billion people around the world with disabilities, AI can provide new levels of independence. AI for Accessibility offers technology seed grants to support promising initiatives that will assist people with disabilities.

AI for Humanitarian Action: Our newest program puts AI in the hands of nonprofits and international organizations working to tackle disaster relief, child welfare, human rights, and assist refugees and displaced people.

32 © 2019 Microsoft Corporation. All rights reserved. anonymized personal data (such as radiology images or personal health records). For more information about AI and GDPR

The Future Computed: Artificial Intelligence Partnership on AI and Its Role in Society The Partnership on AI was established A short non-technical book discussing AI’s to study and formulate best practices on promise and challenge for modern society, by AI technologies, to advance the public’s Microsoft’s President Brad Smith and top AI understanding of AI, and to serve as researcher Harry Shum Link an open platform for discussion and engagement about AI and its influences on Facial Recognition: It’s Time for Action people and society. Link Microsoft President Brad Smith makes the case for careful regulation of AI-based Microsoft AI facial recognition and an ethical approach A more technical introduction to Microsoft’s by governments and corporations to all AI AI platform. Link technologies. Link The AI blog Digital Transformation in the Cloud: What The latest news about AI progress from enterprise leaders and their legal and Microsoft. Link compliance advisors need to know AI Insight Series: A Short Introduction to A non-technical book that lays out the Artificial Intelligence business case for digital transformation and An introduction to AI for legal and explains the foundational requirements of compliance professionals. Link security, privacy, and compliance. Link AI Insight Series: Ethical Considerations A Cloud for Global Good for the Use of AI Microsoft’s policy considerations and The ethical issues around AI for legal and recommendations for creating a framework of compliance professionals. Link laws that extend the benefits of the cloud to all. Link AI Insight Series: AI and the GDPR • targeted online advertising Challenge “Transformation Tuesday” blog series The future of AI under Europe’s General • directA series marketing of blog actions posts suchby Michael as emails sent to Data Protection Regulation. Link selectedcustomersMcLoughlin—Microsoft or prospects on digital transformation, AI, and cloud computing. Link • customized price or product offers made to specific customers based on their profile

• evaluation of individual online applications for employment, loans, insurance contracts,For further health information, contact: benefits, or university admission Michael McLoughlin Director, Public Affairs • certain kinds of scientific research or medicalMicrosoft Corporation diagnosis that involve the analysis of non-