Special Issue Editorial. Artificial Intelligence in Organizations: Current State and Future Opportunities

Home , Superintelligence

MIS Quarterly Executive

Volume 19 Issue 4 Article 4

December 2020

Hind Benbya

Thomas H. Davenport

Stella Pachidi

Follow this and additional works at: https://aisel.aisnet.org/misqe

Recommended Citation Benbya, Hind; Davenport, Thomas H.; and Pachidi, Stella (2020) "Special Issue Editorial. Artificial Intelligence in Organizations: Current State and Future Opportunities," MIS Quarterly Executive: Vol. 19 : Iss. 4 , Article 4. Available at: https://aisel.aisnet.org/misqe/vol19/iss4/4

This material is brought to you by the AIS Journals at AIS Electronic Library (AISeL). It has been accepted for inclusion in MIS Quarterly Executive by an authorized administrator of AIS Electronic Library (AISeL). For more information, please contact [email protected]. Editors’ Comments

Editors’ Comments Special Issue Editorial: Artificial Intelligence in Organizations: Current State and Future Opportunities considerable positive impact on company growth and profitability. Introduction A 2020 Deloitte survey of executives revealed5 that AI is currently being applied in organizations to support the following diverse objectives: making processes more efficient (28%), Artificial intelligence (AI) is typically enhancing existing products and services (25%), defined as the ability of machines to perform creating new products and services (23%), human-like cognitive tasks, including the improving decision making (21%) and lowering automation of physical processes such as costs (20%). Although reducing headcount is a manipulating and1 moving objects, sensing, common objective cited in AI-oriented press, it perceiving, problem solving, decision making was mentioned least in this survey (11%). and innovation. AI2 is currently viewed as the Executives initially focused on using AI most important disruptive new technology for technologies to automate specific workflow large organizations. However, the technology is processes and repetitive work. Such processes still in a relatively early state3 in large enterprises were linear, stepwise, sequential and repeatable. and largely absent from smaller ones other than Currently, however, firms are moving toward technology startups. Surveys suggest that fewer nonsystematic cognitive tasks that include than half of large organizations have meaningful decision making, problem solving and creativity, AI initiatives underway, although the percentage which, until recently, seemed beyond the is increasing over time. scope of automation. AI technologies are also For most organizations, AI projects remain progressively enabling people and machines somewhat experimental—undertaken as a pilot to work collaboratively in novel ways. In or proof-of-concept initiative. Relatively few manufacturing, for example, in order to fulfill organizations have deployed AI on a production customized orders and handle fluctuations in basis, a problem that we describe in greater demand, employees are partnering with robots detail below. Of course, the experimental to perform new tasks without having to manually nature of its use means that many organizations4 overhaul any processes. AI technologies are also have achieved little or no economic return on performing certain tasks autonomously, though their AI investments. However, some analysts complex tasks like driving a car in all conditions suggest that AI adoption will eventually have a remain tantalizingly out of reach. 1 Innovation is defined here as the design, creation, development and/or implementation of new or altered products, services, systems, However, autonomous systems are beginning organizational structures, management practices and processes, or to appear that can perform tasks without any business models; see Benbya, H. and Leidner, D. (2018) “How Alli- human involvement at all because systems are anz UK Used an Idea Management Platform to Harness Employee In- novation,” MIS Quarterly Executive (17:2), 2018, pp. 141-157; Yan, capable of training themselves and adjusting J. Leidner, D. and Benbya, H. “Differential Innovativeness Outcomes to new training data. Consider automated of User and Employee Participation in an Online User Innovation financial trading: because it depends entirely on Community,” Journal of Management Information Systems (35:3), 2018, pp. 900-933. algorithms, companies can complete transactions 2 “Big Data and AI Executive Survey 2019, Executive Summary of much more quickly with AI systems than Findings,” NewVantage Partners, 2019, https://newvantage.com/wp- with systems relying on humans. In a6 similar content/uploads/2018/12/Big-Data-Executive-Survey-2019-Findings- Updated-010219-1.pdf fashion, robots are performing narrow tasks 3 “AI 360: Hold, Fold, or Double Down,” Genpact, 2020, https:// 5 autonomously “Thriving in the Erain manufacturingof Pervasive AI: Deloitte’s settings. State of AI in the www.genpact.com/uploads/files/ai-360-research-2020.pdf Enterprise, 3rd Edition,” Deloitte Insights, Deloitte, 2020, https:// 4 “Notes from the AI Frontier: Modeling the Impact of AI on the www2.deloitte.com/us/en/insights/focus/cognitive-technologies/state- World Economy,” McKinsey & Co., 2018, https://www.mckinsey. of-ai-and-intelligent-automation-in-business-survey.html com/featured-insights/artificial-intelligence/notes-from-the-AI-fron- 6 Davenport, T., & Ronanki, R., Artificial Intelligence for the Real tier-modeling-the-impact-of-ai-on-the-world-economy# World. Harvard Business Review (96:1), 2018, 108-116.

December 2020 (19:4) | MIS Quarterly Executive ix Editors’ Comments

Companies such as Amazon and Google at Dartmouth College in 1956. The program have attempted to create highly ambitious aimed to study the possibility that machine applications of AI, including autonomous intelligence could imitate humans and involved vehicles, unattended retail checkout and drone researchers from various fields including delivery. Some of these “moon shots” have been scientists, mathematicians and philosophers. successful, but some highly ambitious projects, Despite early promises of the practical including cancer treatment, have been largely usefulness of AI, it largely failed to deliver and unsuccessful thus far despite considerable faced several obstacles during the 1960s and expenditures . Less ambitious “low-hanging fruit” 1970s, the biggest of which was the lack of projects have been more successful in many computational power to do anything substantial. firms and are perhaps more consistent with the Research funding gradually stalled and the current narrow intelligence of AI systems. field lost momentum. During the 1980s and Likewise, most autonomous AI applications 1990s, governments and firms made significant remain limited to low-risk areas with limited investments in research on expert systems, costs associated with failure. Although many which revived interest in AI. Machine learning AI systems can do certain things better than and neural networks began to flourish as humans, workers’ trust in AI technology is practitioners integrated statistics and probability still limited because of the issues associated into their applications. At the same time, the with this technology, such as algorithmic bias, personal computing revolution began. Over unexplainable outcomes, privacy invasions and/ the next decade, digital systems, sensors, and or lack of accountability. Consumers are also the internet proliferated, providing all kinds of skeptical about AI and surveys suggest that most data for machine-learning experts to use when or many would not wantMIS autonomous Quarterly Executive vehicles, training adaptive systems. Although the growth disklike dealing with chatbots and so forth. of AI and machine learning has been intermittent This December 2020 over the decades, unprecedented computing special issue is titled “AI in Organizations: capacity and growing volumes of data have Current State and Future Opportunities.” It provided momentum for the recent development details current challenges and implications ofAI artificial Types intelligenceand technologies applications. that may arise from AI applications and ways to overcome such challenges to realize the potential of this emerging technology. The collection of There are many types of AI systems. One papers in this issue (December), combined typology differentiates AI systems based on with a forthcoming (March) article, offers the kind of intelligence they display. A second insights to managers currently implementing typology distinguishes AI applications based digital transformation initiatives driven by on the type of technology embedded into the AI AI technology, to practitioners considering system,Based whereas on a thirdintelligence: is based on the functions implementing AI in their businesses and to performed by the AI. research-oriented faculty and students. In this Philosophical editorial, we first provide a brief history of AI debates on AI are centered on the 7 notion of and an overview of AI typologies. We discuss intelligent machines—that is, machines that the current challenges, implications and future can learn, adapt and think like people. AI types opportunities of AI so that readers are better based on such a notion fall, in general, into equipped to understand the five papers in the three categories: artificial narrow intelligence, special issue. Finally, we summarize the special artificial general intelligence and artificial issue articles and highlight the contributions superintelligence. each makes.Brief History of AI While narrow (or weak) AI is usually able to solve only one specific problem and is unable to transfer skills from domain to domain, general 7 AI Lake,aims B., for Ullman, a human-level T., Tenenbaum, skillJ. and set.Gershman, Once J. “Buildinggeneral AI as an academic field dates back to the Machines that Learn and Think Like People,” Behavioral and Brain 1950s. The term AI was first introduced Sciences (40), 2017, e253 during a multidisciplinary program presented

Table 1: AI Technologies and Domains of Application

Technology Brief Description Example Application

Learns from experience. Highly granular marketing analyses Machine learning Learns from a set of training data. on big data. • Reinforcement learning Detects patterns in data that are not • Supervised learning labeled and for which the result is • Unsupervised learning not known. A class of machine learning that Image and voice recognition, self- learns without human supervision, driving cars. Deep learning drawing from data that is both labeled and unlabeled. Algorithms that endeavor to Credit and loan application recognize underlying relationships in evaluation, weather prediction. Neural networks a set of data through a process that mimics the way the human brain operates. A computer program able to Speech recognition, text analysis, Natural language processing understand human language as it is translation, generation. written or spoken. A set of logical rules derives from Insurance underwriting, credit Rule-based expert systems human experts. approval Systems that automate structured Credit card replacement, validating Robotic process automation digital tasks and interfaces. online credentials. Automatically operated machines Factory and warehouse tasks. Robots that automate physical activity, manipulate and pick up objects.

AI is achieved, it is believed that it might lead provided estimates and outline scenarios for to superintelligence that exceeds the cognitive when technological growth will reach the point performance8 of humans in virtually all domains of singularity at which machine intelligence of interest. This type of superintelligence9 could will surpass human intelligence. This raises emerge following evolutionary and complex philosophical arguments about the mind and the adaptive systems principles. If humans could ethics of creating artificial beings endowed with create AI intelligence at a roughly human level, human-like intelligence. Although the futuristic so the argument goes, then this creation could, literature assumes that AI systems will be able to in turn, create yet higher10 intelligence and perform all tasks as well as or even better than eventually8 Bostrom, N. evolve(2014). Superintelligence further. AI Paths, enthusiasts Dangers, Strate have- humans, this type of artificial general intelligence gies. Oxford University Press. does not yet exist. There are, however, some AI 9 See Benbya, H. Nan, N., Tanriverdi, H. and Yoo, Y. “Complex- 11 ity and Information Systems Research in the Emerging Digital programs, such as the GPT-3 language prediction World,” MIS Quarterly (44:1), 2020, pp. 1-18, for a recent article on application,Based on that technology: are beginning to exhibit some evolutionary principles, and Benbya, H. and McKelvey, B. “Using aspects of general intelligence. Coevolutionary and Complexity Theories to Improve IS Align- A second typology ment: A Multi-Level Approach,” Journal of Information Technology (21:4), 2006, pp. 284-298 for an elaboration of such principles in IT 11 differentiates GPT-3 stands for between generative pre-trained the technologies transformer version that management. three. It is a powerful machine-learning system that can rapidly gen- 10 Hawking, S., Russell, S., Tegmark, M., & Wilczek, F. “Stephen erate text with minimal human input. After an initial prompt, it can Hawking: Transcendence Looks at the Implications of Artificial recognize and replicate patterns of words to work out what comes Intelligence: But Are We Taking AI Seriously Enough?,” The Inde- next, see Thierry, G. “New AI Can Write Like a Human but Don’t pendent, May 5, 2014, https://www.independent.co.uk/news/science/ Mistake that for Thinking, The Conversation, September 17, 2020, stephen-hawking-transcendence-looks-implications-artificial-intelli- https://theconversation.com/gpt-3-new-ai-can-write-like-a-human- gence-are-we-taking-ai-seriously-enough-9313474.html. but-dont-mistake-that-for-thinking-neuroscientist-146082

December 2020 (19:4) | MIS Quarterly Executive xi Editors’ Comments

are embedded into AI systems, which include uses applications such as facial recognition, machine learning (and its subclasses deep speech recognition and computer vision for learning and reinforcement learning), natural identification, authentication, and security language processing, robots, various automation objectives in computer devices, the workplace, technologies (including robotic process home security, etc. While fingerprints have automation) and rule-based expert systems12 (still the longest history14 as a marker of identity and in broad use although not considered a state-of- continue to be used in a number of applications the-art technology). One recent survey suggests across the world, other bodily markers, such as that all contemporary AI technologies (machine the face, voice and iris or retina, are proliferating, learning, deep learning, natural language with significant research exploring their processing) are either currently being used or potential large-scale application. Meanwhile, the will be used within a year by 95% or more of ubiquity of facial images and voice recordings large adopters of AI. Table 1 below provides tagged with people’s names on the internet, briefBased definitions on andfunction: the domain of AI technology alongside algorithms to transform such data into applications. biometric recognition systems, has accelerated This distinction the use of such data at a larger scale—for differentiates between four types of AI: example, to identify suspects, monitor large conversational, biometric, algorithmic and events and surveil protests. Such large-scale use robotic. These categories overlap somewhat; for has triggered calls for regulation to introduce example, conversational and biometric AI already newAlgorithmic laws, reform AI existing laws, or ban the use of makeConversational extensive use AIof algorithmic AI models, and such data in some contexts. robotic AI is increasingly doing so as well. revolves around the use of refers to the general machine learning (ML) algorithms—a set of capability of computers to understand and unambiguous instructions that a mechanical respond using natural human language. Such computer can execute. Some ML algorithms can systems include both voice- and text-based be trained on structured data and are specific to technologies and vary largely based on their narrow task domains, such as speech recognition capability, domain and level of embodiment. and image classification. Other algorithms, Simple conversational AI is mainly used to especially deep learning neural networks, handle repetitive client queries whereas can learn from large volumes of labeled data, smart conversational AI, enabled by machine enhance themselves by learning, and accomplish learning and natural language processing, has a variety of tasks such as classification, the potential to undertake more complex tasks prediction and recognition. For example, neural that involve greater interaction, reasoning, networks can analyze parameters of bank clients prediction and accuracy. Conversational AI has such as age, solvency and credit history, and been used in many different fields, including decide whether to approve loan requests. Such finance, commerce, marketing, retail and networks can also use face recognition to let healthcare. Although the technology behind only authorized people into a building or predict smart conversational agents is continuously outcomes such as the rise or fall of a stock based under development, they currently do not have on past patterns and current data. Despite the full human-level13 language abilities, sometimes potential of ML algorithms, there are concerns resultingBiometric in AI misunderstanding and user that, in some cases, it may not be possible to dissatisfaction. explain how a system has reached its output. : Biometrics relies on techniques SuchRobotic algorithms AI may also be susceptible to to measure a person’s physiological (e.g., introducing or perpetuating discriminatory bias. fingerprints, hand geometry, retinas, iris, facial : Physical robots have been used image) or behavioral traits (e.g., signature, voice, for many years to perform dedicated tasks in 12 keystroke “Thriving in rhythms). the Era of Pervasive AI-powered AI,” op. cit., 2020. biometrics factory automation. Recently, AI (including ML 13 “What is a Chatbot? All You Need to Know About Chatbots!” 14 Amba K., ed. “Regulating Biometrics: Global Approaches and Botpress: Open-Source Conversational AI Platform, 2018, https:// Urgent Questions,” AI Now Institute, September 1, 2020, https:// botpress.io/learn/what-and-why/ ainowinstitute.org/regulatingbiometrics.html.

16 and NLP) has become increasingly present in reported using AI in products or processes robotic solutions, enabling robots to move past across multiple business units and functions. automation and tackle more complex and high- In order to address such deployment level tasks. AI-enabled robots are equipped concerns, companies need to plan for the with the ability to sense their environment, possibility of deployment from the beginning. comprehend, act and learn. This helps robots Some companies, such as Farmers Insurance, perform many tasks by successfully navigating have a well-defined process17 that, when their surroundings, identifying objects around appropriate, seeks to move projects from the them and assisting humans with various tasks, pilot phase to full deployment. In a survey such robot-assistedCurrent surgeries. Challenges of early U.S. adopter organizations, 54% of executives said that their organization has a AI’s Deployment Problem process for moving prototypes into production and 52% reported having an implementation road map. Such organizational approaches One of the major concerns with AI in may facilitate moving more AI systems into organizations at present is that many systems deployment but attempts may be only in their are only experimental and never deployed in earlyAI Talent stages. Issues production. Pilot AI projects are relatively easy to develop and are only intended to demonstrate that the technology is feasible in concept. Securing a sufficient volume and level Deployment, on the other hand, requires a of human AI talent is a challenge for many variety of tasks and capabilities that may be in organizations—particularly those that are not short supply—for example, integration with in the technology sector. Data scientists and existing technology architectures and legacy AI engineers are still scarce, although many infrastructure, change in business processes and university programs have arisen to train them. organizational culture, reskilling or upskilling Firms that are unable to pay high levels of of employees, substantial data engineering compensation and are not located in technology and approaches to organizational change centers are likely to have difficulty hiring the management. Full production deployment tends desired number of skilled employees. Many to take much longer than pilot projects and cost companies should attempt to not only hire new substantially more. employees with AI skills but to retrain existing Surveys of organizations and market research employees to the degree possible. Even when companies do manage to hire data scientists and reports in the U.S. and globally suggest15 that deployment challenges involving big data and other types of analytical and artificial intelligence AI are widespread. A 2019 survey of large talent, there is little consensus within and across financial services and life sciences firms found companies about the qualifications for such that firms are actively embracing AI technologies roles. The term “data scientist” might mean a job and solutions, with 91.5% of firms reporting with a heavy emphasis on statistics, open-source ongoing investment in AI. However, only 14.6% coding, or working with executives to solve of firms reported that they have deployed AI business problems with data and analysis. Some capabilities for widespread production. A 2019 view the role as focused only on developing global McKinsey survey reported under the models, while others view it as extending to the headline “AI Adoption Proves its Worth, but deployment of models in production. The idea of Few Scale Impact” indicates that between 12% (consumer packaged goods firms) and 54% (high 16 “State of AI in the Enterprise, 2nd Edition,” Deloitte In- tech firms) of firms have at least one machine sights, 2018, https://www2.deloitte.com/content/dam/insights/us/ articles/4780_State-of-AI-in-the-enterprise/DI_State-of-AI-in-the- learning application implemented in a process enterprise-2nd-ed.pdf or product, but only 30% of respondents overall 17 Davenport, T. and Bean, R. “Farmers Accelerates its Time to Impact with AI.” Forbes, August 1, 2018, https://www.forbes.com/ sites/tomdavenport/2018/08/01/farmers-accelerates-its-time-to- 15 “Big Data and AI Executive Survey,” op cit., 2019. impact-with-ai/#51430150b672

December 2020 (19:4) | MIS Quarterly Executive xiii Editors’ Comments

21 data scientist “unicorns” who possess high levels18 learning algorithm can put certain groups at a of all these skills has never been very realistic. disadvantage. This has already been observed The skills taught in university programs in various cases, including algorithms that aimed at training AI-oriented workers vary are used to score job applicants and appear widely and some universities offer multiple to be racist, or algorithms that recommend programs with different emphases. For both sentences to judges and appear to propagate newly hired and experienced employees, titles the preconceptions implicit in past sentencing such as data scientist and AI engineer are not decisions that were used as training data. likely to be a good guide for understanding Algorithmic bias can also lead to consequences actual capabilities. Further, activities involved distributed across large subsections of society in the deployment of AI systems and related by affecting the type of information to which organizational change issues may not be taught people are exposed. This happens, for example, at all by many technically focused programs. when machine learning algorithms behind There is an increasing need for a new type of social media propagate fake news or enable the professional who can understand business targeting of individuals for political campaigns. problems and translate them19 into algorithmic To reduce potential algorithmic bias, managers problems and, vice versa, explain technical will need to be proactive by performing small- insights to business20 managers. scale experiments and simulations before Initiatives exist in the early stages to implementing such algorithms, regularly standardize the different types of data, analytics, evaluating the dataset used for training, and AI roles and requisite skills across organizations; using human reviewers who regularly provide however, developing new standards typically feedback to system designers. In politically takes many years. In the meantime, companies and socially sensitive domains like judicial should devote significant attention to classifying sentencing,Unexplainable firms may finddecision it necessary tooutcomes: publish and certifying the different types of AI and data their algorithms to preclude accusations of bias. science jobs needed in their organizations. Companies would also benefit from expanding Potential social dysfunctions resulting their talent pool by working with universities from AI implementation may be caused by directly on educational programs and by decision outcomes of some machine learning building and nurturing communities within their algorithms—deep learning in particular— organizations for employees working in data that cannot be easily explained because of teams. These steps are essential for companies the vast number of feature layers involved in looking to use AI to both improve current their production. This may lead to problematic operations and expand opportunities for digital situations, such as unexplainable evaluations22 innovation.AI and Social Dysfunctions of high school teachers, or parole decisions that cannot be justified and may appear to be unfair. Organizations need to respond to regulators’ Aside from deployment and talent challenges, calls for explainability by avoiding “black box” AI there are a few other potential AI dysfunctions applications and by choosing algorithms whose thatAlgorithmic managers shouldbias: be aware of and make outcomes can be explained. Being open about the plans to avoid. data that is used and explaining how the model The employment of AI works in nontechnical terms is also necessary to systems in classification or prediction tasks ensure customers’ trust and to avoid potential often comes with the risk of algorithmic bias, dysfunctions triggered by a lack of transparency. 18 which Davenport, means T. “Beyond that the Unicorns: outcomes Educating, of Classifying, the machine and Indeed, in some industries such as banking, Certifying Data Science Talent,” Harvard Data Science Review, regulators often force firms to use explainable May 19, 2020, https://hdsr.mitpress.mit.edu/pub/t37qjoi7/release/2 algorithms. 19 Henke, N., Levine, J. and McInerny, P. “You Don’t Have to 21 Davenport, T. The AI Advantage: How to Put the Artificial Be a Data Scientist to Fill this Must-Have Role,” Harvard Business Intelligence Revolution to Work, 2018, MIT Press. Review, Feb. 5, 2018, https://hbr.org/2018/02/you-dont-have-to-be- 22 O’Neil, C. Weapons of Math Destruction: How Big Data a-data-scientist-to-fill-this-must-have-analytics-role Increases Inequality and Threatens Democracy, 2016, Broadway 20 See, for example, https://www.iadss.org/ Books.

Blurring accountability boundaries Implications : As AI is used to enhance or even automate decision- Despite the existing challenges, AI has making procedures, the issue of accountability the potential to dramatically change how the arises. Who is responsible if a traffic accident workforce is structured, how jobs are designed, involving a driverless car occurs? Who is how knowledge is managed and how decisions responsible for approving parole to a criminal are made. Such changes will have broader who eventually commits another crime? Who implications for organizations and societies, is responsible for big financial losses incurred many of which have yet to be understood or by algorithmic trading? These are only a few realized. However, the most common effects are examples of cases in which accountability likely to be on how work is conducted in the boundaries are blurred. Managers will need to AI and the future of work: future. proactively focus on the reasons and processes Recent that may lead to potential harm. They should also developments in AI are already affecting the carefully consider how they engage the different Automating work tasks workplace in different ways. actors that directly or indirectly interact with : AI will have a outcomes produced by AI systems (AI developers23 significant impact on several occupations by andInvaded designers, privacy business users, institutions) and automating mundane tasks and rendering clarify responsibility and legal liability upfront. various human skills obsolete. Given that AI can : Ethical issues arise even perform tasks that previously required human before any action is recommended or performed judgment, the effects of AI-enabled automation by AI systems, with privacy 24 being reported differ from26 those of past technologies, as one of the main ethical considerations particularly regarding their impact on knowledge underlying AI implementation. Data is the workers. AI introduces new threats to the primary resource fed into AI systems and is authority of professionals such as physicians, often perceived to be a source of competitive lawyers, consultants and architects, whose advantage. AI’s need to process increasingly large expertise, judgment and creativity have thus far amounts of data thus conflicts with the right to been highly valued and considered irreplaceable. maintain control over data and its use in order While the need for such professions will not to preserve privacy and autonomy. Organizations disappear in the near future, the changing nature need to ensure that their data practices comply of such work is already a reality. There are many with relevant policies on the use of personal data predictions27 about how AI will impact such work, (e.g., GDPR in EU countries) and avoid potential but thus far job losses have been relatively privacy violations. Developing auditable Changing expertise minor. algorithms and performing algorithmic audits : AI technology that is on them to identify what data is used and what capable of automating some tasks is already variables feed into decision-making processes active in the workplace. In law firms, for example, represent helpful solutions that25 can be used to a plethora of applications have been developed increase transparency regarding how consumer for automating due diligence and contract data is processed and used. Overall, openness review tasks that were previously performed by about how data is handled is essential for junior lawyers. In sales, conversational AI can ensuring customer trust. now automate various tasks that previously had 26 See Davenport, T. Thinking for a Living: How to Get Better Performance and Results from Knowledge Workers, 2015, Harvard Business School Press; Benbya, H., “Knowledge Management 23 Dourish, P. “Algorithms and their Others: Algorithmic Culture Systems Implementation: Lessons from the Silicon Valley,” 2018, in Context,” Big Data & Society, 3(2), 2016, 1-11. Neal-Schuman Publishers; and Faraj, S., Pachidi, S. and Sayegh, K. 24 Kinni, T. “Ethics Should Precede Action in Machine Intel- “Working and Organizing in the Age of the Learning Algorithm,” ligence.” MIT Sloan Management Review, June 29, 2017, https:// Information and Organization (28:1), 2018, pp. 62-70. sloanreview.mit.edu/article/ethics-should-proceed-action-in-ma- 27 For one prominent prediction, see Frey, C. B. and Osborne, M. chine-intelligence The Future of Employment: How Susceptible Are Jobs to Comput- 25 Mittelstadt, B. “Automation, Algorithms, And Politics | Audit- erization, Oxford Martin School working paper, 2013, https://www. ing for Transparency in Content Personalization Systems,” Interna- oxfordmartin.ox.ac.uk/downloads/academic/future-of-employment. tional Journal of Communication (10), 2016, Article 12. pdf

December 2020 (19:4) | MIS Quarterly Executive xv Editors’ Comments

Organizational Implications

to be carried out by account managers. While such automation can increase the efficiency The introduction of AI is associated with of operations and decrease labor costs, they significantChanging changes authority in how organizationsarrangements are threaten to create gaps for professionals by managed. automating processes that had previously been : used to acquire knowledge about customers Unavoidably, as discussed above, expertise or to developed expertise. This will eventually is redefined and the knowledge and skills of lead to changes in the knowledge base required technology practitioners such as machine for affected occupations and could potentially learning experts, data scientists and data even trigger their restructuring. For example, analysts will become increasingly valued in the in the legal profession, law graduates now often workplace, which may lead to a restructuring find it necessary to develop data science skills of authority arrangements across all levels of andAugmenting technical skillsprofessionals instead of following the an organization. On a tactical level, technology traditional career path of a lawyer. practitioners will likely gain authority and : In many cases, control over work design and decision-making AI systems are not yet able to replace human processes, given that they have the ability to experts but can augment human work by prescribe how AI systems will affect operations. supporting experts’ decision-making processes. On a more strategic level, new roles will be For example, in the medical field, while there added to boards and upper echelons, triggering was initially some concern that AI could fully questions related to the established regime: replace radiologists, it is now clear that the For example, where does the CIO’s jurisdiction role of AI 28 will be to augment the work of end and the CDO’s start when it comes to radiologists. Nevertheless, as AI systems are planningChanging a major coordination digital transformation through introduced to support the diagnostic capabilities implementing AI technology? of radiologists, several unintended consequences : The use of AI to have arisen—for example, radiologists may algorithmically manage work will lead to confront communication barriers in interactions fundamental changes in organizational design with data scientists, and conflicts between AI and coordination. For instance, work tasks and their29 own diagnoses may cause radiologists must be redefined so that they can be broken to question their own capabilities versus those down into smaller subtasks that can then be of the AI. This becomes even more complicated algorithmically31 assigned to workers on digital considering the potential inscrutability of labor platforms such as UpWork or Amazon machine learning algorithm functions, meaning MTurk. Machine learning algorithms can that specific outcomes often cannot be easily be used to coordinate more proactively by traced or explained. In any case, the nature of analyzing historical data to predict the need work is changing dramatically and, while many for skills and expertise in future projects. observers predict that the combination of human Furthermore, practitioners and managers and machine30 intelligence will always reign will need to collaborate with new experts supreme, we have yet to see how “augmented entering the workplace in data processing, professionals” will carry out their work and what algorithm development, data visualization and further implications will arise for the workplace, so forth. Collaboration among individuals with organizations28 Davenport, T. andand Dreyer institutions. K. “AI will Change Radiology, but it different types of expertise can make work Won’t Replace Radiologists”, Harvard Business Review, March 27, coordination more challenging and may result 2018, https://hbr.org/2018/03/ai-will-change-radiology-but-it-wont- replace-radiologists in Changing a substantially valuation different schemes execution of an 29 Lebovitz, S. Lifshitz-Assaf, H. and Levina, N. To Incorporate organization’s operations and services. or Not to Incorporate AI for Critical Judgments: The Importance of : The way that Ambiguity in Professionals’ Judgment Process, NYU Stern School of Business, January 15, 2020, available at SSRN: https://ssrn.com/ performance is evaluated is also changing abstract=3480593 substantially, as employees are now being 30 Brynjolfsson, E. and McAfee, A. The Second Machine Age: 31 Faraj, S., Pachidi, S. and Sayegh, K. “Working and Organizing Work, Progress, and Prosperity in a Time of Brilliant Technologies, in the Age of the Learning Algorithm,” Information and Organiza- 2014, Norton. tion (28:1), 2018, pp. 62-70.

assessed by machine learning algorithms, management and governance mechanisms in often with little understanding of the variables place. We have already mentioned those related included in the inherent model or the extent to deployment. In addition, organizations in to which a specific variable might determine a various states of adoption have put in place a specific outcome. Even 32 product quality checks wide range of internal organizational34 structures are increasingly becoming an automated task and roles to manage and govern AI projects. The performed by robots. Such fundamental results of a 2018 survey illuminate a number changes in the values that matter to an of governance mechanisms used to manage AI organization not only substantially impact how projects: respondents indicated that35 45% of firms manage their employees but can also firms had appointed AI champions, 37% of firms lead to countereffects on the employee side. had created36 an AI center of excellence, and 37% For example, Amazon delivery drivers have of firmsDemocratization had developed of a comprehensivedata science andstrategy AI reportedly begun hanging phones on trees for AI. 37 outside of dispatch stations because the closer : proximity to the dispatch network allows33 these Tools like automated machine learning can drivers to receive delivery requests a split- structure and automate the workflow of creating second sooner than other drivers. This is a and implementing a machine learning model. fascinating demonstration of how employees try Such tools can be employed to improve the to Industrial game algorithmic transformations systems in their effort to productivity of professional data scientists maintain some control over their work. or to enable less highly educated “citizen : AI technology data scientists” to complete data science and is currently enabling significant digital AI projects. Several startups and large cloud transformations that not only redefine what vendors have made such capabilities available an organization does but also blur industry and it seems likely that the democratization boundaries. Traditional manufacturing of data science and AI development—i.e., the organizations are currently taking advantage notion that anyone, even those with little to no of machine learning technology to transform expertise, can perform data science if provided their focus from the production of goods to the ampleOngoing data andmodel user-friendly improvement analytics tools— provision of services. GE’s digital transformation will continue to advance. effort is a well-known example of such an : Companies attempt, with AI being the driving force behind that are heavily committed to AI often find that their predictive maintenance services. Such they have many models and algorithms in place, developments invoke a number of questions some of which are in production processes and that must be addressed: Who are the new systems. Since businesses are, to some extent, competitors of such digitally transformed dependent on the accuracy of these models, it organizations? How should such organizations is important to monitor them for “drift” (i.e., be regulated? How do relationships with inaccuracy of predictions) and improve them customers change? Who are the new partners of over time. Vendors are developing tools to suchFuture organizations? Opportunities support this process under the banner of MLOps (machine learning operations), which are most widely used in data- and analytics-dependent As companies continue to use AI, they will industries like financial services. exploreManagement a variety and of governance different directions. mechanisms We 34 Davenport, T. The AI Advantage: How to Put the Artificial suggest several of them below. Intelligence Revolution to Work, 2018, MIT Press. 35 Davenport, T. and Dasgupta, S. “How to Set Up an AI Centre : of Excellence,” Harvard Business Review, January 16, 2019, https:// Leading32 Mahdawi, companies A. “The Domino’s using ‘Pizza Checker’ AI already Is Just the Begin have- hbr.org/2019/01/how-to-set-up-an-ai-center-of-excellence ning: Workplace Surveillance Is Coming for You,” The Guardian, 36 Davenport, T. and Mahidhar, V.“What’s Your Cognitive Strat- October 15, 2019. egy?” MIT Sloan Management Review, 2018, https://sloanreview. 33 Soper, S. “Amazon Drivers Are Hanging Smartphones in Trees mit.edu/article/whats-your-cognitive-strategy/ to Get More Work.” Bloomberg, September 1, 2020, https://www. 37 Sharma, M. “Navigating the New Landscape of AI Platforms,” bloomberg.com/news/articles/2020-09-01/amazon-drivers-are-hang- Harvard Business Review, March 10, 2020, https://hbr.org/2020/03/ ing-smartphones-in-trees-to-get-more-work. navigating-the-new-landscape-of-ai-platforms

December 2020 (19:4) | MIS Quarterly Executive xvii Editors’ Comments

AI explainability and transparency

: As The last paper will be published in the March outlined above, it is now widely known that AI 2021 issue. models can be biased against certain groups Table 2 maps the contributions that each and individuals. Some firms have established paper makes to the special issue along with the AI ethics organizations or “algorithm review type of AI technology it covers. We then briefly boards” to assess transparency issues. Complex discuss each of the papers, outline the challenges models, such as those implicit in deep learning faced by firms in adopting AI technologies and neural networks, may be impossible to interpret offer guidelines to manage such challenges. or explain. Some vendors provide “prediction The first paper in the special issue, explanations” that point out influential variables “Addressing the Key Challenges of Developing or features and their direction of influence, but Machine Learning AI Systems for Knowledge- this is not yet possible for the most complex Intensive Work,” is by Zhewei Zhang, Joe models. Many organizations and researchers are Nandhakumar, Jochem Thomas Hummel and now working on new approaches to improve Lauren Waardenburg. The paper discusses explainability,Reduced requirements but we are only for indata the early stages how a machine learning AI for a legal practice of addressing this issue successfully. firm (LegalTechCo) was developed to help legal : Many AI professionals make faster and better-informed models, particularly deep learning neural decisions. The authors studied the development networks, require large amounts of data in order of the AI system at LegalTechCo over a couple of to be trained effectively. A new deep learning- years. They identified three challenges involved based natural language generation model called in developing machine learning systems. The GPT-3, for example, used billions of words to challenges are related to how to define ML train the model and has 175 billion variables problems, how to manage the training of ML and parameters. Some researchers have argued models and how to evaluate ML AI performance. that the trend toward such volumes of data is The authors propose three guidelines (and unsustainable and that new approaches to AI twelve recommendations) that executives should aim to use less data. However, this trend can use to address the various challenges. is also inSpecial its nascent stages.Issue Papers The guidelines include: 1) co-formulate the appropriate machine learning AI problems; 2) develop machine learning AI through iterative This special issue started out as a refinement; and 3) go beyond the numeric measurements and ask for clues. MISQconversation Executive between the guest seniorJournal editorsof the The second paper, “Unintended Consequences Associationand the editors-in-chiefof Information ofSystems two journals— (MISQE) and the of Introducing AI Systems for Decision Making,” (JAIS)—on is by Anne-Sophie Mayer, Franz Strich and the need to create concerted efforts to contribute Marina Fiedler. This paper focuses on the to both IS theory and practice. This special issue unintended consequences of introducing an is the outcome of this dialogue, which began autonomous AI system in the banking industry. at the 2019 Pre-ICIS Special Issue Workshop It draws on a case study from one of the largest in Munich, Germany. We received over 50 banks in Germany (Main Finance). Main Finance extended abstracts and selected 30 submissions confronted several issues in the small loan for discussion; we received early feedback segment, including: 1) increased competition from the special issue editorial board and the from new market participants due to digitization; participating senior editors from both journals. 2) mismatched personnel resources; 3) high The special issue received a total of 50 default rates; and 4) a decline in profitability. submissions. About half of the submissions were To address the issues faced, the firm introduced sent out for review after the initial screening and, an AI system based on ML to make decisions after three rounds, five articles were accepted for about who is qualified for loans. The authors publication in the MISQE special issue. The first document the implementation of the AI system four papers appear in the December 2020 issue; and its consequences from the perspective of both frontline workers and senior management. While the introduction of the AI system

Table 2: The Focus and Contributions of the Special Issue Papers

Paper Authors AI technology Industry Contribution

Zhang, Nandhakumar, Machine Learning Legal Covers challenges related to 1 Hummel and Algorithmic AI developing machine learning Waardenburg systems

Mayer, Strich and Machine learning Banking Discusses intended and unintended 2 Fiedler Algorithmic AI consequences of introducing an autonomous AI system Asatiani, Malo, Nagbøl, Machine learning Government Offers ways to address 3 Penttinen, Rinta-Kahila Algorithmic AI explainability issues and Salovaara Reis, Maier; Mattke; Machine Learning, Healthcare Explains physician resistance to an 4 Creutzenberg and Natural Language AI virtual agent Weitzel Programming Schuetzler, Grimes, Conversational AI Multiple Offers guidelines to design 5 Rosser and Giboney examples conversational AI systems

enhanced profitability and helped address the though the inner workings are not always main challenges facing loan management, it entirely explainable. The authors build on a six- also resulted in employees’ perceived loss of dimensional framework of an intelligent agent competence and reputation and unpredictability to discuss explainability challenges at DBA: 1) of decisions. From senior management’s the model; 2) the goal; 3) training data; 4) input perspective, the AI system resulted in employees’ data; 5) output data and 6) environment. They loss of critical thinking and expertise and in the further offer guidelines for managers to address misuse of the system. The authors offer several explainability issues: 1) use modular design guidelines to prevent related consequences: to increase AI explainability; 2) avoid online 1) maintain employees’ abilities to reflect and learning if explainability is a priority; and 3) understand underlying processes; 2) understand facilitate continuous open discussion between and guide the shift of employees’ roles; 3) make stakeholders. the AI system as transparent and explainable as The fourth paper, “Addressing User Resistance possible; and 4) reconsider customer groups Would Have Prevented a Healthcare AI Project excluded from the AI. Failure,” is by Lea Reis, Christian Maier, Jens The third paper, “Challenges of Explaining Mattke, Marcus Creutzenberg and Tim Weitzel. the Behavior of Black-Box AI Systems,” is by The authors discuss a case of AI implementation Aleksandre Asatiani, Pekka Malo, Per Rådberg failure in a German hospital. The hospital Nagbøl, Esko Penttinen, Tapani Rinta-Kahila and decided to integrate AI to improve their Antti Salovaara. The authors document ways anamnesis-diagnosis-treatment-documentation used by the Danish Business Authority (DBA)— process with the intent of giving physicians an agency under Denmark’s Ministry of Industry, more time to care for patients and reducing Business, and Financial Affairs—to deal with process costs. A virtual agent based on machine challenges associated with explainability. The learning and natural language processing was availability of large volumes of data-enabled developed to support different activities: 1) DBA to pursue machine learning for core tasks the cognitive agent engages with patients to such as supporting companies’ legal compliance, perform anamnesis, collects data and provides checking annual reports for signs of fraud, and structured documentation; 2) the cognitive agent identifying companies early enough on their applies decision support algorithms to suggest route to distress to ensure that timely support a diagnosis based on the structured recorded can be given. The organization has been able data; and 3) the cognitive agent engages with the to implement AI responsibly and legally even physician to provide treatment options.

December 2020 (19:4) | MIS Quarterly Executive xix Editors’ Comments

However, after nine months of developing the example, upskilling workers to do existing jobs use case and the test version and six months of with AI and retraining and hiring other workers technological testing, the project team realized to fulfillAbout the new theroles thatSpecial AI will demand. Issue that the hospital’s physicians did not want to use Editors the system. While the physicians acknowledged that complementary knowledge supporting the diagnosis decision was valuable to themselves Hind Benbya and the patients, they refused to approve the project. The team decided to postpone the Hind Benbya is a professor and head of IS project indefinitely until they could better and Business Analytics at Deakin University understand the reasons for the physicians’ and is a visiting policy fellow at the Oxford rejection and what steps should be taken to Internet Institute at the University of Oxford. ensure future project success. The authors Hind’s research expertise includes digital document the reasons behind the physicians’ innovation, IT-enabled transformation and rejection of the cognitive agent and offer MIS Quarterly Journal of Management artificial intelligence. Her work has appeared recommendations to address them. Information Systems MIT Sloan Management in , the The fifth paper, “Your Agent Is Ready: Review MISQ Executive Decision Support , Guidance for Designing Conversational Agents,” Systems , and is by Ryan Schuetzler, Mark Grimes, Holly Rosser MISQ Executive , among other outlets. She is currently and Justin Giboney. The paper focuses on chatbot Journal of the Association a senior editor for , guest design. Chatbots are used by organizations to of Information Systems senior editor for the improve business processes, automate routine Journal of Strategic and a member of the interactions and provide an automated social Information Systems editorial board of the touchpoint for customers. The authors build . Hind has been a visiting on their experience with chatbot design and professor at Cambridge Judge Business School, use examples of chatbots across industries to UCLA Anderson School and the London School offer a decision guide about when and how of Economics. She has received several best chatbots should be deployed. The framework paper awards, regularly works with leading presented in the paper asks questions and offers firms in Europe, the U.K. and the U.S. and has considerations that should be discussed early presented her research at premier academic and on in the bot development process, and offers a practitionerThomas H. venues. Davenport number of implicit signals that bots can use to create natural, human-likeConclusion conversations. Tom Davenport is the President’s Distinguished Professor of Information The five papers selected for this special issue Technology and Management at Babson College, along with this editorial provide a variety of visiting professor at the Oxford Saïd Business examples of AI applications across industries School, fellow at the MIT Initiative on the and discuss challenges and implications for Digital Economy and senior advisor to Deloitte organizations. As AI technology is still maturing, Analytics. He teaches analytics/big data in awareness regarding the new management executive programs at Babson, Harvard Business challenges it poses and the implications it School and School of Public Health and MIT raises for the workplace and the organization Sloan School. Davenport pioneered the concept are still emerging, but the most common effect of competing on analytics with his best-selling will likely be on how work is conducted in the 2006 Harvard Business Review article and 2007 future. Therefore, companies need to begin work book. His most recent book is The AI Advantage: now on developing AI applications that create How to Put the Artificial IntelligenceHarvard Revolution Business economic value and that lead to new ways of Reviewto Work. MITHe has Sloan written Management or edited nineteen Review otherThe orchestrating work by humans and machines. Financialbooks and Times over 200 articles for Leaders will need to understand and prepare , , for how AI will impact their workforce by, for and many other publications. He

Wall Street Journal Forbes is a regular contributorConsulting to theNews and . He has been named one of the top 25 consultants by , one of the 100 most influential peopleFortune in the IT industry by Ziff- Davis and one of the world’s top fifty business schoolStella professorsPachidi by .

Dr. Stella Pachidi is a lecturer of information systems at Judge Business School, University of Cambridge. Her research interests lie in the intersection of technology, work and organizing. Currently, her research projects include the introduction of artificial intelligence technologies in organizations, managing challenges in the workplace during digital transformation, and practices of knowledge collaboration across boundaries. She holds a Ph.D. in business administration from VU University Amsterdam, an M.Sc. in business informatics from Utrecht University and an M.Sc. in electrical and computer engineering from National Technical University of Athens. Dr. PachidiOrganization has articlesScience in Informationinformation systemsand Organization and organization and Computers journals inand Human books Behavior including ,

. She has presented her work at various major conferences in the fields of technology and organizations including Academy of Management Meeting, International Conference on Information Systems, European Group for Organizational Studies Colloquium, Process Symposium and others.

December 2020 (19:4) | MIS Quarterly Executive xxi