Cognitive Assistance in Government Papers from the AAAI 2015 Fall Symposium

Cognitive Assistance at Work

Hamid R. Motahari Nezhad IBM Almaden Research Center San Jose, CA 95120 United Stated [email protected]

Abstract their main communication paradigm and infor- Today’s businesses, government and society work and ser- mation sharing, vices are centered around interactions, collaborations and 5) Vast amount of information that are produced, knowledge work. The pace, amount and veracity of data communicated, processed and needed to be man- generated and processed by a worker has accelerated signif- aged by a worker in the work context in order to icantly to the level that challenged human cognitive load and productivity. On the other hand, big data has provided perform work effectively; an unprecedented opportunity for AI to tackle one of the 6) And, the fast pace of the work that has led to main challenges hindering the AI progress: building models shaping new interaction and communication pat- of world in a scalable, adaptive and dynamic manner. In this terns and habit, and in particular the increasing paper, we describe the technology requirements of building use of instant and real-time communication as in- cognitive assistance technologies that assists human work- ers, and present a cognitive work framework that tegrated part of work productivity tools. There are aims at offering intelligence assistance to workers to im- already startups and enterprise applications that prove their productivity and agility. We then describe the innovate by bringing messaging tools and inte- design and development of a set of cognitive services of- grating it into enterprise collaboration platforms. fered by the framework, based on advanced NLP and ma- As the result of these trends, we witness an enormous chine learning methods. The cognitive services help workers in processing and linking information and identifying and shift in the collaboration platforms which is being replaced tracking work items over interactions in communication with integrated, social and stream-driven collaboration channels such as email, social conversations and media, environment where all information such as messages, files chats and messaging and calendar applications. These cog- apps are reporting into an integrated view, and more im- nitive services are designed to be adaptive, online and per- portantly (cognitive) agents (automated bots) are partici- sonalized so that over time adapt to changing environment and knowledge, and the models become personalized pating in conversations among human to facilitate work. through learning preferences and working language and In particular, the growing trend in bringing conversa- style of the subject worker. tional virtual agents into work context presents unique op- portunities and challenges. The opportunities are enormous in terms of capitalizing on these agents to assist humans in Introduction be more productive, and have them to perform jobs that a The nature of work is undergoing tremendous transfor- machine would be superior and better than a human, but in mation. The key drivers of this transformation are as fol- a way that complements human worker. The challenges lows: include characterizing the type and abilities of a human 1) Globalization and geographical distribution of and machine, and investigating the art of possible in tech- workforce (remote and global workforce); nologies to have the two (human and machine) to effec- 2) Faster business cycles and business agility due to tively communicate and collaborate. faster product and innovation cycles, and shorter One other typical challenges of AI, and for building time to market and time to service; cognitive assistant [EDWARD A. FEIGENBAUM 2003] 3) Mobility and the wide adoption of mobile tech- has been characterized as the ability of machines to build a nology among workforce with bring your own de- model of world, their human subject and themselves in a vice becoming the mainstream; scalable, adaptive and dynamic manner. The huge amount 4) Millennials getting into the workforce, who have of data available in the work context, generated as the re- grown up with mobile and social networking as sults of human interactions, workflow, enterprise applica- tions, databases and knowledge bases and the advances in methods for processing and building knowledge models Copyright © 2015, Association for the Advancement of Artificial Intelli- gence (www.aaai.org). All rights reserved. out of unstructured information in systems such as IBM

37 [AAA 2010] has opened up a new era for AI, and current AI technologies. Examples of this category include for enabling technologies for cognitive assistants, towards engaging in creative conversations, emotional intelligence, building such models of world, human subject and cogni- etc. Symbol manipulation also happens in the lowest level tive agents. Nevertheless, this is not a solved problem, and of hierarchical structure of brain function. The higher lev- the problem of developing cognitive assistants requires els of hierarchical structure of brain function involve taking incremental steps in understanding the technology emergent concepts where higher level concepts/ideas com- need, limits and problems that are enabler of building cog- bine, and form complex organisms (take an analogy with nitive assistant. ‘cloud’, a whole, relation to air and water molecules, com- In this paper, we investigate the problem of building ponent). Arguably, there is a subtype of synthetic cognition cognitive assistants that help and complement human that relies on fast and efficient processing of a large workers. A human worker spends on average of 28 hours amount of information, which is out of power of human on collaboration, coordination and communication activi- intelligence, while machines excel at those type of cogni- ties (email –reading and responding-, calendar and meet- tion. It is natural to observe that this is one area where ma- ings, communicating on social media and other communi- chines can complement humans. This is where we define cation tools). This amounts, on average, for 70% of a nor- the scope for cognitive assistance to a person, and in work mal 40-hrs work week for a knowledge worker in private context, in particular. enterprise and also in government sectors. To improve with A software program (agent) can be characterized as in- productivity in this space, we focus on the problem of of- telligent if it can employ computational intelligence tech- fering cognitive assistance over conversations and work niques in order to define a model of the world that facili- organization for a knowledge worker. We start by the ge- tates synthetic cognitive tasks. However, the challenge in neric problem of offering cognitive assistance, characteriz- achieving this type of intelligence is the building of the ing the cognitive abilities of a human and machine (and the knowledge model of the world and the domain that allows ones that each excel at), and then continue by introducing understanding of the meaning of the intellectual tasks. innovative technologies methods for the design and devel- In artificial intelligence literature, there has been consid- opment of a cognitive productivity assistant for human erable amount of research in knowledge representation and workers. knowledge acquisition technologies through reading text and other data types, and methods based on semantic and symbolic representations flourished by the concept of Problem Statement and State of the Art building ontologies in the context of semantic web, and A cognitive agent (CA) is defined as a software tool that linked data. Though, most approaches have not been able augments human intelligence [Engelbart 1962]. To achieve to demonstrate the process of knowledge building can be this objective, a CA should offer complementary cognitive done in a scalable, adaptive and dynamic manner when capabilities to a human by picking up those cognitive tasks entering new domains or adding new information to the that are time consuming, daunting or require high compu- base model. Though, cognitive computing methods, and in tational and cognitive power beyond human intelligence, particular those methods developed in the context of Wat- but in which a machine is greater than human. Let us char- son including deep learning methods, for processing un- acterize the overall cognition capabilities into those that are structured information and building models of the world main differentiator of a machine and human. Cognition (for specific tasks or domains) can be considered as the capabilities are categorized into two major types [Eric first step towards building intelligent methods that can Lord, Science, Mind and Paranormal Experience, 2009]: (i) build models of the world in a scalable manner by pro- analytical cognition, and (ii) synthetic cognition. cessing unstructured information (without the need for Analytical cognitive skills include those that the ma- manual crafting of models – e.g. ontologies). Though, chines excel at them but would take a lot of intellectual these models yet have not been shows to be adaptive and efforts from human. Examples of these capabilities are dynamic. mathematical calculations, making logical decisions in In this paper, we investigate the problem of offering complex situations requiring a series of computation, and cognitive assistance to enterprise workers by offering ana- chess all of which are recognized as computational intelli- lytical cognitive capabilities that supports synthetic cogni- gence. This type involves manipulation of symbols (sym- tive tasks in the context of human collaboration and com- bolic processing) through algorithmic information pro- munication tasks, and organizing work and performing cessing. At this level, the processing units does not know work for human workers to support higher productivity and or care about the “meaning” of symbol. On the other hand, agility. the second type of cognitive capabilities are those that hu- There has been a tremendous progress in the develop- man performs effortlessly but are hard for machines with ment of personal assistants in the industry. In particular,the most popular personal assistants include Apple ,

38 Now, Microsoft , Amazon Echo, , tems, knowledge basis in forms of structured and unstruc- 's , LG's , SILVIA, HTC's tured information. Hidi, Nuance’ , AIVC, Skyvi, IRIS, Everfriend, Evi, In order to support a worker in above three scenarios, we and Alme (patient assistant). There are also productivity define a Cognitive Work Assistant as an intelligent soft- focused agents including Amy (x.ai), Genee for scheduling ware for the work context that offer cognitive services to a assistant, which focus on one specific domain. These ap- worker following the mythology of: monitor, process, rec- plications offer conversational interfaces for human start- ommend and act. In particular, it would offer the following ing by voice (or some text) which is then processed by capabilities: Understands human language; Monitors col- these agents to offer services. A related paradigm is the laboration channels including email, calendar, chat and notion of offering intelligent conversational assistance as a enterprise information sources; Builds a model of the user service. A number of these platforms include Assistant.ai and the world (work context), and is situational aware (human speech conversational services), ChatBots (chat- (context); Offer assistance by pre-processing information, bots.io), telegram platform for including bots in chat con- and presenting information in human understandable for- versations and Watson’s Dialog Service in IBM Blue- mat; Categorizes and filters information; Gathers and or- Mix/Watson Developer’s Cloud. All these offer enabling ganizes related information; Schedules meetings and man- technologies for a personal assistance however, none, yet, ages the time on behalf of its human subject, Identifies addresses the problem of supporting the human workers in requests, and organizing to-dos of its human subject, As- getting work done and improving their productivity. This is sists in performing tasks such as organizing events, travel the focus of this work to offer such cognitive assistance. assistant; And, suggests taking certain actions to its human subject that supports increasing productivity, and growth. We also recognize that there are different roles in the en- Cognitive Work Assistant terprise that may benefit from such a work assistant: man- We envision that need for cognitive assistance of a agers (people who have a human personal assistant), em- knowledge worker at three different domains in private or ployees (who do not have a dedicated human personal as- government sectors: (1) the assistance in on-boarding, sistant), and human personal assistants themselves in get-

Community of cognitive agents that collaborate effectively with one another to support human activities. Assistant’s Cognitive Employee Cognitive Expert Cognitive Agents Agents Agents Interactions types need to be supported: • Cog-to-Cog interactions, • Human-Cog interactions, and • Cog-backed human-to-human interactions Cognitive Assistant Platform

Figure 1. Different types of cognitive work assistants form a community, each specializing in offering one type of service orientation and growth, which entails ability to proactively ting their job more efficiently and effectively. Therefore, point the worker to the right information and material at we define the following three types of personal assistants: the right time, (2) assistance in off-loading work entailed in Cognitive Employee Assistant: These assistants would communication and interactions, which account for a sig- have access to the data space (and applications) that their nificant part of a worker, as it was pointed out earlier in human subject (employees and managers) has access to this paper. In this context, the worker would interact using with the same level of visibility, and offer cognitive work synchronous and asynchronous communication channels assistance to them. such as email, calendar, texting/messaging and network Assistant’s Cognitive Assistant: An assistant to a human information diffusion and consumption. And, (3) the work assistant helps them to become more productive, and focus and project management context in which the worker on work that require human judgment, while more routine would need to interact with organization application sys- requests to them is handled by the cognitive assistant themselves.

39 Expert Process Assistants: This type of assistants are er of an email, participant in a chat session, or receiver of a experts in a specific domain such as travel, human resources (HR), etc. and are accessed by personal assis- tants to serve their human subjects. In Figure 2, we present an archi- Delivery Platforms tecture for a Cognitive Work Assis- tant Platform targeting offering Cognitive Work Assistant Platform cognitive services for improving the Conversation Interface Bot (voice, text) Cognitive Services APIs productivity over communication Personal Email Analytics, Calendar and To-do, Task Context-aware and collaboration tools and plat- Model Builder Auto-Response, Scheduling and Process Information Finder forms in the enterprise. The key Classification Assistant Assistant focus is on offering actionable in- sights integrated into collaboration tools such as emails, chats, work- Watson Apps or Services on BlueMix spaces and other work management Natural Language Parsing Watson Dialogue Concept and Knowledge Questions systems through integrating an in- Toolkit (PoS Tags, and Service Relationship Graph and Answer Dependicy Parser) Extraction Builder telligent agent that processes un- structured (text) and structured in- Enterprise Repositories, Applications and Data Sources formation and identifies action item

(or tasks) that a worker is assigned Feeds Document (requests), or commitments and Repositories collections promises that he makes to others. Assisting a worker requires person- Figure 2. The Cognitive Work Assistant Framework and Platform alizing all such services, and there- fore the “personal model builder” text/sms, etc.). module is responsible for scanning and preparing a model Action recommendation. This module’s functionality is of the world and the user’s work by monitoring the work to determine the type of the actionable statement, and make environment including people he is interacting with, his recommendation to the subject to take an action. Examples communication and conversation, and key entities includ- included adding to to-do list, or updating the status of an ing to-dos, follow-ups, calendar invites, and processes that action in the to-do list, scheduling a meeting, canceling or the person is involved in. rescheduling a meeting. The system has a number of action The cognitive work assistant is architected as an agent types that it recognizes, and a template language that al- that can be offered as a that can interact with lows definition of new action types and corresponding log- users using a voice activated or messaging interface (a chat ic to hand the action by the agent. bot) that can participate in chat sessions. In addition, it Context-aware information finder. This module is an offers the same services via API calls. At a lower level, the advanced search module that proactively scans the infor- cognitive assistant benefits from Watson services on mation space of a user and makes links and connections Bluemix, and a natural language parser that delivers part of among the entities (emails, calendar invite, files, people speech tags, word dependencies to verbs, on top of which and other supported entities). For identifying such links, an advanced natural language model builder extract key the information within the content of the entities, and text elements for actionable statements within conversations, and the role of the target users will be considered. and are delivered to the higher level modules within the cognitive work assistant. In the following, we provide an overview of key components of the cognitive work assis- References tant. EDWARD A. FEIGENBAUM, Some Challenges and Grand Actionable Insight Analytics over Conversations. The Challenges for Computational Intelligence, Journal of the ACM, conversation among workers include exchange of work Vol. 50, No. 1, January 2003, pp. 32–40 item and action requests and promises. The module related David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, to actionable insight over Email, Chat and other communi- David Gondek, Aditya A. Kalyanpur, Adam Lally, J. William cation channel focuses on identifying from the text, action- Murdock, Eric Nyberg, John Prager, Nico Schlaefer, and Chris able statements that are of type request, promise or ques- Welty, Building Watson: An Overview of the DeepQA Project tions addresses to the audience of the conversation (receiv- AAAI Magazine, 2010.

40