Machine Learning, Reasoning, and Intelligence in Daily Life: Directions and Challenges

Machine Learning, Reasoning, and Intelligence in Daily Life: Directions and Challenges Eric Horvitz Microsoft Research Redmond, Washington USA 98052 [email protected] Abstract An example of an implicit integration of ambient learning Technical developments and trends are providing a and reasoning is the effort by our team to create a probabil- fertile substrate for creating and integrating ma- istic action prediction and prefetching subsystem that is em- chine learning and reasoning into multiple applica- bedded deeply in the kernel of Microsoft’s Windows Vista tions and services. I will review several illustrative operating system. The predictive component, operating research efforts on our team, and focus on chal- within a component in the Vista operating system called lenges, opportunities, and directions with the Superfetch, learns by watching sequences of application streaming of machine intelligence into daily life. launches over time to predict a computer user’s application launches. These predictions, coupled with a utility model 1 Reflections on Trends and Directions that captures preferences about the cost of waiting, are used Over the last decade, technical and infrastructural develop- in an ongoing optimization to prefetch unlaunched applica- ments have come together to create a nurturing environment tions into memory ahead of their manual launching. The for developing and fielding applications of machine learning implicit service seeks to minimize the average wait for ap- and reasoning—and for harnessing automated intelligence plications to be ready to use after launching actions. to provide value to people in the course of their daily lives. These developments include (1) technical advancements in Moving from the depths of operating system kernels to the machine learning and reasoning, (2) the growth in CPU and infrastructure of a city, there is great opportunity for collect- memory capabilities within commonly available devices and ing data and learning predictive models from constellations platforms, (3) the connectivity, content, and services pro- of sensors embedded throughout a large-scale region. The vided by the evolving Web, and (4) the increasing availabil- JamBayes traffic forecasting service [Horvitz et al. , 2005] ity of data resources, including corpora of behavioral data serves as an example of an explicit extension of ambient collected via inexpensive sensors, and through ongoing in- intelligence to familiar views of digital maps that display the teraction with software and services. flow of traffic in urban areas. The JamBayes client, operating on desktop systems and portable devices, accesses the 1.1 Panoply of Applications and Services JamBayes predictive traffic service that employs machine Opportunities for integrating applications of machine intel- learning and reasoning about the context-sensitive flow of ligence into the daily lives of people are growing with the traffic. The system overlays predictions about future traffic increasing popularity of computing systems, the widening conditions on a digital traffic flow map. The system com- diversity of web services, the growing popularity of portable bines multiple variables that consider key contextual evi- devices that contain general-purpose operating systems, and dence as well as the dynamics of flow across a greater urban ongoing innovations in human-computer interaction— area to predict when free-flowing traffic will likely become including the increasing prowess of speech recognition, jammed and how long it will be until current jams melt handwriting, and sketch-understanding interfaces. away. A descendant of JamBayes developed by our team named ClearFlow provides users with context-sensitive Various examples of the integration of automated learning routing based on reasoning about the current and future and reasoning into daily life have been appearing as implicit flows on road segments within a regional traffic system. and explicit extensions to traditional systems and services, and also in prototypes and systems that provide qualitatively Beyond serving as an overlay of predictive services on traf- new kinds of experiences. I will review several projects and fic flow maps, JamBayes also provides a qualitatively new efforts undertaken by our team that highlight directions and functionality—surprise detection and surprise forecasting . approaches to introducing potentially valuable machine We have been pursuing research on surprise forecasting, learning and reasoning into the daily lives of people. aimed at developing and fielding sensing and reasoning systems that have the ability to detect and to alert users when [Horvitz et al., 1999], the larger cross-platform alerting sys- current or future events will likely surprise them . Such rea- tem named Notification Platform [Horvitz et al. 2003], and soning promises to provide significant value to people in the several of our Bestcom efforts. Research in this realm high- course of daily life as it explicitly considers the misalign- lights the potential for machine learning and reasoning to ment between computational forecasts about the world and play a significant role with enhancing communications. the inferred expectations of people with regards to important outcomes. JamBayes’ surprise modeling considers a user’s The Notification Platform continues to build and triage in- context-sensitive expectations about current and future traf- formation from multiple sources within a unified inbox, and fic, and infers when situations that it is inferring would reasons about the best devices and modalities to employ. likely be interpreted as anomalous or “surprising” from the The related Bestcom effort (for best-means communica- user’s perspective. Users can configure the system to gen- tions ) centers on assisting people to establish communica- erate alerts when particular kinds of surprises occur, based tions with one another. Within Bestcom research, we have on context. In operation, the surprise modeling and analy- investigated the promise of systems with the ability to un- sis is used to let people know when current or projected derstand communication preferences, the goals of a com- flows (both high and low) on routes that they have ex- munication, current and future availabilities, and context. pressed interest in would likely surprise them. Prototypes explore how agents, working on behalf of a contactor and contactee, can negotiate about the best time and Let us now move to systems that learn and reason about a type of communication to undertake, and then execute con- computer user’s activities and that can provide qualitatively nections. In the general Bestcom methodology, a contactor new experiences for users. In the Web Montage prototype requests a communication with a contactee (the contactor [Anderson and Horvitz, 2002], data collected in the back- “Bestcoms” the contactee), and an optimization is executed ground about a user’s desktop activity is analyzed via ma- based on identity, context, goals, connectivity, and devices chine learning, and predictive models are constructed that available now and in the future. Actions may range from a are used to generate personalized adaptive portals , pages real-time voice call to a rescheduling of a voice or video- that are composed dynamically by coalescing automated conference for a later time. Various prototypes have been clippings from multiple web sources. The system builds created, including systems that perform smart routing of models of the cost of navigation and the value of content to calls based on context, executing on a portable device ( e.g., triage information based on time of day and ongoing activi- the Bayesphone [Horvitz, et al., 2005]), and systems em- ties, including for example, the current and recent patterns ploying a larger infrastructure, such as the Bestcom-ET sys- of topics at focus of attention on the desktop. Content drawn tem fielded internally at Microsoft [Horvitz et al ., 2003c]. in a selective manner from multiple sources is arranged and laid out via an optimization of expected value. Machine learning and reasoning can provide value with the overall coordination of people in the course of daily life. Another example of overlaying intelligence to provide new The Coordinate system [Horvitz et al., 2002] was initially kinds of experiences is the MemoryLens effort [Horvitz et developed to support the Notification Platform and Bestcom al. , 2004], which has been exploring the use of machine efforts, but the work also led to standalone presence and learning and reasoning to model aspects of human memory. availability forecasting services that provide people with The LifeBrowser prototype developed within the Memory- new kinds of awareness. Coordinate continues to collect Lens effort employs inferences about the memorability of data about the presence and activities of people at different events and items to provide new kinds of browsing and locations and devices, and employs machine learning and searching experiences such as variable density timelines. It reasoning to perform availability forecasting. The system also uses the inferences about landmarks to extend familiar can pass its inferences to computational agents, or provide search functionality by integrating a backbone of memora- to people (who have been granted access privileges), such ble personal events and activities into the display of time- predictions as the time until a user will return to their office, sorted search results. read email,

Load more