Wright State University CORE Scholar The Ohio Center of Excellence in Knowledge- Kno.e.sis Publications Enabled Computing (Kno.e.sis) 2010 Computing for the Human Experience: Semantics-Empowered Sensors, Services, and Social Computing on the Ubiquitous Web Amit P. Sheth Wright State University - Main Campus, [email protected] Follow this and additional works at: https://corescholar.libraries.wright.edu/knoesis Part of the Bioinformatics Commons, Communication Technology and New Media Commons, Databases and Information Systems Commons, OS and Networks Commons, and the Science and Technology Studies Commons Repository Citation Sheth, A. P. (2010). Computing for the Human Experience: Semantics-Empowered Sensors, Services, and Social Computing on the Ubiquitous Web. IEEE Internet Computing, 14 (1), 88-91. https://corescholar.libraries.wright.edu/knoesis/543 This Article is brought to you for free and open access by the The Ohio Center of Excellence in Knowledge-Enabled Computing (Kno.e.sis) at CORE Scholar. It has been accepted for inclusion in Kno.e.sis Publications by an authorized administrator of CORE Scholar. For more information, please contact [email protected]. Amit Sheth, "Computing for Human Experience: Semantics empowered Sensors, Services, and Social Computing on Ubiquitous Web," in IEEE Internet Computing - Vision Issue (V. Cerf and M. Singh, Eds.), vol. 14, no. 1, pp. 88-91, Jan./Feb. 2010. Semantics & Services Computing for Human Experience Semantics-Empowered Sensors, Services, and Social Computing on the Ubiquitous Web Amit Sheth • Wright State University n his influential paper “The Computer for focusing on humans interacting with a technol- the 21st Century,” Mark Weiser talked about ogy or system, CHE will feature technology-rich I making machines fit the human environ- human surroundings that often initiate interac- ment instead of forcing humans to enter the tions. Interaction will be more sophisticated and machine’s environment.1 He noted, “The most seamless compared to today’s precursors such as profound technologies are those that disap- automotive accident-avoidance systems. pear. They weave themselves into the fabric of Many components of and ideas associated everyday life until they are indistinguishable with the CHE vision have been around for a from it.” Weiser’s vision, outlined two decades while. Here, I discuss some of the most impor- ago, led to ubiquitous computing. Now, we must tant tipping points that I believe will make CHE again rethink the relationship and interactions a reality within a decade. between humans and machines — this time, including a variety of technologies, includ- Bridging the Physical/Digital Divide ing computing technologies; communication, We’ve already seen significant progress in tech- social-interaction, and Web technologies; and nology that enhances human-computer inter- embedded, fixed, or mobile sensors and devices. actions; the iPhone is a good example. Now We’re on the verge of an era in which the we’re seeing increasingly intelligent interfaces, human experience can be enriched in ways we as exemplified by Tom Gruber’s Intelligence at couldn’t have imagined two decades ago. Rather the Interface technology (http://tomgruber.org/ than depending on a single technology, we’ve pro- news/sdforum-dec13.htm), which has demon- gressed with several whose semantics- empowered strated contextual use of knowledge to develop convergence and integration will enable us to intelligent human–mobile-device interfaces. We’re capture, understand, and reapply human knowl- also seeing progress in how machines (devices edge and intellect. Such capabilities will conse- and sensors), surroundings, and humans inter- quently elevate our technological ability to deal act, enabled by advances in sensing the body, with the abstractions, concepts, and actions that the mind, and place. Such research supports the characterize human experiences. This will herald ability to understand human actions, including computing for human experience (CHE). human gestures and languages in increasingly The CHE vision is built on a suite of tech- varied forms. The broadening ability to give any nologies that serves, assists, and cooperates physical object an identity in the cyber world with humans to nondestructively and unobtru- (that is, to associate the object with its repre- sively complement and enrich normal activi- sentation), as contemplated with the Internet ties, with minimal explicit concern or effort of Things, will let machines leverage extensive on the humans’ part. CHE will anticipate when knowledge about the object to complement what to gather and apply relevant knowledge and humans process. intelligence. It will enable human experiences Human-machine interactions are taking that are intertwined with the physical, concep- place at a new level, as demonstrated by Psyler- tual, and experiential worlds (emotions, senti- on’s Mind Lamp (www.psyleron.com/lamp.aspx), ments, and so on), rather than immerse humans which shows connections between the mind in cyber worlds for a specific task. Instead of and the physical world. Soon, computers will be 88 Published by the IEEE Computer Society 1089-7801/10/$26.00 © 2010 IEEE IEEE INTERNET COMPUTING © 2010 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. Computing for Human Experience able to translate gestures to concrete personality account for much of the will all come together to enable a actionable cues and understand per- background knowledge in a single system that makes conclusions and ceptions behind human observations, mind. Machines also must leverage decisions with human-like intuition, as shown by MIT’s Sixth Sense proj- background knowledge for effective but much more quickly than humans. ect (www.pranavmistry.com/projects/ perception.3 So, effective perception sixthsense). requires a framework for represent- Semantics at an In addition, interactions initi- ing background knowledge. Extraordinary Scale ated in the cyber world are increas- Semantic computing, aided by ing and becoming richer. Examples From Perception Semantic Web technology, is an range from a location-aware system to Semantics ideal candidate framework for mean- telling a smart phone user about a John Locke, Charles Peirce, Ber- ingful representation and sharing of sale item’s availability nearby to trand Russell, and many others have hypotheses and background knowl- advanced processing of sensor data extensively and wonderfully written edge. Together with semantic com- and crowd intelligence to recom- about semiotics — how we construct puting, the large-scale adoption mend a road rerouting or to act on and understand meaning through of Web 2.0 or social-Web technol- behalf of a human. This bridging of symbols. A key enhancement we’re ogy has led to the availability of the physical/digital divide is a key already seeing is the humanization multi modal user-generated content, part of CHE. of data and observation, includ- whether text, audio, video, or sim- ing social computing extending ply attention metadata, from a vari- Elevating Abstractions semantic computing and vice versa. ety of online networks. The most That Machines Understand Metadata is no longer confined to promising aspect of this data is Perception is a key aspect of human structural, syntactic, and seman- that it truly represents a population intelligence and experience. Elevating tic metadata but includes units of and isn’t a biased response or arbi- machine perception to a level closer observations that convey human trary sample study. This means that to that of human perception will be experience, including perceptions, machines now have at their disposal a key enabler of CHE. In 1968, Rich- sentiments, opinions, and intentions. the variety and vastness of data and ard Gregory described perception as Soon, we’ll be able to convert the local and global contexts that a hypothesis over ob servation.2 Such massive amounts of raw data and we use in our day-to-day processing hypothesis building comes naturally ob servations into symbolic representa- of information to gather insights or to people as an (almost) entirely sub- tions. We’ll make these representations make decisions. conscious activity. Humans often more meaningful through a variety of We also see a move from document- interpret the raw sensory observa- relationships and associations we can and keyword-centric information pro- tion before recognizing a conscious establish with other things we know. cessing that relies on search-and-sift thought. On the other hand, hypoth- We’ll then be able to contextually to representing in formation at higher esis building is often cumbersome leverage all this to improve human abstraction levels. This involves mov- for machines. Nevertheless, to inte- activities and experience. ing from entity- or object-centric grate human and machine percep- CHE will bring together many processing to relationship- and event- tion, the convergence must occur at current technological advances in centric processing. This, in turn, this abstraction level, often termed capabilities that are easy and natural involves improving the ability to situation awareness. Therefore, for humans but harder for machines, extract, represent, and reason about regardless