From the Editor in Chief Editor in Chief: M. Satyanarayanan ■ Carnegie Mellon and Intel Research ■ [email protected]

Augmenting Cognition

M. Satyanarayanan

n his futuristic essay “As We May highly demanding cognitive environment quality of life. It can also significantly IThink,” written nearly 60 years ago, such as an aircraft cockpit or a nuclear reduce the attention demanded from care- Vannevar Bush imagined the existence submarine’s control room. If presented givers. Indeed, the Applications depart- of a device called a “Memex” that would unfiltered, the total volume of raw data ment in this magazine’s inaugural issue extend and amplify human thought.1 available in these settings would over- described how an elder care facility in This is one of the earliest descriptions of whelm a human operator, hurting his or Oregon uses pervasive computing tech- using computing to augment human her ability to perform essential cognitive nologies. Recognizing the growing impor- cognition. Until then, computing devices functions. Only by keeping this data tance of the topic, this issue focuses on the were seen primarily as engines that could fairly unobtrusive and by spontaneously role that pervasive computing technolo- reduce the drudgery of laborious hand alerting a user when attention is neces- gies can play in elder care. Our guest edi- calculations. It would take another sary can we make such environments tors, Sumi Helal, Gregory Abowd, and decade, until the mid 1950s, for Herbert hospitable to human cognition. The clue Andrew Sixsmith, represent considerable Simon and Allen Newell to explore the to pervasive computing’s potential here research experience in this domain (for use of computing devices for emulating lies in the phrase, “when attention is nec- more information, see the sidebar). higher cognitive functions such as rea- essary.” Sensing and context-awareness, Looking toward the future, we can soning and problem solving,2 thus key aspects of pervasive computing, are envision computing technologies con- founding the field of artificial intelli- essential to any system that can selectively verging in tantalizing ways to augment gence. But the Memex stands as an imag- alert a user. cognition. For example, imagine a wear- inative first step toward answering the Decline in cognitive function due to old able computer with a head-up display in question, “How can computers help age or to diseases such as Alzheimer’s is the form of eyeglasses and with a built-in humans be smarter?” one of the low-hanging fruits ripe for per- camera for continuous face recognition. Today, we do not see any Memexes vasive computing. Even modest improve- This would offer the essentials of an aug- around us. But a reasonable approxima- ments in an elderly person’s cognitive abil- mented-reality system to aid cognition. tion to Bush’s vision does exist. Instead ity can significantly improve that person’s When you look at a person, his or her of individual devices, the World name could pop up, possibly with Wide Web serves as a collective additional cues to guide your Memex for society. Mechanisms greeting. Such “magic glasses” such as bookmarks and personal could transform your environ- annotation files accomplish a ment into a helpful smart envi- weak form of the personalization ronment. Imagine “water me” Bush described. Information popping up when you glance at stored in databases and file sys- your thirsty houseplant, or “take tems can further customize your me out” appearing when you personal “Memex.” look at your long-suffering dog, or “don’t forget my birthday” CAN PERVASIVE appearing when you say good-bye COMPUTING HELP? to your spouse as you leave for What is pervasive computing’s work! That would be useful for role in this arena? Consider how anyone—and especially valuable instrumentation is organized in a for the elderly.

4 PERVASIVEcomputing Published by the IEEE CS and IEEE ComSoc ■ 1536-1268/04/$20.00 © 2004 IEEE FROM RESEARCH TO REALITY FROM THE GUEST EDITORS Today, this is clearly science fiction. Obvious challenges exist in basic tech- The natural declines in motor, sensory, and cognitive abilities associated with aging impact nologies such as image recognition quality of life and independence. Statistics also suggest that our health and caregiver systems (which is currently nowhere near ade- will be stretched to the limits by the increasing size of the elder population, which accounts quate for these scenarios). Additionally, for the largest growing segment of all populations in the US and many countries worldwide. difficult engineering challenges exist in In this special issue, we focus on applying pervasive computing research in the domain of building a system whose form-factor, “healthy” or “successful” aging. Several research universities and numerous major industries size, weight, and battery life make it as have realized the opportunities pervasive computing can bring in terms of cost-effective and unobtrusive as a pair of eyeglasses yet scalable solutions to successful aging. with sufficient computing power to run We have accepted five papers and invited William Mann, an occupational therapy professor algorithms for, say, image recognition. from the University of Florida, to introduce the aging population and their needs. One promising approach is to extend a The authors of “Automated Analysis of Nursing Home Observations” show preliminary wearable computer’s capabilities by results of the CareMedia project at Carnegie Mellon University, which uses computer vision using nearby computation servers, a techniques to track the progress of patients and staff at a skilled nursing facility for patients concept known as cyber foraging.3 This with advanced dementia. issue’s Wearable Computing department In “Technology for Care Networks of Elders,” the authors describe some of the work in explores cyber foraging’s role in extend- Intel’s Proactive Health Initiative, which aims to provide information systems to an aging ing battery life. What will such a system adult’s supporting social networks. do if you go to a place with no compute “Cyber Crumbs for Successful Aging with Vision Loss” evaluates the viability and utility of resources nearby? The most likely fall- an indoor orientation and wayfinding system for people with visual impairments. back is for the “magic eyeglasses” to The authors of “Aware Technologies for Aging in Place: Understanding User Needs and behave as plain eyeglasses—no longer Attitudes” present work from the Aware Home Research Initiative at , evaluating, annotating the physical world. Then, from the user perspective, three services designed to offset some of the instrumental and upon returning to a resource-rich envi- social problems encountered in later life. ronment, cyber foraging would bring Finally, “A Smart Sensor to Detect the Falls of the Elderly” presents an intelligent fall detec- back the “magic.” tor system based on a low-cost, passive infrared sensor array. They demonstrate how they It is hard to predict when systems of analyzed user needs to arrive at a suitable functional specification for the sensor. this kind will become off-the-shelf prod- ucts. At the moment, they do not exist even as lab prototypes. Fortunately, com- Gregory D. Abowd is an associate professor in the College of Computing and pelling visions of the future have a habit the GVU Center at the Georgia Institute of Technology. His research inter- of becoming true sooner than most peo- ests involve applications research in , concerning both ple think possible. So, I am counting on HCI and software engineering issues. He received his DPhil in computation magic eyeglasses to become available by from the . He is a member of the IEEE Computer Society the time I am desperately in need of them, and ACM. Contact him at [email protected]; www.cc.gatech.edu/ say a decade or two at most. Sooner fac/Gregory.Abowd. would be better—on some hectic days, I could already use them! Sumi Helal is a professor in the Computer and Information Science and Engi- neering Department at the University of Florida. He is also the founder, pres- ident, and CEO of Phoneomena. His research interests span the areas of REFERENCES mobile and wireless computing and networking, collaborative computing, and Internet applications. He received his PhD in computer sciences from 1. V. Bush, “As We May Think,” Atlantic Monthly, July 1945, pp. 101–108; www. Purdue University. He is a senior member of the IEEE and a member of the theatlantic.com/unbound/flashbks/ ACM and Usenix. Contact him at [email protected]; www.cise.ufl.edu/~helal. computer/bushf.htm.

2. A. Newell and H.A. Simon, “The Logic Andrew Sixsmith is a lecturer in social gerontology in the Department of Theory Machine: A Complex Information Primary Care at the University of Liverpool, UK. His research interests focus Processing System,” IRE Trans. Informa- on the quality of life of older people, social care evaluation, and the use of tion Theory, vol. 2, 1956, pp. 61–79. new technology to facilitate independent living. He is a member of the British 3. M. Satyanarayanan, “Pervasive Comput- Society of Gerontology and the International Society for Gerontechnology. ing: Vision and Challenges,” IEEE Per- Contact him at [email protected]. sonal Comm., Aug. 2001, pp. 10–17.

APRIL–JUNE 2004 PERVASIVEcomputing 5