Designing Energy and User Efficient Interactions with Mobile Systems Lu Luo CMU-ISR-08-102 April 2008 School of Computer Science Institute for Software Research Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Daniel P. Siewiorek, Chair Bonnie E. John Priya Narasimhan Diana Marculescu, Department of Electrical and Computer Engineering Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy. Copyright © 2008 Lu Luo This research was sponsored by the Defense Advanced Project Agency (DARPA) under Contract No. NBCHD030010, the National Science Foundation (NSF) under Grant Nos. EEEC-540865 and 0205266, the Office of Naval Research (ONR), N00014-03-1-0086, and HP Labs. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies or endorsements, either express or implied, of DARPA, the NSF, ONR, HP Labs, Carnegie Mellon University, the U.S. Government, or any other entity. Keywords: Mobile computing, energy efficiency, user interaction, user interface design, cognitive modeling, ubiquitous computing Abstract Mobile computing has thrived to provide unprecedented user experiences beyond the boundary of desks and wires. The increasing demand for mobility faces two major challenges: reduced form factor and limited energy supply. Although each challenge has been addressed by significant research efforts, the correlation between them is underexplored. Energy efficiency should be integrated as an important metric of user interaction design in mobile systems. By carefully considering the specific requirements of the user and the context of usage, energy efficiency can be achieved without sacrificing user performance and satisfaction, and interaction design that facilitates user efficiency can also promote energy efficiency. This dissertation starts with a detailed study of typical workload and screen usage that shows that users seldom use the entire screen on most workloads. Hence, Dark Windows, an energy efficient display design is presented and implemented to optimize display power consumption by adjusting the color and illumination of different screen regions according to the user’s workload. This dissertation next presents AstroRDS, a mobile computing system and network infras- tructure that displays documents and information from user’s mobile devices on ambient ubiqui- tous display resources, thus facilitates much better viewing experience of the user with compara- tively ease of use. The energy consumption and network usage of AstroRDS is several orders of magnitudes less than VNC remote desktop protocol on viewing and controlling tasks, and with similar initial loading overhead. A common challenge in these two efforts is the high cost of measuring user experience and energy consumption. This dissertation further presents KLEM, a quantitative methodology based on cognitive modeling techniques that can predict both user performance and system energy consumption from story boards of an interactive task during early stages of design. KLEM can be used as a convenient tool to compare and make early decisions among different design options, as well to resolve potential design issues before investing in actual user testing and iterative development. While KLEM is presented with a focus on handheld devices, the methodology can be applied on any interactive mobile system. iii iv Acknowledgements More than once I was warned against writing a PhD dissertation whilst working full-time, I am glad that I managed it in three months. This dual-task period has been very demanding indeed, but I found it exciting to get continuously inspired by new ideas from work for my writing, as well as to see how my dissertation research may be further extended and applied in the real world. Because of this I wish to express my sincere appreciation to my advisor, Prof. Dan Siewiorek for encouraging me to reach out beyond graduate school and collaborate with industrial labs on various projects that I was interested in, meanwhile keeping me focused on the right track of my dissertation work. Dan’s insights, methods and knowledge not only made invaluable contributions to this dissertation, but also influenced my development as a research professional. Most importantly, I have been fortunate to work with an advisor like Dan, who was always there for my questions and concerns, who never failed to correct even the tiniest typos in my writings, and who treated all his students with care and the warmth of family. My thesis committee is quite special in that except for Dan, all other three members are female. I may not have worked directly with all of them, but I regard them as my role models for developing a research career as a woman. I am indebted to Prof. Bonnie John for teaching me cognitive modeling for one semester and shaping the research on KLM for mobile user interfaces. The KLEM method would not be applicable without the ever improving CogTool provided by Bonnie’s group. Besides providing prompt and insightful feedback on my research progress, Prof. Diana Marculescu helped validating on the energy characterization methods used in my research, and Prof. Priya Narasimhan’s “How to Write a Good (no, Great) PhD Dissertation” presentation served as a great guidance during my dissertation writing. I was fortunate to have had help and friendship from so many talented people at Carnegie Mellon University. I have built Chapter 4 upon the work in collaboration with Harvey Vrsalovic and Matthew Hornyak. As office mates we pulled many all-nighters together building systems and running experiments, and as good friends it was always fun to exchange crazy ideas and to have technical and profesisonal discussions with them. During the years of my PhD study, Rong Zhang, Joshua Anhalt, and Ravi Mosur helped me a lot building a speech decoder built on CMU Sphinx, which familiarized me with the basics of speech recognition systems. On building cognitive models, Peter Centgraf provided the EventLogger for Palm PDAs. Gus Preva, Alex Eiser, Don Morrison, and Jason Cornwell have helped me setting up CogTool or adding new modeling features for me. My graduate study would not have been possible without the guidance from Profs. Bill Scherlis, David Garlan, and Mary Shaw, who established the software v engineering PhD program, and it was a truly great experience for me to grow with the program as one of the first batch of PhD students. My special thanks to Ms. Laura Forsyth, an amazing lady and former car racer, who is always there when I need help. I would also like to thank Prof. Asim Smailagic, Connie Herold, Helen Higgins, Walter Schearer, Marian D’Amico, Alicia Brown, and all SCS and ICES supporting staff that have made my school life so much easier. My PhD research also involved collaboration with many industrial researchers. In summer 2002 I worked as an intern at the HP Labs (former Compaq Western Research Lab) with Drs. Bob Mayo and Partha Ranganathan and Chapter 3 is built upon this work. The IBM PhD Fellowship funded me for the school year 2002 to 2003, and in summers 2003 and 2004 I worked as an intern at the IBM T. J. Watson Research Center with Drs. Chandra Narayanaswami, Michael Olsen and Marcel Rosu. I learned a great deal from all my internship mentors and would like to thank them for the valuable research experience at such world-class institutes. Truly great friends are hard to find, difficult to leave, and impossible to forget. My life at CMU would have been much less enjoyable without the great friendships from Madhavi, Stella, Joy, and many other people. I would also like to thank all my colleagues and friends in Pittsburgh and New York area, you mean a great deal to me. Finally, I would like to thank my parents for loving me, trusting me, and supporting me all the time, and my wonderful husband Edward for filling my life with with so much love, care, and laughter, you make me a better and happier person everyday. Lu “Annie” Luo April, 2008 vi Contents 1 Introduction 1 1.1 Energy efficiency as a design metric . 2 1.2 Energy evaluation during early design . 4 1.3 Roadmap . 5 2 Energy and Mobile User Interaction 7 2.1 Interfacing hardware . 8 2.2 Interfacing software . 10 2.3 Interfacing human . 11 2.4 Summary . 12 3 Energy Adaptive Display 13 3.1 User study on screen usage . 13 3.1.1 Method . 14 3.1.2 Results . 14 3.1.3 Summary of user study . 19 3.2 Energy-adaptive display sub-systems . 20 3.2.1 System design . 20 3.2.2 Implementation . 21 3.2.3 Evaluation method . 22 3.3 Experimental results . 25 3.3.1 Power benefits . 25 3.4 User acceptance evaluation . 27 3.4.1 Method . 27 3.4.2 Results . 28 3.5 Discussion . 28 3.6 Summary . 31 4 Ambient Display 33 4.1 System architecture . 34 4.1.1 Remote display system . 34 4.1.2 Discovery protocol . 35 vii 4.1.3 Software architecture . 36 4.1.4 Lightweight user interface . 37 4.1.5 Ambient display and display protocol . 38 4.1.6 Limitation . 38 4.2 Experimental results . 39 4.2.1 Network usage analysis . 39 4.2.2 Power consumption analysis . 40 4.3 Summary . 43 5 Predicting Mobile User Performance 45 5.1 Modeling user performance . 46 5.1.1 Cognitive engineering models . 46 5.1.2 The GOMS model . 48 5.1.3 The Keystroke-Level Model . 49 5.1.4 Modeling parallel activities . 50 5.2 Verifying KLM on pen-based interfaces . 50 5.2.1 Task definition . 51 5.2.2 Model creation . 52 5.2.3 User study . 53 5.2.4 Result analysis . 54 5.3 Estimate system response time . 56 5.4 Summary .