Integrating Usability Models into Pervasive Application Development Paul Holleis München 2008 Integrating Usability Models into Pervasive Application Development Paul Holleis Dissertation an der Fakultät für Mathematik, Informatik und Statistik der Ludwig-Maximilians-Universität München vorgelegt von Paul Holleis aus Bad Reichenhall München, den 15.12.2008 Erstgutachter: Prof. Dr. Albrecht Schmidt Zweitgutachter: Prof. Dr. Heinrich Hußmann Externer Gutachter: Prof. Dr. Antonio Krüger Tag der mündlichen Prüfung: 19.01.2009 To my dad, who would have enjoyed reading this thesis. Table of Contents vii Table of Contents 1 INTRODUCTION AND STRUCTURE .......................................................................................................... 1 1.1 GOALS AND CONTRIBUTIONS .......................................................................................................................... 1 1.2 STRUCTURE ................................................................................................................................................. 2 2 DEVELOPING PERVASIVE APPLICATIONS ................................................................................................ 3 2.1 PERVASIVE COMPUTING ................................................................................................................................. 3 2.1.1 Brief History and Overview .............................................................................................................. 3 2.1.2 Applications and Related Terms ...................................................................................................... 5 2.2 APPLICATION DEVELOPMENT PROCESS ........................................................................................................... 11 2.2.1 Prototyping .................................................................................................................................... 13 2.2.2 Implementation ............................................................................................................................. 15 2.2.3 Deployment ................................................................................................................................... 15 2.2.4 Evaluation ...................................................................................................................................... 16 3 USER MODELS FOR UI DESIGN .............................................................................................................. 19 3.1 USER MODELS – OVERVIEW ......................................................................................................................... 19 3.1.1 Descriptive Models ........................................................................................................................ 20 3.1.2 Predictive Models .......................................................................................................................... 21 3.2 COGNITIVE USER MODELS ............................................................................................................................ 24 3.2.1 GOMS: Goals, Operators, Methods, Selection Rules ...................................................................... 26 3.2.2 KLM: Keystroke-Level Model .......................................................................................................... 29 3.3 DISCUSSION AND APPLICATIONS OF THE GOMS FAMILY OF MODELS ................................................................... 32 3.4 KLM EXTENSIONS FOR ADVANCED MOBILE PHONE INTERACTIONS ...................................................................... 36 3.4.1 Physical Mobile Interactions .......................................................................................................... 36 3.4.2 Model Parameters ......................................................................................................................... 37 3.4.3 User Studies for Time Measurements ............................................................................................ 41 3.4.4 Evaluation of the Extended KLM .................................................................................................... 49 3.4.5 Discussion and Related Work ........................................................................................................ 50 3.5 FURTHER KLM EXTENSIONS ......................................................................................................................... 52 4 TOOLS FOR RAPID APPLICATION DEVELOPMENT .................................................................................. 55 4.1 A REVIEW OF EXISTING PROTOTYPING TOOLKITS .............................................................................................. 55 4.1.1 Hardware-focused Toolkits ............................................................................................................ 55 4.1.2 Software-focused Toolkits ............................................................................................................. 58 4.1.3 Toolkits Tightly Combining Hardware and Software ..................................................................... 68 4.2 TOOLKIT REQUIREMENTS FOR PERVASIVE APPLICATIONS .................................................................................... 70 4.3 EITOOLKIT – DESIGN DECISIONS .................................................................................................................... 84 4.3.1 Envisioned Application Scenarios ................................................................................................... 84 4.3.2 Requirements Identification .......................................................................................................... 85 4.4 ARCHITECTURE AND IMPLEMENTATION ........................................................................................................... 87 viii Table of Contents 5 PROTOTYPING USING EITOOLKIT AND USER MODELS ........................................................................... 95 5.1 DESCRIBING APPLICATION SEMANTICS WITH STATE GRAPHS ............................................................................... 95 5.1.1 Graph Theoretical Foundations ...................................................................................................... 95 5.1.2 Definition of the State Graph ......................................................................................................... 96 5.1.3 Advantages of Using State Graphs in User Interaction Design ...................................................... 97 5.2 GRAPHICAL, STATE-BASED APPLICATION DEVELOPMENT ..................................................................................... 98 5.2.1 Example 1: Output-state-based Development ............................................................................... 98 5.2.2 Example 2: Trigger-action-based Development ........................................................................... 102 5.3 COMBINING MODELS AND STATE-BASED PROTOTYPING TOOLS ......................................................................... 104 5.3.1 KLM Component for Combining Prototyping with User Modelling .............................................. 104 5.3.2 Example 1: Integration into the d.tools Environment .................................................................. 106 5.3.3 Example 2: Integration into the Eclipse Environment .................................................................. 109 6 CASE STUDIES – APPLICATIONS BASED ON THE EITOOLKIT .................................................................. 111 6.1 DEVICE SPECIFIC APPLICATIONS ................................................................................................................... 111 6.2 TECHNOLOGY ENABLING APPLICATIONS ........................................................................................................ 112 6.2.1 Example Projects Using the Particle Microcontroller Platform .................................................... 112 6.2.2 Connection to Third Party Platforms and Components ................................................................ 119 6.3 DATA VISUALISATION TOOL WITH EXCHANGEABLE COMPONENTS ...................................................................... 121 6.4 WEARABLE COMPUTING ............................................................................................................................ 123 6.4.1 Related Work within Wearable Computing ................................................................................. 123 6.4.2 Touch Input on Clothing ............................................................................................................... 126 6.4.3 Touch Input on Mobile Phone Keypads ........................................................................................ 138 7 PROTOTYPING MOBILE DEVICE APPLICATIONS ................................................................................... 143 7.1 INTRODUCTION AND RELATED WORK ............................................................................................................ 143 7.2 CREATING PROTOTYPES OF MOBILE PHONE APPLICATIONS ............................................................................... 147 7.2.1 Generating the Application Behaviour ......................................................................................... 148 7.2.2 Analysing Tasks during Application Creation ..............................................................................