Prof. James Landay CSE 441 – Spring 2012
Hall of Fame or Hall of Shame?
Mobile Computing & • Windows Mobile Communication home screen • What will I use this phone for other Prof. James A. Landay than as a phone? University of Washington CSE 441 Spring 2012
* Based on slides of Prof. Scott Klemmer, Stanford
Spring 2012 CSE 441 - Advanced HCI 2
Hall of Shame!
• Too many clicks to do Mobile Computing & anything • Calendar Communication – tap “Start” – scroll through icons to find the one I want Prof. James A. Landay – less if used it recently University of Washington (then at top, but still CSE 441 several clicks) Spring 2012
* Based on slides of Prof. Scott Klemmer, Stanford
Spring 2012 CSE 441 - Advanced HCI 3
Outline
• History of miniaturization & mobility • Palm Pilot • iPhone
Spring 2012 CSE 441 - Advanced HCI 5 6
1 Prof. James Landay CSE 441 – Spring 2012
Sony Walkman (1979)
Spring 2012 CSE 441 - Advanced HCI Spring 2012 CSE 441 - Advanced HCI 8
Car Phone (1980s-90s) 6 billion Mobile Phones Worldwide (2011)
Mobile Cellular Subscriptions per 100 inhabitants Developed 118 Developing 79 World 87
Data courtesy ITU (International Telecommunication Union), 2011 http://www.itu.int/ITU-D/ict/statistics/at_glance/KeyTelecom.html
Spring 2012 CSE 441 - Advanced HCI 9 Spring 2012 CSE 441 - Advanced HCI 10
“You Will” – the future comes Mobile Design Evolving Rapidly! slower than we’d like
Newton (1993) Palm Pilot (1997) iPhone (2007)
Spring 2012 CSE 441 - Advanced HCI 11 Spring 2012 CSE 441 - Advanced HCI 12
2 Prof. James Landay CSE 441 – Spring 2012
There was the Newton … The Newton Had Problems… Design Issues • Physical size – too big • Connectivity – not much • Recognition – relied on it too much, didn’t work well enough
Apple Newton MessagePad The Newton OS GUI Photograph of Newton screen screen displaying displaying a Note Checklist, some with text, “ink “Hey, Take a memo “Beat Up “Baahh!” bullet points checked text”, a sketch, & on your Newton” Martin” and/or "collapsed" vectorized shapes The Original Apple Newton's handwriting recognition was made light of in The Simpsons
Source: The Simpsons, Wikipedia episode Lisa on Ice Spring 2012 CSE 441 - Advanced HCI 13 Spring 2012 CSE 441 - Advanced HCI 14
Prototyping the Palm The Palm Pilot Improved… hardware, form factor, software
Design Wins • Physical size: fits in the front pocket • Connectivity: easy sync
• Recognition: simple graffiti Jeff Hawkins, Palm
Graffiti Rob Haitani, Palm OS [Designs] what should be most prominent based on frequency of use, and makes most often used interactions accessible in a single step. Pocket size HotSync Palm OS Source: Palm 1000 Retrospective, Palm V, Rob Haitani in Moggridge, Designing Interactions. Ch. 3. From the Desk to the Palm. http://www.designinginteractions.com/interviews/RobHaitani Spring 2012 CSE 441 - Advanced HCI 15 Spring 2012 CSE 441 - Advanced HCI 16
Palm Pilot Prototypes
http://www.computerhistory.org/collections/accession/102716262
CSE440 - Winter 2012 User Interface Design, Prototyping, and Evaluation 17 Spring 2012 CSE 441 - Advanced HCI 18
3 Prof. James Landay CSE 441 – Spring 2012
Technology Trends
processing
Spring 2012 CSE 441 - Advanced HCI 19 Spring 2012 CSE 441 - Advanced HCI 20
Technology Trends Technology Trends
disk bandwidth
Spring 2012 CSE 441 - Advanced HCI 21 Spring 2012 CSE 441 - Advanced HCI 22
Technology Trends Technology Trends
RAM Devices on Internet
Spring 2012 CSE 441 - Advanced HCI 23 Spring 2012 CSE 441 - Advanced HCI 24
4 Prof. James Landay CSE 441 – Spring 2012
Technology Trends Technology Trends
Imaging Resolution
Display Resolution
Spring 2012 CSE 441 - Advanced HCI 25 Spring 2012 CSE 441 - Advanced HCI 26
Technology Trends Technology Trends
Unaided human abilities Size of pockets
Spring 2012 CSE 441 - Advanced HCI 27 Spring 2012 CSE 441 - Advanced HCI 28
What will we do with Mobile?
• The same applications? • Different ones?
• Some of both is most likely… but the context & constraints differ
Spring 2012 CSE 441 - Advanced HCI 29
5 Prof. James Landay CSE 441 – Spring 2012
Malaysia Integrated Compass. Why?
Spring 2012 CSE 441 - Advanced HCI 32
Grameen Telecom Village Phone What Makes Mobile Design Exciting? Many Design Choices • Think different from GUI/Web • Swiss army vs. dedicated • Pen/speech/touch/gesture modalities • Integrate with other real-world tasks • Social apps
Always in your pocket* or w/ you!
*often not true for women Spring 2012 CSE 441 - Advanced HCI 33 Spring 2012 CSE 441 - Advanced HCI 34
What Makes Mobile Design Difficult? Mobilize ≠ Miniaturize Design constraints • Limited attention/Interactions bursty – sometimes not true (people use phones stationary sometimes for long times) – see Ubicomp ‘2006 paper by Patel • Screen size small • Form factor • Limited network connectivity • Speech / pen / multimodal
Spring 2012 CSE 441 - Advanced HCI 35 Spring 2012 CSE 441 - Advanced HCI 36
6 Prof. James Landay CSE 441 – Spring 2012
Mobile Usage Context Limited Attention & Input Interaction
• Mobile device always with user & on • Minimize keystrokes • Use gives clues to context… • Provide overview + detail – Calendar • Understandable interface at a glance – Job schedule • Design with tasks – Repair man example… • Minimum set of functions • Location gives many contextual cues – .. • Simple activity inference gives context – Driving? Adapt how? Spring 2012 CSE 441 - Advanced HCI 37 Spring 2012 CSE 441 - Advanced HCI 38
Example approach: Nokia Navi-Key Mobile Input: Lots of Research
Reducing number of buttons
Source: Scott Jenson, The Simplicity Shift. Cambridge University Press, 2002.
Spring 2012 CSE 441 - Advanced HCI 39 Spring 2012 CSE 441 - Advanced HCI 40
Disambiguation w/ Dictionary Disambiguation w/ Predictive • Dictionary based (such as T9, Pocket PC) – e.g., 2-2-5-3 • Predictive (such as Letterwise) – able 2-2-5-3-0 – e.g., t-h- – cake 2-2-5-3-N-0 – e A% – bald 2-2-5-3-N-N-0 – i B% – calf 2-2-5-3-N-N-N-0 – o C% – u D% • Lots of “N” = Next – …
Source: Microsoft, MacKenzie, I. S., Kober, H., Smith, D., Jones, T., Skepner, E. (2001). LetterWise: Prefix-based disambiguation for mobile text input. Proceedings of the ACM Symposium on User Interface Software and Technology - UIST 2001, pp. 111-120. New York: ACM. Source: Microsoft, MacKenzie, I. S., Kober, H., Smith, D., Jones, T., Skepner, E. (2001). LetterWise: Prefix-based disambiguation for mobile text input. Proceedings of the ACM Symposium on User Interface Software and Technology - UIST 2001, pp. 111-120. New York: ACM. Spring 2012 CSE 441 - Advanced HCI 41 Spring 2012 CSE 441 - Advanced HCI 42
7 Prof. James Landay CSE 441 – Spring 2012
Dictionary vs. Predictive Case Study: iPhone Input
Design distinctions • Multi-touch Input • Disambiguation of input • Animations
Figure 11. Comparison of entry rates (wpm) with practice for LetterWise, T9, and Multitap. (Note: LetterWise and Multitap figure are from Figure 6. Simulated T9 figures are from Figure 10 with 0.85 frequency of words in dictionary) Internet + Predictive Multi-touch | Mac Music + Touch keyboard OS X | Wireless | Source: MacKenzie, I. S., Kober, H., Smith, D., Jones, T., Skepner, E. (2001). LetterWise: Prefix-based disambiguation for mobile text input. Proceedings of the ACM Accelerometer | Symposium on User Interface Software and Technology - UIST 2001, pp. 111-120. New York: ACM. Phone Spring 2012 CSE 441 - Advanced HCI 43 Spring 2012 CSE 441 - Advanced HCI Proximity Sensor 44
iPhone Typing Algorithm State of the Art: Shapewriter
• Model where a user touched on the screen • Model the layout of keys and what keys surround the touch • If word not in dictionary (or if an extremely unlikely word), present alternative • While user types, dynamically adjust (invisible) target sizes of keys • User can accept by simply tapping ‘Space’
Spring 2012 CSE 441 - Advanced HCI 45 Spring 2012 CSE 441 - Advanced HCI 46
System/Service Design The Future: Sensor Networks
Live Ad Hoc Sensor A handful of Network showing network sensor Light Intensity 'dots'
Lots of 'dots' - getting ready for the big demo
Source: UC Berkeley Smart Dust Program, Largest Tiny Network Yet, http://webs.cs.berkeley.edu/800demo/`
Spring 2012 CSE 441 - Advanced HCI 47 Spring 2012 CSE 441 - Advanced HCI 48
8 Prof. James Landay CSE 441 – Spring 2012
The Future: Mobile Everywhere Information Appliances • A 2002 study calculated there were around 4.2 million CCTV • Mobile devices with dedicated purpose cameras in the UK - one for every 14 people. • “If you go forward 50 years, you are probably talking about one
million forms of sensor per There could be one Amazon Kindle eBook person in the UK,” he said. million sensors per UK resident by 2057 • This was a conservative estimate, he said. “More aggressive” calculations suggest there could be 20m sensors per person.
Source: BBC, “Sensor rise powers life recorders”
Spring 2012 CSE 441 - Advanced HCI 49 Spring 2012 CSE 441 - Advanced HCI 50
9