How we Got Here and Where to Take it
Scott Hudson HCI Institute Carnegie Mellon Instagram: Two guys, two years à 30 million users (now 300 million)
What made that possible? Enablers
“Making it easy to make things easy” àAdvances in UI Software
Starting with tools… Starting in the early and mid-80’s, technical HCI community had a very clear goal:
Make it easy enough to create these “new graphical user interface thingees” to allow people with minimal programming skills to do it (quickly & well)
At the time it took days of work by programmers with fairly rare skills to build even simple interfaces Starting in the early and mid-80’s, technical HCI community had a very clear goal:
Make it easy enough to create these “new graphical user interface thingees” to allow people with minimal programming skills to do it (quickly & well)
At the time it took days of work by programmers with fairly rare skills to build even simple interfaces A Great Concept: User Interface Management System (UIMS) Analogy to DBMS Major line of work: Layered modular abstractions like other big systems like e.g., compilers A Great Concept: User Interface Management System (UIMS) Analogy to DBMS
for (i=0; i<9; i++) { Major line of work: Layered modular if (i== 42) {
abstractions like other big systems Lexer like e.g., compilers Parser
Optimizer (optional) AST
Once you work out the task Code Generator
abstractions can refine each layer, etc… 0100101010 10101110 Lots of Systems Were Produced With This Mindset
TIGER [Kasik ‘82] Lots of Systems Were Produced With This Mindset
TIGER [Kasik ‘82]
Syngraph [Olsen ‘83] Lots of Systems Were Produced With This Mindset
TIGER [Kasik ‘82]
Syngraph [Olsen ‘83]
IPDA [Olsen ‘84] Lots of Systems Were Produced With This Mindset
TIGER [Kasik ‘82]
Syngraph [Olsen ‘83]
IPDA [Olsen ‘84] ADM [Schulert ‘85] Lots of Systems Were Produced With This Mindset
TIGER [Kasik ‘82]
Syngraph [Olsen ‘83]
IPDA [Olsen ‘84] ADM [Schulert ‘85] GWUIMS [Sibert ‘86] Lots of Systems Were Produced With This Mindset
TIGER [Kasik ‘82]
Syngraph [Olsen ‘83] AIDE [Hix ‘86]
IPDA [Olsen ‘84] ADM [Schulert ‘85] GWUIMS [Sibert ‘86] Lots of Systems Were Produced With This Mindset
TIGER [Kasik ‘82]
Syngraph [Olsen ‘83] AIDE [Hix ‘86]
IPDA [Olsen ‘84] ADM [Schulert ‘85] GWUIMS [SibertA ‘L86]GAE [Bergeron ‘87] Modules and Layers Boil Down to Something Like This
Input Application UI Core Appl User Output Interface And this led directly to… Very little we see today…
at least not directly, it took a lot more progress first (Resulting interfaces were typically mediocre) Because we had missed something really fundamental Our CS intuition/methodology told us that if we could understand and decompose “the input/processing/output problem” of UI we’d have it
But we missed that this little bit on the left (“the user”) actually changes everything
User User Centered Systems Becomes a much harder problem… e.g., can’t open the user’s head and pour in the right mental model To do well you have to structure each system around the user’s concepts (AND do all the rest)
à
The user profoundly impacted what should have been purely technical issues In the end the big advance wasn’t a particular architecture, algorithm, or system
It was really a change in mind set à To be user centered even in what seemed like strictly technical issues
Led to adopting interdisciplinary methodologies (and a whole new set of issues to be addressed) At the Same Time…
Lots of other research was going on before, during, and after UIMS work
The shift to a user centered mindset helped all of it à We now see the fingerprints of the work all over modern interactive systems UIMS Work in the Overall UI
SmallTalk Research/Product Flow
Java Industrial Toolkits ••• [Sun, Apple, …] Swing
Early Academic Toolkits Adv. Academic Toolkits Many [CMU Andrew, MIT X Windows, …] [Stanford, CMU, UofAZ, GATech, …] Current Toolkits Input Academic UIMS Work Models Interactive systems infrastructures today are converging and we can trace many things we see there back through research toolkits that followed UIMS work Examples
Three major research toolkit efforts through the 90s Stanford [Linton et al.]: Interviews and Fresco CMU [Myers et al. ]: Garnet and Amulet UofAz/GATech [Hudson et al.]: Artkit and subArctic
Some traceable contributions : Concurrency model [subArctic] à Swing à many Layout abstractions [Interviews] à Swing à many Layout w/ constraints [Garnet, subArctic, Others*] à Adobe, Apple Input dispatch model (heavily modified/adapted) [Garnet/Amulet, Artkit/subArctic] à Swing à many Resizable rich icons [subArctic] à Apple, Android … Animation abstractions [Artkit, Amulet] à Android, … Examples
Three major research toolkit efforts through the 90s Stanford [Linton et al.]: Interviews and Fresco CMU [Myers et al. ]: Garnet and Amulet UofAz/GATech [Hudson et al.]: Artkit and subArctic
Some traceable contributions : Concurrency model [subArctic] à Swing à many Layout abstractions [Interviews] à Swing à many Layout w/ constraints [Garnet, subArctic, Others*] à Adobe, Apple Input dispatch model (heavily modified/adapted) [Garnet/Amulet, Artkit/subArctic] à Swing à many Resizable (9-part) icons [subArctic] à Apple, Android … Animation abstractions [Artkit, Amulet] à Android, …
*esp. Alan Borning’s work at UWash Other Tool Successes: UI “Builders”
Tools to let you draw the (graphical parts of the) interface
SOS (INRIA) [Hullot ‘81] Trillium (PARC) à NeXTInterface Builder [Henderson ‘81, ’86] Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in “Making it easy to make things easy”
à Nearly every modern IDE now supports this Microsoft produced a whole series of products: Visual Basic, Visual C++, … Visual Studio where the “Visual” part was a UI Builder Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in “Making it easy to make things easy”
à Nearly every modern IDE now supports this Microsoft produced a whole series of products:
Visual Basic, Visual C++, … Visual Studio where the “Visual” part was a UI Builder Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in “Making it easy to make things easy” Microsoft
à Nearly every modern IDE now supports this Microsoft produced a whole series of products:
Visual Basic, Visual C++, … Visual Studio where the “Visual” part was a UI Builder Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in “Making it easy to make things easy” Microsoft Apple
à Nearly every modern IDE now supports this
Microsoft produced a whole series of products: Visual Basic, Visual C++, … Visual Studio where the “Visual” part was a UI Builder Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in “Making it easy to make things easy” Microsoft Apple
Google à Nearly every modern IDE now supports this Microsoft produced a whole series of products:
Visual Basic, Visual C++, … Visual Studio where the “Visual” part was a UI Builder Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in “Making it easy to make things easy” Microsoft Apple
Google à Nearly every modern IDE now supports this Microsoft produced a whole series of products:
Visual Basic, Visual C++, … Visual Studio where the “Visual” part was a UI Builder Principle: Visual things should be expressed visually (with minimal code) This turned out to be extremely important in “Making it easy to make things easy” Microsoft Apple
Google à Nearly every modern IDE now supports this
Microsoft produced a whole series of products: Visual Basic, Visual C++, … Visual Studio where the “Visual” part was a UI Builder More Successes: Interaction Techniques Dozens (if not 100s) of interaction techniques in the literature “Pinch” gesture (simultaneous scale and translate) More Successes: Interaction Techniques Dozens (if not 100s) of interaction techniques in the literature “Pinch” gesture (simultaneous scale and translate)
Dates to Kruger’s 1983 VideoPlace system Speaking of Scale: Zoomable Interfaces (ZUIs)
PA3D [Bederson’94] Pioneered using the dimension of scale for interaction à Never run out of space Last Example
Touring Machine Project (Columbia) [Feiner et al. ‘97] Last Example
Touring Machine Project (Columbia) [Feiner et al. ‘97]
Mobile computation Last Example
Touring Machine Project (Columbia) [Feiner et al. ‘97]
Mobile computation Wireless communications Last Example
Touring Machine Project (Columbia) [Feiner et al. ‘97]
Mobile computation Wireless communications GPS Last Example
Touring Machine Project (Columbia) [Feiner et al. ‘97]
Mobile computation Wireless communications GPS Handheld Display & Input (pen rather than touch) Last Example
Touring Machine Project (Columbia) [Feiner et al. ‘97]
Mobile computation Wireless communications GPS Handheld Display & Input (pen rather than touch) Last Example
Touring Machine Project (Columbia) [Feiner et al. ‘97]
Mobile computation Wireless communications GPS Handheld Display & Input (pen rather than touch) + See Through Display Last Example
Touring Machine Project (Columbia) [Feiner et al. ‘97]
Mobile computation Wireless communications GPS Handheld Display & Input (pen rather than touch) + See Through Display Taking a user-centered approach leads to moving from
“How do I implement it?”, “What’s the algorithm/architecture?” to
“If I had
And the UI part is simple enough that many people we would think of as non-programmers can do it àthat is a critical game changer that leads to an app for everything A Success Story That Led to Apps (and Many Other Things Interactive) Today we have the tools that were the goal of the early work à you can create a highly usable small interface in a few hours (sometimes minutes) rather than a few days (or weeks or months)
And the UI part is simple enough that many people we would think of as non-programmers can do it àthat is a critical game changer that leads to an app for everything And There are Many Other Success Stories I Didn’t Talk About And There are Many Other Success Stories I Didn’t Talk About
Wearables
Thad will talk about that next… And There are Many Other Success Stories I Didn’t Talk About
Wearables
Context Aware Computing à
Google Now And There are Many Other Success Stories I Didn’t Talk About
Wearables
Context Aware Computing à
à Internet of Things
Personal Tracking (Quantified Self) And There are Many Other Success Stories I Didn’t Talk About
Wearables
Context Aware Computing
Recognition Based UI à
Microsoft Kinect And There are Many Other Success Stories I Didn’t Talk About
Wearables
Context Aware
Recognition Based UI
Visualization à Overall the mindset change from a systems view to a user-centered view has been a game changer which has amplified the impact of many things
Even more so than individual technologies, ideas that amplify other ideas are what we should be after Where Does This Go Next?
Lessons from UI tools point to possibilities for “Democratization of Computational Power”
àCan we make what used to take teams of programmers something that e.g. every mom and pop small business can bring to bear on their own specialized problems?
We now know a lot about how to make tools usable –I think we can do this. Questions and Discussion?
Scott Hudson, CMU Contact: http://scotthudson.com