A Brief History of Human-Computer Interaction Technology Brad A. Myers 44 interactions...march + april 1998 article Research in human–computer interac- Figure 1 shows the time span for some of the technologies discussed in this article. tion (HCI) has been spectacularly suc- including when they were introduced. Of course, a deeper analysis would reveal signifi- cessful and has fundamentally changed cant interaction among the university, corpo- rate research, and commercial activity lines. It Brad A. Myers computing. One example is the ubiqui- is important to appreciate that years of Human–Computer research are involved in creating and making Interaction Institute tous graphical interface used by these technologies ready for widespread use. School of Computer The same will be true for the HCI technolo- Science Microsoft Windows 95, which is based gies currently being developed that will pro- Carnegie Mellon vide the interfaces of tomorrow. University on the Macintosh, which is based on Clearly it is impossible to list every system Pittsburgh, PA 15213- and source in a paper of this scope, but I have 3891 work at Xerox PARC, which in turn is tried to represent the earliest and most influ- ential systems. Further information can be [email protected] based on early research at the Stanford found in other surveys of HCI topics (see, for example, [1, 11, 36, 41]). Another useful Research Laboratory (now SRI) and at resource is the video All The Widgets, which shows the historical progression of a number Massachusetts Institute of Technology. of user interface ideas [27]. The technologies discussed in this paper Another example is that virtually all soft- include fundamental interaction styles such as ware written today employs user interface direct manipulation, the mouse pointing toolkits and interface builders—concepts that device, and windows; several important kinds were developed first at universities. Even the of application areas, such as drawing, text remarkable growth of the World Wide Web is editing, and spreadsheets; the technologies a direct result of HCI research: applying that will likely have the biggest impact on hypertext technology to browsers allows one interfaces of the future, such as gesture recog- to traverse a link across the world with a click nition, multimedia, and three-dimensionality; of the mouse. More than anything else, and the technologies used to create interfaces improvements to interfaces have triggered this using the other technologies, such as user explosive growth. Furthermore, the research interface management systems, toolkits, and that will lead to the user interfaces for the interface builders. computers of tomorrow is being carried out today at universities and a few corporate Basic Interactions research labs. Direct Manipulation of This paper attempts to briefly summarize Graphical Objects many of the important research developments The now ubiquitous direct manipulation inter- in HCI technology, emphasizing the role of face, where visible objects on the screen are university research, which may not be widely directly manipulated with a pointing device, recognized. By “research,” I mean exploratory was first demonstrated by Ivan Sutherland in work at universities and government and cor- Sketchpad [47], the thesis of his doctoral dis- porate research labs (such as Xerox PARC) sertation in 1963. Sketchpad supported the that is not directly related to products. By manipulation of objects using a light pen, “HCI technology,” I am referring to the com- including grabbing objects, moving them, puter side of HCI. A companion article on the changing size, and using constraints. It con- history of the “human side,” discussing the tained the seeds of myriad important interface contributions from psychology, design, ideas. The system was built at Lincoln Labs human factors, and ergonomics would also be with support from the U.S. Air Force and the appropriate. National Science Foundation (NSF). interactions...march + april 1998 45 KEY: University Research Corporate Research Figure 1. Approximate time lines showing where Commercial Products and when work was performed on some major technologies discussed in this article. Direct Manipulation of Graphical Objects 1945 1955 1965 1975 1985 1995 The Mouse 1945 1955 1965 1975 1985 1995 Windows 1945 1955 1965 1975 1985 1995 Text Editing 1945 1955 1965 1975 1985 1995 HyperText 1945 1955 1965 1975 1985 1995 Gesture Recognition 1945 1955 1965 1975 1985 1995 46 interactions...march + april 1998 article William Newman’s Reaction Handler [33], NLS project (funding from ARPA, NASA, created at Imperial College, London during and Rome ADC) [10]. It was intended to be a 1966 and 1967, provided direct manipulation cheap replacement for light pens, which had of graphics and introduced Light Handles been used at least since 1954 [11, p. 68]. [32], a form of graphical potentiometer that Many of the current uses of the mouse were was probably the first “widget.” Another early demonstrated by Doug Engelbart as part of system was AMBIT/G (implemented at the NLS in a movie created in 1968 [9]. The Massachusetts Institute of Technology’s mouse was then made famous as a (MIT) Lincoln Labs in 1968 and practical input device by Xerox funded by the Advanced Research PARC in the 1970s. It first Projects Agency (ARPA). It “THE SKETCHPAD appeared commercially as part employed, among other inter- of the Xerox Star (1981), the SYSTEM MAKES IT face techniques, iconic repre- Three Rivers Computer sentations, gesture recog- POSSIBLE FOR A Company’s PERQ (1981) nition, dynamic menus with MAN AND A COM- [25], the Apple Lisa items selected using a point- (1982), and the Apple ing device, selection of icons PUTER TO CON- Macintosh (1984). by pointing, and moded VERSE RAPIDLY and mode-free styles of Windows THROUGH THE interaction. Multiple tiled windows David Canfield Smith MEDIUM OF LINE were demonstrated in coined the term “icons” in his DRAWINGS.” Engelbart’s NLS in 1968 1975 doctoral thesis on Pyg- [9]. Early research at Stan- — IVAN SUTHERLAND malion [44] (funded by ARPA [47, P. 329] ford on systems like COPI- and National Institute of Mental LOT (1974) [49] and at MIT Health – NIMH). Smith later popu- with the EMACS text editor (1974) larized icons as one of the chief designers [46] also demonstrated tiled windows. of the Xerox Star [45]. Many of the interac- Alan Kay proposed the idea of overlapping tion techniques popular in direct manipula- windows in his 1969 doctoral thesis [17], and tion interfaces, such as how objects and text overlapping windows first appeared in 1974 in are selected, opened, and manipulated, were his Smalltalk system [12] at Xerox PARC, and researched at Xerox PARC in the 1970s. In soon afterward in the InterLisp system [50]. particular, the idea of “WYSIWYG” (what Some of the first commercial uses of win- you see is what you get) originated there with dows were on Lisp Machines Inc. (LMI) and systems such as the Bravo text editor and the Symbolics Lisp Machines (1979), which grew Draw drawing program [11] out of MIT Artificial Intelligence (AI) Lab pro- The concept of direct manipulation inter- jects. The Cedar Window Manager from Xerox faces for everyone was envisioned by Alan Kay PARC was the first major tiled window manag- of Xerox PARC in a 1977 article about the er (1981) [48], followed soon by the Andrew Dynabook [18]. The first commercial systems window manager [35] by Carnegie Mellon to use direct manipulation extensively were the University’s (CMU) Information Xerox Star (1981) [45], the Apple Lisa (1982) Technology Center (1983, funded [54], and the Macintosh (1984) [55]. Ben by IBM). The main commercial Shneiderman at the University of Maryland systems popularizing windows coined the term “direct manipulation” in 1982, were the Xerox Star (1981), the identified the components, and gave psycholog- Apple Lisa (1982), and most ical motivations for direct manipulation [43]. important, the Apple Macin- tosh (1984). The early versions of Mouse the Star and Microsoft Windows The mouse was developed at Stanford were tiled, but eventually they sup- Research Laboratory in 1965 as part of the ported overlapping windows like the interactions...march + april 1998 47 Lisa and Macintosh. The X Window System, a based display editors that was widely used current international standard, was developed [51]. The Hypertext Editing System [53, p. at MIT in 1984 [42]. For a survey of window 108] from Brown University had screen edit- managers, see [26]. ing and formatting of arbitrary-sized strings with a light pen in 1967 (funding from Applications IBM). NLS demonstrated mouse- Drawing Programs based editing in 1968 (Figure 4). Much of the current technology TECO from MIT was an early for drawing programs was “THE DIFFICULTY screen editor (1967), and demonstrated in Sutherland’s OF MANKIND’S EMACS [46] was developed 1963 Sketchpad system PROBLEMS WAS from it in 1974. Xerox [47]. The use of a mouse to PARC’s Bravo [11, p. 284] manipulate graphics was INCREASING AT A was the first WYSIWYG demonstrated in NLS GREATER RATE THAN editor-formatter (1974). (1965). In 1968 Ken It was designed by Butler OUR ABILITY TO Pulfer and Grant Lampson and Charles Bechthold at the Nation- COPE. (WE ARE IN Simonyi, who had started al Research Council of TROUBLE.) [SO] I ... working on these con- Canada built a mouse out cepts about 1970 while at of wood patterned after COMMITTED MY Berkeley. The first com- Engelbart’s mouse and CAREER TO mercial WYSIWYG edi- used it with a key-frame tors were the Star, ‘AUGMENTING THE animation system to draw LisaWrite, and, later, all the frames of a movie. A HUMAN INTELLECT.’” MacWrite. For surveys of subsequent movie in 1971, — DOUG ENGELBART, ON THE text editors, see [24] and [53]. Hunger, won a number of NLS SYSTEM [8, P. 189] awards and was drawn using a Spreadsheets tablet instead of the mouse (fund- The initial spreadsheet was Visi- ing by the National Film Board of Calc which was developed by Canada) [3].