historical perspective Early Visions of Interactivity: The In(put)s and Out(put)s of Real-Time Computing Andrew Utterson a b s t r a c t Analyzing technical and other texts of the late 1960s and early 1970s, this paper explores hile the notion of interactivity is today a gineers, articulated a desire to inter- the early discourses of inter- W activity—including writings by familiar and widely discussed aspect of the discourses and act with computers as quickly and aesthetics of computing, it was 40 years ago, in the late 1960s easily as they might converse with Charles Csuri, J.C.R. Licklider, and early 1970s, that artists and technologists first explored people or work with other tools. Michael Noll, Ivan Sutherland and other notable figures—via this phenomenon in detail. Notably, these endeavors fore- Considerable possibilities were seen the intersecting fields of comput- grounded a certain type of interactivity, in a conception of in this mode of computing; input ing and the arts, with a particu- the term that differs from contemporary usage; it primarily would be transmitted to the display lar emphasis on the dynamic involved aligning the artist with the role of programmer and in less than the fifteenth of a second (in this instance, a disjuncture) between visionary ideas and the a more fluid mode of input and output in the creative act of for which persistence of vision oc- technical preconditions neces- programming. This focus was a precursor to the more complex curs, with computations processed sary for their realization. interactivity of an end-user’s engagement with a computer pro- fast enough to give the impression cess flexible enough to allow feedback and control. of an immediate response. Such im- Provocative proclamations emerged, most notably in the mediacy was defined as real time: context of technological developments in computer input and “That is,” according to Charles Csuri of Ohio State University, output that were part of a growing culture of so-called real- “time which is ‘real’ because the moment the user makes a de- time computing. While interactive interfaces were not alto- cision also becomes the moment of materialization” [1]. This gether new, advances in input and output devices and related type of operation was still a rarity, despite notable precedents programming occasioned a heightened interest in real-time such as Whirlwind and the Semi-Automatic Ground Environ- computing and in the significant potential of artists’ interac- ment (SAGE) [2] (Fig. 1), as well as the Digital Equipment tive engagement with computers. Corporation (DEC) PDP-1, the first commercially available In research laboratories—not least (but not exclusively) computer to foreground a more direct and responsive rela- in the U.S.A., site of a number of high-tech hubs—and via tionship between operator and computer. a flurry of technical and other papers, artists and technolo- gists attempted to grapple with the computer’s changing na- ture. These efforts were part of a concerted shift away from punch cards (in terms of input) and printed paper (in terms Fig. 1. saGe: an early example of real-time computing that is not a of output) and toward sequential batch processing as a modus part of the arts environment. the saGe system allowed interactive operandi. input from its operators, who monitored the visualization of flight paths through a graphics-based interface used to select and interact Looking back, what models of interactivity were proposed? with individual trajectories using a light-gun device. (courtesy of In what ways did these conceptions anticipate the discourses ibM archives) and practices of today? What dynamic existed between concep- tual developments and the technical preconditions necessary for their realization? Batch Processing vs. real-time comPuting In the late 1960s and early 1970s, artists and technologists alike sought a more direct, flexible, intuitive relationship with computers. Artists of various kinds, alongside scientists and en- Andrew Utterson (educator), Roy H. Park School of Communications, Ithaca College, Ithaca, NY 14850, U.S.A. E-mail: <[email protected]>. See <www.mitpressjournals.org/toc/leon/46/1> for supplemental files associated with this issue. © 2013 ISAST LEONARDO, Vol. 46, No. 1, pp. 67–72, 2013 67 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00487 by guest on 27 September 2021 from responsive enough for the specific needs of filmmakers and other artists. Ivan Sutherland pointed to a similar difficulty when he described a bespoke programming language designed and employed by Knowlton for the creation of computer animation (Fig. 3). “The hard part of using Knowlton’s language is figuring out what the picture is go- ing to look like as you write,” Suther- land complained, describing what he perceived as one of a number of “unre- solved problems” in the area of computer graphics. Since you want an artistic picture, you have to make a little sketch ahead of time on graph paper. You then pick off the coordinates from the sketch and jot the coordinates down on a piece of paper to give to the computer. Knowlton’s lan- guage has the motion capability but it doesn’t have the online characteristics. I should be able to sit at the console and draw what I want. I should be able to ad- just it until it is artistic. I don’t care about the exact coordinates and I shouldn’t have to write them down [4]. Such ideas reflected Sutherland’s ear- lier work at the Massachusetts Institute of Technology (MIT), where he pioneered a key interactive design program. Intro- duced in 1963, Sketchpad [5] (Fig. 4), as Fig. 2. ibM 29 card punch, circa 1964: the card punch machine is emblematic of the era Sutherland’s software was known, func- of batch processing when code would be keypunched into cards (or tape). after the punch- tioned as part of a system that combined ing was done, a dedicated system operator would sequentially schedule and batch tasks for a display and a light pen—technology computer execution at a later time. (courtesy of ibM archives) derived from the earlier Whirlwind and SAGE—with the more general appa- ratus of a keyboard and control board. Although conceived in the context of Real-time computing was crucial to ful scheduling, over the load placed on electrical engineering, Sketchpad never- the development of the discourse con- the computer. This was in diametrical op- theless conveyed a significant aesthetic cerning the computer as a potentially position to the mode of real-time com- dimension. Line drawings could be gen- interactive machine, based on the de- puting, which requires a processor that erated interactively by using the light velopment of computer graphics as an can respond immediately (or as nearly pen directly on the display, in conjunc- intuitive interface between input and so as to be perceived as such), process- tion with a control board comprising a output. While one can imagine a mode ing data as fast as it is input, in order to series of knobs, buttons and switches, of interactivity that does not occur in real enable direct control by, and feedback the results of the manipulation of which time, or real-time operations that do not to, the operator. could be viewed nearly instantaneously. involve a significant degree of interactiv- In batch operation, a disparity existed The user could draw simple geometric ity, the discourse of the late 1960s and between the speed of thought and the shapes but could also manipulate and early 1970s witnessed an effort to con- ability to implement these thoughts. For modify these designs in a series of ways: ceptualize these principles as deeply example, Ken Knowlton of Bell Labora- moving an object around the screen, intertwined. tories stated, regarding his collaboration storing it in memory, zooming in and Real-time computing represented a with filmmaker Stan VanDerBeek: out and so on. “The Sketchpad system,” departure from the existing practice of Sutherland proclaimed, “by eliminating sequential batch processing, a type of When VanDerBeek says “let’s try this typed statements (except for legends) combination,” my immediate reaction computing that emerged as a response is to think for a while about what this in favor of line drawings, opens up a to the high cost of computers and the means, and in the meantime his mind new area of man-machine communica- length of time that one would typically is already off on something else. With tion” [6]. An interactive interface was take to complete a task, due to con- the present batch system, we do perhaps seen as fundamental to the prospect of straints of processing power, memory equally well at discovering interesting liberating the operator to engage with designs; with a much-to-be-preferred in- and so on. Tasks would be batched to- teractive system he’d be far ahead [3]. the computer in unprecedented ways— gether, coded using a card punch (Fig. whether in electrical engineering or 2) and processed in sequence at a later The use of punch cards as input and beyond. time. The major benefit of this approach batch processing as the computational In contrast to this perceived ideal, the was the control, achieved through care- means of dealing with this input was far computers first encountered by artists of 68 Utterson, Early Visions of Interactivity Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00487 by guest on 27 September 2021 Fig. 3. Kenneth c. Knowlton, Collideoscope (detail, 1966). a frame Fig. 4. ivan sutherland’s 1963 sketchpad design system was used illustration from Collideoscope, a computer animation produced for the real-time creation, manipulation and visualization of by stan vanDerbeek (with Knowlton) using an extended process geometric shapes on a display.