Compatability and Interaction Style in Computer Graphics
Total Page:16
File Type:pdf, Size:1020Kb
Compatability and Interaction Style In Computer Graphics George W. Fitzmaurice and Bill Buxton sented -- in fact, many prefer not to. However applications such as computer aided design, AliaslWavefront Inc. providing access and acquiring such under- modeling, compositing and animation. standing is often necessary for users to achieve Introduction their goals. Thus, we must find ways of Example I:APL and Teletype Recent trends in human computer interaction exposing the deep structure to the user in I/O Compatibility -- High have focused on representations based on ways that are compatible, intuitive and efficient, UI to Manipulate Deep Structure -- High physical reality [4, 5, 6, 8].The idea is to provide and which enable the user to work at this level UI to Manipulate Surface Structure -- Low richer, more intuitive handles for control and when appropriate. manipulation compared to traditional graphical To explore this issue, we examine three To begin, consider the use of the programming user interfaces (GUIs) using a mouse. This styles of interaction in terms of their ability to languageAPL in computer graphics in the trend underscores the need to examine the support manipulation of both the deep and 1960s.The language was cryptic and terse, but concept of manipulation and to further under- surface levels of the graphics. We do so by the matrix handling was wonderful."The stand what we want to manipulate versus what considering the issue of compatibility between power ofAP/comes from its direct manipula- we can easily manipulate. Implicit in this is the input and output devices in addition to the tion of n-dimensional arrays of data.The APL notion that the bias of the UI is often incom- ability of the user to manipulate internal repre- primitives express broad ideas of data manipu- patible with user needs. sentations ("deep structure") as well as lation.These rich and powerful primitives can The main goal of UI design is to reduce external representations ("surface structure") be strung together to perform in one line complexity while augmenting the ability of of the application data. This is somewhat akin what would require pages in other program- users to get their work done. A fundamental to the model-view-control organizing struc- ming languages"[I] Interaction was via an IBM belief underlying our research is that ture of SmallTalk [7]. Let us look at each one Selectric typewriter-like terminal, and complexity lies not only in what is purchased of these compatibilities. compared to the card-punch readers typical of from the software and hardware manufac- • Input with output devices: Does the input the era, it was very interactive. turers, but also in what the user creates with device match the capabilities of the it. It is not just a question of making buttons output display? I/O Compatibility -- High and menus easier to learn and more efficient • User interface with the ability to manipulate The input devices (characters on the to use. It is also a question of"Given that I've internal representation (access to "deep keyboard) and the output (printed characters) created this surface in this way, how can it now structure"): Can the user interface effec- had a strong compatibility. IBM Selectric type- be modified to achieve my current design tively manipulate the internal representa- writers were very popular and familiar to objective?" (The observation is that how the tion of the application data stream? users. Moreover, the symbolic nature of the user created the surface in the first place will • User interface with the ability to manipulate language lent itself to typing. affect the answer to the question.) Our thesis external representation (access to "surface is that appropriate design of the system can structure"): Can the user interface effec- UI to Manipulate Deep Structure -- High minimize both kinds of complexity: that tively manipulate the artifacts generated There was a high compatibility in the manipula- inherent in accessing the functionality provided by the internal representations? tion of the deep structure of the graphics.This by the vendor, and that created by the user. For example, let us consider a form-based was by virtue of the language's facility in manip- The literature focuses on the former. In what UI on a database of records that use an ulating n-dimensional arrays, performing matrix follows, we investigate some of the issues in alphanumeric keypad input device. From the multiplication operations and programming achieving the latter. In so doing~we structure our first perspective, this has good compatibility new functionality. discussion around questions of compatibility. since the primary type of data being input and displayed is alpha-numeric.The deep structure Three Perspectives on might be considered the layout of the forms, UI to Manipulate Surface Structure -- Low and what attributes of the underlying database However, there was a strong incompatibility in Compatibility the manipulation of the surface structure of When we consider manipulation in the are exposed and in what relationship. Since the graphics.APL processed numeric arrays of context of 2D and 3D graphics applications, layout, at least, is a spatial thing, the keyboard data, not graphics images per se. While the we often consider properties of the input would likely have low compatibility, compared numeric arrays may represent a graphical device such as degrees of freedom, and how to a mouse, in terms of interacting at this level. image, such as a set of curves, a user could not well the device is capable of moving points or Finally, in interacting with the fields exposed, directly adjust the contour of a curve within the shapes in this space.That is, how well can the the surface structure, there is medium to good graphical domain. user move and adjust interactive widgets as compatibility. In entering the alphanumeric data Consequently, the user was forced to func- well as directly transform geometry (e.g., the compatibility is high. However, tabbing from field to field using the keyboard may tion exclusively at the abstraction level of the adjust points, curves and surfaces). What is graphics, manipulating the internal representa- often not considered, however, is the ease of often be less compatible than doing so by tions, or deep structure. The user interface manipulating the underlying structure of the selection with a mouse. was designed and optimized to facilitate the graphics (e.g., the deep structure of the geom- In this paper we explore the progression of manipulation of the graphics as represented by etry, as represented by the scene graph, etc.), manipulation as it relates to input devices, the APL notation, not by way of the pictures even when these are data that were directly or deep structure and surface structure. We do themselves. Nevertheless, APL was used in the indirectly created by that same user. this by giving two historical examples and then discuss a new trend which we call "interactive 1970s to produce a number of innovative Many users, especially artists, do not under- animations such as by Judson Rosebush's stand the deep structure, or how it is repre- assemblages." We will frame our discussion within the context of sophisticated graphics company, Digital Effects Inc. 64 November1998 Computer Graphics Example 2: GUI Direct Manipulation my goals in a macro sense. Consider or abstraction for the task at hand.The bias is AliaslWavefront's Maya animation package [9], towards the surface structure. Consequently, I/O Compatibility- High as an example. In this application, 3D geometry as the complexity of the (user created) graph- UI to ManipulateDeep Seruccure-- Low is available to the user in a number of repre- ical scene, object or animation increases, the UI to ManipulateSurface Structure -- High sentations. One is a 3D perspective view. effectiveness of the UI breaks down.Without a Another is as a 2D dependency graph which grasp of the deep structure and an effective In this example we leap ahead approximately reflects the deep structure of the graphics. handle on it, users are mired in too much 20 years and consider the shift towards direct Users are able to manipulate the graphics detail, detail which becomes overwhelming man/pulat/0n through the development of the within this view. However, while this exposes regardless of how simple any single step of that graphical user interface (GUl).The GUI signifi- the internal representation to the user in a detail is. cantly improved our ability to manipulate the form that can be manipulated, actually doing so surface structure which was so lacking in the can still be quite cumbersome. Resolving the Dilemma previous example. This is sometimes due co a notational issue: Given these two examples, we are presented the graphical representation or interaction with a dilemma. How can we design interactive I/O Compatibility- High techniques available may not be appropriate to systems which maintain the balance of manipu- The high compatibility of the input and output the task at hand. Other times, however, it is lability and compatibility over deep and surface devices with GUIs is based on the use of due to the data itself being structured in a way structure? Already we have alluded to a hybrid graphical metaphors. Tools and other entities that is incompatible with the manipulation that approach (Houdini and Maya) which offers both are represented graphically, often as icons, and the user wants to perform (even though it was scripting and direct manipulation solutions. But one interacts with them using generic actions likely that same user who structured ir that from the perspective of artists, direct manipu- such as pointing and dragging. Interaction is way.) lation is generally preferred since it is closer to mediated through the use of a graphical input The former notational issue can often be how they work in the physical world.