Art and Science in Visualization Victoria Interrante 1 Introduction Visualization research and development involves the design, implementation and evaluation of techniques for creating images that facilitate the understanding of a set of data. The first step in this process, visualization design, involves defining an appropriate representational approach, determining the vision of what one wants to achieve. Implementation involves deriving the methods necessary to realize the intended results – developing the algorithms required to create the desired visual representation. Evaluation, or the objective assessment of the impact of specific characteristics of the visualization on application-relevant task performance, is useful not only to quantify the usefulness of a particular technique but, more powerfully, to provide insight into the means by which a technique achieves its success, thus contributing to the foundation of knowledge upon which we can draw to create yet more effective visualizations in the future. In this chapter, I will discuss the art and science of visualization design and evaluation, illustrated with case study examples from my research. For each application, I will describe how inspiration from art and insight from visual perception can provide guidance for the development of promising approaches to the targeted visualization problems. As appropriate I will include relevant details of the algorithms developed to achieve the referenced implementations, and where studies have been done, I will discuss their findings and the implications for future directions of work. 1.1 Seeking Inspiration for Visualization from Art and Design Visualization design, from the creation of specific effective visual representations for particular sets of data to the conceptualization of new, more effective paradigms for information representation in general, is a process that has the characteristics of both an art and a science. General approaches to achieving visualizations that ‘work’ are not yet straightforward or well- defined, yet there are objective metrics that we can use to determine the successfulness of any particular visualization solution. In this section I will discuss ways in which the practices and products of artists and designers can help provide inspiration and guidance to our efforts to develop new, more effective methods for communicating information through images. Design, as traditionally practiced, is a highly integrative activity that involves distilling ideas and concepts from a great variety of disparate sources and assembling them into a concrete form that fulfills a complex set of objectives. It is an inherently creative process that defies explicit prediction or definition yet whose results are readily amenable to comprehensive evaluation. Across disciplines, from graphic arts to architecture, the art of design is primarily learnt through practice and review and the careful critical study of work by others, and expertise is built up from the lifelong experience of “training one’s eyes”. Providing a “good environment for design” is critical to enabling and facilitating the process of design conceptualization. Creative insights are difficult to come by in a vacuum – designers typically surround themselves in their work area with sketches, images, models, references, and other material that has the potential to both directly and indirectly provide inspiration and guidance to the task at hand. In addition, designers rely heavily on the ability to quickly try out ideas, abstracting, sketching out and contemplating multiple alternative approaches before settling upon a particular design approach. In visualization research, we can take a similar approach to the problem of design conceptualization – drawing inspiration from the work of others and from the physical world around us, and experimenting with new combinations and variants of existing techniques for mapping data to images. We can also benefit from establishing fertile design environments that provide rich support for design conceptualization and varied opportunities for rapid experimentation. Finally, it can sometimes be useful to work with traditional materials to create approximate mockups of potential design approaches that allow one to preview ideas to avoid investing substantial effort in software development in what ultimately prove to be unproductive directions. Turning now from the process of design to the product, there is, in a wide variety of fields, from art to journalism, from graphic design to landscape architecture, a long history of research in visual literacy and visual communication through drawings, paintings, photographs, sculpture and other traditional physical media that we have to potential to be able to learn from and use in developing new visualization approaches and methods to meet the needs of our own specific situations. In computer graphics and visualization, as in art, we have complete control not only over what to show but also over how to show it. Even when we are determined to aim for a perfectly physically photorealistic representation of a specified model, as in photography we have control over multiple variables that combine to define the ‘setting of the scene’ that creates the most effective result. In many cases this includes not only selecting the viewpoint and defining the field of view, setting up the lighting, and determining the composition of the environment, but also extends to choosing the material properties of the surfaces of the objects that we wish to portray. For practical reasons of computational efficiency or because of the limitations of available rendering systems we often choose to employ simplified models of lighting and shading, which can also be considered to be a design decision. In addition, we may choose to use non physically based ‘artificial’ or ‘artistic’ enhancement to emphasize particular features in our data, and we may selectively edit the data to remove or distort portions of the model to achieve specific effects. Through illustration we have the potential to interpret physical reality, to distill the essential components of a scene, accentuate the important information, minimize the secondary details, and hierarchically guide the attentional focus. In different media, different methods are used to draw the eye to or away from specific elements in an image, and in each medium different styles of representation can be used to evoke different connotations of meaning. When seeking to develop algorithms to generate simplified representations of data or models, it’s useful to consider where artists tend to take license with reality. They have similar motivations to avoid the tedium and difficulty of accurately representing every detail in a photorealistic manner, but at the same time they need to indicate enough detail, with enough fidelity, to meet the expectations of the viewer and communicate the subject effectively. Numerous texts on methods of illustration present various artists’ insights on this subject [e.g. Loomis, Watson, Guptill, McCloud]. Vision scientists have also considered this question, from the point of view of seeking to understand how the brain processes various aspects of visual input and it is interesting to note the connection between the findings in perception and common practices in artistic representation. For example, people are found to be highly insensitive to the colors of shadows [Cavanagh], being willing to interpret as shadows patches whose luminance, and general geometry, is consistent with that interpretation, regardless of hue, and to be broadly tolerant of inconsistencies among shadows cast from different objects in a scene [Ostrovsky et al.], despite the significant role that cast shadows play in indicating objects’ positions in space [Kersten]. Although there is much about the processes of vision and perception that remains unknown, research in visual perception has the potential to make explicit some of the intuition that artists rely upon to create images that ‘work’. 1.2 Drawing Insight for Visualization Design from Research in Visual Perception In addition to seeking inspiration from art for the design of effective methods for conveying information through images, it is possible to use fundamental findings in human visual perception to gain insight into the ‘science behind the art’ of creating successful visual representations. This can be useful because, although it is often, but not always, possible from informal inspection to determine how well a single, particular visualization meets the needs of a specific, individual application, or to comparatively assess the relative merits of alternative visualization solutions for a particular problem, it is much less straightforward to achieve a comprehensive understanding of the reasons that one particular visualization approach is more successful than another, and even more difficult to uncover the theoretical basis for why certain classes of approaches are likely to yield better results than others. From a fundamental understanding of the strengths and weaknesses and abilities and limitations and basic functional mechanisms of the human visual system, we have the potential to become better equipped to more accurately predict what sorts of approaches are likely to work and which aren’t, which can be of immense benefit in helping us determine how to guide our research efforts in the most promising directions and to avoid dead ends. Mining the vision research literature for insights into a