rsity Comp ve ut ni er U S c n ie w n o c r e B 2 5 4 t 0 h 9- AN 7 NIVERSARY conduit! Volume 13, Number 1 Department of Computer Science Summer, 2004 25th Anniversary issue Brown University technologies, such as virtual reality (VR) or vol- ume rendering. It is difficult for artists to get in- ARTISTIC COLLABORATION volved in design in these visual spaces since, with rare exceptions, one needs to know how to pro- IN DESIGNING gram in order to create within them. Unfortunate- VR VISUALIZATIONS ly, these are also the types of technologies that offer great potential for visualizing many of to- day’s complex datasets [4]. Additionally, they are probably the technologies in which we can most benefit from artistic insight, since guidelines for good visual depiction are far less developed in un- conventional visual spaces, such as virtual reality, than in more traditional 2D media. We begin by describing one of our recent major collaborative efforts, a class on designing virtual Daniel Keefe David Karelitz Eileen Vote David Laidlaw reality scientific visualizations that was co-taught with professors and students from Brown’s com- This article describes some of the lessons we have puter science department and from the Rhode Is- learned from our collaborations with artists on land School of Design (RISD)’s illustration visualization problems. Over the past several department. Many of the experiences and conclu- years, we have worked closely with artists to de- sions relayed here are the results of this class. We velop, refine, and critique visualizations ranging then discuss three important themes that we have from archaeological dig data from the Great Temple of Petra site in Jordan to the fluid dynamics and wing bone shape defor- artists routinely provide an mations that begin to explain how bats, the only flying mammals, stay aloft. Perhaps, unique source of visual insight the most important conclusion we have drawn from this experience is that artists and creativity for tackling difficult can fill an important role in the visualiza- tion design pipeline. In our experience, art- visual problems...we need more ists routinely provide an unique source of visual insight and creativity for tackling appropriate design tools to difficult visual problems. They are also ex- pertly trained in critiquing and refining vi- support them and their role sual works, an essential task in the iterative visualization process. derived from our experiences, all motivated by a The second major conclusion we have drawn from desire better to facilitate artistic collaborations. In our collaborations with artists is that we need some cases, these themes can be thought of as more appropriate design tools to support them and guidelines for software tools that may aid collab- their role. We discuss here the experiences that led oration. In others we are not yet ready to offer a us to this conclusion along with some of the tools guideline, but we have at least identified issues we have developed to facilitate working with art- that were major factors in our efforts and deserve ists. The lack of appropriate design tools is partic- consideration before working with artists on visu- ularly evident in visualizations using new alization problems. Brown University, Box 1910, Providence, RI 02912, USA Figure 1. An art student’s visualization design of 2D steady fluid flow past a cylinder. Courtesy of Deborah Grossberg Teaching Art to Computer artists were adept at investigating visual problems like this one when we could clearly convey the Scientists, Computer scientific goals and constraints of the problem. Science to Artists, and Collaboration was sometimes difficult to manage. Fluid Flow to Everyone In early assignments, such as in Figure 1, the right Our interdisciplinary visualization class brought tools for the job were colored pencil, oil paint, artists and computer scientists together to solve gouache, watercolors, and Photoshop. In later as- visualization problems driven by science. Stu- signments, the essential tool for the job moved dents worked in teams on visualization and design closer and closer to programming. At this point, assignments. We began the semester with 2D fluid the art students often had visual insights to offer flow visualization assignments, as in Figure 1, but had difficulty conveying them. It was easy for and gradually built up to the final projects, which the non-programmers to feel left out of the loop. were virtual-reality visualizations of pulsatile As Fritz Drury (the RISD illustration professor blood flow through a branching coronary artery. who co-taught the class) remarked, the program- We found more obstacles to collaboration as we mers are the ones with the ultimate power: they moved towards VR and more complex data, as have the final say about what ends up on the discussed below. Despite these obstacles, the stu- screen. dents learned how to collaborate with one another, One device that helped us keep artists, computer learned to value what each discipline (computer scientists, and fluid flow researchers on the same science and art) could offer to the project, and pro- page is the critique, a common teaching tool in art duced some very interesting visualizations. classes. All the class work was displayed on a Although artists rarely work with complex scien- wall, as seen in Figure 2, and as a class, we dis- tific data, they do train to convey information ef- cussed important design lessons in relation to each fectively through imagery, given the constraints work. We critiqued the work both from a visual imposed by their media, employers, or audience. and a scientific standpoint. Visually, we explored In this abstract sense, normal artistic practice is color, scale, form, metaphor, and narrative. Scien- not such a far cry from typical visualization de- tifically, we learned about the data we were trying sign tasks. The images to represent and critiqued the work on the basis of in Figure 1 show one art how truthfully and completely the science was student’s early visual- represented, given the tasks our scientists wished ization design assign- to perform. We have now adopted ‘crits’ into the ment. We asked the visualization development process for many of students to create a vi- our projects. sualization and legend that convey eight con- How Can Artists Approach tinuous variables de- Design Problems in VR? scribing a steady, 2D fluid flow in a single As we move from 2D visualizations into more picture. This is a very complex 3D situations such as virtual reality, col- difficult visual prob- laboration with artists becomes much more diffi- lem; in fact, it is still be- cult to facilitate. The first theme we have derived Figure 2. Students prepare for a critique of ing actively researched from our class experiences (along with other col- arterial blood flow visualization designs in the visualization com- laborative efforts) is that visualization design munity. We found that should occur within the visualization target medi- conduit! 2 Figure 3. A CavePainting visualization design of bat flight data, snapshots from a 3D VR program. The bat appears to fly into the page in these snapshots, but viewers walk around the entire model when seen in VR um. This sounds simple, but it has fairly signifi- assumes symmetry between the two wings, the cant ramifications for the visualization media we artists chose to represent different aspects of the often use. For example, it is very difficult for any- data on each side of the bat. On the left side of one, and nearly impossible for an artist who is not these images, flow close to the wing is described a programmer, to create visualizations or simply by color and texture along the wing surface. Vor- experiment with design ideas in virtual reality. tex cores and vortical structures in the flow behind the bat are also represented. Changes in bone A starting approximation for designing within VR shape at two distinct times during a wing beat cy- is to design with more traditional, often 2D, media cle are shown on the right side of the images along and hope that some of these design ideas will with a 3D trace of an important bone joint through translate to VR. We were forced to take this ap- the wing beat cycle. proach during many of the class assignments. The difficulty is the drastic difference between what The basis for our VR design tools is the Cave- we can convey on paper and what we can convey Painting program [3], a tool intended for artists to in VR. We use a four-wall Cave VR display envi- use inside the Cave environment to create free- ronment for much of our research. So much form 3D objects. It has been described as a form changes when we enter the Cave: scale, interac- of zero-gravity sculpture. Artists interact with the tion, stereo vision, vividness of color, and con- system by moving a tracked paintbrush prop trast. When designing traditionally with an eye through the air to create 3D ‘paint’ strokes. (Fig- towards VR, we ure 6 shows an artist using the system.) The are left with the ‘paintings’ are actually 3D models, since each a good 2D design does problem that a brush stroke exists in 3-space. The intuitive inter- good 2D design face of the system makes it easy for artists to pick not necessarily translate does not neces- up and quickly begin modeling in the Cave. sarily translate into a good 3D, much into a good 3D, There are several benefits to working directly in much less VR, the Cave with a tool like CavePainting. The most less VR, design design.