The Art and Science of Visualization: Metaphorical Maps and Cultural Models

TA_2-2_Layout 9/11/04 3:42 pm Page 71

Technoetic Arts: A Journal of Speculative Research Volume 2 Number 2. © Intellect Ltd 2004. Article. English language. doi: 10.1386/tear.2.2.71/0 The Art and Science of Visualization: Metaphorical Maps and Cultural Models

Donna J. Cox University of Illinois

Abstract Keywords The author has collaborated in research teams to visualize supercomputer simu- data-visualization lations and real-time data. She describes these collaborative projects that metaphor employ advanced-technology graphics and novel digital displays that include mapping large-format IMAX film, high-definition television productions, and a museum astrophysics digital dome at the American Museum of Natural History. The popularity of virtual reality these images and the function that they provide in popular culture are discussed. postcolonial She also describes two key technologies that she was part of designing: IntelliBadge(tm), a real-time visualization and ‘smart’ tracking system; and Virtual Director(tm), a virtual camera choreography and remote collaboration system. The process of data-visualization involves the mapping of data from numerical form into an iconic representational form in the attempt to provide humans with insight and understanding of a phenomenon. This is discussed in the context of metaphor, cognition, and postcolonial theory. Because data-visualizations carry the ‘weight of scientific accuracy and advanced technology’, most general audiences confuse these visualizations as ‘real’; however, it is argued that data-visualizations are models and metaphors, not reality. In metaphor theory, the mapping of attributes from one domain of information into another is how humans understand, create, and engender new meaning. Data mapping is correlated with this theory. The author explores how the use of mapping information is culturally contingent. In the spirit of scientific inquiry, she deconstructs the very professional activity for which she is most famous.

1. Background of visualization and technology projects Since 1985, I have collaborated with scientists and computer technologists to visualize scientific data from supercomputer simulations at the National Center for Supercomputing Applications (NCSA) at the University of Illinois, Urbana-Champaign. ‘Renaissance Teams’ are teams of specialists including artists and designers who collaborate to solve problems in the visualization of data. The process of scientific visualization involves the translation of numbers into computer graphics and other computer-mediated imagery. As an artist, I have participated in these collaborations in several different areas that include the production, direction, design, colour, and editing of the visuals. Over the years, general audience popularity of these visualizations has increased. These modern scientific visualizations

TA 2(2) 71–79 © Intellect Ltd 2004 71 TA_2-2_Layout 9/11/04 3:42 pm Page 72

have shaped the views of many people. My NCSA team and I have collaborated on several major projects. In addition, we have developed innovative software and new technologies. I will describe several of these projects below. In 1994, I was Associate Director for Scientific Visualization and Art Director for the PIXAR/NCSA segment of a large-scale film project. Cosmic Voyage, an IMAX movie about the relative scale of things in the universe, was nominated for an Academy Award in 1996. Over 6 million people have seen this IMAX film. A typical IMAX screen is about 70 feet across; the film is more than 10 times the emulsion area of a regular Hollywood 35mm movie. An IMAX theatre is an immersive space because the audience’s field of view is totally surrounded by image and audio. Computational science and visualization was an important part of the making of Cosmic Voyage. The advanced technologies of supercomputing and visualization were employed to artistically render images of galaxies colliding in swirling paint- like effects. I collaborated with artists, scientists and technologists to realize an unprecedented number of computer-graphic visualizations for the IMAX film. In addition, we were developing new technologies at the University of Illinois to help create animations for it. In 1996, Robert Patterson (NCSA), Marcus Thiébaux, then a student at the University of Illinois, and I created Virtual Director(tm), a software framework that operates in the CAVE, a room-size, virtual environment with a rear-screen projection system that allows one to see in three-dimensional stereo. Virtual Director(tm) is a choreography and navigation system that enables the user to control the virtual camera, to record frames and to see the recording on a virtual television screen in the CAVE. Virtual Director(tm) also provides capabilities to collaborate over the Internet so that users can interact together even though they may be located at great distances from each other geographically. Initially, we used Virtual Director(tm) to create scenes for Cosmic Voyage; however, we have been developing and expanding upon this software since 1994. We also used Virtual Director(tm) to interactively work with scientists at the Hayden Planetarium. Each user has an independent point of view and can navigate independently while creating and sharing camera paths. Users share the same visual ‘space’ and see the same environment, and they can fly to different locations within that space. When users meet in cyberspace, they see their collaborators as avatars. Avatars are visual metaphors for humans in cyberspace. The original Eastern mythological meaning of ‘avatar’ is the incarnation of God on earth. In virtual reality, avatar is the incarnation of the human in virtual space. In the CAVE using Virtual Director(tm) software, a user is represented over the network as an avatar and can see other avatars floating and flying in ‘cyberspace’ space. To create the following shows and exhibits with the Hayden Planetarium, we used Virtual Director’s remote virtual collaborative capabilities over the Internet: from our CAVE in the University of Illinois (in the middle of United States of America) to the New York City digital dome. My team worked from Illinois and collaborated in real time with the Hayden

72 Donna J. Cox TA_2-2_Layout 9/11/04 3:42 pm Page 73

Planetarium staff to design and choreograph camera paths through the synthetic astrophysical space. The Hayden Planetarium is using the interactive Virtual Director(tm) for evening public interactive shows where the audience can control the digital dome and the Milky Way galaxy model. This virtual-reality technology has provided a method of creating animations for many visualization projects since 1993. We have collaborated with the Hayden Planetarium, American Museum of Natural History, in New York City in the production of two ‘space’ shows in their large digital dome theatre; and also for their Big Bang Theatre exhibit. A ‘space’ show is here defined as a digital image playback with audio and music, and people pay to view the show. The first ‘space’ show was called Passport to the Universe, narrated by actor Tom Hanks, and opened at the Millennium 2000 New Year’s donor celebration. The second ‘space’ show, The Search for Life, narrated by actor Harrison Ford, opened in February 2002. Both of these high-resolution, digital shows are exhibited in the upper hemisphere of a large digital dome (over 9 million pixels), which provides an immersive experience to 440 people during each 17-minute show. We created digital visualizations of the large-scale structure of the universe as well as the local galactic structure near the Milky Way galaxy. Brent Tully, an astronomer from the University of Hawaii, provided mapped locations of galaxies from telescopic data. My NCSA team and I created digital images of a voyage through the cosmos arriving at the large-scale structure of the universe. Over 2.5 million people have seen this exhibit in the last couple of years. We also collaborated with the Hayden Planetarium to provide imagery for their Big Bang Theatre exhibit, which occupies the lower hemisphere of the digital dome structure. Modern Big Bang scientific theorists believe that the universe formed over 15 billion years ago and that hot, dense gas formed stars and protogalaxies that congregated along filaments. Astronomers view today’s galactic filamentary structure of the universe through telescopes. Choreographer Robert Patterson, software developer Stuart Levy and I worked with astrophysicist Dr Michael Norman to visualize over 500 gigabytes of simulation data to show the evolution of the universe following the Big Bang. The audience of 200 people can peer over a railing into a large bowl-shaped digital display to view the digital animations. Looking into the Big Bang digital display bowl is reminiscent of peering into a boiling cauldron where hot gas produces strings of galaxies. Poet Maya Angelou narrates while the audience watches the formation of the universe in the Big Bang cauldron. This scientific narrative of creation draws upon the latest technology and scientific theory. In addition to museums and planetariums, we have also developed visualizations for broadcast television shows. We produced visualizations for the high-definition (over 2 million pixels) television Public Broadcasting System (PBS) NOVA/WGBH show, Runaway Universe. I was producer and art director for the NCSA visualizations for this one-hour special. The story describes how scientists map the universe and require visualizations. Patterson, Levy, and I visualized scientific and astronomical data for this

The Art and Science of Visualization: Metaphorical Maps and Cultural Models 73 TA_2-2_Layout 9/11/04 3:42 pm Page 74

programme that was aired in November 2000 and again in July 2001. Several of the scientific models were the same as in the Hayden exhibits. The show takes us on a virtual voyage from the early universe, to the formation of our galaxy, out of the Milky Way to the Virgo cluster of galaxies. This journey is made possible through the use of digital computer-graphics images, telescopic star catalogue data, and supercomputer simulations. In addition, we also produced visualizations for the Discovery Channel’s Unfolding Universe, that premiered in June 2002. We created over seventeen scenes using data from five scientists and involving several different ren- derers. These visualizations included a boiling red giant star, tours of super clusters of galaxies, colliding galaxies, and the first star. A very recent project, IntelliBadge(tm), involved real-time digital visualization during an academic conference, ‘Supercomputing 2002’ (SC02), held on 16-24 November at the Baltimore Convention Center, in Baltimore, Maryland. About 1000 conference attendees volunteered to carry Radio Frequency Identification (RFID) tags during the events. Our team, NCSA’s Experimental Technologies group, developed an entire system to provide attendees with value-added services and real-time visualizations of their flow patterns during the conference. This system included a real-time database, interactive and playback visualization software, and a web applica- tion. It gathered radio frequency data, tracked people by their professional interests, monitored the flow of people in conference events, provided conference statistics, and showed iconic visualizations representing the flow of people through the physical space of the Convention Center. Participants were then able to log into the system to check statistics and gather information, either at the kiosks or remotely through the IntelliBadge(tm) website. Several visualization schemes were used to show the flow of people at the conference. A scheme called ‘How Does Your Conference Grow’ provided a garden metaphor with each flower representing a conference event room. Each flower was scaled according to the number of people in the room. Each flower had petals that would grow or shrink according to peoples’ professional interests as people entered or exited the rooms. The rate at which people flowed in and out of the rooms was represented by the rate of ants entering and leaving the flowers. The real-time visualization of the data of people and their interests moving throughout the physical space employed similar techniques as the other data-driven visualizations. The garden metaphor was poetic and playful; it was a delight to those who viewed it. The metaphor was in great contrast to other more conventional visualizations such as animated bar charts; yet it was data-driven. Instead of the numbers coming from astrophysics simulations, the numbers were being generated from people moving within a physical space and correlat- ing those numbers with their personal profiles. These visualizations were shown in various locations throughout the Convention Center. The popularity of data visualizations and their use in public spaces have increased dramatically since the advance of computer graphics, supercomputing, and the Internet. Millions of people have viewed visualizations that

74 Donna J. Cox TA_2-2_Layout 9/11/04 3:42 pm Page 75

provide narrative and visual metaphors. People have viewed these images in large-display, general-audience, immersive environments. These visualizations are shaping the reality of cultural beliefs and provide people with a scientific view of reality. Because they carry the ‘weight of scientific accuracy’, most people will believe these visualizations represent the ‘true’ view of reality, but these visualizations are just models and metaphors, not reality. To clarify this point, the process of visualization will be analysed in the next section.

2. The process of visualization: mapping numbers into pic- tures Data-visualization is the process of using computer-mediated and digital technologies to display quantitative and qualitative information. More specifically, scientific and information visualization is the process of transforming a system of mathematical and scientific models, observations, statistics, assumptions, instrument recordings, theories, and other data into animated and interactive visuals that are rendered using two- and three- dimensional computer graphics. ‘Render’ is defined as the process of creating the ‘appearance’ of the data. The term ‘data-visualization’ is used here to specify this process of transforming scientific numerical or information data (often called ‘informatics’) and to distinguish data-visualization from other visualizations such as non-data-driven artwork. The process of data-visualization involves the process of mapping of data into another form in an attempt to provide humans with insight and understanding of a phenomenon. The numerical data from observations (e.g. telescopes) or computations (e.g. supercomputers) is digitally ‘mapped’ (translated) into images, sounds, or other sensory output. This ‘data-mapping’ is the process of making a representation of numbers and correlated facts according to some convention of representation. What is visualized in science are usually natural phenomena, measurements of phenomena, or physically and mathematically approximate models of phenomena. Scientists observe the natural world and its phenomena and formulate theories and models to understand and predict phenomena such as thun- derstorms. Computational scientists develop mathematical models that are computed in supercomputers to describe and predict natural phenomena such as the formation of galaxies. Data can be gathered from other sources such as the IntelliBadge(tm) database that correlates locations of people and their interests. Each scientific discipline or information discipline (informatics) has its own conventions of representations. There are some basic techniques of data visualization that I will describe here. When numbers are mapped from instruments such a magnetic imaging resonance, supercomputer simulations, or other numerical information, the data is often ‘edited’, scaled or down-sampled. Down-sampling means that the data is sampled to a reasonable portion. Datasets are often terabytes of digits and are too large to be visualized at one time. A process of one-to-one mapping is almost impossible because the size of the data is very large and almost always too much data to be displayed in any digital

The Art and Science of Visualization: Metaphorical Maps and Cultural Models 75 TA_2-2_Layout 9/11/04 3:42 pm Page 76

pixel image. Numerical information is often altered in order to fit into the constraints of computer graphics, colour ranges, and pixel resolution. Icons are generated to represent data. As I mentioned above, each discipline has developed visual icons or ‘conventions of representation’. For example, a bar chart is a conventional icon used to relate and compare quantities. In fact all representations of information require humans to understand one domain of information in terms of another domain. It is this process of data-mapping and ‘loosing’ information or the human interpretation of that information that most people do not understand when they view digital visualizations. The fact remains that all visualizations are models or metaphors of reality; they are not, in fact, the reality. These data- visualizations help to organize, analyse, synthesize, and explore hypothe- ses, and communicate. Data-visualization requires the process of ‘editing’ and making decisions to ‘leave out’ some of the data, because not all of the data can be visualized at once. Interactive techniques such as ‘progressive disclosure’ and ‘layering’ of information have been developed to show as much of the multidimensions of data as possible. Currently, one of the great challenges to the visualization community is how to integrate data from many different sources. However, the point that I am trying to make is that the visualization is a ‘visual model’ and it is a data map. The map is not the territory. And the data map has attributes that are often lost in the mapping process. The art of data-visualization is in the translation of these billions of numbers into visual information that humans can understand. Most of the time, this process involves the invention or use of icons or visual metaphors to help communicate information in ways the humans can digest it. In data-visualization, the design of icons and metaphors often employs computer graphics which has its own level of constraints that include rendering limitations and computing power. This mapping of data into visual information is where we find relationships with contemporary metaphor theory.

3. Metaphor theory and cultural contingency Metaphor is not just a linguistic trick; discourse on metaphor has engen- dered a paradigm shift in the way that we think about creativity. Over the last five decades, researchers have generated more than 10,000 articles and books on metaphor and related studies. In particular, most of these writ- ings have concentrated on linguistic metaphor and its connection to cognitive processes. Borrowing from the definition of George Lakoff and Mark Johnson (Lakoff and Johnson 1980), metaphor is more about thinking than about language. Metaphor involves the cognitive process of understanding one domain of information in terms of another domain of information. For example, ‘man is a wolf’ is an English metaphor where a new understanding of man is provided by the cognitive process of mapping characteristics about ‘wolf’ (e.g. beast, hungry, dangerous) onto the concept of ‘man’. The domain of information about man is understood in terms of the domain of information concerning ‘wolf’. To expand upon this basic idea, Lakoff and

76 Donna J. Cox TA_2-2_Layout 9/11/04 3:42 pm Page 77

Johnson have delineated thousands of linguistic metaphors that we use to understand information. They defined ‘conceptual metaphors’ as those linguistic connections that have become so useful and common, that they are now understood as literal language. Such metaphors as ‘time is money’ is integrated into the American culture to the point that we understand time in terms of money and conceptualize time as being ‘spent’, ‘saved’, or ‘wasted’. Basic conceptual or conventional metaphors structure our every- day thinking. We interpret these metaphors literally as a part of speech. For example, ‘argument is war’ formulates how we think about argument. We ‘defend’, ‘strategize’, ‘attack’, and ‘defeat’ arguments. Conceptual metaphors shape the realities that we live by shaping our understanding of basic concepts that we attempt to verbalize. A primary characteristic metaphor involves how novel they are. The range from literal or ‘conventional’ metaphors to the more novel and figurative metaphors is called the metaphor-content continuum: verbal metaphors such as ‘books are fresh fruit’ are considered more figurative than ‘his time was well-spent’. Time being understood in terms of money has become embedded as literal language in American culture; and thus, is interpreted as literal language. The greatest bone of contention is what con- stitutes the literal end of this metaphor-content continuum. Most theorists agree that fresh and novel verbal metaphors can eventually evolve within culture and move from the novel end of this continuum to the literal, as cultural accommodation reduces novelty to literality. While there has been extensive research on verbal metaphors, few have focused on the visual counterparts. Visual metaphors impact on our understanding as linguistic metaphors have been demonstrated. Charles Forceville (Forceville 1996) analysed pictorial metaphors in advertising and focused on figurative, static images. His comparison reveals the visual sim- ilarities to the linguistic non-arbitrary metaphor. ‘Mapping’ is employed in the presentation of visual metaphors. One system of attributes maps onto another to provide a new relationship of meaning. This is not a one-to-one mapping. Some attributes are mapped and some are lost. This selective mapping directly relates to the data-visualization process. Geographic maps are good examples of how literal (conventional) visual metaphors have developed into coherent and consistent system. Here I contend that modern maps are conventional metaphors (in the Lakoffian sense). Their novel origins have been lost over time (due to famil- iarity). The histories of map-making before the 1960s shows a cultural bias of Western European thinking (Bagrow and Skelton 1964). The projection systems, lines on maps, and scales can be traced back to a disciple of Aristotle and Greek mathematics. We literally interpret such conventions today, though their origins reveal one set of visual metaphors to represent maps. Indigenous peoples navigated land and sea with competency and accuracy; but they used a different variety of materials and visual idioms. Early cartographers’ criticisms ‘that these savages couldn’t draw in perspective’ are unfounded. For example, the Marshall Islanders designed intricate patterns from palm fibre and shells to represent wave crests, navi-

The Art and Science of Visualization: Metaphorical Maps and Cultural Models 77 TA_2-2_Layout 9/11/04 3:42 pm Page 78

gation sites, and the mariner’s direction. These accurate and useful maps provided alternative visual metaphors for navigating land and water. Likewise, the Aztec and Mayan maps were accurate though they employed different projections and icons. Cortez conquered the great Aztec civiliza- tion using their indigenous maps painted on cloth. Afterwards, the invaders systematically destroyed these early meso-American maps and replaced them with Spanish Western European maps. Map-making has primarily been dominated by Western European thinking. We can locate a variety of visual methods that enabled different groups of people to navigate their terrain, find locations in the land they explored, and mark there territories with different measuring devices. In Guns, Germs, and Steel Diamond argues that indigenous peoples are as intelligent as white people with lots of technology though there are differences in the way of describing the world and the motivations of the cultures. Today’s methods for spatially quantifying information have roots in civilizations that are not more intelligent, but more dominant. Postcolonial historical perspectives of ancient map-making unfairly crit- icize early American Indians and Pacific Islanders that these savages ‘could- n’t draw in perspective’. After surveying ancient maps, we see a variety of visual methods that enabled indigenous people to mark terrain using alternative metaphors. The difference between primitive approaches and modern maps is in the coherency of the system and the motivations of the culture, not the superiority of the visual models and metaphors. Navigating the highways of life would involve very different visual icons, projections, and embodied experiences if Aboriginal primitive culture had colonized the world instead of Europeans. Contemporary astronomy and astrophysical maps of the universe fit into the evolution of cartography. These are maps wherein we are continu- ing to use translations that are consistent within our cultural bias. The weight of science and technology behind these images affect the audience’s interpretation. Little attention is given to the assumptions that are hidden behind the concept network of beliefs, the inadequacies of technologies, perspective projection, incomplete scientific models, errors in instruments, and limitations of computer-graphics devices. In the humanities and arts, we recognize that scientific narratives are positioned within the cultural, militaristic and political domains from which they evolve. When we take these metaphoric relationships too seriously, we undermine our creative thinking and our realities become calcified within consistent metaphoric myths. I contend that the process of data-visualization is ‘metaphorical in nature’ and that through cultural accommodation, our realities are being created. In conclusion, data-visualizations have provided beautiful images and narratives. It is a magical to sit in a museum and experience virtual voyages through stars and galaxies. But in the interest of metaphor theory and postcolonial discourse, we must recognize that these data-visualizations are culturally contingent and that there may be alternative ways of viewing the universe. Data-mapping cannot be an arbitrary process; however, the cre-

78 Donna J. Cox TA_2-2_Layout 9/11/04 3:42 pm Page 79

ativity is in the awareness that there are alternatives that will work. ‘Thinking out of the box’ is itself a metaphor that describes how creativity requires new perspectives.

References Bagrow, L. and Skelton, R.A. (1964), History of Cartography, Cambridge, MA: Harvard University Press. Diamond, J. (1997), Guns, Germs, and Steel: The Fates of Human Societies, New York: W.W. Norton and Company. Forceville, C. (1996), Pictorial Metaphor in Advertising, London: Routledge. Lakoff, G. and Johnson, M. (1980), Metaphors We Live By, Chicago: University of Chicago Press.

Suggested Citation Cox, D.J.(2004), ‘The Art and Science of Visualization: Metaphorical Maps and Cultural Models’, Technoetic Arts 2: 2, pp. 71–79, doi: 10.1386/tear.2.2.71/0

Contributor Details Donna J. Cox is a recognized pioneer in computer art and scientific visualization. Since 1985, she has been a professor in the School of Art and Design and research artist/scientist at the National Centre for Supercomputing Applications (NCSA). Her collaborative works have been featured in art and science museums, PBS television productions, planetaria, and IMAX theatres around the world, and she has authored many papers on scientific visualization, critical theory, and information design. Contact: NCSA, 605 East Springfield,,Champaign, IL, 61820, USA E-mail: [email protected]

The Art and Science of Visualization: Metaphorical Maps and Cultural Models 79