Macalester College DigitalCommons@Macalester College

Books

2016 Rupture of the Virtual John W. Kim Macalester College, [email protected]

Follow this and additional works at: http://digitalcommons.macalester.edu/books Part of the Digital Humanities Commons

Recommended Citation Kim, John W., "Rupture of the Virtual" (2016). Books. Book 1. http://digitalcommons.macalester.edu/books/1

This Book is brought to you for free and open access by DigitalCommons@Macalester College. It has been accepted for inclusion in Books by an authorized administrator of DigitalCommons@Macalester College. For more information, please contact [email protected]. Rupture of the Virtual

L

M

G

John Kim Rupture of the Virtual

John Kim 1

Introduction 1

2 A New Materialism for New Media Studies 19

3 The Origin of the See-Through Graphical Interface 43

4 Encounters with the Material: Krzysztof Wodiczko and Site-Specific Media Art 69

5 Materiality in Site Responsive Media Art 95

6

Rupture of the Virtual 121

7

Conclusion 145

Notes 153 Index 167 1 Introduction OPENING media been guided by those things in our immediate physical surroundings? In this The human sensorium has always been mediated. (Without the book, I develop a concept of the material “medium” of air or water, the anthropoid ear finds it impossible in order to investigate the development of to hear.) But over the past few decades that condition has greatly modern media and computing devices that intensified. Amplified, shielded, channeled, prosthetized, simu- enhance knowledge and forms of contact lated, stimulated, irritated—our sensorium is more mediated today with one’s physical surroundings. The than ever before. Yet it bothers us less... The microspeaker in the term material is used here to distinguish ear, the drug in the blood, the nanosurgical implant, the simulated it from the virtual, which has had a lasting taste in the mouth—these “enhancements” no longer provoke the influence over ideas concerning the exten- apocalyptic excitement they did even a few years ago.1 sion and enhancement of perception, like McLuhan’s and Jones’. The virtual, however, Caroline Jones’s observation about the is a notoriously flexible term, used in a augmentation of human sensory perception wide variety of contexts.3 The next chapter echoes an earlier one. Marshall McLuhan’s specifies a particular definition of the term famous claim that the media are extensions applied in this book. I use the term broadly of our senses was an early observation and in ways comparable to Anne Friedberg about how the media function as a kind of in her book Virtual Window, in which she “electronic nervous system” and provide us focuses on the “liminally immaterial” image with post-human capacities to see and hear cast by media technologies and their use in things not bound by spatial or temporal the construction of virtual spaces that have constraints.2 Jones’ microspeaker in the ear little or no corollary to those that exist in is conceptually linked to McLuhan’s earlier one’s physical surroundings.4 claim that the television camera enables us to The emphasis on the virtual in media see things happening on the other side of the research has occluded insights into the sta- globe through discussions of the computer tus of the material. A core theoretical argu- screen as a window to virtual spaces, and ment made in this book is the necessity of films as portals to earlier times and places. rethinking the origin of computing interfaces While this claim about the media contin- with an emphasis not on their construction ues to influence their characterization, this as a virtual window,5 but on enhancing book explores the media’s role in produc- knowledge of those things in one’s imme- ing knowledge about what is immediately diate physical environment. In light of the under our noses, our physical environment. proliferation of media that project visual We are accustomed to thinking about the information into the view of one’s immediate media as extensions of human perception physical surroundings—a class of computer beyond what we can perceive with unaug- user interface technologies I refer to as mented sensory organs. But to what extent See-Through Graphical Interfaces (STGI)—a has the origin and construction of the notion of the material is needed more

2 Rupture of the Virtual 3 Introduction than ever. STGIs include a range of devices the raw material or mode of expression used and phenomena as diverse as Augmented in an artistic or creative activity, such as the Reality, outdoor projections, site-specific medium of painting. In Raymond Williams’ media art, Head-up Displays in aircraft and account of the modern usage of the word, cars, and others. Recent media research he notes that medium has come to stand in has turned its attention to one’s physical for the means of communication that makes environment and its implications for the possible or facilitates the transmission of a design and construction of the media and message: computing devices. By understanding the origin of STGIs, it becomes possible to locate This was particularly clear when the word acquired the first ele- what has been at stake in their development, ment of its modern sense in the early seventeenth century. Thus specifically their enhancement of knowledge to the Sight three things are required, the Object, the Organ and and control over one’s immediate surround- the Medium’. Here a description of the practical activity of seeing, ings. As I argue in Chapter 3 in a history of which is a whole and complex process of relationship between the STGI, the Head-up Display is a progenitor the developed organs of sight and the accessible properties of of modern Augmented Reality-type displays things seen, is characteristically interrupted by the invention of a and was originally developed for military third term which is given its own properties, in abstraction from fighter aircraft. The Head-up Display pre- the practical relationship. 7 dates the invention of many devices integral to the design of the digital computer and its A medium stands between a message and interface. its receiver. It intervenes in communication, but is also necessary for its transmission. To use a cybernetic formulation, a medium is DEFINING The difference between the material and the any intermediate substance through which TERMS: THE MEDIUM AND virtual lies at the intersection of a related messages are transmitted from sender to THE IMMEDIATE linguistic and philosophical distinction, the receiver. For Caroline Jones, this intermediate one between medium and the immediate.6 substance can be as basic as air and water, Medium derives from the classical Latin, the media for sound, or pigment and canvas medius, for middle, center, midst, interme- as the medium of painting. Raymond Williams diate course, intermediary. A medium is a focuses on the more modern usage of the thing, person, or process that links other term when defining medium as a channel things, persons, or processes separated by of transmission. For Williams, a medium is distance, time or inconvenience. A medium any means of communication that augments has taken on a wide variety of meanings that or extends the conveyance of a message range across temporal and spatial references beyond human capabilities, which explains both literal and figurative. A medium can be the adoption of the word as applied to the a person who links the past and the present, rise of newspapers and other forms of mass the living to the dead; and it also refers to communication.8

4 Rupture of the Virtual 5 Introduction The immediate, on the other hand, them well... For the porcelain vase is truly porcelain; those olives derives from the Latin root, mediare, to be in are really separated from the eye by the water in which they float... the middle, intercede, or to act as an inter- Chardin can deceive you and me whenever he wishes. 10 mediary. Immediate is the negation of the adjectival form of mediare, which describes Diderot’s appreciation for Chardin’s still a person or thing in its relation to another lifes lies in their ability to fool a viewer into as not having a middle or intermediary.9 mistaking the mediated for the immediate. Like medium, the immediate also has a wide Despite having an immediate relation to the variety of meanings that range across tem- objects (Diderot perceives them with his own poral and spatial references. Immediate can eyes), he imagines that the paintings could refer to people, such as laymen who have be mistaken for the things they represent. a direct relationship with the gods (with- Chardin’s still lifes fool viewers into thinking out the need of an intermediary, such as a that they are unmediated, that is, not repre- priest); immediate can also refer to inter- sentations of fruits and the accessories of vening time, as in to do something without the meal done in the medium of painting, but delay. In the spatial senses of the term it can the actual objects themselves. Both aspects exhibit two features: unmediated and prox- of the meaning of immediate are evident in imity. By unmediated, the person or thing is this account of Chardin’s paintings: Diderot’s perceived directly by the perceiver without proximity to the still lifes allows him to per- the use of an external medium. By proximity, ceive them with his own eyes. Nonetheless, the person or object is within close distance he imagines that it is possible to mistake from perceivers such that they can view or them for the unmediated. hear with their own senses. Indeed, Diderot appraises the quality What is immediate, however, is more of Chardin’s paintings on the basis of their complicated than this definition, because likeness to the unmediated things they medium and the immediate are often con- represent. In this way the medium and ceived of as mutually determinative; one is the immediate mutually constitute each defined against the presence or absence of other. The paintings Diderot so admires can the other. An illuminating example of this is be understood as a mediated immediacy. in Denis Diderot’s reflections on trompe-l’œil Diderot’s examination both frames and painting in the mid-eighteenth century: produces a notion of immediacy through an examination of the medium of painting. There are several small paintings by Chardin... Nearly all represent This manner of thinking is comparable to fruit and the accessories of a meal. They are nature itself; the Derrida’s analysis of the dialectical interplay objects seem to come forward from the canvas and have a look of speech and writing in Dissemination.11 of reality which deceives the eye... When I look at other artists’ In this text, Derrida studies Plato’s text paintings I feel I need to make myself a new pair of eyes; to see Phaedrus, which concerns Plato’s bias Chardin’s I only need to keep those which nature gave me and use towards speech as a privileged means of

6 Rupture of the Virtual 7 Introduction communication in Greek thought. Writing secondary, because it is communicated in is understood to be subordinate, because a medium, and therefore, a less immediate writing was conceived to be in dialectical form of communication. But, as Derrida tension with speech. Writing is mediated, observes, this distinction is artificial, delayed and distanced from the sender in because it is based on a groundless pre- contrast to the immediacy of spoken words. sumption that speech is an unmediated form In the section on the “Pharmakon” in of communication. Speech, of course, is artic- Dissemination, Derrida focuses on Plato’s ulated in the medium of spoken language. definition of the “pharmakon,” which Plato uses as a metaphor for writing, but also defines as a drug, recipe, philtre, charm, THE MATERIAL: The interdependency of the medium and A MEDIATED medicine, spell, artificial colour, and paint. IMMEDIACY? the immediate has influenced the study of The pharmakon is ambivalent, because it is the historical development of media tech- situated in a flickering and disorienting play nologies and the perceptual logics by which of philosophical oppositions: they operate. In their influential book on emerging technologies, Remediation, Jay If the pharmakon is ‘ambivalent,’ it is because it constitutes the Bolter and Richard Grusin note that the medium in which opposites are opposed, the movement and the existence of the logic of immediacy in the play that links them among themselves, reverses them or makes construction of modern media interfaces one side cross over into the other (soul/ body, good/ evil, inside/ in a way that restates Derrida’s position on outside, memory/ forgetfulness, speech/ writing, etc.).…The phar- mediation.13 They argue that the invention makon is the movement, the locus, and the play: (the production of new media has often been guided by an of) difference... Contradictions and pairs of opposites are lifted impulse to erase or dissimulate the recog- from the bottom of this diacritical, differing, deferring, reserve. nition that it is produced as a medium. In Already inhabited by différance, this reserve, even though it other words, media aspire to a transparency ‘precedes’ the opposition between different effects, even though such that viewers do not recognize the it preexists differences as effects, does not have the punctual mediation. Though Bolter and Grusin make simplicity of a coincidentia oppositorum. It is from this fund that use of the term immediacy, it is meant not dialectics draws its reserves. 12 as an absence of mediation, but a sensibility that emerges when new media forms refer Speech and writing are discrete forms of to older ones. When new media are popu- communication, but Derrida observes that larized, older media appear by comparison their meaning is staged in dialectical ten- to be more familiar, intuitive and immedi- sion with each other. Speech is that which ate. A (unintentional or intentional) design is inside, linked to memory, immediate and strategy is to integrate earlier media forms presence, while writing is conceived as its into new ones in order to produce a sense binary opposite: outside, forgetting, medi- of immediacy in a user’s interactions with it. ated and absence. Writing in this sense is This integration of earlier forms is what they

8 Rupture of the Virtual 9 Introduction term remediation. As an example of remedi- indistinguishable from that which we expe- ation, Bolter and Grusin consider cut-scenes rience through our sensory organs. As I will in video games, non-interactive film clips argue in the next chapter, this has often that are inserted into what is otherwise an been rendered to the argument that every- interactive experience.14 Because cut-scenes thing is always already virtual, for even our are an older non-interactive medium, they perception of the material is conditioned by foreground what is distinctive about the a technics of perception.15 As I argue in this newer. By referencing them, video game play book, a concept of the virtual has frequently gains a heightened immediacy. been defined by how human perception has The implication of Bolter and Grusin’s been augmented by technological enhance- argument is that which we understand ment. The concept of the virtual has been to be immediate is often the result of the immeasurably productive for the study remediation of an earlier form, that is, what of the media, but it has also constrained we take to be immediate is always already inquiry. New media research has been mediated. This is a logic that Derrida’s founded on an endemic exclusion of a con- deconstructionist analysis of mediation cept of the material with the virtual, mainly helps to reveal. Despite the fact that both understood as the conceptual obverse of the speech and writing are forms of mediation, material and the material perceptible only in speech is attributed a heightened sense of its disappearance. To move beyond oversim- immediacy, because of its difference and plified accounts of the disappearance of the presumed anteriority to writing. Bolter and material, we must recognize ways in which Grusin’s position relies on the same under- our perception of it both conditions and is lying conceit: because mediation always influenced by our interaction with media already constrains our perception of the technologies. A substantive understanding world, it is impossible to conceptualize an of the material can reveal previously over- immediate relation to it. Immediacy only looked ways to investigate the media. exists in a mediated form, because the Because a concept of the material has immediate is discursively produced by the been developed in a wide variety of disci- logic of remediation. plines, including philosophy, political econ- This tendency to distill questions of omy, geography and theology, it is important immediacy to that of another form of media- to put in place clear boundaries around what tion pervades media research. It is not only is meant with the term. Each field offers a present in Bolter and Grusin’s account of version of the concept that is specific to its remediation, but it is also an assumption in epistemological position, and though I draw Jones and McLuhan’s account of the aug- on some of these fields, my definition fore- mentation of human’s sensory abilities. That grounds the media’s role in conditioning and which we experience through our electronic reproducing our interactions with the mate- nervous system has an ontological status rial. The next chapter develops this claim

10 Rupture of the Virtual 11 Introduction in more detail, but for the sake of clarity, it in an awareness of the disciplinary function is helpful to introduce it briefly here. The of site, or site’s disciplinary role in the forma- material is understood here as those objects, tion of individual subjecthood; and finally, structures, and others that comprise one’s as an encounter with the materiality of site immediate surroundings. Immediate is through a recognition of the formal differ- meant in the sense of proximity; the objects, ence between the virtual and the material. In structures and others that one interacts light of the third opening, I examine cultural with in physical form on a regular basis. practices that strive for an awareness of the In the chapters that follow, I consider a material in order to produce a critical recog- range of topics that become visible with the nition of the virtual as a cultural condition. introduction of a concept of the material in To the extent that they are able to produce a the study of modern media. The opening critical position relative to the virtual, such chapters identify and analyze a new class of practices are said to rupture the virtual. interface technologies, which I refer to as the See-Through Graphical Interface. Existing research has either overlooked entirely or ORGANIZATION Chapter 2 is a theoretical introduction to underacknowledged STGI’s centrality to OF THE BOOK the concept of the material. It begins with the development of modern media. I argue an analysis of the concept’s endemic exclu- that this oversight is a consequence of the sion or erasure from new media theory. emphasis on a concept of the virtual in The ascendency of a concept of the virtual media research. The latter part of this book in theoretical and historical research on examines cultural practices that make use of the development of new media technol- this type of interface in order to encourage a ogies has occluded the consideration of critical awareness or understanding of one’s a concept of the material. Recalling the relationship to the material. I investigate earlier discussion of the immediate and the site-specific media art, specifically artists mediated, the material has been defined as who utilize a STGI type interface (such as the conceptual obverse of the virtual with Augmented Reality) in order to reveal how the material’s presence only perceptible their work has been inspired by an explora- in its disappearance. Three influential tion of site and its materiality. accounts of the virtual in media studies are In its analysis of a concept of the mate- reviewed (the history of technologies of rial and the role that media and computing the virtual, embodiment and informatics, devices have in producing knowledge of it, and post-structuralist theories of digital I argue that this knowledge takes form in media) in order to demonstrate how each three distinct ways. First, in a recognition is grounded in an exclusion of the material. of the sociocultural conditions of site, or Chapter 2 also offers a definition of the mate- an understanding of the social or cultural rial that responds to but is not informed by conditions of a particular location; second, these exclusions.

12 Rupture of the Virtual 13 Introduction On the basis of this definition for the other military STGIs are aids for enhancing material, Chapter 3 examines how a concept knowledge and control over one’s material of the material can provide insights into environment, site-specific media artists, by the history and theorization of media and contrast, have employed media technologies computing interfaces and the visual logics as part of an aesthetic strategy to open a by which they operate. I argue that it is critical intervention into one’s immediate necessary to define a new class of computer physical surroundings. Wodiczko’s 35mm interface technologies in order to distinguish slide and video projections have been char- between technologies of the virtual from acterized as “interrogative,” that is, critical technologies that can enhance knowledge of social and political problems associated over the material. This class includes with site. Building upon discussions of site devices as diverse as Augmented Reality, specificity in art, this chapter argues that his Head-up Displays in aircraft and cars, pre- projections can be read as an interrogative dictive gunsights, and others. Chapter 3 address of three different conditions of site, also explores the history of technologies its sociocultural conditions, its disciplinary that mediate a relationship to the material function, and its materiality. by focusing on a study of the first STGI, the Jason Farman, Nick Kaye, Gabriella Mark II Gyro Gunsight, a predictive gunsight Giannachi,17 and others have recently insisted developed by the British military in the that there is a gradient between the real and latter half of World War II and a predecessor the simulated, the prerecorded and the live, to modern Head-up Displays. In tracing the the virtual and material. The material and the origin of the STGI to military technologies, virtual do not exist in simple binary opposi- the chapter considers the military’s ongoing tion. Accordingly, the STGI can be used in a role in the research and development of wide range of hybrid applications that vary computing interfaces. Paul Virilio exercised from the critical experience of the material, lasting influence over the theorization of the as in Wodiczko’s projections, to an expansion visual logic of cinema and military technol- of the virtual into the experience of one’s ogies; what he has referred to as the “fateful physical environment. Nowhere is this more confusion of eye and weapon.”16 in evidence than in future-oriented game Chapters 4 through 6 extend the discus- designs for Augmented Reality devices, which sion of the STGI and its implications for media insert computer generated objects and char- research by considering cultural practices acters into the view of one’s surroundings.18 that employ STGI type interfaces. Chapter 4 In Chapter 5 I continue to examine media analyzes Krzysztof Wodiczko’s site-specific art, but consider examples that reflect this media projection artwork and contends that range of hybridity; artists who create sited Wodiczko has utilized STGIs for aesthetic works but do not necessarily strive for a investigation of our interactions with the critical interrogation of the material in all material. While the Head-up Display and the ways explored in previous chapters. I

14 Rupture of the Virtual 15 Introduction employ the term site responsive to charac- the rupture of the virtual. One of the recur- terize the works discussed in this chapter. rent themes of the book is the potential for Site responsive is intended as a more inclu- site specificity to open an encounter with sive notion of how media art responds to the material and critical reflection on the vir- location. It is a more open-ended category tual as a cultural condition. The Conclusion for media art that includes both site-spe- suggests that this type of critical interven- cific art and work that is agnostic about tion is not limited to the realm of art, rather, site specificity. I consider three groups of it can be discovered in cultural practices media artists who offer diverse models of beyond the aesthetic. (Examples as diverse site responsiveness. In the work of Janet as urban farming are considered.) The rise Cardiff and George Bures Miller, and the art of practices that strive for an encounter with collectives, Minneapolis Art on Wheels and the material can be understood as harness- Mobile Experiential Cinema, we see a grow- ing a resistance to rise of the immateriality ing interest in using See-Through Graphical and heightened mediation of experience. type interfaces for the creation of immersive This resistance does not rely on a nostalgic media experiences away from the space of return to some pre-virtual idyll, but as a the gallery or desktop computer. critical awareness of the social and eco- Since 2003, I have been utilizing STGI nomic forces that have contributed to this technology in my own art practice. Chapter condition. The Conclusion also reflects on 6 introduces this work by reviewing two forces poised to mitigate the power of site site-specific installations. These projects specificity in art, in particular, the rising are practice-based research vehicles for popularity of outdoor media and cinematic exploring questions surrounding the material projection in urban spaces. I urge caution in and the critical possibilities contained in its any confusion of outdoor projection with the theorization. My practice has been inspired kinds of site-specific projection considered by an effort to draw out what I have found in this book. When not deployed in a way successful in existing site-specific artworks, that realizes the critical potential of site specifically how they foreground the material specificity, outdoor media threaten to proj- as a response to the rise of the virtual as a ect the spectacle out into the materiality of cultural condition. I develop a notion of a public space. “rupture of the virtual” that identifies cultural practices that open an encounter with the material as a critical intervention into the virtual. The work considered in this chapter strives to develop a repertoire of techniques that produce this type of intervention. Finally, the Conclusion offers linked speculative claims about the possibilities for

16 Rupture of the Virtual 17 Introduction 2 A New Materialism for New Media Studies INTRODUCTION When thinking about the media, there is a basis of the work is a piece of technology that tendency to focus on those elements that Herczka and Frid designed: a set of off-the- distinguish them from other forms of rep- shelf video goggles that is modified to receive resentation: that they cast light and sound, a live video feed transmitted wirelessly by a the immateriality of their screen-based remote camera operator. When the camera images, and the constructed environments operator stands directly behind viewers in which we make ourselves captive to wearing the goggles, they can see themselves these images. Stanley Cavell’s poetic reflec- moving through physical space (see Figure tion on cinema distilled an early fascination 1 and 2). The live feed it provides is remi- with the philosophical and phenomenolog- niscent of the disorienting perspective fre- ical questions posed by our interactions quently adopted by cinematographers (think, with the media: Gus Van Sant’s extended steadicam shot through high school hallways in Elephant A screen is a barrier. What does the silver screen screen? It (2003)) and first person video games. But screens me from the world it holds—that is, makes me invisible. in Reverse Avatar, viewers are seeing them- And it screens that world from me—that is, screens its existence selves in the position of the protagonist. The from me. 19 interface has a derealizing effect on viewers’ perception of themselves not only because Cavell’s lines were penned in reference to the they are interacting with their immediate magic of cinema and its immaterial imagery surroundings through a live video feed, but that draws us imaginatively into the world on because of the out-of-body perspective the the display, but keeps us at arm’s length from device affords: viewers see themselves seeing it. His sentiment that the screen is a barrier the world—the protagonist in their own live prefigures an early definition of the virtual, as cinematic drama. the “liminally immaterial.”20 The virtual world As Herczka and Frid’s work illustrates, depicted is a space of illusion that does not screens no longer just screen us from the have physical form outside of its immaterial world. Screens can screen our world, that is, existence on the screen. the world immediately around us. The screen In contrast to Cavell’s Cartesian musings is not always an opaque barrier, but can be a about the screen, media artists have recently media-enhanced window onto our surround- been experimenting with emerging visual ings. As I establish later in this chapter, in technologies in an effort to explore their contemporary research on emerging inter- capacity to unsettle our perception of our face technologies, like Augmented Reality, immediate surroundings. Mateusz Herczka where attention is being paid to how the and Pär Frid’s new media art project, Reverse boundary between the computer generated Avatar (2010), for example, challenges the and the real are being blurred (due to the manner in which we relate to our physical arrival of mobile computer interface technol- environment through media interfaces. The ogies), 21 a concept of the virtual continues to

20 Rupture of the Virtual 21 A New Materialism for New Media Studies predominate. The virtual guides theoretical lost its descriptive power,”24 because of the and historical research into interactive, proliferation of meanings surrounding the immersive new media technologies, but this term.25 Rather, the influence of the concept research is founded on conceptual exclu- is evident precisely in its ubiquity today. sions that have limited the breadth of how The virtual is used across a wide spec- we understand new media. trum of inquiry, each definition reflecting This chapter addresses new media the- the nuances of the approach. Given this ory’s endemic exclusion and erasure of the diversity, it would be fruitless to try to material. It has been precisely through its con- generalize across them all. The intention ceptual erasure that the material has served here is neither to collapse the distinctions to draw attention and research to a corre- between the various definitions for the vir- sponding rise of the virtual. To employ an tual, nor to imply that this reading applies idea drawn from deconstruction, the material to all of them without regard to their differ- has been “supplemental”22 to the celebrated ences. The choice of these three influential definition of the virtual, and the material has accounts of the virtual illustrates the yet to be theorized apart from its contingency. existence of a pattern of supplementarity This has limited the ways in which to conceive established early in new media research, of the media’s role in shaping one’s knowledge one that still exerts influence over the use and awareness of the material. An examina- of the concept today. tion of the concept can open new pathways On the basis of this analysis, a defi- for media research, that include substantive nition of the material is offered, one that ways to rethink the history and function of the responds to, but is not informed by, these computer interface and the existence of media exclusions. The material, like the virtual, is and cultural practices that enable us to rec- deeply embedded in many areas of spec- ognize and reflect upon the materiality of our ulation, from political economy to philos- physical surroundings. ophy and theology. The conceptualization I demonstrate this claim by exam- developed here resonates with some of ining three influential accounts of the those found in other fields, but these con- virtual in which an idea of the material nections are not explicitly explored. has had a role in constituting knowledge precisely through its exclusion. These three accounts of the virtual include HAUNTOLOGY OF The virtual made an early appearance THE REAL: post-structuralist theories of digital media, THE VIRTUAL in post-structuralist and deconstructive history of technologies of the virtual, and AS GHOSTLY accounts of the rise of new media technol- embodiment and informatics. This is only SUPPLEMENT ogies and attendant cultural and economic a selection of the many definitions for the transformations. In Specters of Marx, Derrida virtual,23 but I would not go so far as Anne links the virtual to our experience and con- Friedberg, who claims that the term “has ception of the real, specifically the manner

22 Rupture of the Virtual 23 A New Materialism for New Media Studies in which the virtual constitutes our under- of new media are increasingly central to standing of the real, but is overlooked in our the formation of the real. He associates the perception of it.26 This absence constitutes virtual with a variety of contemporary devel- a gap in our understanding of the real; a opments, including “spectral effects, the gap, which in the course of his investigation, new speed of apparition (we understand this is unveiled as a blind spot that configures word in its ghostly sense) of the simulacrum, knowledge in Western thinking. In what the synthetic or prosthetic image, and the follows, I reveal how, despite the recognition virtual event, cyberspace and surveillance, of the role of the logic of supplementarity in the control, appropriations, and specula- the theorization of the virtual, the definition tions that today deploy unheard-of pow- of the virtual itself was contingent on an ers.” 29 The nature of electronic information is exclusion of a concept of the material. an immateriality that is becoming central to To gain insight into the excluded status contemporary social, economic and cultural of the virtual, Derrida associates the vir- developments in the West. For Derrida, the tual with ghosts, those immaterial beings coinage “virtual reality” best captures the that haunt the living yet are excluded nuances found in his redeeming of the from them. “What is a ghost? What is the concept of the virtual. It is not just a tech- effectivity or the presence of a specter, that nological device that enables the creation of is, of what seems to remain as ineffective, immersive computer worlds. The neologism virtual, insubstantial as a simulacrum?”27 also captures the virtual’s relationship to Despite their marginal status, Derrida the real in contemporary society: reality is observes that Shakespeare’s ghosts are possessed by virtual processes to such an frequently characterized as being in pos- extent that the virtual has come to govern session of knowledge or understanding developments in the West. The virtual is a that surpasses the everyday: “A specter... supplement to the real in that the material is unreal, a hallucination or simulacrum is integral to it, but has been excluded from that is virtually more actual than what is consideration. so blithely called a living presence.”28 As Derrida’s characterization of the virtual ghosts have the ability to upset the natural is emblematic of a variety of postmodernist order of things because of their possession readings of contemporary media culture of knowledge that surpasses the living, the during the period. In his review of this virtual, when acknowledged as integral to work, Mark Poster compares Derrida’s and the constitution of knowledge about the Jean Baudrillard’s respective positions on real, similarly has the potential to destabi- the virtual in order to highlight their pre- lize what we understand about reality. occupation with the concept.30 In contrast Derrida destabilizes the assumption that to Derrida, Baudrillard’s reading of virtual the real is the locus of activity and meaning reality as a metaphor for our cultural con- with his insistence that the immaterial forces dition is an extension of his earlier ideas

24 Rupture of the Virtual 25 A New Materialism for New Media Studies on simulation. Without going into too much to dispense with a materialist understanding detail, Baudrillard contends that electronic of the world in which meaning is associ- media, consumer capitalism and commod- ated with the appearance of phenomena. ity culture have come to replace religion For Baudrillard, the claim that all reality and other traditional sources of value that is virtual reality is based in his insistence once grounded truth and meaning. Because that reality is constructed out of virtual, it retains an antiquated sensibility in immaterial processes, including the flow of which words had stable or fixed referents, electronic images, commodity culture and Baudrillard rejects the term “reality” as an financial capitalism. Because it overlooks accurate characterization of postmodern immaterial processes, a materialist under- culture. With the triumph of electronic standing of the world does not recognize media and consumer capitalism, simulation how reality has been dematerialized with is preferable to reality, because signifiers the advent of simulation. For Derrida as well, have become unhinged from their referents. the virtual is the supplement to the real in In this context, Baudrillard sees virtual real- that our notion of reality is founded on the ity as nothing less than the technological endemic exclusion of the immaterial. Derrida realization of simulation. undermines a materialist position with his insistence that the real is always already With Virtual Reality and all its consequences, we have passed determined by the virtual. But because over into the extreme of technology, into technology as an virtual processes are immaterial and thus extreme phenomenon. Beyond the end, there is no longer any hidden from view, they remain excluded reversibility; there are no longer any traces of the earlier world, from intellectual speculation. nor is there even any nostalgia for it.31 Derrida and Baudrillard shift attention to the virtual as the locus of activity that consti- Because virtual reality environments are tutes meaning. Next, I consider two additional constructed by the hands of human design- concepts of the virtual that were influenced ers, all experience within them is the playful by post-structuralism’s work. Though they construction of postmodern culture. In differ in terms of their approach, the accounts virtual reality, simulation becomes reality; share a preoccupation with foregrounding a the word made immaterial flesh. concept of the virtual. They demonstrate how In both Derrida and Baudrillard’s work, the virtual has been integral to social, cultural the concept of the virtual is overdeter- and economic transformations in which we mined, referring to a wide swath of epochal are in the midst. As a result, the virtual comes economic and cultural changes. Their ver- to define new media research and is no longer sions of the concept destabilize prevailing peripheral to their study. conceptions of the real that do not take into account the epistemic transformations occurring with the virtual’s rise. They seek

26 Rupture of the Virtual 27 A New Materialism for New Media Studies THE HISTORY OF In the years since Derrida and Baudrillard’s Friedberg and Grau refer to as “spaces of TECHNOLOGIES OF THE VIRTUAL speculations on the virtual, theorists have illusion.” Their historical work addresses refined the concept’s usage to such an extent the separation of the virtual from one’s that it no longer aspires to characterize physical environment in technologies, like epochal cultural or philosophical shifts. An the GUI or video games, in which code influential version of the concept is found in comes to “define the entire informational the description of perceptual changes and or perceptual environment for the user.”33 technological innovations that have been Though it draws on the physical world involved in the rise of interactive, immersive for metaphors (such as the window, the media environments.32 The virtual has been desktop, etc.), the GUI does not have to applied to the construction of computer-gen- correspond to our surroundings or the erated environments, such as video games laws that govern them. These studies of the and Virtual Reality, but it is also located in virtual frequently culminate in an extended the rise of panoramas, camera obscuras examination of the promise and potential and other early media devices integral to of Virtual Reality, which is undersood as the development of contemporary spaces further evidence of the concept’s primacy. of immersion. In her historical study of the VR is read as the realization of what was rise of the virtual, Anne Friedberg charts inherent in earlier media forms: the possi- its history from Renaissance perspective bility of the creation of fully immersive and devices to today’s Graphical User Interface interactive virtual worlds and the superses- (GUI). Renaissance painters used enframing sion of the real by the virtual.34 devices, a kind of picture frame, as a window Consider again Stanley Cavell’s onto physical space as an aid to painting philosophical musings quoted earlier. I according to the laws of perspective. For suggested that they be read as an early Friedberg, today’s GUI finds its origins in definition of the virtual with Cavell’s these Renaissance devices in that the GUI has emphasis on the screen as a barrier. The become a metaphorical window onto infor- “world” to which he refers is one of cine- mation space, a virtual space that no longer matic projections—a virtual world created needs to obey the laws of perspective. out of imagery projected onto the silver If we venture a different look at this his- screen. There is, however, a crucial ambi- tory and consider the status of the material, guity in his use of the word, identifiable we can recognize how the material has been now that we’ve considered the status of the elided from accounts of technologies of the material in the literature on the virtual. It virtual. In this history of new media, there could also refer to the world outside the has been an emphasis on the historical and theater, that is, one’s physical environ- perceptual development of the virtual in the ment. Here, in the darkened room of the creation of ever more sophisticated immer- theater, that which is screened from me is sive and interactive mediated spaces, which the world that I inhabit, the material world.

28 Rupture of the Virtual 29 A New Materialism for New Media Studies This ambiguity reinforces a claim about the of the material into a marker of virtual virtual suggested earlier: screens capture information. In that the virtual is privileged our gaze in fixed attention. So drawn into over other possible explanations, this char- this immaterial world, we are screened from acterization is evidence of the continuing our surroundings on the other side of the emphasis on the virtual in research on projections’ borders and theaters’ walls. contemporary interface designs. The virtual has been instrumental in promoting an understanding of the per- ceptual and technological changes that EMBODIMENT In contrast to the two areas reviewed above, have taken place with the rise of modern AND THE the research on new media and embodiment MATERIALITY media, but only those technologies that fit OF INFORMATICS applies a different set of theoretical assump- into this particular account of the virtual tions in its work on the virtual. There have have been included. In other words, the been two interrelated positions in this area virtual has been instrumental in producing of research: the claim that our interactions a kind of teleological thinking that sees new with digital communications lead to a kind of media’s history written from the vantage disembodiment because of the immateriality point of the virtual. It is a history premised of information, and research on how the on the leading role that the virtual has had, body and information technologies mutually which organizes historical and theoretical constitute each other, because information insights around it. We can see this ten- technologies extend humans’ perceptual dency continuing in research on emerging capacities. Though they differ in terms of technologies. In recent investigations of how they understand the virtual, what they mixed reality interfaces, for example, there share is a conception of the material as is a tendency to conceive of interfaces like coextensive with the human body at the Augmented Reality as an outgrowth of exclusion of a concept for the material as earlier technologies of the virtual. Jason outside of the perceiving subject. Farman notes how in Augmented Reality In her account of the former position, the material is subsumed by a virtual or Emily Apter argues that we leave our informational dimension.35 In Augmented bodies behind (that is, at the computer Reality and related mixed reality type inter- terminal) in online interactions. For Apter faces, one’s physical surroundings become and others, this disembodiment is poten- an “information interface”36 in that one’s tially liberatory, because we can free geographical location becomes an indexical ourselves from the rigid markers of the marker to online information accessible body’s constraints (i.e. race, gender, dis- by mobile computing devices. Augmented ability). This celebration of disembodiment Reality is an interface technology that is rooted in the belief that humans could enables the material to be colonized by escape the essentializing forces that con- information through the transformation struct the body through the use of digital

30 Rupture of the Virtual 31 A New Materialism for New Media Studies technologies. Digital technologies can embodying tendencies of new media.41 enable a “postidentitarian politics rooted Munster observes the manner in which the in the obsolescence of racial and ethnic body, rather than excluded or disembodied categorizations.”37 in one’s use of various new media technol- While Apter depicts the body and ogies, is directly involved in the interaction. informational systems as in opposition, Indeed, Munster uses the term material Katherine Hayles has posed a more dynamic interchangeably with the body and the cor- way to conceive of this relationship. Instead poreal. The “materializing” of new media (to of portraying technology as a disembodying which the title of Munster’s book refers) is force, Hayles has insisted that human and precisely the inclusion of the body’s phys- information systems are in a co-evolution- io-motor capacities or physical presence in ary dance, which she has referred to as an new media interactions. “intermediating dynamic.”38 In earlier work, In his study of embodiment and new especially “The Materiality of Informatics,” media, Mark Hansen has traced this interest she offered a preliminary version of this in the body to a more general “turn to the idea with her insistence that the human body” in cultural theory, culminating in the body exists as a kind of constraint on the 1990s with the development of the “cultural formation of information systems, because constructivist paradigm.”42 The adoption of technologies are only adopted when they cultural constructivism in feminist and queer can be integrated into existing human func- writings on technology then gave rise to the tioning through a dual process of “inscrip- figure of the cyborg reclaimed as a synecdo- tion” and “incorporation.”39 She uses the che for emancipation from the reigning essen- example of the development of audiotape tialisms that dominate racial, ethnic, gender with its capacity to record and objectify and sexual categories. Despite characterizing the human voice. Its adoption “changed the her manifesto as an “ironic political myth,”43 relation between voice and body” producing Donna Haraway’s fantastical depiction of the a “new kind of [posthuman] subjectivity” cyborg in “A Cyborg Manifesto” accelerated that she identifies in examples of modernist into a proto-utopianism surrounding late fiction and poetry.40 twentieth and early twenty-first century Hayles’ insistence that information information and communication technologies. systems only develop within the constraint From the Internet to cellphones to pros- of the materiality of the body has influ- thetic enhancement of the body, new digital enced research in this area. It has shifted technologies became the site of growing discourse to ways in which the human body fascination, because they were perceived as is a site for the inscription of informational enabling a liberation from the constraints of systems. In Materializing New Media, for the everyday. Work, like Apter’s, is born from example, Anna Munster explicitly relates a belief in information and communication the two in her observations about the technologies’ liberatory potential.

32 Rupture of the Virtual 33 A New Materialism for New Media Studies Hansen shares a conception of the body computers, and virtual reality, bridge this gap that renders it as a raw materiality that is by extending the body’s perceptual capabili- given extension or enhancement with infor- ties into the realm of the virtual. mation and communication technologies. Thus, when Hansen writes that “virtual But dissimilar to cultural constructionists, reality comprises something of a reality test Hansen grounds his work in an updating of for the body,”46 he is referring to this differ- a phenomenological account of perception, ence between the materiality of the body and specifically a reassessment of Merleau- the virtuality of sense data. Virtual reality Ponty’s notion of the “technics” of human is a reality test for the body, because visual embodiment. For Merleau-Ponty, technics images are given actuality by the perceiving refers to any technologies that enhance subject. In positing the body’s difference human perceptual capabilities. Hansen’s from the virtual in this way, Hansen’s work is approach to human perception is an effort to comparable to other theories of embodiment illustrate how technology is “always already that conceive of the body and new media as embodied,”44 that is, human perception in opposition, with the body the domain of always already bears the mark of technolog- materiality and media that of virtuality. ical enhancement. The body’s perceptual Slavoj Zizek’s critique of cultural con- capabilities are the frame within which structionism can help identify what is media technologies are developed and inte- limited about the depiction of the body grated into human perception. as the locus of materiality. Employing The reason for the interdependence of language familiar to deconstructionists, human perception and media technologies Zizek insists that cultural constructionism that enhance it derives from the difference is founded on an “exclusion / foreclosure between the body and the realm of sense that grounds this very horizon.”47 The data, two phenomena that differ in form. effort to unmask and resist all essentialist According to a neo-Bergsonian account of positions regarding the body forecloses perception, sense data is the realm of the the possibility of recognizing the existence virtual, because visual and auditory stimuli of a background from which subjectivity are immaterial when they enter perception. arises. For Zizek, this excluded background Sense data becomes “actual” or “real” by is capitalism, which for the purposes here means of human perception, which attends is less relevant, but by applying his argu- to and fixes images in memory.45 The body, it ment to the role that the body has in the can be said, gives reality to the virtuality of literature on embodiment and new media, a the image through embodied perception. Put related critique can be made. In this work differently, there is a gap between the body the differences between material/virtual, and the virtuality of sense data which needs inside/outside, and body/environment are to be bridged in order for sense perception reified to such an extent that the subject to occur. Media technologies, such as film, is considered material and everything

34 Rupture of the Virtual 35 A New Materialism for New Media Studies outside the subject (because it enters the virtual. This position does not recognize that body as sense data) is virtual. the construction of the body presupposes Zizek’s accusation that cultural con- the existence of forces external to it that structionism fails to acknowledge the exis- ground the possibility for its materiality. In tence of a background that makes possible the next section, I offer an alternative for- subjectivity is applicable to the research mulation of the material, one that posits its on embodiment in that it relies on an idea presence outside the perceiving subject and of a gap between the body and the virtual. foregrounds the media’s role in enhancing However, this view overlooks the extent to knowledge of the material. which there is a determining background or environment that makes possible the materiality of the body. That the bound- RE-MATERIAL- Derrida’s observation about the exclusion ary between the body and environment is IZING NEW of the virtual from intellectual speculation MEDIA STUDIES porous is familiar to cultural theory. Gender, must be reconciled with the concept’s cen- for example, is a performative act made up trality in new media research. The virtual is of behaviors learned via sense perception. not only presumed to inhabit the everyday, As Judith Butler has argued, the border but it has a role in radically reshaping it. The between the inside and outside is blurred, as concept has moved from periphery to center, external norms or forces constitute the indi- from exclusion to inclusion, and continues vidual and have determining effects on the to influence the way in which we understand subject’s body in terms of how it expresses media culture. In enabling researchers to itself and functions as a gendered being.48 trace tendencies in the historical develop- If external norms and forces have a role in ment of new media, it has simultaneously constituting the materiality of the body, then constrained inquiry because it has been any simple opposition between body and founded on an endemic exclusion of a con- environment disregards the complexity of cept of the material external to the perceiv- the body’s materiality. ing subject. The material has been conceived Returning to Zizek’s critique, we can of as the supplement to the virtual with the conclude that theorists of embodiment do material perceptible in its disappearance. not properly acknowledge the existence of In contrast to the virtual, the material has background forces that ground the materi- not been given sufficiently considered as ality of the body. In contrast to other areas a constitutive element in our interactions of research in new media, the research with media technologies. To move beyond on embodiment offers a conception of the accounts of the disappearance of the mate- material, but one that is too restrictive. The rial, its total mediation, or association with material is portrayed as coextensive with the body, we must recognize ways in which the perceiving subject, and what is outside our perception of it both conditions and is the subject is consigned to the realm of the influenced by our interaction with media

36 Rupture of the Virtual 37 A New Materialism for New Media Studies technologies. Developing a substantive them without the aid of the binoculars, it is understanding of the material can open new included in the definition of one’s immediate ways to investigate the media. environment. By contrast, video chatting In order to provide a manageable defi- with friends on the other side of the world nition of the material, it is important to might give one a powerful feeling of close- put in place clear boundaries. A concept ness to them, but they are not physically of the material has been utilized in a wide proximate without great effort and time. variety of disciplines, including philosophy, The latter example brings to mind an political economy, geography and theology. important distinction: physical proximity Each field, of course, offers a version of is often conflated with, but should not be the concept that is particular to its episte- confused with, perceived or felt closeness. mological position. The definition offered It is important to distinguish between them here does not try to integrate these diverse conceptually. Felt closeness does not require approaches, nor even explicitly engage with physical proximity. One can feel an attach- them. Though it draws on some of these ment to a place, for example, despite the fact fields, this definition is appropriate to the that one may never have visited there. The study of the media, one that foregrounds the difference between proximity and closeness manner in which the material both condi- employed here is a technical distinction tions and is produced in our interactions not an evaluative one –what is not being with the media. suggested is a preferability of one over the The material is understood here as the other. The media have a power to shrink presence of objects, structures, and others one’s sense of distance and time to such an that comprise one’s immediate surroundings. extent that one can feel close to things and Immediate is meant here in the sense of others who are far away, but the definition physical proximity, or the objects, struc- here strictly concerns physical proximity. tures and others that are within the range A related problem exists in recent intellec- of one’s perceptual senses. This can include tual efforts to reclaim the material. In fields as things that are perceived with the aid of diverse as anthropology, geography, and oth- devices that augment sensory perception, ers, there has been a call to return to a mate- such as eyeglasses, telescopes, megaphones rial analysis of things and objects, in order to and shotgun microphones, provided they distance research from the perceived excesses can also be experienced without augmen- of the social and cultural constructedness tation with minimal effort. An example of phenomena. Sometimes referred to as the of an experience that is immediate, but “materialist return,”49 this position skirts close augmented, is the use of high-powered to a longing for an immediate, lived experience binoculars to look at trees at the edge of of the world that is oftentimes nostalgic for a one’s physical surroundings. Because one perceived closeness of objects and environ- could move closer to them in order to see ment. A problem with the materialist return is

38 Rupture of the Virtual 39 A New Materialism for New Media Studies its epistemological basis. This sense of close- class of interface technologies I refer to as ness is grounded in a presumption that mean- See-Through Graphical Interfaces, computing ing emanates from the objects and things interfaces that overlay graphical information themselves. Critical of a similar tendency, into one’s field of vision, and the reasons for the nineteenth-century German philosopher distinguishing them from technologies of Arthur Schopenhauer took to task a compara- the virtual. Instead of opening up windows ble materialist position, one in which “every- onto virtual spaces, See-Through Graphical thing objective, extended, active, and hence Interfaces enhance knowledge over things in everything material” is regarded “as so solid a one’s immediate physical surroundings. basis for its explanations.”50 Schopenhauer is suspicious of this position for it imagines the material as a ground from which truth derives with materialist philosophy’s task to uncover this meaning. The definition of the material offered here, however, does not assume that the material is the ground of meaning. Like the virtual, the material is embedded in diverse systems of signification and is the conflu- ence of complex forms of meaning, histories and cultural legacies of which one may not be aware. These systems of signification constitute the way in which one evaluates, understands, sentimentalizes and relates to one’s physical surroundings. The media is one system of signification that has a role in constraining and enabling one’s experience and understanding of the material. The material is defined here as that which is in our immediate physical surroundings, but the forces that condition the material itself can be non-local and immaterial. In order to develop the significance of this definition for the study of new media, the next chapter turns to an examination of the role that computing interface tech- nologies have had in enhancing knowledge over the material. I focus specifically on a

40 Rupture of the Virtual 41 A New Materialism for New Media Studies 1 3 Mateusz Herczka and Pär Frid, Reverse Avatar, 2010 The Origin of the See-Through Graphical Interface

2 Mateusz Herczka and Pär Frid, Reverse Avatar [alternate view], 2010

42 Rupture of the Virtual In this chapter I define a new category of there has been a growing body of research media and computing interface devices, the about these technologies. See-Through Graphical Interface (STGI), in order to acknowledge its unique history and mode of visuality that is distinct from BETWEEN THE In the past few decades technologists technologies of the virtual. In contrast to VIRTUAL AND and research scientists who work at the THE MATERIAL media that are a virtual window onto con- intersection of mobile computing, vision, structed worlds or spaces, STGIs project interface design and computer graphics graphical information into the view of one’s have been conceptualizing and forecast- immediate physical surroundings. The last ing the arrival of a wide variety of mobile chapter detailed the legacy of a concept of interface devices, including but not limited the virtual in research on media and inter- to Augmented Reality. Milgram and Kishino, face technologies, which has resulted in an for example, exercised an early influence on impoverished conception of the material. the theorization of Augmented Reality with By extension, this has led to the under-the- their taxonomy of visual display technolo- orization and recognition of the historical gies, in which they conceive of Augmented importance of computing interfaces that Reality as a “subset” of Virtual Reality tech- enhance knowledge over the material. There nology.51 By contrast, Ron Azuma, a pio- are a number of computing devices, such neer and innovator of Augmented Reality, as gunsights developed during World War locates AR on a spectrum ranging from II, that precede the invention of many of the the completely synthetic (Virtual Reality) hardware and software innovations associ- to the completely real (telepresence) with ated with the development of technologies AR situated between these two poles.52 of the virtual. Steve Mann has been developing and using Defining a new category of interfaces computing devices that technologically is required in order to reveal this history enhance his vision since the 1970s. As and underscore how they pose a mode distinct from the others, Mann prefers the of visuality that is distinct from other terms mediated reality, which includes both screen-based media. I refer to these devices virtual and augmented reality. For Mann, as STGIs, because users are able to see mediated reality is a term that emphasizes the environment through the computing the use or attributes of the media interfaces device; users’ surroundings are the “back- instead of their technical details.53 ground” on which computer-generated or There is little consensus among tech- virtual objects are projected. This cate- nologists either about the future direction gory includes technologies as diverse as of these technologies, or the concepts Head-up Displays, Augmented Reality, mili- and terms to describe, contextualize and tary gunsights and others. With the arrival analyze Augmented Reality-type devices. of interfaces like Google Glass, especially, For their part, media and visual studies

44 Rupture of the Virtual 45 The Origin of the See-Through Graphical Interface researchers have offered limited help in is encoded, organized, and made subject to this regard and have generally been slow algorithmic analysis. to acknowledge their distinctiveness from Lev Manovich identified “numerical other digital technologies. One reason for representation” as one of the main princi- this is what Nathan Jurgenson has referred ples of new media, a concept that can be to as a prevailing “digital dualism” within usefully applied to this analysis.56 According scholarship that has constrained thinking to Manovich, what is distinctive about new about digital media. This is a tendency to media is their ability to represent or encode render digital processes and the physical analog information or processes as numer- world in a binary opposition with digital ical (i.e., binary) data. While Manovich had technologies conceived as enabling virtual in mind the migration of analog media to experiences (i.e., immaterial, disembodied, digital formats (e.g., digital photography, digitally fabricated), while the physical digital audio, and digital video), his insight world exists separate from the virtual. also illustrates an important dimension of According to Jurgenson, this has resulted in AR. As maintained above, AR engineers are the under-theorization of computing tech- developing techniques to encode one’s mate- nologies that mediate our relationship to our rial surroundings so that they can be subject physical surroundings, like AR, for they pose to computer-based algorithmic analysis and a mode of visuality distinct from existing interpretation. In Manovich’s terms, in order technologies of the virtual.54 for such a system to be feasible, the numer- While the virtual, broadly conceived, ical representation of one’s material sur- concerns media that involve a distancing, roundings is required. Users access this data, if not a complete separation, from one’s when retrieving information about their material surroundings, with AR the material environment through an AR device. In this figures importantly in one’s interaction with way we can understand AR as the expansion the medium. If we look more closely at how of the virtual into the material, insofar as some researchers speculate AR will func- users’ material surroundings are encoded as tion, we can recognize how the relationship a numerical representation, and interactions between the material and the virtual is more with the environment are mediated by this complicated than it at first appears. AR engi- representation. neers foresee the development of computer Jason Farman’s Mobile Interface Theory vision and recognition algorithms (a devel- is a forward-thinking account of the informa- opment that is years from realization, if the tional representation of physical surround- technical challenges are ever surmounted) ings in interface technologies, like AR, and it that would enable software to recognize is an example of work that strives to move objects, people and spaces in one’s mate- beyond the pervasive dualisms of which rial environment.55 Such a system would Jurgenson is critical. Farman considers a require that a user’s material environment number of different kinds of mobile media

46 Rupture of the Virtual 47 The Origin of the See-Through Graphical Interface technologies, including Augmented Reality mobile digital computing and the effect enabled smartphones, SixthSense, and that it is having on our experience of the locative games, and their effect on our expe- material environment. This is, however, rience of the physical world. In his analysis too limited an account of the history of of Augmented Reality specifically, Farman these technologies. In order to make notes the growing interpenetration of mate- sense of them, this article situates AR as rial space by an informational or virtual part of developments dating back as far dimension. He writes: as World War II with the invention of the first STGI, a predictive gunsight found in The move from personal computing to pervasive computing, a military airplanes. A precursor to mod- shift characterized by the move from immobility to mobility, ern Head-up Displays and AR, the gun- has allowed for online space to interact with material space in sight provided users with an information unprecedented ways.57 layer in the experience of their physical environment, which enhanced knowledge Farman notes an erosion of the distinction over their surroundings. The next section between the material and the informational begins with an overview of the military’s with the rise of mobile computing. Physical ongoing role in the research and devel- surroundings are an “information inter- opment of STGIs for the technological face,” because one’s geographical location enhancement of vision, a research pro- becomes an indexical marker to online gram that introduces the case study of information accessible by mobile computing this chapter. devices.58 While technologies of the virtual entail the creation of environments that have no necessary corollary to the material AUGMENTED In July 2009, soldiers of the 5th Brigade, REALITY AND world, with AR a distinction between the THE LAND WAR- 2nd Infantry Division of the U.S. Army were material and its numerical representation is RIOR PROGRAM deployed to Afghanistan under President integral to users’ interaction with the inter- Obama’s call for a troop surge to counter- face. The presentation of computer gener- act a rapidly deteriorating military situa- ated graphical information in AR constitutes tion in the country.59 What was noteworthy an information layer overlaid onto the expe- about this division was that each soldier rience of the material environment. Users of was outfitted with the latest in military AR devices still move through and interact hardware and armament, a suite of tech- with their material environment, which the nologies known as the Land Warrior sys- interface facilitates. tem. The Land Warrior system has been Though it is prescient in its analysis in research and development since 1993, of the perceptual changes taking place when the military made an initial call for with new interface technologies, Farman’s proposals for contractors to develop dif- argument is contingent on the rise of ferent components of the weapons system.

48 Rupture of the Virtual 49 The Origin of the See-Through Graphical Interface The call described the Land Warrior sys- into university, industrial and military tem in the following terms: research.63 With the headgear subsystem of the The LWS [Land Warrior System] will be used by the dismounted Land Warrior program the military con- war fighter at the tactical level of war. The LWS will include an tinues its longstanding influence on the integrated complement of weapon, protective clothing and head- research and design of computer devices for gear… to enhance his lethality, command and control, survivabil- the technological enhancement of military ity, mobility, and sustainment.60 troops.64 According to General Dynamics, the chief contractor on the subsystem of the As Chris Gray observed in his chapter on Land Warrior project, the headgear consists the “Cyborg Soldier” in Postmodern War, of a Head Mounted Display (HMD) attached the military has long experimented with to a mobile computer, running specialized new warfare technologies, with the sol- software and connected to an array of dier’s body frequently the site of prosthetic sensors (see Figure 1).65 The display has enhancement and experimentation.61 From multiple functions: with its integrated GPS, devices that augment a soldier’s sight soldiers can call up their precise geograph- and communication abilities to drugs that ical position on a digital map and get infor- prevent stress and increase endurance, mation on the positions of enemy targets military research programs have aimed to and friendly troops. In a realization of Paul augment a soldier’s lethality and reliability Virilio’s observation about the linked devel- in combat through integration with tech- opments of military and cinematic modes of nology. According to Gray, in this sense, visuality—what he has referred to as “the the soldier has long been a human-ma- fateful confusion of eye and weapon”66— chine hybrid, or “a cybernetic organism the HMD is a remote display for a camera (cyborg) model of the soldier that com- mounted on a gun, allowing soldiers to aim bines machine-like endurance with a their weapons without looking through redefined human intellect subordinated the gunsight. Soldiers literally see from the to the overall weapon system.”62 The Land perspective of the firearm, as if they were Warrior program fits squarely within the their own gun. The HMD can also connect military’s “cybernetic” research programs to a night vision camera, allowing soldiers of the type about which Gray writes. One to see in the dark, and with the aid of ther- part fantasy: the cyborg soldier is pure mal imaging sensors, even to look through ideology, an expression of the military’s walls. Lastly, the HMD is an integrated faith in technology’s ability to enhance the communication device; it enables superiors mobility, lethality and commandability of to transmit data to soldiers in the form of combat troops; and one part reality: an digital images, video, data messaging, and active program that has the material result other kinds of combat information, keeping of channeling billions of taxpayers’ dollars soldiers up-to-date on battlefield conditions.

50 Rupture of the Virtual 51 The Origin of the See-Through Graphical Interface In this way, the HMD is a repository of infor- manner.68 Soldiers are able to keep their mation from a variety of sources, including eyes on their material surroundings when commanders, other troops, and surveillance retrieving information, lessening the dangers data from satellites and airborne sources, of being distracted while in combat. connecting soldiers to a real-time battlefield When the Land Warrior system is situated information network. The soldier, in effect, in the context of the military’s longstanding becomes a node in an information network role in developing computing technologies,69 and can be remotely controlled according to the military’s foray into display interfaces changes in the combat situation. should come as little surprise, for the While the Land Warrior HMD now func- AR-HMD is deeply connected to earlier war- tions like a highly mobile computer display, fare technologies used for graphical interface the U.S. government has been funding an and visualization purposes. There are clear active research program in Augmented connections between HMDs for foot soldiers Reality in an effort to craft the next gener- and the Head-up Display (HUD) found in the ation of the interface.67 The HMD currently cockpits of many military and commercial designed for the Land Warrior system aircraft.70 The HUD is an interface device basically functions like a sophisticated that projects graphical information into version of the eyepiece of a video camera; users’ view of their material surroundings. the display takes up the entirety of the users’ For pilots, the HUD makes available real-time field of vision, allowing no unmediated visual combat and navigational information on a access to their surrounding environment. transparent screen. With the Land Warrior An AR-enabled HMD (or AR-HMD), on the system the miniaturization of computing and other hand, facilitates interaction with users’ display technology has enabled the military immediate physical surroundings. Users to re-imagine the role of the HUD on the are able to see their environment through battlefield; computing devices that were once the headset. Information is superimposed restricted to the cockpit of an airplane can directly onto the battlefield itself, allowing now be carried on the body. In this sense, the soldiers to access a wealth of battlefield AR-HMD is a wearable version of the HUD. information, such as the locations of friendly Both the HMD and the HUD are computer and hostile forces, improving their under- interfaces that enhance a users’ knowledge of standing of their geographical position and their material environment. the potential dangers in the environment. By projecting computer-generated or virtual objects into soldiers’ field of vision and thus THE MARK II It is difficult to determine when precisely onto their view of their physical surrounds, GYRO GUNSIGHT: the term was coined, but in the decades A PREDECESSOR the AR-HMD is said to improve “situational TO THE HEAD- following World War II, military researchers awareness” by providing battlefield infor- UP DISPLAY and personnel employed the term Head-up mation to soldiers in a less distracting Display to refer to a variety of interface

52 Rupture of the Virtual 53 The Origin of the See-Through Graphical Interface devices to improve the visual presenta- The Mark II was developed in 1942, and tion of flight and targeting information. In production orders were taken in that year.77 1958, J.M. Naish,71 a British research sci- By 1944 the gunsight was widely found in the entist for the Royal Aircraft Establishment cockpits of Spitfire and Hurricane aircraft.78 at Farnborough (referred to as R.A.E. Elements of the gunsight were the outcome Farnborough), the premier military aircraft of technological developments going back research facility through most of the early decades,79 and many of the innovations used twentieth century, penned the earliest writ- in the device were found in other combat ten definition of the term: technologies during the period. The Mark II, however, was the first STGI, because it was The essential feature of the pilot’s instrument presentation is the the first “computing” device to employ a pro- ability to provide certain information from the so-called “head- jection screen on which graphical informa- down” display of flight instruments, without the pilot needing to tion was displayed that enhanced operators’ deflect his attention from the head up position... With the difficul- performance at their task.80 ties of providing safe automatic control at low level this “head-up” For fighter pilots, the task is shooting display must as a minimum give a clear unambiguous flight direc- down enemy aircraft, and for this purpose tor display that can be used for long periods without fatigue.72 the Mark II performed startlingly well. The gunsight was painstakingly tested, studied Naish was well-positioned at R.A.E. and analyzed and found to dramatically Farnborough to work on the HUD, for the improve targeting accuracy. One popular facility was at the forefront of the research account of the gunsight summed up its effi- and development of predecessors to the cacy in the following way: “Combat results, technology. In a series of technical papers reports the British Air Commission, show that he wrote while at R.A.E. Farnborough, that the fighter aircraft are now destroying Naish proposed inventions crucial to the nearly twice the number of Luftwaffe aircraft development of a flight-ready Head-up since the introduction of the new gunsight.”81 Display. These included a “superposed It is impossible to assess the factuality of projection from a bright cathode ray tube”73 these claims, but it was an opinion shared or a display which the pilot sees “‘out of by military staff as well. Writes Trafford the corner of his eye’ using the para-foveal Leigh-Mallory, the Air Marshal of the Royal region of the retina.”74 According to Naish,75 Air Force, about the Mark II: these designs were basically enhancements to existing gunsighting technologies used During nearly 1200 actual attacks against enemy aircraft [the in the cockpits of military aircraft.76 Naish’s study of the Mark II Gyro Gunsight] has shown that 21% resulted acknowledgement of this connection links in complete destruction of the enemy aircraft... The general his work on the HUD to earlier World War II introduction of the sight into squadrons should result in a marked military technologies such as the aforemen- improvement in the standard aiming accuracy by fighter pilots in tioned Mark II Gyro Gunsight. combat with both enemy fighters and bombers. I consider that

54 Rupture of the Virtual 55 The Origin of the See-Through Graphical Interface Mark II Pilot’s Gyro Gunsight to be potentially one of the most for the lead time, in order to hit the target. valuable air fighting assets which has been developed during the In aerial dogfights, however, not only is the war, and I am sure that its introduction throughout my command enemy moving, but the firing aircraft as is a pressing need.82 well, which adds a layer of complexity and increases the kinds of error in estimating The “pressing need” for the gunsight was when and where to fire. The Mark II Gyro a dramatic increase in the complexity and Gunsight calculated the amount of angle of speed of warfare during World War II, espe- deflection, bullet drop and range needed to cially in aerial combat. Prior to World War II, hit a moving target, and it did this through pilots were trained to target enemy aircraft the presentation of a moving aiming mark, by making mental calculations on when and or graticule, which a pilot lined up with where to fire. During World War I, for exam- enemy aircraft (see Figure 2). ple, pilots would bomb “low and slow,” that is, It is worth pausing here to consider in at relatively low altitudes and slow speeds,83 more detail how the gunsight aided targeting because the runs could depend on a bom- through a study of its construction and oper- bardier’s intuition and training. Intuitive ation. The Mark II consisted of three basic bombing and gunnery was effective, partly elements: the “computer unit,”84 the graticule, because aircraft were slow moving. World and the reflector screen (see Figure 3). The War II saw the introduction of the British “computer unit” —the brains of the gunsight— Spitfire and the German Messerschmitt was an analog computing device consisting of Bf109s, aircraft that could fly in excess of a gyroscope and a series of electromagnets, 300 M.P.H.; due to their improved speed which provided the offset information for the and maneuverability, pilots could no longer graticule. The principle behind the use of gyro- reliably make mental calculations on when scopes for predictive targeting was well estab- and where to fire at enemy aircraft. lished by this period. A gyroscope resists an What was needed, then, was a device airplane’s axial movement, and by connecting that could take the guesswork out of aiming, the moving graticule to the gyroscope, the one that could automatically compute the graticule would respond to an airplane’s move- information needed for accurate targeting ment.85 Pilots could dial in the type of aircraft of moving aircraft. The Gyro Gunsight against which they were fighting, which would did just this. In aerial gunnery, there are adjust for the enemy’s speed by changing the primarily three ballistics challenges in strength of the four electromagnets aligned targeting: lead or “angle of deflection” firing, around the gyroscope. These electromagnets bullet drop, and range. The angle of deflec- account for range and gravity by offsetting the tion is the amount of “lead” needed to fire movement of the gyro. bullets that could intercept a moving target. The reflector screen consisted of a glass When firing at a moving target, one needs panel about 4 inches tall by 2 inches wide. to shoot in front of the target, or account Pilots looked through this screen when

56 Rupture of the Virtual 57 The Origin of the See-Through Graphical Interface aiming their guns. A light bulb housed within the sight: in high speed aircraft susceptible the computing unit back lit the moving grat- to sudden and unpredictable movements icule and projected its image onto the trans- especially during combat, there was a high parent screen. When looking through the potential for facial injury, as pilots had to put reflector screen, pilots would see an image their face up to the peephole sight in order of the graticule superimposed into their line to target enemy aircraft.86 of sight. By aligning the graticule with the The Mark II gunsight, by contrast, enemy aircraft, pilots would have the proper addressed these issues by eliminating the amount of lead time computed for deflection peephole entirely and replacing it with a firing. The predecessor to the Mark II (the transparent reflector screen. In comparison Mark I Gyro Gunsight) did not utilize a reflec- to the peephole, the reflector sight could tor screen. Instead, the sighting device was a almost be used as a continuous extension peephole, comparable to scopes commonly of pilots’ normal flying position. Though found on rifles today. When Virilio noted the the sight was in no way perfect, pilots no similarities between the technologies of cin- longer had to lean forward when looking ema and warfare, it was on the basis of com- through the gunsight. They could remain in parisons between the peephole sight found in their normal flying position and operate the bombsight technologies (such as the Norden gunsight from there. Just as importantly, the bombsight) and the eye piece of a movie reflector screen also improved pilots’ field camera. In this, the Mark I Gyro Gunsight was of view. Unlike the restricted aperture size similar to other gun and bombsights during of the peephole, the reflector screen was the period in its use of a peephole sight. considerably larger and better integrated into The transition from the peephole sight pilots’ forward field of view. The design was to the Mark II’s reflector screen directly such that it made for a more seamless presen- relates to the thesis of this chapter regard- tation of targeting information; information ing the insertion of a layer of graphical was projected directly into the pilots’ line of information into the pilots’ experience of sight. Put into terms defined earlier, the Mark their material surroundings. The reasons II gunsight overlaid information into the pilots’ for the redesign concerned the restrictive view of their surroundings. By improving aperture size of the peephole, which forced their ability to shoot down enemy aircraft, pilots to reposition themselves to operate this display of visual graphical information the gunsight. Pilots had to lean forward to enhanced pilots’ control over their material look through the sight, which took them out surroundings in quite a literal way. of their normal flying position, making them less prepared for evasive maneuvering and, thus, more susceptible to counterattacks. ENLARGING THE As early as February 1941, British military This problem is embodied in Clarke’s obser- TRANSPARENT researchers explored the possibility of SCREEN vation about the inherent dangers in using enlarging or even completely removing the

58 Rupture of the Virtual 59 The Origin of the See-Through Graphical Interface reflector screens used in aircraft gunsights This would build upon the visual advantages in order to increase the area in which a pilot noted in the transition from the peephole to could target an enemy aircraft and reduce the reflector sight; a pilot would no longer the gunsight’s obstruction of the forward have to look through a constricting device in field of view. In reference to the bulk and order to target an enemy plane. size of an early reflector gunsight, the Chief The proposal to use the windscreen as the Superintendent of the R.A.E. Farnborough reflector screen was enthusiastically received. remarked, “the layout of cockpits in some The Wing Commander in charge of the trials of the new S.S.F. [Single-seater Spitfire for a prototype of the new gunsight wrote: Fighters] is rather congested” and it would “improve the general ‘view’”87 to redesign The opinion of pilots of the Unit and others who have flown the gunsight.88 He proposed removing the the aircraft is that the forward view is very much improved and reflector screen entirely and using the psychologically the installation is good because the pilot does cockpit windscreen instead as the reflective not have to concentrate on the sight so much as previously, as he surface for the gunsight projection. He wrote, is not conscious of looking through a reflector glass.90 “As all of these planes [Single-seater Spitfire Fighters] have B.P. [bulletproof] screens the No phrase better expresses the forces that led question of flexibility does not arise. Sight to the development of the STGI than the test production [e.g. gunsight production] will pilot’s observation that he was “not conscious not be interfered with as the existing optical of looking through a reflector glass” when unit can be used as the projector and the using the windscreen gunsight. Not satisfied B.P. screen as the reflector.”89 To accomplish with the larger window utilized by the reflec- this task, researchers at R.A.E. Farnborough tor screen, researchers desired to further embarked on an ambitious development pro- enlarge the aperture size of the interface moti- gram that lasted through the end of the war. vated by the desire to use the entire cockpit The reasons for the removal of the reflec- windscreen as a surface of projection. To not tor screen directly relate to the idea of the be conscious of looking through a reflector See-Through Graphical Interface as a device glass is a sought after attribute of the design to project graphical information directly into of the STGI. If realized, pilots are cognizant the pilots’ view of their material environ- only of their material environment and the ment. Turning the windscreen itself into a information superimposed onto it. The inter- reflector screen would transform the entire face literally disappears from view, an attri- cockpit into a transparent interface for the bute still pursued today in the design of STGIs: computing unit. Likewise, the windscreen now, Head Mounted Displays can track users’ would become the reflector screen, enlarg- head movements enabling information to be ing the area on which to project the sighting accessible from a full 360 degrees. graticule and eliminating awkward and ineffi- Despite the Chief Superintendent’s enthu- cient sighting mechanisms of earlier designs. siasm for the project, because of numerous

60 Rupture of the Virtual 61 The Origin of the See-Through Graphical Interface technical hurdles, it did not prove easy to information into a pilot’s line of sight as an develop a prototype suitable for mass pro- aid for targeting moving enemy aircraft. In duction. In order to be effective as reflector making this case, I focused on a few aspects screens for the gunsight, the bulletproof of the Mark II’s design: the transition from windscreens had to be manufactured to a peephole sight to a transparent reflector such a high level of precision that they were screen and subsequent enhancements to impossible to produce in quantities needed enlarge the screen. Both were characterized for the war effort. In addition, the light bulbs by efforts to make the interface device itself used in backlighting the aiming mark were disappear, so that graphical information not of a sufficient brightness to make the could be presented to pilots in such a way as gunsight useful for daytime flying.91 to minimize the obstructions to their field of view and improve pilots’ capacity to act on this information. FROM GUN- With the war’s end there was a diminished The design of this first STGI was intended SIGHTS TO sense of urgency surrounding the devel- to enhance knowledge over one’s material HEAD MOUNTED DISPLAYS opment of new gunsight technologies, but environment, a form of visual augmentation the British military continued to invest that links the Mark II gunsight to subsequent considerable resources into Head-up Display STGIs, such as the HUD and AR-HMD. From research and other devices for the visual the first STGI to today’s Land Warrior pro- presentation of flight and targeting infor- gram, the military has shown an abiding inter- mation in aircraft cockpits. As should now est in technologies that enhance the capacity be evident given the history of the gunsight to act in one’s material environment. One can provided in this chapter, R.A.E. Farnborough imagine what the future holds for this technol- was pivotal to the development of the HUD ogy. When the military develops successful both in the decades following World War II technologies, consumer applications are likely and the design of predecessor technologies, to follow. AR-based devices and software such as the Gyro Gunsight. J.M. Naish’s are poised to transform consumer interac- work at the facility, discussed earlier in this tion with computing devices, for there are chapter, was an extension of research at the a growing number of AR applications avail- facility that had already been full bore in the able for the iPhone and Android operating service of the war effort. systems for a wide variety of entertainment This chapter has demonstrated how a and consumer purposes, including Google’s new category of media interface devices recent announcement of Project Glass (see facilitates the identification of computing figure 4). The mobile foot soldier will become technologies going back to the first half of an AR-HMD enhanced consumer, hunting the the twentieth century. The Mark II Gyro most spectacular sales on the shop floor. Gunsight was the first STGI, for it was This historical study has far-reaching the earliest device to project graphical implications for the concept of the virtual

62 Rupture of the Virtual 63 The Origin of the See-Through Graphical Interface as it has been defined in media and visual that extend the virtual to the material, the studies. The virtual has generally been reverse will be explored: aesthetic practices linked to the opaque screen of media tech- that enable an encounter with the material nologies. In subsuming the STGI to research in virtual and virtualized experience. Such on the opaque screen, researchers have practices could enable potent recombina- oversimplified the relationship between tions of materiality, the virtual and embod- the material and virtual in light of See- iment and, even, pose challenges to their Through Interface devices. As suggested current configuration in such a way as to here, the STGI expands upon the history of encourage critical reflection on our shared the Graphical User Interface by identifying cultural condition. a class of interface technologies that have been mainly under-theorized. This history complicates a dualistic conception of the relationship between the virtual and the material, because the STGI imposes an information layer between users and their physical surrounding by projecting graphi- cal information into their field of view. The insertion of this layer complicates any simple opposition between the material and the virtual, for the material itself becomes an integral dimension of the interaction. Finally, the study presented in this chapter concerns a particular vision for the technology wedded to military research and development. It is, clearly, a deeply instru- mental one, for with the STGI, the military aims to augment soldiers’ lethality in the theater of combat by making their surround- ings subject to a form of total information awareness. This vision for the technology is dangerous and limited, but it does not proscribe alternative uses for it. The next chapter considers critical engagements with the medium, such as artistic practices that utilize the interface in ways that trouble or directly contradict the military’s deploy- ment of the STGI. Instead of applications

64 Rupture of the Virtual 65 The Origin of the See-Through Graphical Interface 1 3 A soldier mod- This diagram de- eling the Land picts the optical Warrior System. system of the GGS. According to Clarke, “The lamp shines through the graticule (G) onto the gyro L mirror, which reflects the image of the graticule onto the fixed M mirror (M) which reflects through the lens (L) onto the pilot’s G reflector screen” (Clarke, 1994: 171). Diagram adapted by author.

2 4 A gyro gunsight Announced in in the cockpit April 2012, Google of a Spitfire Glass™ is Goo- airplane. It is un- gle’s foray into clear what model Augmented Reali- this one is. ty Head Mounted Display technolo- gy. The AR-HMD allows one to navigate Google’s suite of services, including Google Maps, Shopping and Chat.

66 Rupture of the Virtual 67 The Origin of the See-Through Graphical Interface 4 Encounters with the Material: Krzysztof Wodiczko and Site-Specific Media Art Since 1981, Krzysztof Wodiczko has been explored many different forms, spanning crafting site-specific outdoor projections, art objects and interventionist public per- artworks that have drawn attention to a formance to image and video projections. viewer’s surroundings in order to provide Since the 1980s, Wodiczko has become a critique of the public function of often widely known for his image and video overlooked monumental architecture.92 His projections onto public monuments and work addresses these overlooked structures the facades of urban architecture. Using in hopes of compelling viewers to connect high-powered projectors, these projections their relevance to ongoing social and polit- consist of imagery that examines or reveals ical problems. As distinct from the military the social, cultural, and/or political signifi- uses and applications put to See-Through cance of site. In his initial work, these projec- Graphical Interface technology discussed in tions consisted mainly of static imagery, but the last chapter, Wodiczko’s public artwork his more recent work makes use of moving is a prescient encounter with technologies images. to visualize information away from the In one of the more provocative readings immobile screens of desktop computers and of Wodiczko’s work, Dick Hebdige suggests televisions. While See-Through Graphical that Wodiczko seeks to cultivate a feeling type technologies are already being mined for of the uncanny as a political tactic in his their military, entertainment and commercial artwork.93 The uncanny arises out of his art potential, Wodiczko’s projects, by contrast, objects, because he takes everyday objects are distinctive for their critical address of the and transforms them into something altered, experience of our built environment. Building unexpected, haunted. With his Homeless off of research on site specificity in art, in Vehicle, for example, Wodiczko turns a shop- this chapter I argue that his media work is ping cart with its familiar use by the home- a cultural practice that opens an investiga- less as a mode of transport and habitation tion of three interrelated aspects of site: its into a striking militarized vessel “as if trans- social and political meaning, its disciplinary planted from another dimension.”94 Hebdige function, and its materiality. By opening an quotes Patrick Wright’s characterization of encounter with the materiality of site, work it as “unlike anything that has ever existed like Wodiczko’s is a critical address of the before, and yet deliberately engineered out virtual in our current cultural condition. This of resemblances to things familiar.”95 analysis challenges how mobile technologies, Though he diverges from Freud’s cele- like STGIs, are characterized today, as tech- brated examination of the term, Hebdige nologies that speak to the heightened expan- draws on Freud’s concept of the unheimlich, sion of the virtual into the experience of the or the uncanny, in his study of Wodiczko’s physical environment. art. For Freud, the uncanny object “proceeds Krzysztof Wodiczko has been making from something familiar which has been public art since the 1960s. His work has repressed.”96 The repressed in Wodiczko’s

70 Rupture of the Virtual 71 Encounters with the Material work is the social antagonisms and con- contributes to their spectral uncanniness is flict surrounding the original object, what that the site becomes bathed in the light of Wodiczko refers to as the “problem issues”97 these ghostly projections; the site becomes that are of topical concern in his artwork. haunted by the images. As ghosts are lim- With its reimagining of the everyday shop- inal beings, neither dead nor alive, neither ping cart, the Homeless Vehicle casts an physical nor fully disappeared, so too are indirect light on the rampant homelessness projections, which are neither material nor and social displacement Wodiczko observed fully immaterial, insofar as we are able to in his visits to the United States.98 The see the light and experience them.100 As I raw materials for his projects are those argue later, the projections’ formal differ- overlooked everyday objects that are open ences from the site (what Friedberg101 calls secrets about the failings of society. their “liminal immateriality” in contrast to The unheimlich is key to understanding the site’s materiality) heightens the projec- Wodiczko’s site-specific projections as well. tions’ uncanniness. For Freud, places and built environments If the uncanny is a return of the can also be unheimlich. The unheimlich repressed, what is the repressed of site? In derives from the German word, heimat, other words, what is revealed about site in which refers to one’s hometown or birth- Wodiczko’s projections? In order to answer place and the feeling of closeness and these questions, I move away from the connection to it. Thus, a common defini- literature on the uncanny in cinema theory tion of the unheimlich is the experience of and focus on the role of place in site-spe- one’s hometown as a haunted, unfamiliar cific media work, because what is distinc- place. Freud quotes the Grimms’ German tive about such work is that it occurs in Dictionary about the close connection public settings and not the darkened rooms between the unheimlich and heimat, “At of theaters. I link the “problem issues” times I feel like a man who walks in the Wodiczko identifies of topical concern in his night and believes in ghosts; every corner work to Miwon Kwon’s conceptualization of is heimlich and full of terrors for him.”99 interrogative site-specific installation art, In contrast to his art objects, Wodiczko’s or work that interrogates the sociopolitical projections produce the feeling of the conditions of site.102 Kwon’s work on site unheimlich in their capacity to alter one’s specificity has been influential, because perception of site. There is something she identifies site-specific art’s break from undeniably creepy about the content of earlier public artworks103 that did little to some of the projected imagery: skeletal respond to the sociocultural conditions of hands holding instruments and machine site. In distinguishing between site-specific guns, eyes without faces, and hands with- and site-generic art in terms of how respon- out arms or bodies that grasp handguns sive the artwork is to the conditions of site, abound in his work (see Figure 1). But what Kwon insists that the site-specific work only

72 Rupture of the Virtual 73 Encounters with the Material makes sense in and is appropriate for the second, a rhetorical device for speaking particular site for which it was crafted.104 with clarity at a distance.”105 In his later If we apply Kwon’s conceptual approach work the “concepts” that are visualized to Wodiczko’s work, we can understand refer to hot button “problem issues”106 of Wodiczko’s projections as one of the earli- concern to Wodiczko, such as homeless- est examples of interrogative site-specific ness, war and militarization, migrant labor, media art. But because Kwon writes primar- and others. Critics have mainly focused ily about pieces that originate out of the on these problem issues and how his work plastic arts, there are distinct limitations in critically engages with them (Hebdige on her approach to the analysis of the formal homelessness;107 Deutsche on redevelop- characteristics of media work. Therefore, it ment and gentrification;108 Smith on spatial is necessary to go beyond Kwon’s concep- differentiation109), for the visual language tualization of site specificity and recognize that he develops for the projections’ overt that Wodiczko’s projections not only address content derives from reflection on these the sociocultural conditions of site, but issues. In his projection on the Arco de la are responsive to the site in the form of a Victoria (Victory Arch) in Madrid, Spain critique of the disciplinary function of built (1991), for example, Wodiczko produced environment and an indictment of our obliv- visual imagery critical of Spain’s military ion to its social and political history. What involvement in the Gulf War. On the sup- is distinctive about Wodiczko’s art is that porting columns of the arch, two skeletal these multiple ways of engaging with site are hands were projected: the right one, hold- marshaled in the creation of an encounter ing a machine gun; the left, a red gas pump. with materiality, an encounter that is fore- Above the skeletal hands on the arch’s attic grounded by the immaterial projection of was projected the question “¿CUANTOS?,” in the sociopolitical conditions of site. In order reference to the cost of war in terms of the to develop this claim, the following two numbers of dead, the price of oil, the cost sections investigate how Wodiczko’s work of war, etc. engages with the sociopolitical conditions of The Arco de la Victoria piece, like much site and interrogates the built environment of his projection work, is responsive to the as a social formation that exerts disciplinary site in that the location is both the inspira- power over inhabitants. tion for the artwork’s content and is used as the surface for projections. The content interrogates the arch’s function as a memo- ENGAGEMENT Wodiczko refers to his own work as a prac- rial to war, while simultaneously recalling WITH THE tice of interrogative design, which Rosalyn the bloody legacy of Spanish militarism SOCIOPOLITICAL CONDITIONS Deutsche has observed consists of “first, and focusing attention on the wars in which OF SITE a symbolic operation by which concepts Spain continues to participate. This link- are visualized as external realities and, age between site and content is integral

74 Rupture of the Virtual 75 Encounters with the Material to Wodiczko’s art, because he seeks to site, Federal Plaza in downtown New York City, address the unconscious power of forgotten was heavily used by pedestrians and people monuments: on work breaks. Serra’s piece was a purpo- sive interruption of the space, a massive Franco, who ordered the monument to be built, is petrified within 12-foot-high, 120-foot-long metal sculpture it but people pretend not to see him. I don’t want to bring him back that effectively cut the plaza in two. Because to life—that doesn’t interest me. What interests me is seeing the of the disturbance it created and its per- connection between this monument and present day concerns.110 ceived lack of aesthetic and social value by a vocal public, the installation became mired in Despite their scale and imposing presence, “rancorous and vehement” controversy, which public monuments, like the Arco de la Victoria, ultimately led to its early removal in March are “neglected” or “overlooked,”111 because 15, 1989 after five years of public hearings, they disappear into the background and lawsuits, and media coverage.114 Serra’s piece are no longer seen. Wodiczko is fascinated was interrogative, because it was a critique of by how their power accrues in their disap- the sociopolitical conditions of public space, pearance. Monuments represent norms and like parks and plazas, which are imagined values that are preserved in the public’s as a “coherent spatial totality.” According mind despite a sense that society might have to Kwon, Tilted Arc “literalized the social superseded them. His Arco de la Victoria pro- divisions, exclusions, and fragmentation that jections link historical aggressions to Spain’s manicured and aesthetically tamed public continuing participation in wars, opening up spaces generally disguise.”115 reflection on how war and militarism con- In light of Kwon’s account, Wodiczko’s tinue to inform the social imagination. projections can also be characterized as Put another way, Wodiczko’s work interrogative in that they highlight the socio- interrogates the overlooked sociopolitical political conditions of public monuments conditions of site. When understood in this and architecture. Jeffrey Skoller has simi- way, Wodiczko’s work fits Miwon Kwon’s larly noted that avant-garde filmmaking has “paradigm” of interrogative site-specific been preoccupied with memorializing the art,112 where site “is imagined as a social sociocultural history of place through cine- and political construct,” and the function matic reflections on the present.116 Whereas of the artwork is “a proactive interrogation,” the films that Skoller discusses engage this which “manifest[s] a judgment (presumably through dynamic montage of contemporary negative)—about the site’s sociopolitical and historical film footage, the Arco de la conditions.”113 The interrogative public artist’s Victoria projection breaks from cinematic relationship to the site is not complementary, form by drawing attention to overlooked but “antagonistic.” Kwon analyzes Richard structures themselves by overlaying imagery Serra’s Tilted Arc (1981-1989) as an exemplar directly onto site. By projecting his work of this type of public art (see Figure 2). The onto war monuments, which are intended

76 Rupture of the Virtual 77 Encounters with the Material to memorialize past wars and the dead who and reproducing social norms in society. His were lost in them, Wodiczko’s work implies work interrogates the built environment, spe- Spain’s continuing possession by militarism. cifically its urban design, as a social formation This critique of shrouded, hidden values, like that exerts disciplinary power over inhabi- militarism, is characteristic of Wodiczko’s tants. About this power, Wodiczko writes: pieces, which expose the conditions of the site and suggest how these structures speak What is implicit about the building must be exposed as explicit; to values still enshrined in the present. the myth must be visually concretized and unmasked. The absent- Kwon reads the development of interrog- minded, hypnotic relation with architecture must be challenged by ative site-specific art primarily in historical a conscious and critical public discourse in front of the building.118 and institutional terms. Its formation was a reaction to the prevailing “assimilationist” Wodiczko draws on Foucault’s influential tendency in public art, one that gravitated studies of the history of modern disciplinary towards “accessibility” and “usefulness” in the institutions in defining this exercise of state decades before the 1980s.117 Serra challenged power in the construction of space and the the assumptions of this assimilative paradigm implantation of this power in individuals. “As in public art and posed a counter-model of Michel Foucault taught us, extensive expe- site specificity. Though Kwon’s is a useful rience and intensive disciplining processes historical periodization, there are distinct have, since the eighteenth century, trans- limitations in locating the interrogative func- formed our bodies from rural to urban, from tion of site-specific work solely in the way it undisciplined to disciplined... Architecture responds to or reacts against paradigms inter- was a didactic mission.”119 Here, Wodiczko is nal to the field of public art. I propose two paraphrasing Foucault’s well-known analysis alternate ways to read Wodiczko’s work: as a of the birth of modern discipline, in which critical address of the disciplinary function of he recounts the origin of Jeremy Bentham’s site and an investigation of materiality. Both panopticon in the late eighteenth century as move beyond Kwon’s historical periodization a physical structure that both incarcerated and suggest ways in which his interrogations the bodies of inmates and served as a potent are more provocative than their critique of symbol of the modern state’s consolidation the sociopolitical alone and open up ways to of power to discipline individuals. Foucault’s further understand their distinctiveness as a observations about the birth of modern formal exploration of materiality. disciplinary institutions are echoed in Wodiczko’s claim that, “Our position in soci- ety is structured through bodily experience ENGAGEMENT If the content of Wodiczko’s projections with architecture.”120 War monuments, banks, WITH SITE reveals the sociopolitical conditions associ- museums—the design of urban space—are ated with site, the location and placement of more than simply concrete forms; they are the work address the site’s role in enforcing fetishistic reminders of the unconscious

78 Rupture of the Virtual 79 Encounters with the Material forces that regulate behaviors. A public one to look closer and scrutinize one’s built structure, like a war monument, speaks to environment, especially those details that the legacy of militarism, but it also has a role are generally unseen. One might notice, for of implanting these values as norms within example, the triumphal arch under which us, because the design of our built environ- one walks on a daily basis and appreciate ment is part of the symbolic universe that details, like the impressiveness of scale interpellates individuals as subjects.121 or the complexity of ornamentation, but What is counterintuitive about this power structures become unheimlich (one “actually is that it is exercised in those places we sees them”123) when one recognizes their feel most comfortable. Familiar spaces are disciplinary function within the constellation where a maximum amount of discipline is of forces that converge on an individual in in evidence. The effective hegemonic func- the experience of site. It is in these moments tioning of public monumental architecture when we “expose” site’s function as “explicit,” is precisely in its capability to go unnoticed. “unmask” its “myth,”124 and engage in criti- Referring to Guy Debord’s work on psychoge- cal discourse about it. Wodiczko’s work on ography, Wodiczko characterizes urbanism and with site, in other words, is a traumatic (which includes city planning, public archi- resurfacing of its repressed. tecture, urban design, etc.) as “the material From this perspective, it is necessary foundation” for those “symbolic, psychopo- to deepen Miwon Kwon’s claim that inter- litical and economic forces” which interpel- rogative site-specific art engages its socio- lates individuals.122 Our habitual movements political conditions. Her account does not through space are evidence of the existence sufficiently account for how artwork might of these forces over our seemingly autono- be more sophisticated in its interrogation mous decisions. We have internalized the of site beyond an identification of explicit state’s norms, and this is illustrated by the social or political issues found within it. fact that we do not balk at the war monu- Wodiczko’s work expresses a concern with ments or the ornate churches we pass daily. the exercise of social power and disciplinary Wodiczko’s projections are interven- force that converge at the focal point of tions into this daily remonstration of site. What substantiates this claim is that disciplinary force. Recalling Hebdige’s despite the fact that the projections’ content characterization of Wodiczko’s work as has shifted among different problem issues opening the door to the unheimlich, his pro- over the years (e.g. militarism, migrant labor, jections trouble the familiarity of home. By homelessness, etc.), his work continues to be dressing the familiar in foreign projections, situated in sites that have overt disciplinary it is changed; it becomes unfamiliar. Home significance. This is an indirect acknowl- is made unheimlich, because it is dispos- edgement of the difficulties in challenging sessed of its cozy familiarity. For those who sociopolitical conditions in face of the dis- are receptive, this is a haunting that compels ciplinary function of site. Critical discourse

80 Rupture of the Virtual 81 Encounters with the Material is blocked when we are unable to recognize environment. In one of the more sharp- how the built environment enforces and minded theorizations of this kind of inter- reproduces the social norms against which face, Jason Farman has insisted that mobile artwork struggles. computing enables a “collaboration of mate- The shifting subject matter of Wodiczko’s rial and digital spaces.”125 One’s geograph- work also suggests the ephemerality of ical location, for example, is increasingly interventions based in video projection. In integrated into our interactions with mobile contrast to the materiality of site, video technologies. This is akin to Giannachi and projection is immaterial, temporal and Kaye’s insistence on the growing hybridiza- temporary. In other words, site is not only tion of virtual and physical presence with sedimented social power, but its material mixed reality media usage.126 form is the surface for projections. The next As an illustration of this collabora- section considers at length the formal quali- tion, Farman examines Streetmuseum, an ties of Wodiczko’s projections, where I draw Augmented Reality app for the iPhone on conceptual frameworks familiar to new released by the Museum of London in 2010.127 media studies and argue that his projections Streetmuseum enables users to view histor- onto site are an investigation of site’s materi- ical photographs and paintings in the muse- ality. A point of entry into this aspect of his um’s collection superimposed on their actual work is the recognition that his projections location. When standing outside Buckingham seem a prescient investigation of the possi- Palace, for example, one is able to view a pic- bilities of recent digital technologies, such as ture of British suffragettes being arrested for Augmented Reality. Wodiczko’s works are an protesting. Looking through a mobile device exploration of the aesthetic possibilities in at the specific location, it is as if the image the superimposition of virtual images onto were projected onto the site itself.128 one’s built environment, a kind of informa- I address differences between them tional interface with one’s physical surround- towards the end of this section, but with ings that is becoming widespread with the regard to the experiences each provides, popularization of mobile computing devices. there are points of comparison between Augmented Reality applications, like Streetmuseum, and Wodiczko’s site-specific ENGAGEMENT Let’s first flash forward to research on projections. Both superimpose visual imag- WITH MATERI- contemporary media technologies in which ery into the view of one’s surroundings, and ALITY there is growing interest in alternatives to the content is dependent upon the viewer’s our interaction with data beyond the opaque physical location. Additionally, in both cases window of the computer screen. Of particu- the imagery is not only intended to draw lar interest are mixed reality interfaces, like attention to its content, but also to the built Augmented Reality, that have the potential environment.129 The overlaying of images to reconfigure our experience of the built invites viewers to discover something about

82 Rupture of the Virtual 83 Encounters with the Material the site that is not apparent to the naked eye, While it has been productive for the study such as some detail about the social or polit- of various developments in mediated cul- ical significance of the location, prompted ture, the concept of the virtual has led to through curated visual content that illumi- an impoverished idea of media’s role in nates the site’s meaning. connecting us to our physical surroundings. The conclusion Farman draws about the In new media research, the material has role of mobile computing in mediating site, primarily been conceived in its disappear- however, is different from those drawn about ance. By applying insights derived from the Wodiczko’s outdoor projections in previous analysis of Wodiczko’s site-specific media sections. Farman recognizes in such tech- projection, we can discover resources for an nologies the growing interpenetration of alternative account to the material that does material space by an informational or virtual not presume its disappearance. dimension. He writes: Chapter 2 reviewed research on the virtual and documented how the concept The move from personal computing to pervasive computing, a has been instrumental in revealing trans- shift characterized by the move from immobility to mobility, formations brought about by media and has allowed for online space to interact with material space in informational processes, but this research unprecedented ways.130 has relied on a systematic exclusion of a concept of the material. Examples of this Farman identifies a substantive erosion of proliferate in the literature: from Baudrillard the distinction between the material and the and Derrida’s early insistence that the informational with the rise of mobile com- virtual has come to determine cultural puting. One’s physical surroundings become processes to more recent work on disem- an “information interface”131 because a bodiment in online environments. The person’s geographical location is an indexi- virtual has enabled researchers to identify cal marker to online information accessible the myriad ways in which social, economic by mobile computing devices. Farman writes and cultural phenomena are being dema- of the “collaboration” of the material and the terialized with the advent of informational informational, but the focus is on the ways processes. This has constrained inquiry by in which physical space is being subsumed providing a singular interpretation of these to informational processes. developments. The material has either been Farman’s reading does not consider with completely excluded, or acknowledged only enough complexity our relationship to the in its demise. In other words, the virtual has material, because it draws heavily from been conceived of as the conceptual obverse existing concepts of the virtual in new media of the material with the material perceptible studies. It has been argued that digital only in its disappearance. In this regard, technologies have extended and enhanced the material is a kind of “mute facticity”132 the possibility of virtual-type experiences. shaped or acted upon by the virtual and

84 Rupture of the Virtual 85 Encounters with the Material has not been given consideration as a con- immateriality of the projected content. Both stitutive element of interactions with media require the viewer to venture out into the technologies.133 built environment in order to see the imagery When considered in this way, it should projected onto site. The contrast between come with little surprise that Farman depicts the light and impermanence of the projected the material as subject to colonization by images and the site’s materiality—the surface informational processes. The hybridization of for the projections—is a significant part of the material and the informational results in what makes the experience compelling. In the transformation of physical environment order to characterize this dimension of the into an “information landscape,”134 akin to work, let’s reconsider Anne Friedberg’s defi- existing accounts of the precedence of the nition of the virtual, introduced briefly in the virtual. What’s needed, however, is an analy- first half of this chapter: sis that recognizes with more complexity our interaction with the material, and Wodiczko’s Any representation or appearance (whether optically, techno- work can help us refine our thinking. Before logically or artisanally produced) that appears ‘functionally or developing this account, it is important to effectively but not formally’ of the same materiality as what it acknowledge differences between Wodiczko’s represents. Virtual images have a materiality and reality but of a projections and AR-enabled interfaces, like different kind, a second-order materiality, liminally immaterial.135 Streetmuseum. Whereas Streetmuseum is available on Internet-capable mobile comput- As a metaphor substitutes one meaning with ing devices, Wodiczko uses high-powered another, a projected image stands in for 35mm slide projectors, only beginning to use the thing itself. Though this comparison is video projectors in the 1990s. Because of the “functionally” accurate, a difference between nature of the projections, Wodiczko’s work is a metaphor and a projected image is that a staged as public events with large numbers of projection also differs formally from the thing people in attendance. With AR, on the other it represents. The projected image is liminally hand, the content is available to the public immaterial, made of light and darkness. (to be more precise, only those who have One way to conceptualize the history of smartphones), but experienced privately on a the media is as a record of trying to cover handheld device. While Streetmuseum draws over this formal difference between the on historical photographs and paintings from liminally immaterial projected image and the the museum’s collection, Wodiczko stages thing it represents, to create an experience scenes to illustrate the conditions of site. in which the virtual image is mistaken for Despite these technical differences, when the material object. In her discussion of the we consider the formal characteristics of various technological and perceptual devel- both, similarities emerge. What is striking is opments around the virtual image, Friedberg their shared exploration of the formal differ- reflects on the architectural details of the ences between the materiality of site and the movie theater and how historical changes to

86 Rupture of the Virtual 87 Encounters with the Material its design were made to enhance its capacity materiality. Under illumination, the mate- for enhancing the illusion of cinema.136 The rial is brought forth as that which exists construction of movie theaters facilitates a beneath the light of projections. In this way, viewer’s willing suspension of disbelief in the Wodiczko’s work creates awareness of the virtuality of the images. With their darkened formal difference between the projected interiors and prosceniums that suggest the- images and the site, the virtual and the atrical stages, movie houses are crafted so material. Wodiczko’s work is an encounter that spectators can temporarily lose them- with materiality, one that is foregrounded by selves in the world on display. Spectators the immaterial projection of the sociopoliti- suspend their recognition of the virtuality cal conditions of site. of the images and enter them in imaginative What is not being suggested is that the construction as real places. material gives meaning to site, or is a locus of The site specificity of media projec- contact or interaction that is a more authentic tion can upset expectations about the relation to one’s built environment. In both suspension of the material during media of these senses, the material is imagined as spectatorship. The settings in which viewers a kind of ground or truth from which mean- experience Wodiczko’s projections are not ing derives. Critical of a similar tendency, cinema’s silver screens and darkened the- Schopenhauer once wrote that “everything aters. Wodiczko’s projections take place in objective, extended, active, and hence every- public locations in order to address the site thing material” is regarded “as so solid a basis and its sociopolitical conditions. He projects for its explanations.” This skepticism with onto the built environment, and the over- materialist explanation and the diverse forms laying of virtual images onto site highlights it can take is illustrative of the confluence the formal difference in the materialities of of complex structures of meaning, histories site and image. (It is as if the theater itself is and cultural processes that attend our under- illuminated and the screen left intentionally standing of the material. Like the virtual, dark.) For uninformed viewers, the reali- the material is constructed in interweaving zation that the projections’ sociopolitical systems of signification, which fashions the content refers to the structures under illumi- manner in which we evaluate, understand, nation can come as a surprise. (One expe- sentimentalize and relate to our built environ- riences his work when attention is arrested ment. Though the theorization of the concept by the uncanniness of the projections; when of the virtual has led to a refined understand- one’s movement through space is altered.) ing of how the media are integral to our inter- This uncanniness draws attention to the actions with the world, it has simultaneously buildings, the monuments, and the built obscured a concept of the material and the environment serving as the screen for pro- different modalities of contact with it. jections. The virtual images float spectrally on the buildings’ surfaces, which is a solid

88 Rupture of the Virtual 89 Encounters with the Material RUPTURE OF I want to open up the implications of this which media technologies are seen to THE VIRTUAL: TOWARDS MEDIA reading of site-specific media art by expand- enhance or expand the reach of virtual PRACTICES OF ing upon the conclusion drawn at the end of processes. Despite my insistence on the MATERIALITY the last section: that Wodiczko’s projections conceptual limitations in this application of are an encounter with materiality, one that is the concept to new media, this should not foregrounded by the immaterial projection be confused with a rebuke of the concept’s of the sociopolitical conditions of site. Three usefulness in understanding and character- linked speculative claims about the impli- izing contemporary culture. Indeed, many cations of this conclusion for critical media of the premises found in the theorization practices are explored in this section. The of the virtual are compelling, especially the first claim applies this notion of an encoun- underlying claim that our cultural condition ter to a range of cultural practices, not just is one increasingly subject to forces of virtu- Wodiczko’s projections. His work is a power- alization—the argument that most, if not all, ful example situated in media practice, but experiences are inflected by the virtual to this encounter is not exclusive to his work some degree. This is evidenced by a reliance alone. To the extent that a range of cultural on the media to extend one’s sensory per- practices share this preoccupation with the ception of the world and the dematerializa- material, I argue that such encounters are tion of forms of social interaction and work. responses to the growing experience of the Wodiczko’s art is situated in the experi- virtual as a cultural condition, what I earlier ence of the virtual in contemporary culture, referred to as the mounting immaterializa- and invites an interrogative address of it by tion of social, economic and cultural phe- creating an encounter with the material. The nomena because of informational processes. overlaying of projected images onto the built When contextualized in a way that enables environment foregrounds the site’s materi- us to link the work to broader concerns, ality, preventing the site from receding into these encounters can produce a critical darkness. Such an encounter with site’s understanding of our condition relative to materiality challenges viewers’ expectations the virtual. Finally, I develop a brief typology about the circulation of images in a culture that organizes cultural practices according of heightened virtuality. This is to say that to their capacity for a critique of the virtual a recognition of the role of the material in and define a type of practices that “rupture Wodiczko’s projections invites an interroga- the virtual.” tive address of the virtual. This recognition In the last section I argued against the distances viewers from an embeddedness view that AR is simply a technological in the virtual and contributes to a critical expansion of virtual processes, a trans- position regarding it. formation of the built environment into The sociopolitical content of Wodiczko’s an “information landscape.” This position projections facilitates this encounter but results from a concept of the virtual in is not necessary to it. The content is a

90 Rupture of the Virtual 91 Encounters with the Material reflection of Wodiczko’s aesthetic choices about the poetry of the late nineteenth and and is not the only way to activate site. early twentieth century, can be a guide for Content, in other words, is a tactic, not a key understanding this type of work. Adorno to unlocking the meaning of his chosen site. resists interpretive positions that portray One can imagine a range of practices that lyric poetry as “the most delicate, the most produce a recognition of the material with a fragile thing” that seemingly rises above the range of content, including—but not limited “bustle and commotion of the everyday.”137 to—the construction of site-specific content Instead, Adorno offers an opposing conclu- that relies on fictional narratives concerning sion about lyric poetry. He contends that it material objects in the site, and the stripping is precisely deadlocked social antagonisms away of sociopolitical content in order to that constitute lyric poetry as the most foreground formal differences between the delicate, the most fragile thing. Because of virtual and material. This invites analysis this, analyses that take into account social of a range of contemporary media and context are of the utmost importance. performance artists, who have explored the When Adorno’s insight is applied to cultural material in their site-specific work. Chapter practices that engage the material, we can 5 examines work by artists including Janet conclude that the more practices are able Cardiff, George Bures Miller, MAW, Daniel to open encounters with the material with- Dean and Ben Moren. Media artists, however, out veiling sociopolitical content, the more are not alone in their preoccupation with sharply defined their critique of our cultural this form of critique; it is possible to expand condition. I contend that they “rupture the the horizon of this research to include non- virtual,” because of their more realized art based cultural practices. The conclusion critique of our cultural condition relative to of this book reflects upon examples of this. the virtual. Site-specific media projections during the Occupy Wall Street protests, for example, helped to illuminate the insight that space can be more fluid, contestable, or transiently occupied, despite the disciplinary pressures exercised upon the proscribed use of site. In this regard, cultural practices that aim to refine their content into a direct interrog- ative address of the virtual are compelling, precisely because they have the potential to reveal in less overdetermined ways the viewer’s relationship to the material. Though penned in a different context, Theodor Adorno’s essay “Lyric Poetry and Society,”

92 Rupture of the Virtual 93 Encounters with the Material 1 5 Krzysztof Wodiczko, Arco de la Victoria, Madrid, 1991. © Krzysztof Materiality in Wodiczko. Courtesy Galerie Lelong, New York Site Responsive Media Art

2 Richard Serra, Tilted Arc, 1981. Image: © G. Lutz.

94 Rupture of the Virtual Krzysztof Wodiczko’s projection work estab- multiplicity of ways, we can extend the utility lished an early and influential approach of a concept of the material in the examina- to site-specific public media art. Since tion of media that frames a relationship to the 1980s, there has been a rapid expan- one’s physical surroundings. sion in the number, variety and technical sophistication of outdoor media art. Today there are a number of artists who engage JANET CARDIFF AND GEORGE in projection-type work but operate with BURES MILLER: different models for site specificity. This AUDIO AND chapter investigates three groups of media VIDEO WALKS artists and collectives who create work that make use of site in innovative ways, but in We are all searching for the real, for the authentic experience... configurations differing from the examples that we are actually on this earth … We are building a simulated previously examined. In contrast to the experience in the attempt to make people feel more connected previous chapter’s focus on site specificity to real life.139 in media art, I employ the term site respon- sive to characterize the works discussed in Cardiff and Miller have been creating this chapter. Site responsive is intended as artwork since the early 1990s, and their a more inclusive notion of how media art projects span a wide variety of forms, from responds to location. While site-specific art gallery installations and sonic environments exhibits the characteristics that were dis- to outdoor media pieces. I focus on a body cussed in Chapter 4,138 site responsive media of work they refer to as audio and video art, by contrast, may exhibit some (but not walks, because, in light of the site-specific all) of these characteristics. Site responsive, work discussed in the last chapter, they in other words, is used here as a more open- make use of comparable sensory and media ended category for media art that includes elements. The manner in which they have both site-specific art and work that is agnos- spoken about their intentions for the work, tic about site specificity. furthermore, resonates with the discussion An investigation of the variety of site of the material found in Wodiczko’s projec- responsive models for media art will sup- tions. As suggested in the quotation above, port the discussion of the material and the Cardiff and Miller characterize their own media’s role in constituting knowledge about work as an effort to “make people feel more it. The works examined here include ones connected to real life” or inspire a “renewal ones that critically investigate the material to of a sense of place.”140 In an examination of those that blur or hybridize viewers’ percep- a selection of their audio and video walks, tion of their physical surroundings in such I argue they can also produce an opposite a way as to extend the virtual out into site. effect: the extension of mediated experi- By examining projects that engage site in a ence out into the material, which blurs the

96 Rupture of the Virtual 97 Materiality in Site Responsive Media Art boundary between the real and the invented. soundtrack of an audience around you. The While Wodiczko’s site-specific projections third is a… sound environment of scenes foreground the sociocultural conditions of that unfold around you in the darkness.”141 site and its materiality, Cardiff and Miller’s The audience strains to distinguish extend the virtual out into the participant’s between the movie’s soundtrack and exter- interactions with site. I begin with an exam- nal noises, real and fake audience noises, ination of one of Cardiff and Miller’s gallery between sounds occurring inside or outside based projects, which illustrates a preoccu- the structure. The piece unsettles conven- pation with the virtual and its relation to the tions surrounding spectatorship in theatri- material that is central to their audio and cal settings, which makes the experience video walks. discomforting. The audience’s attention Originally created for the Canadian jitters between the film’s soundtrack and Pavilion at the Venice Biennale in 2001, the recorded audience noises. This shuffling Paradise Institute (2001) consists of a ply- of attention thwarts one’s ability to settle wood structure 17 x 36 x 10 feet with two into the immersiveness of a cinematic expe- doors that grant access to viewers (see rience. Paradise Institute is less about the Figure 1). When viewers enter, they find narrative content of the film being screened themselves on a theater’s balcony with 16 than this struggle to comprehend the mul- seats. It is a model of an old-fashioned movie tiple soundtracks. In other words, the piece theater in extreme forced perspective and examines the audience’s positionality as crafted in miniature to give the illusion that spectators in the constructed environment one is in a full-sized movie theater despite of the theater by unsettling the expectations the sturcture’s actual scale (see Figure 2). A that normally govern it. movie is screened and participants listen In Jonathan Crary’s analysis of modern to the soundtrack on headphones provided spectatorship, he charts how our capac- at each seat. Cardiff and Miller constructed ity for attention is the outcome of over a the soundtrack in such a way as to confuse century and a half of construction.142 That viewers about which sounds are part of the we can sit in fixed attention for the dura- movie’s storyline and which are recorded tion of a Hollywood movie hardly seems a ambient sounds that are outside the movie’s feat of disciplined self-control, but when narrative, such as the sounds of coughing, understood alongside myriad ways in chit-chat and rustling, a ringing cell phone, which focused attentiveness is demanded and a woman who whispers intimately into in contemporary forms of consumption one’s ear. In interviews, Cardiff and Miller and labor, such as full-time computer work have said that there are, in fact, three and assembly line manufacturing, the discrete soundtracks to the piece. “One importance of the construction of attentive level gives you the experience of the film subjects becomes clear.143 As part of his soundtrack. The second level is... a 3D analysis of the modern attention economy,

98 Rupture of the Virtual 99 Materiality in Site Responsive Media Art Crary also notes the rise of psychological are created for these precise locations, yet disorders associated with an inability to unlike walking tours, Cardiff and Miller’s properly embody attentiveness, including pieces tell a story about the place that is forms of schizophrenia and Attention frequently fictional, but at times contain Deficit Disorder.144 Symptoms associated fragments of factual details. with these disorders can include hyperac- To date, Cardiff and Miller have made tivity, disorientation, confusion, inability to over 25 audio and video walks and have concentrate or focus, and others. These are developed a number of techniques to precisely the kinds of effects that Cardiff deepen a listener’s involvement in the nar- and Miller produce in Paradise Institute. The rative. Given the large number of walks, it disruptions to the narrative make it impos- is difficult to generalize the techniques that sible to focus on the plotline, creating a they employ. I focus on a particular example, disorientation akin to the disorders asso- Her Long Black Hair (2004), an audio walk ciated with inattentiveness. What an inter- commissioned by the Public Art Fund for loper whispers into your ear, for example, Central Park in New York City, to exemplify becomes part of the unfolding sequence some of these techniques. The piece has the of events occurring on the screen. The sensibility of a detective story, because lis- screen does not define the entirety of the teners are given a packet of photographs at audience’s experience of the piece, for the the start of the tour, which includes images outside breaks in as a constitutive element. of a mysterious woman with long black hair This preoccupation with the inside (see Figure 3). Cardiff’s narratorial voice and the outside, the constructed and the also has a conspiratorial tone to it, making real, sets up a discussion of Cardiff and it seem like we are in pursuit of someone or Miller’s audio and video walks, which cul- something. There is no discernible narrative tivate similar tactics. In contrast to their structure to Her Long Black Hair, which gallery-based work, the walks draw on a consists mainly of the narrator’s recollec- tradition of recorded driving and walking tions and thoughts about various locations. audio tours.145 At the start of a walk, partici- In Cardiff and Miller’s own words the piece pants are given portable audio players with is an immersive “investigation of location, headphones (in later video walks people are time, sound, and physicality, interweaving given portable video players, such as digital stream-of-consciousness observations with video cameras or video iPods). Prerecorded fact and fiction, local history, opera and narrators (frequently Cardiff herself) entreat gospel music, and other atmospheric and listeners to follow their voice to various cultural elements.”146 locations around the site. At stops along the Both Her Long Black Hair and Paradise way, listeners pause to look and listen to Institute cultivate disorientation about what particular details about their surroundings. is happening inside and outside the struc- The walks are site-specific in that the stories ture of the work. While participating in Her

100 Rupture of the Virtual 101 Materiality in Site Responsive Media Art Long Black Hair, listeners hear gunshots environment and what is pre-recorded, in the distance, which the narrator claims, simulated and virtual. implausibly, is the sound of wild goats and Despite the virtual reality of their audio pigs being slaughtered. (This claim is only and video walks, Cardiff and Miller have also slightly credible. There were pig farms on claimed that they offer a “renewal of a sense the land that eventually became Central of place”147 or a sense of “connection” to a Park, but it is doubtful that wild goats or participant’s environment. Cardiff and Miller pigs were ever shot in the park). Music have spoken about how their work is driven drifts in—its origin unclear—the sound by an impulse to reconnect people to the could be coming from within the park or physical world: it may be part of the recording. One of the photographs provided for the walk has the Walkmans have always been criticized for creating alienation, semblance of a historical snapshot of a but when I first discovered the walking binaural technique I was crowd sitting in the park, further confusing attracted to the closeness of the sound and the audio bridge participants about the walk’s basis in fact between the visual, physical world and my body. To me it was or fiction. This continual shifting between about connection rather than alienation. I think it’s partially the real and virtual, invented and factual, because the audio on the CD meshes with the audio in the “real” pre-recorded and live, is typical of Cardiff environment but it’s also because sound does come into your and Miller’s audio and video walks. unconscious more directly than visual information.148 Underscoring their recognition that the walks blur the real and the invented, Following up on this discussion of connec- Cardiff and Miller have variously referred tion, Cardiff discusses a possible confusion to their audio and video walks as “physical between the invented and the real in the cinema,” “virtual reality,” “3D narrative walks: experience,” and “film soundtracks for the real world.” Though Cardiff and Miller “There’s a green door to your right, turn left at this corner, around refer to their work as virtual reality, it is the post. Watch out for the bike!” Some of what I say reinforces a term they use knowingly as a metaphor, what you see in front of you so that my description parallels the for none of it employs head mounted voice inside your own head: “Yes there is a green door, yes there displays or computer-generated virtual is a corner, I’ll turn left.” You are lulled into a complacency but environments. What they mean is that then Bam! There is a disjunction: “Where is the bike?” I’m not their work is a kind of immersive cinema sure why this technique interests me, but I think it works to push in which our perception of reality comes and pull people in and out of the experience.149 to be experienced as mediated. As virtual reality substitutes the material with com- What Cardiff might be describing here is a puter simulation, in Cardiff and Miller’s version of the concept of salience, which is work it becomes difficult to distinguish used to describe objects and ideas that take between what is occurring in one’s physical on heightened importance or fascination

102 Rupture of the Virtual 103 Materiality in Site Responsive Media Art relative to other things in one’s environ- in renewing a sense of place echoes ment. When a narrator speaks knowingly Wodiczko’s preoccupation with encoun- of a “green door,” for example, it gives tering the material. Instead of producing listeners a sense of that object’s height- an awareness of site in ways that were ened significance within the context of discussed with regard to Wodiczko’s the story. (The next time a green door site-specific work, Cardiff and Miller’s appears in the story, it is encountered audio walks make use of one’s material with anticipation.) What is problematic surroundings as the site for the creation about Cardiff’s claim, then, is that a of an invented narrative. The audio walks thing’s salience primarily exists relative are narrative structures that reference to the particular narrative in which it is site and create salience surrounding told or contained. (All green doors do not particular objects within the work. The take on heightened importance in one’s walks are thus site responsive, because life; only those in the story.) Because the they respond to and refer to one’s sur- walks are closed narratives, the objects roundings, but they create a sense of and themes highlighted within them are disorientation about what is inside and salient only within the semiotic structure outside, the invented and the real. To the in which they are contained. Though extent that we can refer to Cardiff and Cardiff and Miller might wish that their Miller’s media walks as virtual construc- walks produce a recognition or “renewal” tions, this disorientation is an effect of of place and things outside of the walks’ the blurring of the virtual with the mate- confines, the salience of objects exists rial. The material is enfolded as salience only within the semiotic closure of a nar- into the closure of the virtual. rative and does not extend beyond it. This semiotic closure is further evi- denced in passing details found in some MINNEAPOLIS Since the early 2000’s, a number of art of Cardiff and Miller’s walks that occur ART ON WHEELS groups have been exploring outdoor mobile in public spaces outside the protective projection as an artistic medium. Wodiczko’s confines of parks, buildings or forests. In early projections were necessarily sta- them the narrator reminds the listener to tionary, because they utilized large, high look both ways before crossing the street. wattage slide or video projectors. Newer (In a similar way she reminds listeners groups have made use of advances in projec- to close a door gently when on a library tion technology, such as lightweight, lower walk.) Participants are so immersed in wattage projectors that can be run off of the closed narrative of an alternate real- batteries and deployed as mobile projection ity that they forget the rules and dangers units. The rise of a DIY software program- by which this one operates. Cardiff and ming culture has also led to the creation of Miller’s claim that they are interested tools to enhance forms of interactivity and

104 Rupture of the Virtual 105 Materiality in Site Responsive Media Art immersion in outdoor projections. From includes a laptop, video projector, amplified virtual graffiti to bicycle-based mobile “hap- speakers, battery power to provide electric- penings,” there has been a proliferation in ity to all of this equipment (see Figure 4). models for mobile outdoor projection, which The MBU became a platform for the creation provide additional insights into site specific- of a wide range of outdoor projections, ity and site responsiveness considered here. such as L.A.S.E.R. Tag, software that allows The next two studies are drawn from a participants to draw on walls with swaths of burgeoning media art scene in Minneapolis, projected color using only a laser pointer. Minnesota, where I have worked since 2009. MAW’s version of the MBU was built Minneapolis has become a hub of creativity during a class on mobile projections at around outdoor media arts. Steve Dietz, for- the University of Minnesota in 2008 (see mer curator of new media at the Walker Art Figure 5). They began “outings” with the Center and the founder of the SJ01 Biennial, unit that spring. Initially, they created the has had a significant role in this. Since 2011, content using a mixture of digital video files he has organized Northern Spark, a nuit and employed off-the-shelf tools, such as blanche (dusk-to-dawn) outdoor art festival, Flash. By the summer, Momeni had written which focuses predominantly on outdoor a preliminary version of the software that media art.150 The two artist groups discussed came to define MAW’s projection work, next have both been commissioned by Livedraw. Originally written in MaxMSP (a Northern Spark. Minneapolis Art on Wheels newer version is written in C++), Livedraw is (MAW) and Mobile Experiential Cinema a software package that provides users with (MEC), which was founded by MAW mem- real-time video performance tools for mixing bers, explore the possibilities in outdoor hand-drawings and real-time keying, layering media projection. Though both groups work and sequencing (see Figure 6). This allows with site-specific media projection, MAW has users to create in real-time a wide range generally been less concerned about issues of projected content, including short video concerning site specificity while MEC contin- loops, hand drawings, and playback of prere- ues to refine and explore new possibilities corded video. for immersive, mobile video projection in MAW has organized hundreds of bicycle site responsive and site-specific ways. outings. Each one varies in content and MAW draws from a flexible group of col- duration, and as the following examples laborators.151 Inspired by a visit to the Graffiti demonstrate, they are often guerilla in Research Lab in New York City, which had nature. The group does not seek permission recently unveiled its Mobile Broadcast Unit to project on public walls, and because of (MBU), Ali Momeni founded the group in this, outings generally take place in out-of- 2008. The MBU is a utility tricycle outfitted the-way locations (i.e. under bridges and on with a toolkit of technologies for mobile commercial buildings that are closed for the outdoor video projection; the basic setup night) in order to avoid attention from the

106 Rupture of the Virtual 107 Materiality in Site Responsive Media Art authorities. Despite the potential for run-ins A second memorable public projection, with authorities, they have also organized in October 2010, was more orchestrated. outings in public, high traffic areas, where MAW has worked with institutions to create they can attract large audiences. As is the large-scale, public projections. “Seaworthy” nature of MAW’s work, the projectionists was a multiple projection piece commis- actively seek and encourage participation sioned by the University of Minnesota for by pedestrians, soliciting short poems, their Twin Cities campus (see Figure 7). drawings, etc. They have also asked the Like the Lyndale projection, “Seaworthy” public to perform in the projections. This was participatory, consisting of wireless content gets composed into a collage-like projections of audience members’ faces, video piece. who were asked to close their eyes while In conversations with Ali Momeni, he reflecting on the prompt, “Think of the last highlighted a handful of outings he thought time you were at the sea.” were most successful. He recalled a series As both of these examples demonstrate, of projections on a building located at the participation is central to the construction intersection of 26th Street and Lyndale and experience of MAW’s outdoor pro- Avenue South, a busy stretch of the Uptown jections. Audience interaction is actively neighborhood in Minneapolis during cultivated in the design of their work. What 2008. The guerilla nature of these urban made these projects particularly success- projections gave rise to unexpected and ful was the public’s unforeseen levels of impactful encounters. What made this surprise and interaction. For MAW, site Uptown performance especially memorable is the location for social encounters and for Momeni was the arrival and response participatory interaction with a public of the Minneapolis police, which came that is drawn into the work. In this sense, in response to MAW’s activities. In the MAW’s engagement with mobile outdoor ensuing conversation between the officers projection has more in common with and the group, one member of MAW, using relational art, what Nicolas Bourriaud has the MBU projected on a public wall, wrote defined as “a set of artistic practices which the phrase, “I’ve lost everything, and I take as their theoretical and practical feel great.” This prompted a conversation point of departure the whole of human between the police officers in which one relations and their social context, rather remarked how a friend had recently said than an independent and private space.”152 something similar. The phrase became a In MAW’s work, the material becomes a disarming gesture that got the officers to screen for projections that facilitates active become participants in the work and to social interactions and participation. recognize the value of MAW’s public projec- tions. The officers allowed MAW to continue their projections and left the scene.

108 Rupture of the Virtual 109 Materiality in Site Responsive Media Art MOBILE Daniel Dean and Ben Moren are both long- site-specific, mobile cinematic work. The EXPERIENTIAL CINEMA time contributors to MAW. Sharing an inter- narrative structure of the piece is orga- est in experimental filmmaking, they formed nized around a small group of young adults Mobile Experiential Cinema (MEC) in 2011 who are discussing a missing friend. In the for a Northern Spark commission to explore structure of the piece, the pretext for mov- site-specific projection. They have subse- ing from site to site is to follow the group’s quently collaborated on several works that recollections about their friend at the exact explore comparable themes and ideologies. locations in which they occurred. At each MEC makes use of the Mobile Broadcasting site, the audience views a series of pre-re- Unit developed by MAW as a platform for corded video shorts, which are projected their own work, but Dean and Moren make directly onto the materiality of site. The significant changes to the outfit to facilitate opening sequence of Second Bridge is Wider, their divergent approach to site-specific, But Not Wide Enough, for example, occurs responsive projection. under the Hennepin Bridge, where the The last section highlighted how site audience discovers the namesake for the was not a central concern in MAW’s public piece on a memorial plaque (see Figure 8). projections. It is this relative disinterest that A video sequence is projected directly onto was one of the inspirations for the formation the concrete bridge supports: a recording of MEC. Dean and Moren maintain a more of the characters, in the exact same loca- consistent interest in site responsiveness and tion, recounting their memories of their site specificity in their work. In an interview missing friend. with Dean and Moren, Moren stated that in From the bridge, the audience is led by addition to a meaningful engagement with bicycle around Minneapolis to various loca- site, they were interested in examining three tions where additional pieces of the story elements through their work: the urban unfold. Even Minneapolis natives are unfa- environment, story and narrative elements, miliar with many of the sites. Dean said and live action sequences. In contrast to the that their selection of sites was informed earlier case studies, MEC’s work explores the by an interest in “non-places” or “unpro- ways in which the virtual and the material grammed” locations. Moren added that the intersect in media interactions. Site can be Situationist dérive was an inspiration for the screen for projection that extends the the identification of these non-places.154 virtuality of a narrative cinematic experience. In a mix of technological savvy and ludic Their site responsive work explores the blur- psychogeography, Dean and Moren first ring or hybridization of the virtual and the scouted sites by scouring Google Street real, presence and simulation, live and pre-re- View. Because Street View would often corded in media performance and art.153 show buildings that were no longer stand- Second Bridge is Wider, But Not Wide ing or contain other digital mistakes, they Enough is the title of Dean and Moren’s first would explore the areas in person in order

110 Rupture of the Virtual 111 Materiality in Site Responsive Media Art to select specific sites that fit their criteria structure.156 The story concerns a kidnapping of unprogrammed non-spaces.155 and attempted ransoming. Similar to the The final sequence of The Second Bridge Second Bridge, there is a hybridization of the is one of the most memorable of the piece, virtual and the material, the simulated and the because it contains the most intriguing real. The audience witnesses the kidnapping sequences from the perspective of the as a live-action sequence at the opening of site-specific filmic and narrative effect the piece and over the course of the work that they’ve created. While in front of an becomes involved in an attempt to recover the abandoned building close to the Mississippi kidnapped person by collecting a ransom and River, the audience is shown a disorienting giving it to the kidnappers (see Figure 9). short video. The video is projected onto the Dean and Moren’s work with MEC exists building’s facade, which has two staircases. alongside other media artists who explore Because the recorded video is of the same the relationship between the virtual and the facade, when the video is projected onto real through their intentional blurring and the wall, it is misaligned with the actual hybridization. In an extended examination of staircases. This produces a sense of visual British new media performance and the art feedback, a visual disorientation that is group, Blast Theory, Benford and Giannachi accentuated in multiple physical and tem- note how the group’s performances unsettle poral dislocations that occur during the our distinctions between the virtual and real, segment. A man, who might be the missing live and the mediated, and the physical and person who dominates the storyline, picks simulated. In Blast Theory’s performance himself off of the ground in front of the piece, Can You See Me Now?, the group projection wall. Is it a homeless person create a mixed reality game that blurs online accidentally caught in the wrong place at play and real action. Groups of participants the wrong time, or an actor planted there are divided into two, one seated at computer in advance? A cameraman steps out into terminals and the other navigating the city the audience, and his footage is transmit- using handheld computers with wireless ted live into the video. This adds a layer of network connections and GPS.157 One group temporal dislocation to the piece, because tries to track down the other who moves it blurs the line between the pre-recorded avatars through a virtual model of the same and the live. town. In light of Blast Theory’s work and oth- Dean and Moren expand upon their ers like it, Benford and Giannachi argue that repertoire of techniques for site responsive artists are making use of mobile and ubiq- cinema in a second project, The Parade (2012). uitous computing in order to “span physical Whereas Second Bridge’s had a loose narrative environments and virtual worlds.”158 construction organized around a series of rec- When understood in this context, Dean ollections, The Parade, by contrast, has a clear and Moren are similarly engaged in the and conventional “Hollywood style” narrative creation of work that hybridizes space, time

112 Rupture of the Virtual 113 Materiality in Site Responsive Media Art and presence. But as a broad-stroke account of site, especially those with which viewers of some of the transformations taking place have little or no familiarity. with the use of mixed reality in performance In sum, The Parade is comparable to and art, Benford and Giannachi’s thesis Cardiff and Miller’s site responsive work in about hybridization fails to capture some of that it offers a constructed narrative about the nuance in Dean and Moren’s work. Dean site. Site, in other words, becomes the and Moren’s curation of non-places in their screen for projections. The virtual imagery work, for example, suggests a fascination blurs the distinction between invented with physical environment in site responsive and real history, actors and real action, media that goes beyond hybridization. In projection and the conditions of site. Dean choosing sites with which the audience has and Moren hybridize the material with the few associations, the environment becomes virtual through the projection of images a “blank” site/screen for their work. This onto the materiality of site. This approach allows them to reimagine site as a screen for to outdoor media art contrasts with the the projection of the virtual for the duration site-specific work discussed in Chapter 4. of the performance. This examination of alternative artistic The hybridization of site is also in evi- practices that make use of outdoor projection dence in other details, especially in their highlights the variety of site responsive work use of Hollywood-style narrative in their possible with the medium. The technologies second piece, The Parade. David Bordwell do not predispose art making towards partic- has described classical Hollywood style as ular aesthetic practices. There are, however, an “aesthetic system” that includes a number “real” stakes in art making, particularly in work of filmic and narrative conventions, including that strives to open an encounter with the storyline development, cinematographic materiality of site. The next chapter argues techniques, film length, color correction that while outdoor media projection may not standards, etc.159 As the dominant filmic con- incline art making toward particular forms of vention, the Hollywood-style has led to the site specificity, the expansion of the virtual cultivation of what Bordwell, drawing on cog- into the material is compatible with existing nitive psychology, refers to as “schemata,”160 social and capital relations. Thus, site-specific or cognitive shorthands the brain uses to artwork that strives for an encounter with filter, process and interpret sensory informa- the material is a resistant engagement with tion. These schemata facilitate ego-identifica- the medium that attempts to mount critical tion with a film’s dramatic content. Schemata opposition to these relations. aid in imaginative integration into the piece, because viewers unconsciously rely upon them when forming impressions of a film. The Parade draws on these conventions to aid in the construction of the viewers’ perception

114 Rupture of the Virtual 115 Materiality in Site Responsive Media Art 1 4 Janet Cardiff and Graffiti Research George Bures Lab’s Mobile Miller, Paradise Broadcast Unit Institute, 2001

5 2 Minneapolis Janet Cardiff and Art on Wheels’ George Bures iteration of the Miller, Paradise Mobile Broad- Institute [alter- cast Unit nate view], 2001

3 Janet Cardiff and George Bures Miller, Her Long Black Hair, 2004

116 Rupture of the Virtual 117 Materiality in Site Responsive Media Art 6 8 A screenshot Mobile of an early Experiential version of the Cinema, Second custom-written Bridge is Wider, software pack- but not Wide age, Livedraw. Enough, 2012. Image: Ben Moren

9 Mobile Experiential Cinema planning diagram, The 7 Parade, 2013. Minneapolis Image: Trevor Art on Wheels, Burks Seaworthy, 2010

118 Rupture of the Virtual 119 Materiality in Site Responsive Media Art 6 Rupture of the Virtual 2004. November 8. 2:25 P.M. Headphones are retrieved once the noise that pumped through At first the road looks like it has been passed through a mesh them stops. Mid-sentence the talk continues… But the underlying screen. Everything is two dimensional—the sky is painted on the problem is the same. The wrong questions are being asked. We are same plane as the trees, the buildings, the road. Minor adjustment not interested in recovering reality from its shroud of virtuality. We to the eye’s perception of depth allows one to pass through the need to concern ourselves with other problems; other questions screen and restore perspective. It is no different than looking need to be asked; other issues attended to. The fixation of atten- through a picture window or coming out of a daydream in the tion—the sedentarization of daily work and leisure life— has been passenger seat of a car. realized in the diffusion of the personal computer on which both A voice comes in over the headphones. Take a right into the work and entertainment are performed. All time has been monopo- trees. Nothing more until well within the boundary of the wooded lized by the forces of production that has led to the construction of area when the voice returns. The tone is now militant. Modern desires that connives us to stay in our seats. Docility and spectator- entertainment technologies have transformed life into a passive ship are hand-in-glove in the production of today’s spectator, who experience of the everyday. Today’s cineplexes are designed with sits immobilized in hypnotic attention at the cinema, in front of the the efficiency of slaughterhouses. They are thinly disguised mecha- television, and now before the computer terminal. nisms for creating heightened levels of docility during consumption. A mistake has often been made in the thought that if it were The realization of this logic is the installation of video screens possible to break the spectator’s gaze from its entrancement, it as distractions from the emergency of experience. Mounted into would be possible to liberate reified experience. This is to say that the seats of cars and airplanes, a passenger could care less if certain types of mobility would be an antidote to docility, freedom the sensation of descent is a pleasurable descent into Heathrow from fixatedness. Isn’t this the logic of the dérive—the roaming International or a crash-landing into the Atlantic. Keep walking. drift that eludes the psychological order of modern life by exploring A feeling that begins in the gut transforms into the sound of movement outside of the control of centralized authority and dom- a distant rumbling not unlike thunder. Keep walking. The sound inant economic force? Mobility has been the experience promised is continuous and starts to advance. It begins in the left ear and by entertainment technologies since their early form as panoramas. approaches from behind. It takes on the quality of an airplane It will become the ideology by which new technologies built upon flying overhead. A look overhead through the trees doesn’t reveal physical mobility, such as the one you are using now, will be popu- anything. Keep moving. The noise continues to grow louder and larized. Transporting a viewer to distant locations from the pleasure louder until, in a panic, the only solution is to rip off the head- of your own home has already been realized. So what if you’re now phones. Relief offered by the quiet is broken by a visual riot as walking in the woods. It’s just as easy for someone who is in nature disorienting as the white noise. Animal alarm, a sensation unfa- to dismiss a preternatural noise as only realistic sound effects, as it miliar to predators. A megashape intrudes on peripheral vision. is for someone who is terrified by the sudden approach of an animal The occlusion: a gray stain with a tapered cylindrical form. It’s too on the screen to proclaim, “It’s only a movie!” The terrifying truth is large to make out any detail, but it’s moving slowly overhead. As it that we would actually prefer that a foreign noise in the woods was passes, the headphones, dangling by its cord, rattle with distortion. a sound effect piped in for our entertainment… A fin shape can be made out, which now resolves into a wing. What seems to be a rudder with working ailerons comes into Above is an excerpt of a guided walk-through view? No longer taking up the entirety of the sky, it is an airplane, of my interactive installation Airplane Tree flying overhead, brushing the treetops yet impossibly passing Minaret (2003), which uses Augmented through them. Reality (AR), a type of See-Through Graphical

122 Rupture of the Virtual 123 Rupture of the Virtual Interface, as its medium. In an examination projections and sounds. When a partici- of this and another site-specific work, this pant is within 50 yards of a predetermined chapter elaborates on claims already intro- geographical spot within the installation, duced in this book, specifically that site-spe- referred to as the “clearing,” the second part cific media art can open an encounter with begins. The audio heard throughout the first the material. This chapter focuses on the part of the project restarts, but this time utilization of STGIs in site-specific works and it does not grow in intensity according to examines the technical details and aesthetic its own internal timing, but is linked to the possibilities of the creative application of the viewer’s physical proximity to the center medium in rupturing the virtual. An ongoing of the clearing. The deeper the participant discussion of mobility is central here, and moves into the clearing, the louder the sound I consider how mobile interfaces, such as grows; farther away, the sound diminishes. Augmented Reality, can facilitate critical It is as if there were a point in the clearing awareness of the virtual. eliciting a noise that grows louder as the Airplane Tree Minaret explores AR’s participant draws near, but the audio is ability to overlay virtual objects onto actually generated by the wearable computer site. Participants are given Head Mounted responding to the viewer’s GPS location, not Displays with integrated headphones by any source external to the participant’s before entering the work, which begins position. With the participant’s approach to when a participant arrives at the space of the center of the clearing, the audio grows in the installation. Audio is fed into the Head intensity until it again reaches a level causing Mounted Display’s headphones, a noise auditory distress. Once within visual range of which crescendos in pitch and intensity. The the center, the participant sees the “minaret,” audio—a dull rumbling similar to the sound a computer generated projection of a metal of an approaching jet engine—is spatialized structure, about 30 feet tall, bearing speak- to sound like it is coming from overhead, ers and reminiscent of a watchtower. but there is nothing in the sky that elicits Before moving into an analytical discus- the noise. When the sound reaches its peak sion of Airplane Tree Minaret, I would like to level, as suddenly as it started, it drops review a second installation of mine in order to a minimum. Simultaneously, a realistic to remark on what both share in terms of site computer generated airplane flies overhead, and site-specificity. Created before Airplane passing improbably through the treetops Tree Minaret, Military Bay (2003) is a visualiza- (see Figure 1). tion of the entire United States military naval The participant interacts with the first capacity set in San Francisco Bay. From pub- part of the installation until her physical licly available data, I collected an inventory of movements through the space trigger the U.S. military watercraft and gathered mea- second, which builds upon the sensory surements of each type of ship. I projected full disorientation already created by the scale, three dimensional computer models of

124 Rupture of the Virtual 125 Rupture of the Virtual these ships onto the part of the San Francisco American viewers. The existing popularity of Bay contained by the Golden Gate, Bay and computer-based flight simulators whets the Richmond Bridges—an area of approximately appetites for such images and creates desire 200 sq. miles (see Figure 2). The military ships for remotely operated entertainment. A step not only covered much of this part of the Bay, beyond the grainy-green video produced by but extended out into the Pacific Ocean past smart bombs, the high-resolution footage the Golden Gate Bridge. On bridges, piers and from these remote-controlled killing machines coastlines, viewing locations were indicated demonstrates the military’s uses for AR. The by sandwich boards with numbers printed technologies underlying tele-operated drones on them. These provided panoramic views of and AR are comparable: in both, a human the installation, though participants were free actor operates in a geographical environment to move about the Bay for alternate vantage where information and action are mediated points. through a computer display. Originally conceived in early 2003, during Military Bay was more than its political the United States’ preparations for its most critique of military power and its assessment recent intervention into the Middle East, of AR as a military-entertainment technology. Military Bay is rooted in a critique of American On a different level, the installation unsettled military force and the impending invasion participants by taking the site (the Bay), of Iraq. It reveals America’s transition into which had fixed associations for local resi- a more aggressive, interventionist phase at dents, and turning it into something foreign the turn of the twenty-first century. By pop- or unfamiliar. The project colonized a space ulating the Bay with weapons of destruction normally protected from military displays normally secreted away from sight, viewers and used for water sports, tourism and were confronted with a visual display of the commerce. The projection of virtual objects military’s monetary and material expenditure onto the Bay transformed one’s relationship and shocked by its scale. to the site. What changed, however, was not The project also referred to AR’s role in the site itself, but one’s perceptions of and the virtualization of war violence by retooling associations with it. a technology that was becoming an enter- tainment medium. Similar to the first Gulf War, the military, with spectacularized fan- Airplane Tree Minaret and Military Bay both fare, released images from the Iraq War and MATERIAL SPACE interrupt a viewer’s experience of place, but broadcast the war as evening entertainment INTRUDES: THE do so in ways that make obvious the artifice RUPTURE OF for the American public. Since the first Gulf THE VIRTUAL of their projections. Computer generated mil- War, military-entertainment technologies had itary ships mysteriously float on the surface evolved; footage from tele-operated drones, of the water unresponsive to the motion of known as Predators, was simulcast to U.S.- the waves. Meanwhile, waves impossibly based control stations and the homes of break through virtual objects. Airplanes

126 Rupture of the Virtual 127 Rupture of the Virtual fly through real treetops and boats pass by contrast, require physical mobility in order through the renderings of military water- to view the pieces. The viewer must move craft. These constitute “imperfections” in through and around the site to experience the projections. Hito Steyerl is fascinated them. As I argue next, mobility and its simula- with the imperfect or “poor” image, because tion have been central to the development of it reveals itself as a “digital uncertainty.”161 modern media, and by looking at the role that These imperfections must be understood virtual mobility has had in the disciplinary not simply as the image’s inability to live production of the modern docile subject, we up to the quality of the original, rather the can recognize how a digital economy has revealing of digital processes that have con- enforced attention and sedentariness. Site- ditioned the image in this way (through com- specific installations that encourage physical pression, data errors in digital copying, etc.). movement can open an encounter with the In Military Bay and Airplane Tree Minaret, a material through a disruption of this enforced digital uncertainty exists at the difference attention and sedentariness. between the virtual and the material. This uncertainty is not a limitation, but an oppor- tunity for site to interrupt projection. The Internet abounds in spatial metaphors As discussed in Chapter 4, STGI-based THE SPATIAL- that promise to take us on “virtual trips,” IZATION OF artwork has the potential to rupture the THE INTERNET “fly-throughs,” “global road trips,” etc. All virtual by opening an encounter with the AS A VIRTUAL this without leaving the comfort of our chair. material. This was articulated in Krzysztof FIX For example, Hyperlapse, a website released Wodiczko’s projection of images onto the in 2013, automates the process of stitching built environment that foregrounded site’s together Google’s Street View photographs materiality. A recognition of the material from to simulate cinematic fly-throughs between behind the screen of the virtual distances any two locations. These fly-throughs create viewers from an embeddedness in the virtual a strangely disembodied experience where and invites an interrogative address of the you float slightly above eye-level162 skimming virtual. Military Bay and Airplane Tree Minaret the roofs of cars and heads of pedestrians. both rupture the virtual by producing a It is the pure visual experience of speed; digital uncertainty that enables the material tourism is no longer about arriving at a to crash through the virtual. destination, but the pure sensation of flying A second way in which these projects over roads and highways.163 Though Street rupture the virtual is the manner in which View photographs are of real locations, one’s they employ mobility. Viewers can circumam- movement is in no way a physical experi- bulate Wodiczko’s projections, but his work ence, but one fabricated by the hardware is stationary. Viewers have restricted viewing and software infrastructure that comprise locations from which to view Wodiczko’s the Internet and computer graphics. One work. Military Bay and Airplane Tree Minaret, selects the destinations one wants to travel

128 Rupture of the Virtual 129 Rupture of the Virtual between on Google Maps (locations which redress the chronic problem of overaccu- exist as representations of places on a com- mulation in capitalism.166 While Harvey has puterized two-dimensional map) and travels focused on spatio-temporal fix’s geographical along a route chosen and constructed by the consequences, the expansion of capitalism Hyperlapse algorithm. While roller coasters through the Internet can be understood as produce a sensation of speed by catapult- its virtual counterpart. After a discussion ing our bodies through the air around, of the expansion of the Internet as a type of Hyperlapse accomplishes this through the spatio-temporal fix, I return to a discussion of visualization of diverse software and hard- its ideological function in the production of ware processes as spatial form. docile bodies and the role that mobility can Hyperlapse is an example of what Wendy play as a critical response to this problem. Chun and others have characterized as the The spatialization of the Internet has spatialization of the Internet, that is, the been central to the rhetoric surrounding the representation of the Internet with spatial Internet since its early popularization. In characteristics. This spatialization is remark- “The Wild, Wild Web: The Mythic American able because it has no basis in the design or West and the Electronic Frontier,” Helen structure of the Internet. In an analysis of McLure identifies the spatialization of the the communication protocols and hardware Internet in a comparison of the characteri- that structure Internet communications zation of the Internet as an e-frontier to the (such as TCP/IP), Chun underscores the rhetoric that helped to inspire the coloniza- differences between the Internet’s construc- tion of the American West. She notes that tion and actual physical space. The dangers both were conceived as open spaces, ripe for in presuming that the Internet has a spatial expansion and investment: dimension belies how fundamentally dissim- ilar they are. While we have a relative free- Elements of space and time define and link both the American dom to roam in physical space, the layers of western frontier and the global e-frontier. [Frederick] Turner’s protocol that comprise Internet communica- “moving frontier” encompassed both physical space and time and tion make information subject to microforms was distinguished by its apparent closure by 1890; in contrast, of control and surveillance.164 the e-frontier is also spatial and temporal but is usually imagined The crucial question, however, is not as new and still wide open territory.167 why the spatialization of the Internet is preserved, despite a recognition that there During the American westward expansion, is “no space in cyberspace,”165 but what prosperity and health were characterized in ideological function spatialization serves. I spatial and temporal terms. (Horace Greeley’s argue that this spatialization has contributed famous urging: “Go West, young man, go to the creation of the Internet as, what David West.”) By moving West, settlers could go Harvey has referred to, a spatio-temporal forward into the future and find happiness fix, that is, a labor and capital investment to and prosperity unattainable to them in the

130 Rupture of the Virtual 131 Rupture of the Virtual corrupted East. A similar frontier mentality is the spatial fix (together, the spatio-temporal present in the hype surrounding the Internet fix). Harvey writes: in works including Howard Rheingold’s The Virtual Community: Homesteading on the A certain portion of the total capital is literally fixed in and on the Electronic Frontier, Bruce Sterling’s The Hacker land in some physical form for a relatively long period of time Crackdown: Law and Disorder on the Electronic (depending on its economic and physical lifetime). Some social Frontier, Katie Hafner and John Markoff’s, expenditures (such as public education or a healthcare system) Cyberpunk, Outlaws and Hackers on the also become territorialized and rendered geographically immobile Computer Frontier, and John Perry Barlow’s through state commitments. The spatio-temporal “fix”, on the other “Declaration of Independence of Cyberspace.” hand, is a metaphor for a particular kind of solution to capitalist McLure’s geographical and spatial crises through temporal deferral and geographical expansion.169 analysis, however, stops short of offering an examination of why a spatial metaphor Temporal deferral consists of large-scale governed the rhetoric around the Internet, investments in infrastructure and projects despite the fact that there is no space whose value take long periods to realize, in cyberspace. David Harvey’s work on if ever. (This can include roads and high- the spatio-temporal fix offers us a way to ways, large-scale energy projects, social conceptualize it. Following Marx, Harvey infrastructures of education, healthcare, observes that capitalism is prone to peri- social services, as well as investments in odic crises due to inherent contradictions state administration, law enforcement and in the capitalist economy. There are a military protection.170) Temporal deferral number these, but the contradictions that displaces the problem of excess surplus relate directly to the spatio-temporal fix capital to the future by dumping it into concern the role of surplus of capital and infrastructure. The spatial fix, on the other labor power. Capitalists continually seek to hand, operates in conjunction with temporal increase their access to capital through a displacement, for temporal displacement mix of tactics, such as technological inno- can occur by “scale enlargement,” that is, by vation, education, the expansion of access expanding into new regions and spaces for to a cheap labor force, etc. An outcome capital expenditures. of this drive for expansion is chronic sur- Harvey’s spatio-temporal fix is built upon pluses of capital and labor. Harvey refers to Marx’s insight that “the need of a constantly this situation as overaccumulation, or “the expanding market for its products chases condition in which too much capital is pro- the bourgeoisie over the whole surface duced relative to the opportunities to find of the globe. It must nestle everywhere, profitable employment for that capital.”168 settle everywhere, establish connections Harvey has identified two historical everywhere.”171 It is a mechanism for the responses to the problem of overaccumula- creation of the necessary infrastructure tion: temporal deferral or displacement and for regions to be integrated into a global

132 Rupture of the Virtual 133 Rupture of the Virtual capitalist economy. China’s integration into These amounts are meager in comparison global capitalism is the most notable spa- to private capital investments in Internet tio-temporal fix of the last few decades. With infrastructure, hardware and software, investments in infrastructure (roads, dams, including fiber optic lines, cellular phone universities, etc.) in the hundreds of billions service networks, data centers, software, of U.S. dollars, China has “the potential to etc. There is no reliable estimate for the total absorb surpluses of capital for several years amount of funding put into the Internet by to come.”172 With this investment in infra- private companies but, given the number and structure, the spatio-temporal fix conjures wealth of businesses involved in provision- into being new cities while redeveloping old ing Internet service, this number would be ones as well. This is why Harvey refers to enormous, far eclipsing government funding. an outcome of the spatio-temporal fix as the Google’s capital investments on servers, data capitalist “production of space.”173 centers and other infrastructural improve- A spatio-temporal fix is also in evidence ments alone exceed $1 billion per quarter177 in governmental and corporate financing of and Google is just one company among many Internet hardware and software technolo- that drives money into Internet technologies. gies, which has been well-documented. The From an economic perspective, the U.S. government provided funding to the spatialization of the Internet encourages its Advanced Research Projects Agency (ARPA, utilization as a spatio-temporal fix for surplus later DARPA) for the invention of the com- capital and labor. Capital expenditures in munication protocols integral to the Internet. and around the Internet have been long-term Estimates put the cost on the low end of investments in the reinvention of capitalism. around $25 million from 1969 - 1990.174 Vint The Internet has become a digital platform for Cerf, ARPA’s program manager on Internet the unprecedented mobility and a playground related projects during that period, puts the and factory for the harnessing of novel forms figure closer to $50 - $70 million.175 As should of labor power and the creation of capital.178 be expected given Harvey’s claim that the spatio-temporal fix is a response to financial crisis, the U.S. government continues to fund Even before the advent of the Internet, the the Internet directly and indirectly at a level DOCILE BODIES media had been rendered in spatial terms. much higher in times of crisis and recession. AND VIRTUAL Oliver Grau and Anne Friedberg179 both MOBILITY In 2009, as part of a major stimulus pack- read the history of modern media as the age intended to redress a global economic construction of virtual spaces of illusion. decline, the federal government allocated Friedberg traces the origin of such virtual $7 billion to high-speed Internet communi- spaces to an unlikely location, the nine- cations for the installation of broadband teenth-century French shopping arcade, access for educational programs and under- which, for Friedberg, was a precursor to served neighborhoods.176 modern media technologies. She compares

134 Rupture of the Virtual 135 Rupture of the Virtual the cinema-goer to the figure of the flâneur, here and now, but paradoxically chains him whose aesthetic pleasure moving through to his seat. This insight is precisely what is halls of the arcade, is akin to a cinemat- at stake in Crary’s research on the history of ic-type experience. As the flâneur wanders attention. He writes: through the arcade’s displays and specta- cles, so too do media technologies (such as Many of the modes of fixation, of sedentarization, of enforced the panoramas, cinéoramas and, eventually, attentiveness implicit in the diffusion of the personal computer the cinema) promise viewers transport or have achieved some of its disciplinary goals, in the production of mobility through virtual spaces.180 what Foucault calls docile bodies. The proliferation of electronic What distinguishes modern media tech- and communication products ensures that docility will always be nologies from the spectacle of the arcade, linked to intensified patterns of consumption.182 however, is that actual physical movement is no longer constitutive of the experience Extending Crary’s observations about docile of mobility. Whereas the flâneur ambled bodies and patterns of consumption to the through the halls of the arcade, today we research on digital labor, we can argue that sit in fixed attention to the screen and virtual mobility is not only integral to pat- experience virtual spaces in the comfort of terns of consumption (of media such as video our chairs. Though mobility is integral as games, films, television, etc.), but to work an aesthetic dimension to modern media in the digital economy as well. During the technologies, such as cinema, it has become, industrial era, Frederick Taylor scrutinized concludes Friedberg, a “virtual mobility,” the movements of assembly line workers insofar as modern media entrain the viewer to extract greater productivity.183 Today, to the screen and do not necessitate actual management analyzes the attentiveness and physical movement. movements of information workers with pre- Jonathan Crary’s study of the disci- cision in order to extract greater efficiencies plinary mechanisms that have led to the in screen time, typing speed, and time-at-key- construction of the modern media spectator board. Crary’s account of disciplinary atten- complements Friedberg’s account of vir- tion is hand-in-hand with Friedberg’s account tual mobility.181 In fact, Friedberg’s notion of mobility: the Internet’s spatialization of virtual mobility would be strikingly contributes to the production of sedentary incomplete without Crary’s insistence that workers in a screen-based economy. Any dis- an individual’s capacity to stare in fixed comfiture with the prolonged docility of mod- attention to a screen has had a contingent ern life is sublimated in the dream of virtual relationship to the development of apparati mobility, an ideological fantasy of freedom of discipline and control in the construction from fixation and enforced sedentariness. of the modern spectator. This is to say that Repression and sublimation, however, the experience of virtual mobility simultane- are not the only responses to media cul- ously frees the viewer’s imagination from the ture. Art movements have long embraced

136 Rupture of the Virtual 137 Rupture of the Virtual movement and mobility as forms of critical way of seeing the street.”186 By compelling resistance to the disciplinary chains of a participants to break out of habitual move- media-based economy. The next section ments through a city, the dérive “alerts reviews the preoccupation with physical people to their imprisonment by routine.”187 movement in avant-garde art as a critical Evidence that we have become sites of artistic practice for resisting docility and discipline and control, according to Debord spectatorship. This will enable us to return and the Situationists, is in our bodily nav- to a consideration of the role of mobility in igation through a city. The route we take site-specific media art as the critical experi- from point A to point B has been chosen ence of space. for us by a multitude of disciplinary forces, such as urban planners, who have designed routes according to the greatest amount of The Situationists were a European radical surveillance and control; by capitalist pres- PHYSICAL artistic and intellectual collective that sures for the maximization of efficiency; by MOVEMENT AS A existed in various forms from 1957 to 1972. our attraction to spectacles, such as the STRATEGY OF DÉTOURNEMENT Producing a body of critical writings and pretty landmark by which we like to pass artistic interventions, they sought to pro- as a morning ritual. What an individual on vide a “radical critique of modernist art a dérive becomes aware of in his rootless practice, a politics of everyday life, and an wanderings is how behaviors are unknow- analysis of contemporary capitalism.”184 ingly governed by the convergence of these The dérive, often translated as the drift, disciplinary mechanisms. Social and capital was a key Situationist aesthetic tactic to relations are inscribed and articulated in upend mid-twentieth century European our very movements through a city, and bourgeois society. It consisted of a wander- they get brought to the surface through ing through a city on foot, “a drifting with- participation in the dérive. More than just out destination, neither going to work nor a tactic for recognizing the existence of properly consuming, a waste of time in the disciplinary mechanisms, the dérive was a temporary economy.”185 Lasting anywhere revolutionary practice for breaking free of between one hour to months at a time, such forces of control. while on a dérive, the drifter could happen Military Bay and Airplane-Tree-Minaret also upon obscure locations, places of ambiance, employ mobility as a basis for their effect. forgotten landmarks, and encounter unfa- Whereas the dérive encouraged participants miliar people and places. to take over the city as a way of overturning There was a critical method informing disciplinary mechanisms, these installations the dérive in spite of its apparent aimless- utilize mobility in order to orient the viewer ness. In his “Theory of the Dérive,” Guy to structures that have come to dominate Debord confidently defined it as “playful our everyday interactions. In contrast to constructive behavior,” which “changes our the disciplinary goals of media technologies

138 Rupture of the Virtual 139 Rupture of the Virtual to create docile bodies, STGI installations art opens an encounter with site and its encourage viewers to wander the space of the materiality in multiple ways. It draws atten- installation and linger in contemplation of the tion to the materiality of site through the relation between the project and site, the site formal differentiation of the virtual (as and their physical location. When viewers media image) and the material (as surface of of the Military Bay installation, for example, projections). This differentiation is a digi- travel to locations around the San Francisco tal uncertainty when the material crashes Bay—frequented predominantly by tourists through the imperfection of the projections. seeking panoramic views of the city—they do By extension, this uncertainty produces an so not because of the attractions of spectacle, awareness of the material as the screen for but the re-envisioning of the space as a site projections. A second way that STGI-based documenting military power. The space of the media art opens an encounter with the installation is appropriated by the project and materiality of site is by challenging the dis- re-configured in the minds of viewers: the Bay ciplinary forces that proscribe one’s interac- is no longer the site of leisure activities, com- tions with site. When employed as a critical merce or spectacle, but the space of possibility, artistic practice, mobility is an effective loosened from the disciplinary mechanisms response to the disciplinary forces required that construct the city for its inhabitants. Is by a digital economy. Against the reproduc- this not a realization of Debord’s rationale for tion, normalization and expansion of forms the dérive as a strategy of détournement? For of fixed attention and sedentariness in labor participants to “[drop] their usual motives for and leisure time, media art literally compels movement and action”?188 viewers to get out of their seats and attend No longer fixed in rapt attention to the to the presence of site beyond the screen. screen, people move through and around In its capacity to open an encounter with their surroundings to encounter space as a the material, site-specific media art enables site of possibility and meaning. The installa- a critical recognition of the virtual as a tions employ physical mobility as a way to cultural condition. Because STGIs construct enable viewers to envision material space experiences situated at the intersection of the as centers of “possibilities and meaning” material and the virtual, it is a medium that is not dominated by ingrained psychologi- well suited to the formation of such a position. cal patterns and fixed social and capital Our interactions with site are conditioned by relations, which cement the experience of virtual processes, and this habitus requires a particular locality.189 In this way Military disruption before such a position can be won. Bay and Airplane-Tree-Minaret loosen the Site-specific media art can provide such a dis- disciplinary goals of modern spectatorship ruption, because it can enable a recognition and open alternate ways of experiencing of the presence of the material. An awareness one’s surroundings. of the material in the experience of site is In sum, site-specific STGI-based media necessary for the articulation of a critical

140 Rupture of the Virtual 141 Rupture of the Virtual position regarding the virtual. Together, 1 John Kim, Site- this recognition and critique constitute a specific Augment- rupture of the virtual. The next chapter, the ed Reality Instal- Conclusion to this book, extends these claims lation, Airplane Tree Minaret, 2003 by exploring how varied cultural practices outside the realm of art can also produce ruptures of the virtual.

2 John Kim, Site- specific Aug- mented Reality Installation, Mili- tary Bay, 2003

142 Rupture of the Virtual 143 Rupture of the Virtual 7 Conclusion In the last decade there has been a marked book, pose an alternative model for media proliferation of art and cultural events that projection, one that takes seriously a are organized around media projection and desire to engage viewers in and about their outdoor screening. In outdoor film events, surroundings. In the face of proliferating digital billboards, guerilla video bombing, ways to extend and enhance virtual expe- interactive projection, video walls, etc., rience, we must attend to practices that projections have moved from darkened draw attention to the material, especially theaters to public squares and parks, back- as a critical address of the virtual in our yards, the sides of buildings, and a myriad lives. Neither retrograde nor nostalgic, such of other outdoor spaces. Nuit blanche art media practices can provide us with an festivals, such as Minneapolis’s Northern awareness of our cultural condition and Spark, Toronto and Paris’s Nuit Blanche, connect us to the social meaning of site and Santa Monica’s Glow and others, have its materiality. become major venues for outdoor projec- In this book I have developed a concep- tions and public media installation art. This tual and critical framework with which to expansion has spawned research initiatives, recognize and interpret media practices frequently referred to as “urban screens,” that strive to open such connections. The and the growth of outdoor projections is material’s exclusion from media research is heralded by advocates and researchers for reinforced by the proliferation of types and their capacity to build “community, local sizes screens, which offer up more immer- identity and engagement.”190 sive virtual spaces. The material’s exclusion Against the perceived benefits advocated has been fundamental to research on the by their supporters, this book has suggested virtualizing forces of contemporary culture. that it is difficult to speak of a singularly This book is obviously indebted to this area positive function for outdoor projections, of research, but I protest against the focus especially one that could broadly result in on technologies of the virtual as prosthetic the creation of community, localism or, even, extensions of vision. I have argued that identity. Indeed, a common use of site in the media have also had a historical and outdoor projection is simply to add ambiance ongoing role in constructing a relationship or mood to events, and in this sense, rather with those things that are in front of our than enhancing localism or community, the noses, that is, our immediate physical sur- media can further distance us from our built roundings. The utility of this concept has environment by projecting the virtual out been documented through the study of the into it. Site recedes with the dimming of lights, historical development of media and com- regardless if the screen is indoors or out. puting perceptual devices that help cognize Work that strives to reference and our immediate surroundings. interrogate the material, like the examples This book has mainly drawn on examples of site-specific media art reviewed in this from media art in making its claims about

146 Rupture of the Virtual 147 Conclusion the material. Beyond these particular exam- that determines their particular function, ples, it is possible to recognize a preoccupa- just the accretion of historical precedents tion with the material in cultural practices that constrain the imaginative uses that outside of the realm of art. In order to make we conceive for them. As Cardiff and Miller such a claim, I want to first recall arguments reveal in their work, we can claim alter- made earlier regarding the material. In native uses for media devices: instead of Chapter 5, I noted Janet Cardiff and George privatization and separation, the Walkman Bures Miller’s appropriation of the Walkman can be employed for empathy, memory and to encourage “a sense of connection to place.” situatedness. Cardiff and Miller’s use of the This need for connection to place can be Walkman and comparable technologies is a read as symptomatic of a range of problems resistant engagement with the medium; they in contemporary life that contribute to a strive to employ such devices in the reverse sense of separation from an intimacy with of how they are frequently employed. our surroundings. Dan Schiller, for example, Countervailing pressures are similarly has written about the Walkman as an exam- poised against the class of technologies I ple of our growing separation and disconnec- have referred to as See-Through Graphical tion from others and our environment, what Interfaces. As discussed in earlier chapters, he refers to, drawing on Raymond Williams, there is commercial and military pressure to as “mobile privatization.”191 Personal media utilize the interface for particular ends, such technologies contribute to a technologically as the exertion of knowledge and control mediated sensory bubble that shields us over the material and the expansion of the from contemporary life by keeping it at an virtual into our experience of the material. imagined arm’s length. From personal enter- Google Glass is the most visible example of tainment devices, like the Walkman, iPod and this tendency in recent years. As suggested smartphones, to the rebooting interest in above, however, resistant engagements with Virtual Reality, we live in the midst of peo- the media are possible despite the pressures ple, places, and histories, but feel a sense of of capital and mass entertainment. Site- disconnection from them. specific media artists have articulated prac- Manuel Castells,192 Sherry Turkle,193 and tices that employ See-Through Graphical others have maintained that this sense of type technologies for purposes contrary to disconnection and dislocation is constitutive how they are being managed by businesses. of life today, because lived experience is My own work with the medium has been everywhere striated by capital and media inspired by the envisioning of alternative flows. While media technologies like the possibilities for STGIs in opposition to grow- Walkman, the computer screen, digital ing normative pressures that constrain their media, and others, are influential in the pro- imaginative possibilities. duction of life’s dislocations, there is nothing inherent to the logic of their construction

148 Rupture of the Virtual 149 Conclusion CULTURAL The latter half of this book’s investigation of knowledge about the conditions in which PRACTICES AS ENCOUNTERS media practices that open encounters with industrial food products are grown and WITH THE the material focused on art to clarify a set made. (Among many issues of concern, this MATERIAL of tendencies concerning the material and includes the use of pesticides, herbicides efforts to bring it to critical consciousness. and fertilizers in industrial farming prac- I focused on three distinct ways in which tices.) The experience of packaged foods in media art have a role in producing knowl- supermarkets is itself an idealized represen- edge of the material. First, in a recognition tation of the food items themselves. They of the sociocultural conditions of site, or are virtualized products; their packaging an understanding of the social or cultural masks their organic origin. Industrial food conditions of a particular location; second, products are promoted, furthermore, in in an awareness of the disciplinary function ways that embed them into spectacularized of site, or site’s disciplinary role in the for- channels of advertising and consumer cap- mation of individual subjecthood; and finally, italism, separating their food content from as an encounter with the materiality of site the conditions in which they are grown. through a recognition of the formal distinc- The argument about the material tion between the virtual and the material. I offered in this book offers a distinctive suggested that such cultural practices are way to conceive of urban farming. The realized in a critical recognition of the virtual movement can be understood as a reac- as a cultural condition. I referred to such tion against the forms of mediation in practices as enabling a rupture of the virtual. contemporary life that extend down to This book’s almost exclusive focus the purchasing and consumption of food on art does not presume to suggest that products. Taking control over the means it is the only cultural practice that opens of production re-materializes food in such encounters with the material. It is possible a way that the social and environmen- to identify a wide range of practices that do tal conditions in which they are grown this as well. To briefly consider one exam- can be revealed. (In many urban plots, ple here, I am interested in how the growing one must test the soil. The discovery of urban farming movement (pursued both as harmful contaminants, such as lead or a collective and an individual activity) is an arsenic, uncover the industrial history of instance of a cultural practice outside of the the site. Testing reveals the social history domain of art that can also be understood of the land.) Farming is also a material as opening an encounter with the material. practice in its tactility. Hands are rubbed Though it is impossible to generalize across raw from pulling weeds and using farm the diverse motivations in peoples’ desire tools; fingernails get caked with dirt, to cultivate, a sentiment one can find in and fingertips feel the dampness of the the movement is that growing one’s own ground. The formal qualities of the mate- food is a form of protest against the lack of riality of the earth are discovered in the

150 Rupture of the Virtual 151 Conclusion practice of farming. This is a materiality that contrasts to the virtualized experi- ence of shopping in supermarkets. A wider Notes net needs to be cast to discover cultural practices that open encounters with the material in ways that differ from art’s aes- thetico-critical tactics. Finally, this book traced the origin of the See-Through Graphical Interface to World War II military technologies. Cinema did not simply have its origins in the turn of the nineteenth century with the invention of devices for play- back, recording and film processing, but in longer historical transformations that estab- lished changes in visual organization and perception. It is these changes that cleared the way for film’s development. Similarly, this book has implications for the history of See-Through Graphical type technologies that originates even before the first half of the twentieth century. In its re-presentation of a viewer’s physical surroundings, the camera obscura, for example, has frequently been associated with the rise of technologies of the virtual. The device, however, clearly connects the viewer relationship to the material as well. In other words, what was the camera obscu- ra’s role in situating the viewer in relationship to her immediate physical surroundings? A comparable question can be asked of early portraiture and landscape paintings. What was the function of landscape paintings that would hang for decades in the environment they depicted? Questions, such as these, remain to be answered in future research on the subject.

152 Rupture of the Virtual 1 Caroline Jones, Sensorium (2006), (1971), 24. the virtual’s influence over ideas 43-44. 5. 20 Friedberg, Virtual Window, 11. and forms found in our physical 59 Michelle Tan, “Pioneering Stryker 2 Marshall McLuhan, Understanding 21 See Jason Farman, Mobile Interface environment. Unit Preps for Afghanistan” (2009). Media (1964). Theory: Embodied Space and 35 See Chapter 4 for a more detailed 60 Commerce Business Daily (1993). 3 See Rob Shields’s (2003) expan- Locative Media (2012); Steve discussion of Farman’s analysis of 61 Chris Gray, Postmodern War: The sive definitions for the virtual. Benford and Gabriella Giannachi, Augmented Reality. New Politics of Conflict(1997). 4 Anne Friedberg, Virtual Window Performing Mixed Reality (2011); 36 Farman, Mobile Interface Theory, 62 Ibid, 196. (2006), 11. and Gabriella Giannachi and 43-44. 63 As Lenoir (2003) has observed, 5 Anne Friedberg (2006) refers to Nick Kaye, Performing Presence: 37 Mark Hansen, Bodies in Code military research and devel- the Graphical User Interface as Between the Live and the (2006), 142. opment of new technologies is a virtual window onto spaces Simulated (2011). 38 Katherine Hayles, Electronic frequently a joint military-educa- of illusion that can have little 22 Jacques Derrida, Of Grammatology Literature (2010), 48. tional-industrial venture. This is corollary to spaces that exist in (1976). 39 Katherine Hayles, How We Became true for the Land Warrior program. the physical world. 23 As Shields, Lister, and others have Posthuman (1999), 193. The various institutions that have 6 The lexical opposition is really suggested, there are a multitude 40 Ibid, 208. received governmental funding between mediated and its of definitions for the virtual 41 Anna Munster, Materializing New through the program include negation (immediate), but as is across different areas of research. Media (2006). Columbia University, University of discussed next, the immediate The intention here is not to 42 Hansen, Bodies in Code, 5-6. Washington, Microvision, General and medium are frequently provide a review of all of them. 43 Donna Haraway, Simians, Cyborgs, Electric, General Dynamics, conceived of as opposing twins. 24 Friedberg, Virtual Window, 7. and Women (2001), 149. Defense Advanced Research Given the media’s relevance to 25 As is reviewed later, this did not 44 Hansen, Bodies in Code, 59. Projects Agency, and others. this distinction, it is appropriate deter Friedberg from submitting 45 Ibid, 123. 64 Though the analysis offered to consider this one, which I her own definition for the term as 46 Ibid, 5. here focuses on the combat foot argue next is also conceived of in the “liminally immaterial.” 47 Judith Butler, Ernesto Laclau soldier, it is worth mentioning opposition. 26 Jacques Derrida, Specters of Marx and Slavoj Žižek, Contingency, that comparable research and 7 Raymond Williams, Marxism and (1994). Hegemony, Universality (2000), 108. development programs have Literature (1992), 158. 27 Ibid, xviii. 48 Judith Butler, Gender Trouble been aimed at upgrading visual 8 Ibid, 159. 28 Ibid, 13. (2006). computer interfaces across 9 See “immediate” in the Oxford 29 Ibid, 54. 49 Sarah Whatmore, “Materialist military vehicles and hardware. English Dictionary (2014). 30 Mark Poster, What’s the Matter Returns” (2006), 600-609. Head-Mounted interface devices 10 Diderot, “Salon of 1763” in Eitner, with the Internet (2001). 50 Arthur Schopenhauer, The World are already found in aircraft and Neoclassicism and Romanticism 31 Jean Baudrillard, Perfect Crime as Will and Representation (2010), land vehicles, some of which I (1750-1850), 58. (2008), 35. 50. discuss below. For an account of 11 Jacques Derrida, Dissemination 32 See Anne Friedberg, Window 51 Paul Milgram and Fumio Kishino, the military’s recent involvement (1976). Shopping: Cinema and the “Taxonomy of Mixed Reality Visual in developing the Head-Mounted 12 Ibid, 127. Postmodern (1994); and Oliver Displays” (1994), 1321. Display, see Grau (2004: 163) and 13 Jay Bolter and Richard Grusin, Grau, Virtual Art: From Illusion to 52 Ronald Azuma, “A Survey of the US Army’s conceptual frame- Remediation (1999). Immersion (2004). Augmented Reality” (1997). work doctrine entitled, “AirLand 14 Ibid, 47. 33 Jay Bolter, et al., “New Media and 53 Steve Mann, “Wearable Battle 2000.” 15 See Jean Baudrillard (2008) and the Permanent Crisis of Aura” Computing as Means for Personal 65 General Dynamics (2010). Mark Hansen (2006). (2006), 22. Empowerment” (1998). 66 Virilio, War and Cinema, 88. 16 Paul Virilio, War and Cinema 34 Friedberg (2006) does consider 54 Nathan Jurgenson, “Digital 67 See Livingston et al. (2003); (1989), 88. the influence of the virtual over Dualism versus Augmented Defense Advanced Research 17 Jason Farman, Mobile Interface our physical environment in her Reality” (2011). Projects Agency (2008); Cameron Theory (2012). Gabriella Giannachi examination of how architectural 55 Ronald Azuma, et al. “Recent (2010). and Nick Kaye, Performing details came to mimic forms Advances in Augmented Reality” 68 Livingston et al., “An augmented Presence (2011). found in cinema, such as the use (2001). reality system for military opera- 18 One example of such a use is the of horizontal windows in buildings 56 Manovich, The Language of New tions in urban terrain” (2003). company Occiptal (http://occipi- that referenced cinema’s wide Media. 69 For studies of the military’s role tal.com/) and the development of format projections. But her 57 Farman, Mobile Interface Theory, in developing early computing AR technology for tablets. analysis is less an account of 39. technologies, see David Mindell’s 19 Stanley Cavell, The World Viewed the material, than an example of 58 Farman, Mobile Interface Theory, Between Human and Machine

154 Rupture of the Virtual 155 Notes (2004) and Paul Edwards’ The Bead: Part 3” (1983); and Price, 89 Ibid. ‘openness’ and unrestricted phys- Closed World (1997). Late Marque Spitfire Aces: 1942-45 90 Wing Commander (name ical access—parks, university 70 Caudell and Mizell (1992) observe (1998). unknown), “Letter from the Wing campuses, civic centers, entrance this basic similarity in their work 79 A comprehensive history of the Commander of the Air Fighting areas to federal buildings, plazas on early Augmented Reality Gyro Gunsight, one that accounts Development Unit” (1943). off city streets, parking lots, devices, yet they do not recognize for its significance in the develop- 91 Instrument & Photography airports” (Kwon 2004: 60). A the historical and theoretical ment of interface technologies for Department, “Bullet proof screen definition of public art I share in significance of this connection. early computing devices, has yet as gunsight reflector” (1943). this chapter. 71 Daniel Weintraub and Michael to be written. R. Wallace Clarke’s 92 Krzysztof Wodiczko and Duncan 104 In the next section I consider Ensing’s anecdotal history of the impressively expansive British McCorquodale, Krzysztof Richard Serra’s Tilted Arc, a HUD is noteworthy for its astute Aircraft Armament is an invalu- Wodiczko (2011), 51. site-specific work of public art acknowledgement of J.M. Naish as able resource for researching the 93 Ibid. made famous by the controversy a “prime mover in the conception historical origin of Gyro Gunsight 94 Ibid, 146. surrounding attempts to relocate and development of head-up in the context of military gunnery 95 Ibid, 148. it, despite the fact that it was a displays” (1992: i). As I document and sighting devices, but his 96 Sigmund Freud, The Standard site-specific work created for the in this chapter, Weintraub and account is unreliable in places. Edition of the Complete location in which it was originally Ensing, however, incorrectly It is insufficiently documented, Psychological Works of Sigmund installed. identify Lt. Col. Paul M. Fitts and some of its historical details Freud (1955), 247. 105 Rosalyn Deutsche, “Architecture as responsible for the earliest and claims I have found to be 97 Wodiczko, Krzysztof Wodiczko, of the Evicted” (1990), 31. reference to the ‘head-up display inaccurate. I have avoided citing 149. 106 Wodiczko and McCorquodale, concept’ (1992: 3) in 1946. I argue passages from his texts, but used 98 Ibid, 172. Krzysztof Wodiczko, 149. here that the Head-up Display has a figure from his book. 99 Sigmund Freud, The Standard 107 Ibid. its origin in the design of the Gyro 80 Royal Aircraft Establishment at Edition of the Complete 108 Deutsche, “Architecture of the Gunsight in 1942. Farnborough, “Gyro gunsight Psychological Works of Sigmund Evicted.” 72 JM Naish, “A study of O.R. 3044 for adoption by U.S.A.: development, Freud, 226. 109 Neil Smith, “Contours of a a navigation, bombing reconnais- and firing trials in America” (1941). 100 I develop this claim in greater Spatialized Politics: Homeless sance and flight control system 81 Science News Letter, “Gunsight for detail later, when I quote Anne Vehicles and the Production of for strike reconnaissance aircraft” Planes” (1944). Friedberg (2006), who makes this Geographical Scale” (1992). (1958), 11. Quotation marks in the 82 Trafford Leigh-Mallory, “Pilot’s point regarding screen based 110 Wodiczko and McCorquodale, original. gyro gunsight Mark II” (1943). media projections. In addition, Krzysztof Wodiczko, 104. 73 Ibid, 36-37. 83 Timothy Moy, “ War Machines: Derrida (1994) also draws a con- 111 Ibid, 104. 74 Ibid, 36. Transforming Technologies in the nection between the virtual and 112 Kwon, One Place After Another, 60. 75 JM. Naish, “The head-up steering U.S. Military, 1920-1940” (2001), 50. ghosts in his examination of the 113 Ibid, 74. display for the T.S.R. II aircraft” 84 Royal Aircraft Establishment at specters of contemporary politics 114 Ibid, 57. (1959). Farnborough, “Gyro gunsight and culture in his Specters of 115 Ibid, 74. 76 In a research paper on the design adoption by U.S.A.: development, Marx. For the sake of brevity, I 116 Jeffrey Skoller, Shadows, Specters, of a “Head-up Steering Display,” and firing trials in America” (1941). refer to screen based projections Shards: Making History in Avant- Naish refers to gunsighting 85 R. Wallace Clarke, “Drawing a as immaterial, instead of liminally Garde Film (2005). technologies: ‘It would then be Bead: Part 3” (1983); Price, Late immaterial. 117 Kwon, One Place After Another, possible to make use of a known Marque Spitfire Aces: 1942-45 101 Friedberg, Virtual Window. 65-66. technique of flight, viz. gun-aim- (1998). 102 Miwon Kwon, One Place After 118 Krzysztof Wodiczko, Critical ing, in which both visual and 86 R. Wallace Clarke, “Drawing a Another: Site-Specific Art and Vehicles: Writings, Projects, instrument information, i.e., target Bead: Part 3,” 202. Locational Identity (2004). Interviews (1999), 47. and market, are used concurrently, 87 WS Farren, “Letter to the Ministry 103 Drawing on the research of John 119 Krzysztof Wodiczko, “Projections” but with the intention of replacing of Aircraft Production” (1941). Beardsley, Donald Thalacker, and (1990), 284. the target by the visual environ- 88 These specific comments were in Harriet Senie, Kwon’s definition 120 Ibid, 284. ment of the aircraft’ (1959: 4). reference to a non-gyro reflector for public art traces its origin 121 Louis Althusser, For Marx (2005). 77 Archibald Sinclair, “Letter from gunsight, but it was held that all to the 1960’s modern public art 122 Hal Foster, Discussions in Archibald Sinclair, Air Ministry, to fighters which employed reflector movement in the United States, Contemporary Culture (1987), 42. J.J. Llewellin, Ministry of Aircraft gunsights could benefit from this in which art was sited “outdoors 123 Wodiczko and McCorquodale, Production” (1942). modification (Armament Division, or in locations deemed to be Krzysztof Wodiczko, 148. 78 R. Wallace Clarke, “ Drawing a 1942). public primarily because of their 124 Wodiczko, Critical Vehicles, 47.

156 Rupture of the Virtual 157 Notes 125 Farman, Mobile Interface Theory, Janet Cardiff & George Bures Second Bridge I had never been to First Quarter 2013 Results” (2013). 45. Miller: Works from the Goetz most of the sites before. 178 See the characterization of the 126 Giannachi and Kaye, Performing Collection (2012), 27. 156 Interview with Daniel Dean and internet as “social factory,” in Presence. 140 Janet Cardiff, Miller, and Christov- Ben Moren. Schultz’s “Digital Labor” and 127 Farman, Mobile Interface Theory. Bakargiev, Janet Cardiff: A Survey 157 Ibid, 27. Terranova ADD. 128 Because of a growing interest of Works Including Collaborations 158 Benford and Giannachi, 179 See Anne Friedberg, Window in Augmented Reality and the with George Bures Miller (2001), 33. Performing Mixed Reality, 23. Shopping and Oliver Grau, Virtual popularity of Streetmuseum, 141 Ibid, 158. 159 David Bordwell, The Classical Art. several other apps for viewing 142 Jonathan Crary, Suspensions of Hollywood Cinema (1985), 4. 180 Friedberg, Window Shopping, 143. photographs and images relating Perception (1999). 160 Ibid, 7. 181 Crary, Suspensions of Perception. to location have become available. 143 I elaborate in more detail on this 161 Hito Steyerl, The Wretched of the 182 Ibid, 37. One that has been garnering idea in Chapter 6. Screen (2012), 32. 183 Marc Andrejevic, iSpy (2007). considerable attention is Local 144 Crary, Suspensions of Perception, 162 At 8.2 feet to be precise - 184 Jonathan Crary, “Spectacle, Projects’ Explore 9/11 app, which 36-38. http://en.wikipedia.org/wiki/ Attention, Counter-Memory” enables users to contribute 145 See, for example, the award-win- Google_Street_View (1989), 97. photos on and around September ning Gettysburg Driving Tour 163 See Virilio, Speed and Politics 185 Simon Sadler, The Situationist City 11 of the area surrounding the (http://www.gettysburgdriving- (2006). (1998), 93. former World Trade Center. tour.com), which takes listeners 164 Wendy Chun, Control and Freedom 186 Guy Debord, “Theory of the 129 Another artist’s work which would on a driving tour of historical sites (2006). Dérive.” illustrate the connection between around Gettysburg, Pennsylvania. 165 Manovich quoted in Chun. Ibid, 39. 187 Sadler, The Situationist City, 93. the two is Shimon Attie, whose 146 Janet Cardiff and Miller, Her Long 166 David Harvey, Spaces of Capital 188 Debord, “Theory of the Dérive.” work is even more closely akin to Black Hair. (2001). 189 Ibid. Streetmuseum. In his The Writing 147 Janet Cardiff, Miller, and Christov- 167 Helen McLure, “The Wild, Wild 190 Urban Screens, “Urban Screens.” on the Wall (1991-1993) project, Bakargiev, Janet Cardiff: A Survey Web,” 475. 191 Daniel Schiller, How to Think about Attie projects pre-World War II of Works Including Collaborations 168 Harvey, Spaces of Capital, 80. Information (2007). photographs of Jewish inhabi- with George Bures Miller. 169 Giovanni Arrighi, “Hegemony 192 Manuel Castells, The Rise of the tants of Berlin’s former Jewish 148 Atom Egoyan and Janet Cardiff, Unravelling,” 53. Network Society (1996). quarter, the Scheunenviertel “Janet Cardiff” (2002), 65. 170 Harvey, Spaces of Capital, 319. 193 Sherry Turkle, Life on the Screen neighborhood, onto the location 149 Ibid, 65. 171 Karl Marx, Communist Manifesto (1995). in which the photograph was 150 I consider at greater length out- (1998), 39. originally taken. door screens in the Conclusion. 172 Harvey, New Imperialism, 123-124. 130 Farman, Mobile Interface Theory, 151 Over the years members of MAW Harvey has written about the 39. have included Ali Momeni, Jenny American frontier only indirectly. 131 Ibid, 43-44. Schmid, Andrea Steudel, Davey In Spaces of Capital Harvey sug- 132 Butler, Gender Trouble, 176. Steinman, Ben Moren, Daniel gests that the American frontier 133 Refer to Chapter 2 where I apply Dean, Aaron Marx, and many constituted a spatial fix to the Derrida’s notion of the supple- others. excess capacities of European ment to the exclusion/erasure of 152 Nicolas Bourriaud, Relational capitalism in an analysis of the material with the ascendency Aesthetics (2002), 113. Johann Heinrich Von Thünen, a of a concept of the virtual. 153 I draw in part on Benford and German economic geographer. 134 Farman, Mobile Interface Theory, Giannachi’s account of hybridiza- 173 Harvey, Spaces of Capital. 237. 53. tion here. See Steve Benford and 174 Sidney Reed et al., “DARPA 135 Friedberg, Virtual Window, 11. Gabriella Giannachi’s Performing Technical Accomplishments,” 28. 136 Ibid. Mixed Reality (2011). This amount is approximately $45 137 Theodor Adorno, Notes to 154 Juleanna Enright, “Film Noir Meets million in 2012 US dollars. Literature (1991), 37. Bike Culture.” 175 Vint Cerf, “Amount of funding 138 Site-specific media art address the 155 All places, of course, have social from ARPA?” (2009). This amount sociocultural conditions of site, and historical meaning, just one is approximately $90 - $125 reveal its disciplinary effects, and not evident to a general public. million in 2013 US dollars. highlight a differentiation of the As a testament to how marginal 176 Amy Schatz, “Broadband Stimulus material and the virtual. these spaces were, however, while Funds Up for Grabs” (2009). 139 Janet Cardiff, Miller and Alberro, participating in The Parade and 177 Google, “Google Inc. Announces

158 Rupture of the Virtual 159 Notes Bibliography & Illustration Credits ARCHIVAL SOURCE the Permanent Crisis of Aura.” Chun, Wendy Hui Kyong. Control and Cardiff.” BOMB.79 (2002): 60-7. Print. National Archive, Kew, London, UK. Convergence: The International Freedom: Power and Paranoia in Eitner, Lorenz. Neoclassicism and Abbreviated NA in the references. Journal of Research into New Media the Age of Fiber Optics. Cambridge, Romanticism, 1750-1850; Sources and The four letter code followed by a Technologies 12.1 (2006): 21-39. Print. Mass.: MIT Press, 2006. Print. Documents. Englewood Cliffs, N.J.: series of numbers refers to the piece Bordwell, David, Janet Staiger, and Clarke, R. Wallace. British Aircraft Prentice-Hall, 1970. Print. reference used at the Archive. Kristin Thompson. The Classical Armament 2 Vol. London: Patrick Enright, Juleanna. “Film Noir Meets Hollywood Cinema: Film Style & Mode Stephens Ltd., 1994. Print. Bike Culture.” l’etoile magazine. BIBLIOGRAPHY of Production to 1960. New York: ---. “Drawing a Bead: Part 3.” Aeroplane June 7 2012. Print. Adorno, Theodor, and Rolf Tiedemann. Columbia University Press, 1985. Print. Monthly April 1983 1983 Print. Farman, Jason. Mobile Interface Theory: Notes to Literature. New York: Bourriaud, Nicolas. Relational Aesthetics. Commerce Business Daily. “Sources Embodied Space and Locative Media. Columbia University Press, 1991. Dijon, France: Les Presses du Réel, Sought Market Survey for Land New York: Routledge, 2012. Print. Print. 2002. Print. Warrior System C41 Components.” Farren WS. “Letter to the Ministry Althusser, Louis, and Ben Brewster. Butler, Judith, Ernesto Laclau, and Commerce Business Daily December of Aircraft Production.” NA: AVIA For Marx. London; New York: Verso, Slavoj Žižek. Contingency, Hegemony, 1 1993. Print. 15/1195. 9 February 1941. Print. 2005. Print. Universality: Contemporary Dialogues Crary, Jonathan. “Spectacle, Attention, Foster, Hal. Discussions in Andrejevic, Mark. ISpy: Surveillance on the Left. London: Verso, 2000. Print. Counter-Memory.” October 50 Contemporary Culture. Number One. and Power in the Interactive Era. Butler, Judith. Gender Trouble: Feminism (1989): 97-107. Print. Seattle: Bay Press, 1987. Print. Lawrence, Kan.: University Press of and the Subversion of Identity. New Crary, Jonathan. Suspensions of Freud, Sigmund, Josef Breuer, and Kansas, 2007. Print. York: Routledge, 2006. Print. Perception: Attention, Spectacle, and James Strachey. The Standard Edition Apter, Emily S. Continental Drift: Cameron, Chris. “Military-Grade Modern Culture. Cambridge, Mass.: of the Complete Psychological Works From National Characters to Virtual Augmented Reality could Redefine MIT Press, 1999. Print. of Sigmund Freud. London: Hogarth, Subjects. Chicago, Ill.: University of Modern Warfare.” New York Times de Souza e Silva, Adriana. “Cyber to 1955. Print. Chicago Press, 1999. Print. June 12 2010. Print. Hybrid: Mobile Technologies as Friedberg, Anne. Virtual Window. Armament Division Royal Aircraft Cardiff, Janet, George Bures Miller, and Interfaces of Hybrid Spaces.” Space Cambridge, Mass.: MIT Press, 2006. Establishment at Farnborough. Notes Carolyn Christov-Bakargiev. P.S.1 and Culture 9.3 (2006): 261-278. Print. Print. on Reflector Gunsight Installations Contemporary Art Center. “Janet Debord, Guy. “Theory of the Dérive”. ---. Window Shopping: Cinema and using Separate Projector and Cardiff: A Survey of Works Including Ed. http://www.cddc.vt.edu/sionline/ the Postmodern. Berkeley, Cal.: Reflector Units. NA:AVIA 15/1195 Vol., Collaborations with George Bures si/theory.html. Web. University of California Press, 1994. November 5, 1942. Print. Miller” New York, NY: 2001. Print. Defense Advanced Research Projects Print. Arrighi, Giovanni. “Hegemony Cardiff, Janet, George Bures Miller, and Agency (DARPA). “Urban Leader General Dynamics. “Land Warrior.” 2010. Unravelling.” New Left Review. 32 Alexander Alberro. Janet Cardiff & Tactical Response, Awareness & Web. . Azuma, Ronald, et al. “Recent Goetz Collection. München: Stiftung 2008 2008. Web. . Performing Presence: Between the IEEE Computer Graphics and Cardiff, Janet, and George Bures Miller. Derrida, Jacques. Dissemination. Live and the Simulated. Manchester, Applications. 21.6 (2001): 34. Print. “Her Long Black Hair.” Web. . Derrida, Jacques. Of Grammatology. Google. “Google Inc. Announces First Teleoperators and Virtual Castells, Manuel. The Rise of the Baltimore: Johns Hopkins University Quarter 2013 Results.” April 18 2013. Environments 6.4 (1997): 355. Print. Network Society. Malden, Mass.: Press, 1976. Print. Web. . Benford, Steve, and Gabriella Giannachi. Virtual World: How Online Fun is the New International. New York: Grau, Oliver. Virtual Art: From Illusion Performing Mixed Reality. Cambridge, Changing Reality. New York: Palgrave Routledge, 1994. Print. to Immersion. Cambridge, Mass.: MIT Mass.: MIT Press, 2011. Print. Macmillan, 2007. Print. Deutsche, Rosalyn. “Architecture of Press, 2004. Print. Bergson, Henri, Nancy Margaret Paul, Cavell, Stanley. The World Viewed; the Evicted.” Exit Art, New York City Gray, Chris Hables. Postmodern War: and William Scott Palmer. Matter and Reflections on the Ontology of Film. Tableaux: Tompkins Square. Exit Art, The New Politics of Conflict. New Memory. London: G. Allen & Unwin, New York: Viking Press, 1971. Print. 1990. Print. York: Guilford Press, 1997. Print. 1911. Print. Cerf, Vint. Amount of Funding from Edwards, Paul. The Closed World: Hansen, Mark. Bodies in Code. New Bolter, Jay, and Richard A. Grusin. ARPA? 2009. . Mass.: MIT Press, 1997. Print. Women: The Reinvention of Nature. Bolter, Jay, et al. “New Media and Print. Egoyan, Atom, Cardiff, Janet. “Janet New York: Routledge, 1991. Print.

162 Rupture of the Virtual 163 Bibliography Harvey, David. The New Imperialism. Military Entertainment Complex.” Mindell, David. Between Human and of Illinois Press, 2007. Print. Oxford; New York: Oxford University Correspondence: An International Machine: Feedback, Control, and Scholz, Trebor. Digital Labor: The Press, 2005. Print. Review of Culture and Society 10. Computing before Cybernetics. Internet as Playground and Factory. ---. Spaces of Capital: Towards a Critical Winter/Spring (2002-2003): 14-16. Baltimore: Johns Hopkins Press, New York: Routledge, 2013. Print. Geography. New York: Routledge, Print. 2004. Print. Schopenhauer, Arthur, et al. The 2001. Print. Lévy, Pierre. Becoming Virtual: Reality Moy, Timothy. War Machines: World as Will and Representation. Hayles, N. Katherine. Electronic in the Digital Age. New York: Plenum Transforming Technologies in the U.S. Cambridge; New York: Cambridge Literature: New Horizons for the Trade, 1998. Print. Military, 1920-1940. College Station, University Press, 2010. Print. Literary. Notre Dame, Ind.: University Lister, Martin, et al. New Media: A Texas: TAMU Press, 2001. Print. Science News Letter. “Gunsight for of Notre Dame, 2010. Print. Critical Introduction. London: Munster, Anna. Materializing New Planes.” Science News Letter. August ---. How we Became Posthuman: Virtual Routledge, 2009. Print. Media: Embodiment in Information 26 1944: 130. Print. Bodies in Cybernetics, Literature, and Livingston, Mark, et al. “An Augmented Aesthetics. Hanover, N.H.: Dartmouth Shields, Rob. The Virtual. London; New Informatics. Chicago, Ill.: University Reality System for Military College Press, 2006. Print. York: Routledge, 2003. Print. of Chicago Press, 1999. Print. Operations in Urban Terrain”. Naish, JM. The Head-Up Steering Silverman, Kaja. The Acoustic Mirror: Hayles, N. Katherine. “The Materiality December 2-5, 2003, Orlando, Flor., Display for the T.S.R. II Aircraft. The Female Voice in Psychoanalysis of Informatics.” Configurations1.1 2003. 868–875. Print. NA:AVIA 6/19907 Vol. , December and Cinema. Bloomington, Ill.: (1993): 147. Print. Lockyer, CCW. Letter to Ministry of 1959. Print. Indiana University Press, 1988. Print. Herczka, Mateusz, and Pär Frid. Aircraft Production. NA:AVIA 15/1195 ---. A Study of O.R. 3044 for a Navigation, Sinclair, Archibald. Letter from Reverse Avatar. 2010. Vol. March 30 1943. Print. Bombing Reconnaissance and Archibald Sinclair, Air Ministry, to Instrument & Photography Department Mann, Steve. “Wearable Computing as Flight Control System for Strike J.J. Llewellin, Ministry of Aircraft Royal Aircraft Establishment at Means for Personal Empowerment.” Reconnaissance Aircraft. NA:DSIR Production. NA:AIR 20/1739 Vol. Farnborough. Bullet Proof Screen as International Conference on Wearable 23/26510 Vol. January 1958. Print. September 10 1942. Print. Gunsight Reflector. NA:AVIA 15/1195 Computing. May 12-13, 1998. Print. Oxford University Press. “Oxford Skoller, Jeffrey. Shadows, Specters, Vol. May 1943. Print. Manovich, Lev. The Language of New English dictionary.” 2000. Web. Shards: Making History in Avant- Jones, Caroline, and Bill Arning. Media. Cambridge, Mass: MIT Press, Poster, Mark. What’s the Matter with the Garde Film. Minneapolis, MN: Sensorium: Embodied Experience, 2002. Print. Internet. Minneapolis, MN: University University of Minnesota Press, 2005. Technology, and Contemporary Art. Martin, Reinhold. “Occupy: The of Minnesota Press, 2001. Print. Print. Cambridge, Mass.: MIT Press, 2006. Day After.” Design Observer. Price, Alfred. Late Marque Spitfire Smith, Neil. “Contours of a Spatialized Print. 12/08/2011 2011.Web. . Reed, Sidney G., Richard H. Van Atta, Social Text. 33 (1992): 55-81. Print. Society Pages. February 24 2011. ---. “Occupy: What Architecture Can Do.” and Seymour J. Deitchman. DARPA Steyerl, Hito. The Wretched of the Screen. Web. . com/feature/occupy-what-architec- Projects. Alexandria, Va.: Institute for Three-Dimensional Display”. Kim, John. “The Origin of the See- ture-can-do/31128/>. Defense Analyses, 1990. Print. Proceedings of American Federation through Graphical Interface: Marx, Karl, Friedrich Engels, and Rheingold, Howard. Virtual Reality. of Information Processing Societies. World War II Aircraft Gunsights E.J. Hobsbawm. The Communist New York: Summit Books, 1991. Print. 1968. Print. and the Status of the Material in Manifesto: A Modern Edition. London; Royal Aircraft Establishment at Tan, Michelle. “Pioneering Stryker Unit Early Computer Interface Design.” New York: Verso, 1998. Print. Farnborough. Gyro Gunsight Preps for Afghanistan.” Army Times. Convergence: The International McLuhan, Marshall. Understanding Adoption by U.S.A.: Development, and April 21 2009. Print. Journal of Research into New Media Media; the Extensions of Man,. New Firing Trials in America. NA:AVIA Taylor, Joy. “Nightmare Japan: Technologies (May 2015) Print. York: McGraw-Hill, 1964. Print. 15/1206 Vol. December 1941. Print. Contemporary Japanese Horror Kwon, Miwon. One Place After Another: McLure, Helen. “The Wild, Wild Web: Royle, Nicholas. The Uncanny. New Cinema by Jay McRoy.” Journal of Site-Specific Art and Locational The Mythic American West and the York: Routledge, 2003. Print. Popular Culture 43.1. 2010. Print. Identity. Cambridge, Mass.: MIT Electronic Frontier.” The Western Sadler, Simon. The Situationist City. Turkle, Sherry. Life on the Screen: Press, 2004. Print. Historical Quarterly. 31.4 (2000): 457. Cambridge Mass.: MIT Press, 1998. Identity in the Age of the Internet. Layar. “What is Layar.”Web. . Milgram, Paul, and Fumio Kishino. Schatz, Amy. “Broadband Stimulus Print. Leigh-Mallory, Trafford. Pilot’s Gyro “Taxonomy of Mixed Reality Visual Funds Up for Grabs.” Wall Street Urban Screens. “Urban Screens. “Web. Gunsight Mark II. NA:AIR 20/1739 Vol. Displays.” IEICE Transactions on Journal. April 15 2009. Print. . October 27 1943. Print. Information and Systems E77-D.12 Schiller, Dan. How to Think about Virilio, Paul. Speed and Politics: An Lenoir, Timothy. “Fashioning the (1994): 1321-1329. Print. Information. Urbana: Ill.: University Essay on Dromology. Los Angeles:

164 Rupture of the Virtual 165 Bibliography Semiotext(e), 2006. Print. com/photos/peosoldier/3882312870/ Virilio, Paul. War and Cinema: The Cockpit of a spitfire airplane. Logistics of Perception. London: Photography by Rottweiler in the Verso, 1989. Print. public domain. http://commons.wiki- Ward, Mark, et al. “A Demonstrated media.org/wiki/File:Spitfire_cockpit. Index Optical Tracker with Scalable Work jpg Area for Head-Mounted Display R. Wallace Clarke, British Aircraft Systems”. Proceedings of the 1992 Armament (1994), 171. Diagram Symposium on Interactive 3D adapted by author. Graphics. 1992. 43-52. Print. Google Glass™. Google and the Google Weintraub, Daniel, and Michael Ensing. logo are registered trademarks of Human Factors Issues in Head-Up Google Inc., used with permission. Display Design: The Book of HUD. “Arco de la Victoria, Madrid, 1991” © Dayton, Ohio: University of Dayton Krzysztof Wodiczko. Courtesy Research Institute, 1992. Print. Galerie Lelong, New York. Whatmore, Sarah. “Materialist Returns: “Tilted Arc, 1981” © G. Lutz. Used with Practicing Cultural Geography in permission. and for a More-than-Human World.” Janet Cardiff and George Bures Miller, Cultural Geographies 13.4 (2006): “Paradise Institute,” 2001. Used with 600-9. Print. permission. Williams, Raymond. Television : Janet Cardiff and George Bures Miller, Technology and Cultural Form. alternate view of “Paradise Institute,” Hanover, N.H.: Wesleyan University 2001. Used with permission. Press, 1992. Print. Janet Cardiff and George Bures Miller, Wing Commander (name unknown). “Her Long Black Hair,” 2004. Used Letter from the Wing Commander of with permission. the Air Fighting Development Unit, Graffiti Research Lab’s Mobile Royal Air Force to the Ministry of Broadcast Unit. Photography by Aircraft Production. NA:AVIA 15/1195 Graffiti Research Lab, Evan Roth, Vol. March 7 1943. Print. and James Powderly in the public Wodiczko, Krzysztof, and Duncan domain. https://www.flickr.com/ McCorquodale. Krzysztof Wodiczko. photos/urban_data/452740396/ London: Black Dog Pub Ltd, 2011. Minneapolis Art on Wheels’ iteration Print. of the Mobile Broadcast Unit. Used Wodiczko, Krzysztof. Critical Vehicles: with permission. Writings, Projects, Interviews. A screenshot of Livedraw. Software by Cambridge, Mass.: MIT Press, 1999. Minneapolis Art on Wheels in the Print. public domain. Wodiczko, Krzysztof. “Projections.” Minneapolis Art on Wheels, Perspecta 26 (1990): 273-87. Print. “Seaworthy,” 2010. Used with Elephant. Dir. Gus Van Sant. HBO Video, permission. 2003. Mobile Experiential Cinema, “Second Bridge is Wider, but not Wide Enough,” 2012. Illustration by Trevor ILLUSTRATION CREDITS Burks. Used with permission. Mateusz Herczka and Pär Frid, Mobile Experiential Cinema, “The “Reverse Avatar,” 2010. Used with Parade,” 2013. Photography by Ben permission. Moren. Used with permission. Mateusz Herczka and Pär Frid, John Kim, “Airplane Tree Minaret,” “Reverse Avatar” alternative view, 2003. 2010. Used with permission. John Kim, “Military Bay,” 2003. “Land Warrior” ©PEOSoldier. Licensed under CC BY 2.0. https://www.flickr.

166 Rupture of the Virtual absence 6, 8-9, 24 150; situational 52; Chardin, Jean- 91, 138, 147 display (see Head 38, 40 abstraction 5 total information 64 Baptiste-Siméon 6-7 cybernetic 5, 50 Down Display, Head essentialize, essential- Adorno, Theodor 93 Azuma, Ronald 45-46 China 134 cyberspace 25, 130 Mounted Display, izing 31, 33, 35 Advanced Research Chun, Wendy 130 cyborg 33, 50 Head-up Display) excess 133 Projects Agency background (interface) cinema 14, 20, 58, dissimulate 9 exclusion 11, 13-14, (ARPA) 134 44, 76 88, 102, 112, 123, Dean, Daniel 92, 110- Do It Yourself (DIY) 22-24, 27, 30-31, 35, aesthetic, aestheti- ballistics 62 136, 152; cinematic 115, 158 105 37, 85, 147 cism 15, 17, 82, 92, Barlow, John Perry 132 29, 51, 77, 99, 110, Debord, Guy 80, docile, docility 122- extension 2-3, 34, 59, 115, 124, 136, 152 Baudrillard, Jean 111, 136; “physical 139-140 123, 129, 131, 135, 97, 147 Airplane-Tree-Minaret 25-28, 85 cinema” 102 “Declaration of 137-138, 140 eye 6-7, 14, 51, 54, 139, 141 Benford, Steve 113-114 cinematography 21, Independence of dogfight 57 122, 129 Afghanistan 49 Bentham, Jeremy 79 114 Cyberspace” 132 dream 122, 138 eyeglasses 38 algorithm 46-47, 130 Bergson, Henri 34 cinéorama 136 deconstruction 10, drone 127 eyepiece 52, 58 alienation 103 binary (numbers) 47 Clarke, R. Wallace 22, 35 drug 2, 8, 50 alternative, alternate binaural 103 58, 67 Defense Advanced dualism, dualistic facade 71, 112 70, 85, 104, 115, 141, Blast Theory 113 cockpit (airplane) Research Projects 46-47, 64 factory (digital) 135 147, 149 body, bodies 21, 31-37, 53-55, 60-62, 66 Agency (DARPA) 134 fantasy 50, 138 Althusser, Louis 80 50, 103 colonization 86, 131 deflection (lead or economy, economic farming 150-151 analog 47, 57 Bolter, Jay 9-10 combat 50-53, 56, 59, 64 angle firing) 56-58 17, 23, 25-27, 80, 123, Farman, Jason 15, 30, Andrejevic, Mark 157 bombsight 58; Norden commodity 26, 27 dematerialize 27, 85 129, 132-135, 137- 47-48, 83-84, 86 Android (Operating bombsight 58 communication 5, dérive 111, 123, 138, 141; attention fiber optic 135 System) 63 Bordwell, David 114 8-9, 31, 33-34, 50-51, 138-140 economy 100 film 2, 10, 34, 77, 112, angle of deflection boundary (between 135, 137; internet Derrida, Jacques 7-10, electromagnet 57 114-115; filmmaking firing 56-58 virtual and real) protocols 130, 134 24-28, 37, 85 Elephant (movie) 21 77, 110 appropriation 25, 148 21, 98 compression (soft- destabilize 24-26 emancipation 33, financial, finance Apter, Emily 31-32 bourgeois, bourgeoisie ware) 128 détournement 138, 140 embedded 23, 40, 91, 27, 134 arcade 136 133-134, 138 computer, computers, Deutsche, Rosalyn 128, 151 flâneur 136 architecture 70, 77, Bourriaud, Nicolas 109 computing 2, 3, 4, 74-75 embody, embodied, flesh 26 79-80 bulletproof 60, 62 14, 21, 22, 25, 28, 31, diacritical 8 embodiment 13, 22, flows (media) 148 Arco de la Victoria 75, Bures Miller, George 35, 45, 46, 48, 51, 52, dialectic 7-8 31, 33-36, 65 food 150-151 77, 94 16, 92, 97-105, 116, 53, 99, 102, 113, 123, Diderot, Denis 6-7 encode 47 foreground 10, 16, 27, Arrighi, Giovanni 133 148-149 125, 130, 132, 137; Dietz, Steve 106 encounter 13, 16, 17, 37, 38, 74, 89-92, 98, artificial 8-9 Butler, Judith 36 “computer unit” 57; différance 8 65, 70, 74, 89-91, 93, 128 Atlantic Ocean 128 desktop 16, 70 digital 31, 33, 46, 47, 105, 109, 115, 124, Foucault, Michel 79, 137 attention, attentive, camera 2, 51, 52, constructionism, 49, 51, 82, 83, 85, 100, 128, 129, 140, 141, Franco, Francisco 76 attentiveness 30, 54, camera obscura 28, 152 cultural 35-36 107, 111, 128, 135, 150, 152 freedom 123 88, 99-100, 123, 129, Canadian Pavilion constructivism, cul- 141; communica- enframing 28 Freud, Sigmund 71-72 136-137, 140-141, 147; (Venice Biennale tural 33 tions 31; computer enhance, enhance- Frid, Pär 20-21, 42 Attention Deficit 2001) 98 contingency 22 4; economy 129, 137, ment 2-4, 11, 14, Friedberg, Anne 3, 23, Disorder 100 capital, capitalist, Crary, Jonathan 141; labor 137; 21, 33-34, 41, 44, 45, 28-29, 73, 87, 135-137 audio walk 101, 105 capitalism 26, 27, 99-100, 136-137 media 13, 22, 148 49-51, 53-55, 59, 63, frontier 131-132 augment, augmented, 35, 115, 131-135, 138, critical, critique 12-13, disappearance 11, 13, 85, 88, 91, 147 augmentation 2, 5, 139, 141, 148, 149, 15-17, 70, 74, 75, 78, 37, 76, 85-86 environment 2, 4, 15, Galerie Lelong 94 10-11, 38, 50, 63, 151; spatio-temporal 79, 81-82, 90-93, discipline, disciplinary 20, 26, 28-29, 35-36, gaze 30, 123 Augmented Reality fix 131-135 96, 115, 124, 126, 13, 15, 70, 74, 78-82, 39, 44, 46-49, 52-53, gender 31, 33, 36 (AR) 5, 12, 14-15, 21, Cardiff, Janet 16, 92, 127, 138-142, 147, 99, 129, 136-141, 150 60, 61, 63, 70, 74, gentrification 75 30, 44, 45, 48, 52, 67, 97-105, 116, 148-149 150, 152; critical disembody, disem- 79-91, 97, 99, 102-104, geography 133; geo- 82-83, 123, 124, 142 Castells, Manuel 148 practice 12-14, bodiment 31-33, 46, 113-114, 127, 128, graphical location avant-garde 77, 138 cathode ray tube 54 16-17, 22, 64-65, 70, 85, 129 146, 148, 151-152 30, 48, 51, 52, 83, 84, awareness, critical Cavell, Stanley 20, 29 74, 90, 92-93, 138-139, disorient, disorienta- ephemeral 82 127, 131 12-13, 17, 22, 89, 105, cellphone 33 141-142, 147-152 tion 8, 21, 100, 101, epistemic 27 ghost, ghostly 24-25, 124, 141, 142, 147, Cerf, Vint 134 culture 22-27, 37, 85, 105, 112, 122, 125 epistemological 11, 72-73; see also

168 Rupture of the Virtual 169 Index specter Heathrow Airport 122 also Graphical User materialist 27, 109, 111, 113, 123, panopticon 79 Giannachi, Gabriella Hebdige, Dick 71, 75, 80 Interface and See- 39-40, 89; material- 124, 128, 129, 131, Paradise Institute 15, 83, 113, 114 hegemony 80 Through Graphical ity 31-32, 34-37, 65, 139, 98-101, 116 Glow Festival 146 Hennepin Bridge 111 Interfaces 70, 73, 74, 78, 82, 87, 148; virtual mobility participation, partici- goggles (video) 21 Herbicide 151 internet 33, 86, 129, 89-91, 98, 111, 115, 129, 135-138 patory 98, 102, 104, Google 63, 130, 135; Herczka, Mateusz 130-132, 134-135, 137 128, 141, 147, 150, modern 3, 4, 5, 9, 12, 107-109, 139 Google Glass 44, 20-21, 42 interpellate 80 151 30, 79, 99-100, 129, peephole 58-59, 61, 63 135-138, 141; mod- perceptual 9, 28-30, 63, 67, 149; Google Hollywood 99, 113-114 interrogative 15, 73-74, Minneapolis Art on ernist 32 31, 34-35, 48, 87, 147 Street View 111, 129 homeless, homeless- 76-78, 81, 91-92, 128 Wheels (MAW) 16, Momeni, Ali 106-108 performance 55, 71, Global Positioning ness 72, 75 105-109 montage 77 92, 107, 110, 113-114 Satellite (GPS) 51, hybrid, hybridizes, Jones, Caroline 2 MaxMSP 107 monument 70, 71, performative (act) 36 113, 125 hybridization 15, McLuhan, Marshall 2 76-80, 88 pesticide 151 Graffiti Research Labs 50, 83, 86, 96, 110, Kim, John 143 McLure, Helen 131, Moren, Ben 92, 110- Phaedrus 7 (GRL) 106, 117 113-115 Kishino, Fumio 45 132 115, 158 Pharmakon 8 Graphical User hyperactivity 100 Kwon, Miwon 73-74, mediated 2, 7-10, 13, movement 70-71, pilot 53-63, 67 Interface (GUI) Hyperlapse 129-130 76-78, 81 28, 45, 47, 85, 97, 76-80, 88 place 39, 72-73, 77, 80, 28-29, 53; see 102, 113, 127, 148; Munster, Anna 32-33 88, 97, 101, 103-104, also See-Through ideology 50, 123 Land Warrior System mediation 9-10, 17, myth 33, 79, 81 127, 130, 138-139, Graphical Interface illusion 88 (LWS) 49-53, 66 37, 151; mediare 6 148; “non-place” 101, graticule 57-58, 60, 67 immaterial 3, 17, 20, layer 48-49, 58, 64, Merleau-Ponty, Naish, J.M. 54, 62 114 Grau, Oliver 29, 135 24-27, 30, 31, 34, 40, 107, 130 Maurice 34 narrative 92, 99, 100, Plato 7-8 Gray, Chris Hables 50 46, 73, 74, 82, 87, lead firing 56-58 Messerschmitt 101, 104-105, 110-115 playground (factory nature 6 as) 135 Greeley, Horace 131 89, 90 Leigh-Mallory, (German aircraft) 56 navigation 53, 139 Poster, Mark 25 Grusin, Richard 9-10 immediate, immediacy Trafford 55 metaphor 8, 26, 87, network (information) posthuman 32 132-133 guerilla 107, 108 2, 3, 4, 6-10, 13, 15, 20, liminal 3, 20, 73, 87 52, 113 postidentitarian 32 Milgram, Paul 45 Gulf War 75, 126 38-41, 52, 147, 152 live 15, 21, 102, 110, non-place 101, 114 postmodern 26, 50; military 14-15, 44, gun 51, 58, 75 immersion, immersed, 112-113 Norden bombsight 58 postmodernism, 49-55, 59, 62-64, 70, gunnery 56 immersive 16, 25, Livedraw 107, 118 Northern Spark 106, postmodernist 25 125-127, 133, 140, gunsight 14, 44, 49, 51, 28-29, 99, 101, 102, 110, 146 power 17, 25, 74, 76, 149, 152; militarism 54-63; gyro gunsight 104, 106, 147 location 13, 16, 30, 48, nuit blanche 106, 146 79-82, 127, 140; labor 75, 76, 78, 80; mili- 14, 53-62, 66 immobile, immobility 75, 78, 83-84, 96, 101, numerical (representa- 132, 135 tarization 75 tion) 47-48 practice (critical) gyroscope 57 48, 70, 84, 123, 133 108, 109, 111, 130, Military Bay 125-129, 12-14, 16-17, 22, imperfect, imperfec- 138, 140, 150 139-141, 143 Obama, Barack 49 64-65, 70, 74, 90, habitus 142 tions 128, 141 locative 48 Miller, George Bures ontological 10 92-93, 138-139, 141- hacker 132 impermanence 87 London 83 16, 92, 97-105, 116, opaque (screen) 21, 142, 147-152 Hansen, Mark 33-35 implant 2 Luftwaffe 55 148-149 64, 82 presence 6, 8, 13, 24, happenings 106 indexical 30, 48, 84 lyric poetry 93 miniaturization 53 optical 60, 67 33, 38, 83, 110, 114, Haraway, Donna 33 informatics 13, 23, 31 Minneapolis 106, 108, organic 151 141-142 hardware 44, 49, 130, information 3, 25, 28, Mann, Steve 45 111, 146 organism 50 privatization (mobile) Minnesota (see 134-135 30-34, 44, 47-49, Manovich, Lev 47 organ (sensory) 2, 148-149 Minneapolis) Harvey, David 131-134 51-64, 70, 84-86, 90, Mark II Gyro Gunsight 5, 11 proscenium 88 Mississippi River 112 haunted 71-73 114, 127, 130 14, 53-59, 62-63 outdoor 4, 17, 70, 84, prosthetic, prostheti- mixed reality 30, 83, hauntology 23 interactive 10, 21, 22, Marx, Aaron 158 96, 97, 105-107, 109, cized 25, 33, 50, 147 113-114 Hayles, Katherine 32 28, 29, 105, 123, 146; Marx, Karl 132 115, 146 protest 83, 92, 147, 151 Mobile Broadcasting Head Down Display interaction 9, 11, material 3-5, 9, 11-17, outside 8, 29, 35-37, protocol (communica- Unit (MBU) 106-108, (HDD) 54 15, 20, 31, 33, 37, 38, 21-40, 44-50, 53, 98-101, 104, 105, 123 tion) 130, 134 110 overaccumulation psychogeography 80, Head Mounted Display 46-48, 52, 64, 82, 58-61, 63-65, 73, Mobile Experiential 131-133 111 (HMD) 51-53, 61-63, 83, 86, 89, 98, 109, 80-93, 96-98, 102, Cinema (MEC) overlay 41, 77, 84, 88, psychology, psycho- 67, 102, 124 141-142 105, 109, 110, 113, 110-115 91, 124 logical 61, 100, 114, Head-up Display interface 3, 9, 14, 21, 115, 124, 126-129, mobile, mobility 21, 123, 140 (HUD) 4, 14-15, 44, 22, 30-31, 44-45, 48, 140-142, 146-152; 30, 45, 47-52, 63, 70, painting 5-7, 28, 83, public art 70, 73, 76, 78 49, 53-54, 62 53, 60-63, 82, 84; see materialism 19; 82-84, 86, 105-107, 86, 152

170 Rupture of the Virtual 171 Index queer 33 140-141, 149, 152 spectator, spectator- total information Walkman (Sony) 103, semiotic 104 ship 88, 99, 123, awareness 64 148-149 race, racial 31-33 sense, sensory (see 136-138, 141 transparent, transpar- wander, wanderings radical 138 organs) specter, spectral ency 9, 53, 58-60, 63 136, 138-140 reality 26-27, 34-35, sensorium 2 24-25, 72, 89 traumatic 81 war 50, 62, 75-76, 126 102, 104, 123 Serra, Richard 76-78, speed 56-57, 59, trompe l’eoil 6 wearable 53, 125 recession 134 94 129-130 Turkle, Sherry 148 Williams, Raymond reflector (screen) signification 40, 89 Spitfire (airplane) 5, 148 57-63, 67 simulacrum 24-25 55-56, 60, 66 ubiquitous (comput- window 2, 21, 28-29, remediation 9-10 simulate, simulated, Steadicam 21 ing) 113 41, 61, 82, 122; vir- remote 21, 51, 127 simulation 2, 15, 97, Steinman, Davey 158 unaugmented 2 tual window 3, 44 representation 20, 103, 113 Steudel, Andrea 158 uncanny 71, 73, 88 windscreen 60-62 87, 130, 151; see site (see site-specific Steyerl, Hito 128 unconscious 76, 80, World War I 56 also numerical and site responsive) Streetmuseum 83, 86 103, 115 World War II 14, 44, 49, representation site-specific, site-spec- subjectivity 32, 35-36 unheimlich 71, 72, 53-54, 56, 62 repressed, repression ificity 4, 12, 14-17, sublimate, sublimation 80, 81 Wodiczko, Krzysztof 71, 73, 81, 138 69-74, 76, 78, 81, 83, 138 United States of 14-15, 69-93 resistant, resistance 85, 88, 90, 92, 96-98, superimpose 52, 58, America 126, 127, 17, 35, 115, 138, 149 101, 105-106, 110-112, 61, 83 131 Žižek, Slavoj 35-36 retina 54 115, 124-125, 129, 138, supplementarity 22-27, unmediated 6, 7, 9, 52 Reverse Avatar 20-21, 141-142, 146, 149, 150 37 unprogrammed 42 site responsive 16, surface 60-61, 75, 82, 111-112; see also Rheingold, Howard 73-75, 95-96, 105-106, 87, 89, 141 non-places 132 110, 112, 114-115 surplus 132-135 Uptown (Minneapolis Royal Aircraft Situationism 111, surroundings (phys- neighborhood) 108 Establishment at 138-139 ical or material) urban 17, 71, 79-80, Farnborough 54, situational awareness 3-4, 12, 15, 20-22, 30, 108, 110, 146, 150; 60, 62 52 38, 40-41, 44, 46-49, urban screens 146 rupture 13, 16, 17, 90, Sixthsense 48 52-53, 58-59, 64, 70, 93, 124, 128, 142, 150 SJ01 Biennial 106 82-85, 96-97, 100, van Sant, Gus 21 slaughterhouse 122 105, 124, 140-141, Venice Biennale 98 San Francisco Bay smart bomb 127 147-148, 152 video 15, 21, 47, 51, 52, 125-126, 140 smartphone 48, 86, 148 surveillance 25, 52, 71, 82, 86, 105-108, Santa Monica 146 software 44, 46, 51, 130, 139 111-112, 127, 146; schemata 114 63, 105, 107, 118, 130, synecdoche 33 walks 97-98, 100-102; Schiller, Dan 148 134, 135 synthetic 25, 45 games 10, 28, 29, 137 schizophrenia 100 soil 151 Virilio, Paul 14, 51, 58 Schmid, Jenny 158 soldier 49-53, 63-64, target, targeting 51, virtual 3-4, 11-17, Schopenhauer, Arthur 66 54-63 20-41, 44-48, 63-65, 40, 89 Sony Walkman 103, TCP/IP 130 70, 82-93, 96, 98, 102, screen 2, 20-21, 29-30, 148-149 technics 11, 34 105, 110, 113-115, 44, 53, 55, 57-64, 67, space 46, 79, 80, 88, technology 3, 9, 11-16, 124, 127-131, 135-137, 70, 82, 89, 100, 109, 92, 114, 127, 130-134, 20-23, 25-26, 28-34, 141-142, 146-152; 110, 114-115, 122-123, 138, 140; material 37-38, 40, 44-51, virtuality 34-35, 128, 136-137, 140-141, or physical 21, 28, 53-55, 58, 62-64, 70, 88, 91, 110, 123; 146-148 48, 83-84, 127, 130; 82-87, 90-91, 105-106, virtualization 91, Seaworthy 109, 118 public 17, 77, 104; 111, 115, 122-124, 126; virtual spaces sedentary, seden- urban 17; virtual 126, 127, 132, 2-3, 20, 25, 28-29, 41, tariness, seden- 2-3, 20, 25, 28-29, 41, 134-136, 44, 83, 136, 147 tarization 123, 129, 44, 83, 136, 147 140, 147-149, 152; Virtual Reality 25-29, 137-138, 141 spatialization 129-131, technologist 45 35, 45, 102-103, 148 See-Through 135, 137 telepresence 45 visuality 44, 46, 51 Graphical Interface spatio-temporal fix television 2, 70, 123, visualization 53, 125, (STGI) 3-4, 12, 14-16, 131-135 137 130 41, 43-44, 49, 55, spectacle 17, 136, temporary 82, 138 60-64, 70, 124, 128, 139-140 territorialize 133 Walker Art Center 106

172 Rupture of the Virtual 173 Index COLOPHON AUTHOR An earlier version of Chapter 2 “A New John Kim Materialism for New Media Studies” was published in communication+1 COPY EDITING as “Whither the Material in New Laura Bigger Media Studies” (Fall 2013). A version Alicia Johnson of Chapter 3 “The Origin of the See- Through Graphical Interface” was DESIGN published in Convergence (May 2015). A Sara Fowler version of Chapter 4 “Encounters with the Material: Krzysztof Wodiczko and ADDITIONAL FUNDING AND Site-Specific Media Art” was published ASSISTANCE in Discourse (Spring 2016). Portions of DeWitt Wallace Library, Macalester Chapter 6 “Rupture of the Virtual” were College presented at the “Critical Studies in New Media” conference at Stanford University in 2004; and as a multimedia performance at the Sarai Institute in New Delhi, India during a residency there in 2009.

COPYRIGHT INFORMATION This book is licensed under the Creative Commons Attribution-Share Alike 3.0 license. (This does not necessarily apply to the figures contained within. Please check the individual copyright designations for them.) You are free: to share – to copy, distribute and transmit the work to remix – to adapt the work Under the following conditions: attribution – You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).

The author has made every effort to secure permission to reproduce the listed material – illustrations and photographs. We apologize for any inadvertent errors or omissions. Parties who nevertheless believe they ISBN 978-0-692-71850-6 90000> can claim specific legal rights are invited to contact the author.

ISBN 978-0-692-71850-6 9 780692 718506

Colophon The cover diagram de- picts the optical system of the Gyro Gunsight, a predictive gunsight de- veloped by the British military in the latter half of World War II and a direct predecessor to the Head-up Display. According to R. Wallace Clarke, “The lamp shines through the graticule (G) onto the gyro mirror, which reflects the image of the graticule onto the fixed mirror (M) which re- flects through the lens(L) onto the pilot’s reflector screen.”