Rhythmanalysis of the Earth, the Animal, and the Machine

Brian House

BA, Columbia University, 2002 MSc, Chalmers University of Technology, 2006 MA, Brown University, 2016

Dissertation

Submitted in partial fulﬁllment of the requirements for the degree of Doctor of Philosophy in Computer Music and Multimedia at Brown University

ii Tis dissertation by Brian House is accepted in its present form by the Department of Music as satisfying the dissertation requirement for the degree of Doctor of Philosophy.

Date______Wendy Chun, Advisor

Date______Todd Winkler, Advisor

Recommended to the Graduate Council

Date______Ed Osborn, Reader

Date______Rebecca Schneider, Reader

Approved by the Graduate Council

Date______Andrew Campbell, Dean of the Graduate School

iii Curriculum Vitae

Brian House (b. 1979, Denver) is an artist who explores the interdependent rhythms of the body, technology, and the environment. His work has been shown by MoMA (NYC), MOCA (Los

Angeles), Ars Electronica, Transmediale, ZKM | Center for Art and Media, Cincinnati

Contemporary Arts Center, and Rhizome, among others, and has been featured in publications including TIME, WIRED, Te New York Times, Neural, Creative Applications, Hyperallergic,

Creator’s Project, and by Univision Sports. His academic writing has been published by

Autonomedia, Journal of Sonic Studies, and Contemporary Music Review. Prior to his doctoral studies, House was a member of the Research & Development Lab at the New York Times. He has also led technology at the award-winning design studio Local Projects and developed interdisciplinary courses at RISD Digital+Media, Parsons Design & Technology, and at

Columbia’s Spatial Information Design Lab. He has been an artist-in-residence at Eyebeam,

MassMOCA, and the Rocky Mountain Biological Lab, and a fellow at the TOW Center for Digital

Journalism at Columbia. House holds a BA in Computer Science from Columbia University in

New York, an MSc in Art & Technology from Chalmers University of Technology in Göteborg,

Sweden, and an MA in Modern Culture and Media from Brown University in Providence.

iv Acknowledgements

Tis work is the result of co-thinking and co-making with my community at Brown and at RISD.

I want to express my gratitude ﬁrst to my mentors: Wendy Chun, Todd Winkler, Ed

Osborn, and Rebecca Schneider. John Cayley, Geoﬀ Cox, Gertrud Koch, Lenore Manderson, Jim

Moses, Neal Overstrom, and Butch Rovan have also given me their invaluable attention and support.

My cohorts in MEME and MCM as well as our inter-institutional critique groups are everywhere in these pages and I am grateful to have shared these years with them. In particular, I want to thank Asha Tamirisa, Elisa Giardina-Papa, Nathan Lee, Peter Bussigel, Nadav Assor, Liat

Berdugo, Ari Kalinowski, David Kim, Jane Long, Nupur Mathur, Lakshmi Padmanabhan, Tomas

Pringle, Marcel Sagesser, Clement Valla, and Brett Zehner for their contributions, as well as to give a shout out to Critical Sofware Ting.

Tese projects could not have happened without the support of Eyebeam, the TOW

Center for Digital Journalism at Columbia, Te Rocky Mountain Biological Laboratory Art-

Science Exchange, Vicki Myhren Gallery, Jeﬀery Keith, Heidi Steltzer, Marguerite Holloway,

Michael Parsons, Matthew Combs, Bo-Won Keum, Will Reeves, Sophia LaCava-Bohanan, Greg

Picard, Shawn Tavares, and Chira DelSesto. And my interdisciplinary studies were possible thanks to the Open Graduate Education Program at Brown.

Tanks to Sue Huang, Jer Torp, and David Feinberg, collaborators and conﬁdants essential to this work. Also to Mark Hansen, for a book, and Dave Hammond, for another.

Lucia Monge has taught me what it means to make contact. What is articulated here is rooted in our collaborative life project, and it has grown only through her patience, brilliance, and bravery—thank you.

Finally, in lieu of any thanks, which can only be insuﬃcient, this dissertation is dedicated to Steve and Susan House, my ﬁrst teachers, who are in all that I do.

v Table of Contents

Rhythmanalysis as a Minor Data Science ………………………………………… 1 Animas ……………………………………………………………………………… 13 Urban Intonation …………………………………………………………………… 33 Everything Tat Happens Will Happen Today ……………………………………… 56

vi List of Illustrations

2.1 Hiking in Gothic, Colorado ………………………………………………………… 14 2.2 Te Animas River in 2015 ………………………………………………………… 16 2.3 Kevin Cooley, Golden Prospects …………………………………………………… 17 2.4 Gov. Hickenlooper at the Animas ………………………………………………… 17 2.5 One month of USGS data “streams” from the Animas River ……………………… 19 2.6 David Tudor, Rainforest IV ………………………………………………………… 22 2.7 Bernard Leitner, Tabla Room ……………………………………………………… 22 2.8 Robert Smithson, Mirror/Salt Works……………………………………………… 24 2.9 David Bowen, Tele-Present Water ………………………………………………… 24 2.10 Animas installed at the Vicki Myhren Gallery …………………………………… 29

3.1 Dr. Parsons at work ………………………………………………………………… 35 3.2 Frequency response of hearing in laboratory animals …………………………… 37 3.3 Ultrasonic recording rig …………………………………………………………… 40 3.4 Spectrogram of audio recording in the Okavango Delta …………………………… 44 3.5 Spectrogram of ultrasonic audio recording in New York City …………………… 47 3.6 Urban Intonation, installation ……………………………………………………… 51

4.1 Brian House, Quotidian Record …………………………………………………… 62 4.2 One year of OpenPaths data for ~4000 residents of NYC ………………………… 63 4.3 Vito Acconci, Following Piece ……………………………………………………… 67 4.4 Vito Acconci, sketches for Following Piece …………………………………………. 68 4.5 Laura Poitras, Disposition Matrix …………………………………………………… 71 4.6 Richard Long, A Line Made By Walking …………………………………………… 75 4.7 Sophie Calle, Suite Vénitienne ……………………………………………………… 78 4.8 LSTM trial output …………………………………………………………………… 81 4.9 Schematic of an LSTM module …………………………………………………… 85 4.10 First photograph, Port Authority …………………………………………………… 89 4.11 Everything Tat Happens Will Happen Today, installation ………………………… 91 4.12 Everything Tat Happens Will Happen Today, detail ……………………………… 91 4.13 Everything Tat Happens Will Happen Today, printed book ……………………… 93

vii Documentation

Video, audio, and additional images of the original artwork described in this text can be found online at https://brianhouse.net

viii Rhythmanalysis as a Minor Data Science Will the (future) rhythmanalyst … set up and direct a lab where one compares documents: graphs, frequencies, and various curves? … Just as he borrows and receives from his whole body and all his senses, so he receives data from all the sciences: psychology, sociology, ethnology, biology; and even physics and mathematics … He will come to ‘listen’ to a house, a street, a town, as an audience listens to a symphony. (Lefebvre [1992] 2004, 22)

Tis dissertation began when I pilfered Rhythmanalysis, by Henri Lefebvre, from the desk of a friend. As a musician, I couldn’t resist. I knew Lefebvre from his idea that everyday spaces are socially constructed, but this work on time proved even more intriguing. In the book, Lefebvre claims that commodiﬁcation—the basis of capitalism—is a matter of substituting the “dramatic becoming” that is our temporal relation to the world with the exchange of static “things.” “Time is money” is only the most obvious formula. Such representations conceal the actual dynamics of

“dressage”—Lefebvre’s term for how, through the repetitions of everyday life, we conform to the rhythms of society and so come to physically embody capitalism’s temporal order. It is in our daily routines, our speech, our gestures. And it is technology for Lefebvre that is the means of this training. Te measured tick of the clock, the continuous ﬂow of images from the television, the mechanical pulse of industrial production alienate us from our lived time. In this, Lefebvre anticipates today’s attention-driven economy and the buzz of the phone against our bodies, always demanding the labor of our “likes.”

For Lefebvre, there is “Nothing inert in the world, no things: very diverse rhythms, slow or lively (in relation to us)” (Lefebvre [1992] 2004, 17). Tis is intuitive, if not precise. Lefebvre locates rhythm—“where there is interaction between a place, a time and an expenditure of energy, there is rhythm” (Lefebvre [1992] 2004, 15)—but he does not deﬁne it. For that we can turn to the classical sense of rhythm as an “order of movement” (Sachs 1953, 17) or to traditional musicology, in which it is an “organization of time accessible to the senses” (Sachs 1953, 15). Both of these resonate with Lefebvre’s use, but I prefer Anne Danielsen’s deﬁnition, for whom rhythm is both a

2 “structuring pattern and the particular quality of a signiﬁcant or expressive variation of this pattern” (Danielsen 2013, 5). In short, it is repetition with diﬀerence. Tis is never, however, a matter of pure abstraction—Lefebvre explains how “Rhythm reunites quantitative aspects and elements … [it] appears as regulated time, governed by rational laws, but in contact with what is least rational in human being: the lived, the carnal, the body” (Lefebvre [1992] 2004, 9).

Consequently, what I think makes Rhythmanalysis most remarkable is that it is presented as a methodology. Te name invokes Sigmund Freud’s psychoanalysis (the goal of which is to reconcile the internal representations of the psyche), as well as Gilles Deleuze and Felix Guattari’s schizoanalysis (a corrective that shows how it is the reconciliation that is the pathology).

Lefebvre’s focus on rhythm parallels Deleuze’s and Guattari’s turn away from idealist semiotics and toward the material and the extra-personal. He writes, for example, “be attentive, but not only to the words or pieces of information … listen to the world, and above all to what are disdainfully called noises, which are said without meaning” (Lefebvre [1992] 2004, 36). Rhythmanalysis, however, centers itself on an almost meditative practice that suggests its purpose is less diagnostic, in the sense of producing actionable knowledge, as it is a means of knowing in itself.

In other words, when it comes to resisting capitalist time, the method is to short-circuit the very epistemology of “things” on which capitalist production depends.

In this respect, Lefebvre is advocating for what might today be called “artistic research.”

Tis term has proved slippery and the subject of much debate, particularly in Europe, where the presence of the art practice PhD is far more established than in the United States. Te artist

Michael Baers has noted that as artistic research has been institutionalized, it has meant that art takes on modes of production typical of academia and is understood through method, outcome, and publishable results (Baers 2011). Artistic or not, knowledge, as a thing, is supposed to be the output of research, and one not incompatible with neoliberal logics of value by which it is validated. However, I think we are better situated by Hakan Topal, in his notes on the thirteenth

3 documenta, who claims artistic research is “a synthesis of the new experience of the artists and intellectual operations they engage” (Topal 2012). He connects this to Henri Bergson’s concept of intuition. Tis is not knowledge, as something known, nor is it an abstract idea. Rather, “It is an attuned empiricism that does not reduce its components and parts but expands them to connect this object to the very universe itself” (Grosz 2011, 48). Tis is precisely the epistemological reversal for which Lefebvre advocates.

Rhythmanalysis is also, distinctly, a “research-based practice.” Tis is the approach of many contemporary artists, myself included, who work with the tools of other disciplines, particularly those of the sciences and engineering. Tis could either be to critically re-frame the discourse of that discipline (“research as art”) or simply to build one’s practice on the means of experience relevant to one’s time. Te intentional, or rather intuitive, “misuse” of tools—whether archives, robots, or genetic sequencing—breaks down their normalized purpose and recognizes their latent expressivity. In this sense, art, in whatever context, is the act of what Deleuze and

Guattari call “becoming-minor.” For example, in the realm of language, those in power might want to determine what counts as proper speech. But to become minor is to “make language stammer, or make it ‘wail,’ … draw from it cries, shouts, pitches, durations, timbres, accents, intensities … Te closer a language gets to this state, the closer it comes not only to a system of musical notation, but also to music itself” (Deleuze and Guattari [1980] 1987, 104).

Rhythmanalysis as a means of knowing, not of generating knowledge, is a research-based practice because it suggests that all the disciplines, from ethnology to physics, might be turned to such

“musical” ends.

Furthermore, all minor forms are situated in relation to a major. In that respect, Lefebvre’s use of the word “data” stands out to contemporary ears, which hear it in the sense of the “digital.”

Tis is because the practice of knowledge today has converged around computing technologies, whether that is a word processor or a deep learning algorithm. On a material level, normalizing

4 knowing to the logic of binary signals privileges data science as what Friedrich Kittler calls

“Wissenswissenschafen,” or a knowledge of knowledges (Kittler 2004). Philip Agre has pointed out, however, the predilection of that discipline to conﬂate epistemology with ontology (Agre

1994). Tat is, the distinction between the world out there and the world as captured in data is lost from the perspective of the machine. Tis, of course, is a temporal operation. Data are produced by sampling what moves in the world into discrete and countable things, and to do so is to claim authority over the ambiguous.

In this sense, data are “visual.” I’m thinking of Donna Haraway’s forceful prose:

The eyes have been used to signify a perverse capacity—honed to perfection in the history of science tied to militarism, capitalism, colonialism, and male supremacy—to distance the knowing subject from everybody and everything in the interests of unfettered power. … The visualizing technologies are without apparent limit … all seems not just mythically about the god trick of seeing everything from nowhere, but to have put the myth into ordinary practice. (Haraway 1988, 581) Te ﬂoating eye doesn’t see its own vantage point, and it doesn’t acknowledge its own agency in producing the knowledge it gains. Vision becomes reiﬁcation—as knowledge, as datum, as commodity. Haraway doesn’t want to do away with the possibilities of science and technology, she wants to reembody them, to trade claims of universality for actual material relationships that undergird our means of knowing. To make data minor via rhythmanalysis, then, would be to restore to them the ambiguous and continually changing—to, perhaps, make data “aural.”

I use “aural” here not to speak, necessarily, of sound, nor is noise only auditory, but this vocabulary puts things back into motion and restores the materiality to their meaning. Teorists including Christoph Cox, Brandon Labelle, Fred Moten, and Jonathan Sterne have all emphasized how thinking through sound radically reorganizes epistemological priorities. Lefebvre, however, is shrewd to avoid the phenomenological limits of sound per se (which has a history of being made into a “thing”),1 when he adopts the language of music to describe the time of the body, to put it in terms of measures and meter, of polyrhythms, of cycles and beats. Music, afer all, is a

1 Pierre Schaeffer’s wideley cited concept of the objet sonore, for instance.

5 practice, or perhaps, as Christopher Small would have it, a verb that encompasses performing, listening, composing, recording, and dancing, forms of cultural knowing whose rhythms ofen sidestep dominant power structures (Small 1998).

In this, I think Lefebvre almost rescues music from itself. I might agree with the maxim, coined by the critic Walter Pater in 1877, that “All art constantly aspires towards the condition of music,” if he had not intended it to mean music was something transcendental.2 As it is, I prefer

Giséle Brelet’s famous maxim that “music is the art of time par excellence,” as it is an art which makes special use of the relationship between technology (instruments, acoustics, encodings) and social dynamics (composer/performer, performer/audience, audience/consumer). Or as Lefebvre puts it, music “has an ethical function. In its relation to the body, to time, to the work, it illustrates real (everyday) life.” Musicians today (especially those that would be “experimental”) too ofen forget the art part, that it’s not a matter of fetishizing sound or fulﬁlling tired representations or schilling product but of making minor the current socio-technical practice of time. Maybe that’s why Lefebvre leaves music to music and founds his own “science,” and it’s why I situate this work not as music but as artistic research, even though secretly I would have it as the former even when

(especially when?) there is no sound involved.

So how are data instruments of musical practice, a medium through which to listen, if they present themselves so readily as “things?” It requires an understanding of data as rhythmic phenomena. And in fact, data, digital or otherwise, are exemplary of what Wendy Chun calls an

“enduring ephemeral,” a thing that persists by its continual repetition, whether a repeated enunciation, the cycle of an electric current through a hard disk, or a mark read and re-inscribed.

We have rhythm’s repetition, which she writes, is “not the evidence of thought wasted but of thought disseminated” (Chun 2008). Always, too, there is its diﬀerence, in the material conditions

2 Anyway, Michael Fried took that for painting in 1967.

6 of data and their potential to produce new eﬀects (which also means there is the possibility they may be forgotten).

Data made musical means they aren’t just “information,” but processes of “transduction.”

Tis is, via Gilbert Simondon, the “operation by means of which an activity propagates itself from one location to another” (Shaviro 2006). the activity being rhythm, and “location” being not only a place but a time. Simondon introduces transduction to explain how a form of matter or energy is only information insofar as it structures subsequent material. Tere is never a datum that exists apart from the temporality of its physical substrate in this chain of relationships. In music, a

“transducer” is a piece of equipment like a microphone, for example, which converts sound waves to an electrical signal. Lefebvre’s own use of “transduction” shows up in Te Right to the City as a form of reasoning—he contrasts it with both induction (in which speciﬁc examples are generalized into a universal theory) and deduction (in which universal assumptions establish a chain of logical inferences). With transduction there is no universal, just the “incessant feedback” through which a concept is bound to the empirical experience of the world (Lefebvre [1968] 1996,

102).

Te concept might even be that of the individual, which through transduction, according to Simondon, both diﬀerentiates itself from and is inexorably included within its environment.

An inﬂuence on Lefebvre is credible when the latter writes:

Every more or less animate body and a fortiori every gathering of bodies is … polyrhythmic, which is to say composed of diverse rhythms, with each part, each organ or function having its own in a perpetual interaction that [simultaneously] constitutes an ensemble [and] a whole. (Lefebvre [1992] 2004, 89) Tis is why Lefebvre insists that for the practice of rhythmanalysis it is “necessary to situate oneself simultaneously inside and outside.” As a practical matter, he notes that “A balcony does the job admirably in relation to the street” (Lefebvre [1992] 2004, 27-28),—it is a technology that lets him become sensitive to this relation between the individual and the collective, a relation that is something very diﬀerent than, say, the exchange of things. So this miniature example shows

7 how rhythmanalysis (like music) is a means of both experiencing the world through transduction and (re-)creating that world via technological practice.

So what socio-technical relationships are at stake today, and from whom have we been estranged by our impoverished notions of time? Lefebvre’s work expands outward from the individual to the city, and though he celebrates the cyclical rhythms of nature, it is more or less humans with which he is concerned. Te crises of capitalism, however, are increasingly understood to be of a more-than-human constitution—and likewise unfold on more-than-human timescales.

We have been, ﬁrst of all, insensitive to the long term consequences of industrial production. Now, as Bruno Latour puts it, “the ground on which … history had always played out, has become unstable … the political order now includes everything that previously belonged to nature” (Latour [2015] 2017)—nature, that is, has gone from being passive to active, as degraded ecologies present a challenge for humans whether by the direct force of the hurricanes symptomatic of climate change or by the poison of concentrated minerals seeping into our water supply. Te overlapping temporalities of geology, ecology, and human industry haunt moments of crisis. Attempts to make them legible—like the use of the term “Anthropocene”—are limited by our inability to feel past a present which is still captured within the rhythms of capitalist consumption.

We are also prone to relegate nature to elsewhere, particularly in the aﬄuent cities of the

West. A box of ingredients for a single meal dropped (by drone?) at one’s doorstep does nothing to implicate us in cycles of ecological exchange, nor does Planet Earth address the barrier produced by the screen. At the same time, many animals—such as the coyote, the falcon, the rat— thrive in urban environments. Seen as pests due to their disrespect of the human-animal divide, they nonetheless persist in attempting to re-normalize interspeciﬁc co-habitation on other than human terms. Our failure to hear their rhythms—which happen simultaneously to ours, though

8 at a diﬀerent register—is a missed opportunity to expand our social understanding past the human.

And meanwhile, the process of dressage continues. Since Lefebvre, however, the temporal order of the machine has shifed. It is less a matter of the imposition of a linear clock from above

—a modernist project that achieved a certain apotheosis with atomic clocks launched into space, which is the basis of our GPS systems—but an appropriation of personal time moment-to- moment. In other words, we have moved from the synchronized time of the mechanical to the asynchronous time of online interaction. Social media platforms such as Facebook and Twitter provide an unceasing stream of opportunities to interact with other individuals, but they alienate us from the lived time of those interactions, regulating them into a perpetual (non-)present. An online “community” is thus totally bound by the logic of exchange. What does it mean, then, when the collective patterns of individuals are re-embodied in an “artiﬁcially intelligent” algorithm which inserts itself into our social reality? Whether it determines what news we see, deﬁnes urban dynamics by the constant manipulation of the map, or provides the voice to which we adapt our speech, AI re-produces a social body limited in its capacity to reinvent its relationships with others. Designed to integrate seamlessly with our lives, we need strategies to sort out this strange AI temporality.

Te rhythms of these three domains—the earth, the animal, and the machine–are indicative of the temporal order of capitalism that moves beyond the human. Each, however, presents a problem of scale in relation to the human body if we are to restore to our own lived time. If rhythmanalysis is a technological practice, we are faced with a technical problem of translating—or, rather, transducing—rhythm into the range of our sensory experience. I have addressed this by making explicit Lefebvre’s suggestion that data are a means of extended listening. I use data to make contact across distances, to re-mediate the range of the ear, and to sediment the movement of the many into the one. And as the materials I’ve used have all been,

9 necessarily, technological, the secondary work of transduction is a matter of moving that tech in the direction of music, to make minor the logics otherwise employed to divisive ends.

Tis has been my approach in the three case studies that are the artistic research of this dissertation and which are dealt with in the essays that follow. I consider these pieces to be

“compositions.” I mean this to bring together the musical sense of the term and its adaptation by

Latour, who speaks of the need to bring things (rhythms?) together without subordinating them to a universal system (Latour 2010). Composition ignores the epistemological separation between nature and culture, which is not real, by eschewing categorization in favor of contact. For a human body, contact is a question of aesthetics, and it’s also a matter of rhythm—because rhythm is not only felt, it is feeling, the change that propagates between bodies and joins them into an ensemble. Rhythm cannot be a universal, because it depends on this heterogeneity.

Rhythmanalysis is the method I have used to investigate the Animas River, urban rats, and recursive neural networks. Te method speaks to art making, not how they are framed within a discourse—the essays do this for each piece on their own terms rather than imposing Lefebvre’s language used here. Tis writing is therefore just one practice among ethnography, biology, media theory, acoustics, and engineering in which I have engaged that are subordinate to the hours, days, and months spent listening. I’ve reached no inductive or deductive conclusions, but I have teased out some of these ensembles’ expressive potential. However, the rhythmanalyst is also you, who is situated both outside and inside the work, and who is, hopefully, moved by it.

10 References

Agre, Philip. 1994. “Surveillance and Capture: Two Models of Privacy.” Te Information Society 10, 101–127.

Baers, Michael. 2011. "Notes From Within the European Artistic Research Debate." In e-ﬂux Journal 26. June.

Brelet, Gisèle. 1949. Le temps musical : essai d’une esthétique nouvelle de la musique. Paris: PUF.

Chun, Wendy Hui Kyong. 2008. "Te Enduring Ephemeral, or the Future Is a Memory.” Critical Inquiry 35 (1), 148–171.

Danielsen, Anne. 2013. Introduction, Musical Rhythm in the Age of Digital Reproduction. New York: Routledge.

Deleuze, Gilles and Felix Guattari. (1972) 1983. Anti-Oedipus: Capitalism and Schizophrenia, trans. Robert Hurley, Mark Seem, and Helen Lane. Minneapolis: Minnesota University Press.

Deleuze, Gilles and Felix Guattari. (1980) 1987. A Tousand Plateaus: Capitalism and Schizophrenia, trans. Brian Massumi. Minneapolis: Minnesota University Press.

Grosz, Elizabeth. 2011. Becoming Undone: Darwinian Reﬂections on Life, Politics, and Art. Durham: Duke University Press.

Haraway, Donna. 1988. "Situated Knowledges: Te Science Question in Feminism and the Privilege of Partial Perspective.” Feminist Studies 14 (3), 575–599.

Kittler, Friedrich. 2004. "Universities: Wet, Hard, Sof, and Harder.” Critical Inquiry 31 (1), 244– 255.

Latour, Bruno. 2010. "An Attempt at a 'Compositionist Manifesto.'" New Literary History 41 (3), 471–490.

11 Latour, Bruno. (2015) 2017. Facing Gaia: Eight Lectures on the New Climatic Regime trans. Catherine Porter. Cambridge: Polity Press.

Lefebvre, Henri. (1968) 1996. "Te Right to the City.” In Writings on Cities trans. Eleonore Kofman, Elizabeth Lebas. Cambridge: Wiley-Blackwell.

Moten, Fred. 2003. “Not In Between: Lyric Painting, Visual History and the Postcolonial Future.” Te Drama Review 47 (1), 127–148.

Pater, Walter. 1877. “Te School of Giorgione.” Fortnightly Review, October.

Sachs, Curt. 1953. Rhythm and Tempo: A Study in Music History. New York: W. W. Norton.

Shaviro, Steven. 2006. “Simondon on Individuation.” Te Pinocchio Teory. January 18. http:// www.shaviro.com/Blog/?p=471

Small, Christopher. 1998. Musicking: Te Meanings of Performing and Listening. Hanover: University Press of New England.

Topal, Hakan. 2012. “Notes on dOCUMENTA (13): Artistic Research.” Deliberately Considered. New York: New School for Social Research.

12 Animas 2.1 Hiking in Gothic, Colorado (Brian House)

My boot splashed in the shallow water and stuck. Tick, orange sediment swirled around it— oxidized iron runoﬀ from a cut somewhere nearby, some industrial monument nestled deep in the Colorado Mountains. Tis geography was familiar to me from my youth, and my wife and I had returned as artists-in-residence at the Rocky Mountain Biological Laboratory. We were hiking with Dr. Heidi Steltzer, a biologist from Fort Lewis College in nearby Durango. “It’s the color of the Animas,” she said, looking at the water.

As Heraclitus famously put it, “You can’t step into the same river twice,” and a river is the perfect example of a thing that is not a thing, a form that is so clearly a process. To “animate” is to

“bring to life,” which for a theorist like Jane Bennett is something “vibratory, liquid, and virtual” (Bennett 2010, 54)—“virtual” here meaning the latent potential of something to both have an affect and be affected. In this respect, the Animas is well-named, because down a river like this one flows both geologic forces and human histories that modulate our environment.

Steltzer’s observation—and her subsequent guidance—ﬂowed into material investigations of sound, metal, and data, eventually arriving at an artwork commissioned by the University of

Denver. I decided that this artwork should also be called Animas, as the piece is not only connected to this river but links agency to “presence”—that quality of art that has us reconsider our relations. Tis text traces the tributaries of that artistic research and describes my attempt to overﬂow representations of nature versus culture.

14 ✧

Te name Río de las Ánimas (“River of Souls”) was chosen for the river by the Spanish explorer

Juan María de Rivera, who in 1765 made an unwelcome incursion into what would become

Colorado. He also noted the presence of gold in the region. Te waves of European invasion that followed displaced the Ute and Navajo nations, and in the so-called Brunot Agreement of 1873 the United States government oﬃcially opened the San Juans to unrestricted mining (Wildfang

2009, 27). Today’s Ute and Navajo reservations are now to the south and west, along the lower

Animas and the larger San Juan River into which it ﬂows. Tey are both now aﬀected by what is released upstream in their prior homelands.

Tese are in the San Juan mountain range, where the Animas gathers together its tributaries, along one of which sits the Gold King Mine. Te mine was active in the latter days of

Colorado’s gold mining boom, and produced upwards of 350,000 ounces of high-grade gold, worth nearly half a billion dollars at today’s prices (Turkewitz 2015). Abandoned in 1923, it is now one of 23,000 inactive mines that dot the landscape, slowly ﬁlling with water from melted snowpack into which leach the heavy metals—sans gold—extracted from the rock and lef behind

(Beaty 2016).

Incredibly picturesque, the economy along the Animas largely depends on tourism, white- water rafing in particular. But the liveliness of the river belies the absence of species such as trout, a signature presence in most Colorado waterways, which disappeared from the upper river when mining in the region began and have not returned since. Tis is because the mines leak. At various times prior to 2015, from 11 to 200 gallons per minute of contaminated water ﬂowed out of the Gold King, but sealing it up had been a game of Wack-A-Mole—for every bulkhead installed, increased pressure caused another leak elsewhere (Tompson 2016). A full-ﬂedged

15 2.2 The Animas River in 2015 (Jerry McBride / AP) cleanup would have required a Superfund designation, but this was unpopular due to the eﬀect it would have had on the tourist industry that depends on the perception of the region as pristine wilderness.

In the summer of 2015, the EPA showed up because it was unclear why water escaping from the mine had actually decreased in the months prior. Te hope was to open a controlled

ﬂow from one of the plugs to release any pressure that might be building up inside—an attempt to mitigate, if not cure, the ongoing problem. Te contractors certainly succeeded in releasing pressure—3 million gallons’ worth of contaminated water exploded through the exploratory hole they had made. Te water washed out the access road, and the crew were forced to leave and let it

ﬂow uncontrolled into the Animas watershed, carrying cadmium, iron, arsenic, copper, maganese, lead, beryllium, aluminum, and zinc. Te most iconic image of the event might have been an Associated Press photograph of three kayakers suspended in what was now, thanks to the high concentration of iron oxide, bright orange water. Before and afer photos gave the river its media moment, with the green pines providing a perfect contrast between what appeared to be untouched nature and the wreckage caused by human industry.

16 Te following year, the Yerba Buena Center for the Arts in San Francisco mounted a show by the photographer and videographer Kevin Cooley, Golden Prospects, which showed the unexpectedly beautiful new color combinations of the altered landscape on monitors throughout the exhibition space, occasionally featuring a friend or two going through the motions of panning for gold. In the artist’s words, the installation “points to the diﬃculties of distinguishing between what is natural and what is caused by human interference, asking us to question whether, and how, to trust the elements that nurture and sustain us.”3 And yet in its meticulously framed and glowing imagery, the work establishes color as the index of precisely such a divide. It is a memorial to the lack of separation, romantic and removed as depictions of the West have always been.

I think Governor John Hickenlooper might have done better than Cooley. In the photo released by his oﬃce, he’s actually drinking straight from the river, a bit of theater that actively considers what is vital and what is inert in the water.

2.3 Kevin Cooley, Golden Prospects (2016) 2.4 Gov. Hickenlooper at the Animas (Durango Herald)

3 See Cooley’s description on his website, http://www.kevincooley.net/golden-prospects/

17 ✧

Tis mesh of interrelations that are the Animas cannot be traced in orange. Steltzer published an op-ed in the Durango Herald a year afer the spill, warning readers “Don’t judge a river by its color” (Steltzer 2016). She explains that a visible color indicates the presence of undissolved metals, which are actually less harmful than those that are dissolved and invisible. Steltzer oversaw the placement of a “sonde” in the river at Fort Lewis College in Durango, which is a bundle of sensors measuring various indicators of water quality. Tis joined similar sondes that were in operation prior to the spill further downstream, placed by the Ute and Navajo nations.

Tey were right to distrust EPA assurances, as an estimated $335 million worth of damages to

Navajo crops and ranches are as yet uncompensated. Afer the event, the United State Geological

Survey also placed sondes at Durango, the town of Silverton, and close to the Gold King in

Cement Creek. Tanks to Steltzer’s introduction, I gained access to each of these data “streams.”

Plotting these data in a visualization over time quickly dispenses with any preconception that a river is unchanging in its continuous ﬂow. Te numbers produce an ensemble of periodicities, punctuations, cadences, builds. It is strikingly rhythmic. Take “conductance,” an indicator of dissolved metals, which stair-steps in daily bursts, only to plunge erratically ever so ofen. Looking at this I realized I’d never thought of a river being connected to diurnal cycles in this way. But the very shape of the waves expresses the cumulative eﬀect of interactions upstream between sunlight, snowmelt, rock, root, marmot, and insect. It is what Gilles Deleuze and Felix

Guattari call an “assemblage,” which as Bennett describes, possess “emergent properties, emergent in that their ability to make something happen … is distinct from the sum of the vital force of each materiality considered alone” (Bennett 2010, 24).

18 2.5 One month of USGS data “streams” from the Animas River (Brian House)

Taking assemblages into account changes how we think about an “event” such as the spill.

Rather than the act of a singular subject upon an object and the cause of various eﬀects, an event becomes the meeting point of virtual and actual, a position in a ﬁeld of potential that is always unfolding the connections between innumerable entities. Te temporality of the event is therefore multiple, as what unfolds does so on multiple timescales. From this perspective, the Gold King

Mine spill cannot be understood as a singular occurrence, but it is a conﬂux of historical processes. Te release of water is a double irruption into the present of both the industrial history of resource exploitation in Colorado and the geologic forces that deposited those minerals. Te dynamics of the ﬂowing water that emerge from those realities are inextricable from present political complications—precarious resident populations, the economics of eco-tourism and agriculture, data politics, and short-sighted government bureaucracy that struggle to reconcile such multiplicity.

Te “Anthropocene” discourse reﬂects the diﬃculty. As a term coined by geologists, the proposed name for our epoch recognizes the need to take into account the long-term consequences of human actions. Resistance to term, however, asserts that it both re-inscribes the very problem of anthropocentrism that it describes and treats all humanity as equally complicit.

19 Marisol de la Cadena prefers the term “anthropo-not-seen,” which describes how “heterogenous worlds that do not make themselves through the division between humans and nonhumans” (Cadena, 2015) are invisible to Western reasoning, to the detriment of all.

Considering the complexity of the Animas (and the reductiveness of my initial data visualizations, as suggestive as they were) I wanted to explore aesthetic strategies other than rendering something visible and which moved toward the “congregational agency” (Bennett 34), as Bennett puts it, that is at work.

✧

Brandon Labelle might as well have been describing a river when he writes that “Sound is intrinsically and unignorably relational: it emanates, propagates, communicates, vibrates, and agitates; it leaves a body and enters others; it binds and unhinges, harmonizes and traumatizes; it sends the body moving” (Labelle ix). Tat is, it animates. Sound is not matter, like water is, but it is a phenomenological experience that describes what water does. It’s promising, then, as a medium of artistic practice, to bring to the fore the dynamics of ﬂow rather than the properties of a static thing.

Te technical practice of sound is the process of transforming from one form of energy into another, called “transduction” (Sterne 2003). Tis might be as simple as a stick struck against a surface, but even that includes the chain of transformations from wood to air to ear to nerve that implicate us in the event. We also cut plastic into a form which can later be mechanically retraced. We then move a gemstone along it to stimulate the electrical potential of a magnetically aligned metal, which results in a charge that ﬂuctuates a much stronger current, moving yet more magnets to push the air and re-create recorded sound. All of these elements—speakers, ampliﬁers, cochlea—are known as “transducers.” And we intuitively understand something about how

20 transducers work when we put on a record, moving along to a beat which we know is no less material than when it was recorded.

However, David Tudor’s Rainforest IV, from 1973, treats transduction as not just a technical means, but as the concept driving the work. Tudor calls on participants to build a series of sculptures from resonant physical materials and suspend them from a ceiling. An audio transducer is then affixed to each of the sculptures. Tis is a technological element like a speaker without the paper cone, and it is designed to send an audio signal into a solid object rather than into the air. Each participant subsequently “programs” their sculpture with audio “collected from natural scientific sources [that] are specific to each instrument, exciting their unique resonant characteristics.” Te transducers animate the sculptures, which subsequently transform the space itself: “wherever you move in the room, you have reminiscences of something you have heard at some other point in the space.”4

In this way, Rainforest builds on the idea of the “expanded ﬁeld” important to 20th- century minimalist sculptors such as Richard Serra, Donald Judd, and Tony Smith, who felt that the relation of the viewer to the piece was as important if not more so than its intrinsic properties

(Krauss 1979). Te critic Michael Fried famously derided this vulgar concern with what he called

“presence,” and declared it more appropriate to theater (Fried [1967] 1998). Fried contrasted presence with “presentness,” a sublime quality in which art is something independent of its relations. If “presentness” is an eternal now, “presence” is something that is animated over time.5

Fried’s tastes may diﬀer, but it is presence that is closer to an aesthetics of assemblage. Tis is especially the case when it becomes tangible through sound, which in retrospect almost seems to have been latent in the minimalists’ work.

4 From an interview with David Tudor by Teddy Hultberg in Dusseldorf on May 17, 1988. https://davidtudor.org/Articles/ hultberg.html 5 Interestingly, this same temporal dichotomy is used by the sociologist Henri Lefebvre, for whom the latter is the failure of theory, art, politics, economics, etc to understand themselves as other than the exchange of empty representations. Presence, on the other hand, emerges from relationships over time. (Lefebvre [1992] 2004)

21 2.6 David Tudor, Rainforest IV (1973) 2.7 Bernard Leitner, Tabla Room (1999)

Sound art such as Tabla Room (1999) from Bernhard Leitner makes the connection even more explicit, with a series of austere steel panels hung from the ceiling intentionally reminiscent of Serra’s sculptures. Like Rainforest, these are resonated by electro-mechanical transducers—this time just bare speakers stuck to the metal via their built-in magnetism—and invite a peripatetic interaction. Te size and position of the panels, not to mention the complex harmonics with which they vibrate, make listening a matter of more than just the ear. Leitner claims that “I can hear with my knee better than with my calves” (Leitner 2008) which testiﬁes to the somatic purpose of the work and its intent to make presence a sensation within the human body.

Leitner’s work, however, is indicative of what Seth Kim-Cohen sees as the phenomenological ﬁxation of sound art. He claims that minimalist sculpture—and visual art in general—was able to move beyond presence to involving itself within a discourse of ideas. But sound art, according to Kim-Cohen, seems to feel “no obligation to point to the world” (Kim-

Cohen 2009, 41), a lack of inclination it has inherited from music. I think Kim-Cohen fails to recognize the special capacity of sound to be something that isn’t limited by representational discourse. But the question remains as to how sound art might extend its particular means of relation beyond the room in which it’s in and to involve animating forces in the larger world.

22 Consider the work of Robert Smithson, who is known for working in both the art gallery and in the open landscape, which he called “non-site” and “site,” respectively. Smithson’s “site” work included such monuments as Spiral Jetty (1970), which foregrounded the process of entropy in the relationship between a human construction and the Great Salt Lake. His “non-site” pieces, however, brought raw materials—dirt, stones, and industrial detritus—indoors. Tese “three- dimensional pictures” produced for Smithson a “‘new sense of metaphor’ free of natural [or] realistic expressive content” (Smithson (1968) 1996). In other words, the pile of earth does not look like the landscape from which it is drawn, but establishes a connection to a literal site beyond the gallery.

Smithson’s interest in entropy already concerns his work with the agency of matter in complex and durational relationships with humans. In that sense, it “sounds,” within both the deep time of geology and the time of human industrial excavation of the earth. And by intervening in these processes, he performs a kind of social transduction. He involves the viewer in ongoing relationships happening elsewhere, not through representation but through an imaginary retracing of the material chain through which they have been brought into contact. In this way Smithson binds the phenomenological experience of dirt to something beyond the frame, including the social context of art and the discursive boundaries between nature and culture within the “site” of the piece.

Electronic media, of course, allows for this binding to be a little more direct. One particularly relevant example that joins site and non-site is Tele-Present Water (2011), which is representative of the work of David Bowen. As an installation intended for a gallery space, it is linked to a buoy floating somewhere on an unknown trajectory through the Pacific Ocean (thus adding another wrinkle to the concept of site-specificity). Te buoy was originally deployed—and anchored—by the National Oceanic and Atmospheric Administration (NOAA), and is collecting data on the frequency and intensity of the waves in which it drifs. It transmits data via satellite,

23 2.8 Robert Smithson, Mirror/Salt Works (1976) 2.9 David Bowen, Tele-Present Water (2011) which Bowen retrieves and scales to the movement of the installation. Tis consists of a grid of plastic tubing suspended in space and ﬂexible at its vertices, which are pulled up and down by a series of mechanical actuators mounted in the ceiling. Te result “recreates the physical movement of the surface of the water”6 as an abstracted geometry animated within the exhibition space.

Bowen’s artistic choices do not create the literal presence that sound does, however.

Instead of emphasizing the material and energetic aspects of transduction, Tele-Present Water draws from the abstraction of cybernetics in which connection is not a matter of transformation, but of transmission. As such, Bowen speciﬁcally empties the physical resonance of the materials in the gallery by making them a grid, minimizing matter to emphasize form, which invokes the vectorial lines of dataﬁed motion that are well established tropes of the digital. Tough the grid is animated, as a puppet is, in the end it’s about distance, not proximity.

6 See Bowen’s description on his website, http://www.dwbowen.com/telepresentwater/

24 However, data remain material. As Bruno Latour puts it, a datum “belongs to matter by its origin and to form by its destination,” and so there is no “rupture between things and signs … We see only an unbroken series of well-nested elements, each of which plays the role of sign for the previous one and of thing for the succeeding one” (Latour 1999, 56). Tat is to say, the matter of waves becomes data for the electromagnetic radio, which is in turn data for the harddisks of the

NOAA database, which is in turn data for the motors. Tese, too, are all transducers, and they are part of the animated matter that sounds in the piece, whether or not it is emphasized.

✧

At the end of 2016, when I was commissioned by DU, I proposed doing something with data from the Animas. With my own artworks that incorporate data, I have always insisted on an output that is materially rich in order to avoid the misimpression that I am presenting data as such. Typically, I have done this by collaborating with classically trained musicians, who have honed the skill of interpreting information through embodied expression. Considering the data streams from the sondes in the Animas, my ﬁrst inspiration was along these lines—the river would become a musical score for an ensemble. I would compress its rhythms to a playable temporal scale, and the musicians would re-animate it in a performance that could happen in the gallery.

Tis idea, however, immediately raised a question. What period of time would I use?

Trough my relationship with Curtis Hartenstine of the Southern Ute Indian Tribe, I possessed data from the time of the Gold King disaster, which would be the most logical source. Making music of just this event, however, in itself felt extractive, sensationalizing a particular moment rather than the assemblage from which it arose. If I was most interested in how the river itself is always animate, and how its complex pulsations work across multiple timescales, I could not take

25 it out of its durational continuity. Te data would have to be real-time, which the USGS sensors made possible. But this made a performance by musicians less feasible, as meaningful oscillations in the river happen over longer durations than a human can play.

Te solution, which feels obvious now, came from parallel artistic investigations. Prior to meeting Steltzer in Colorado, I had taken part in a residency at MASS MoCA, in Western

Massachusetts. It was here that I was inspired to work with audio transducers afer experiencing

Steven Vitiello’s All Tose Vanished Engines (2011). MASS MoCA inhabits a post-industrial complex, and Vitiello “brought back to life”7 the sounds of the factory by placing speakers throughout a defunct boiler house. I found the use of speakers deeply unsatisfying—why not activate the structures themselves to reanimate them? Late at night, I would drag audio gear into an unused space in order to try. Tis consisted of wiring feedback circuits in which an ampliﬁer was routed between a contact microphone and a transducer, both of which I would hold against whatever resonant detritus I could ﬁnd.8

A contact microphone (also a type of transducer) is a piezo-electric disk which picks up vibration in solid material, rather than air. Te diﬀerence in charge between its ceramic and metal layers creates an electrical potential that, when they move against each other, creates the audio signal. Sensing vibration in the same surface upon which I’ve pressed the transducer—and sending that same signal into the transducer—makes the material come alive. Unlike Rainforest, or Tabla Room, or any of countless other examples that play some external audio source through transduced material, feedback means the sound is endogenous to the circuit itself. What becomes audible are the resonant frequencies of the material, in this case scrap metal, old steel oﬃce chairs, and various barrels. No traditional speakers were involved, but the result was quite loud. While it necessitated constant manual adjustment to keep in equilibrium, I was nonetheless surprised that

I had not yet encountered such a beautiful eﬀect in a performance or installation.

7 See MASS MoCA’s press release, http://massmoca.org/event/stephen-vitiello-all-those-vanished-engines/ 8 For documentation of these experiments, see https://vimeo.com/166992042

26 Considering the presence of metal in the Animas as well as the related industrial context of the mining operations, transducer feedback through metal—somehow modulated with the sonde data—became the basic concept of an artwork. Te metal, naturally, would have to be the same as that ﬂowing in the water. If water had brought a “dead” mineral to life (albeit in a way destructive to biological life), transducers could do the same with sound. While extracting minerals from the site was impossible, I was happy to make a referential link by just using the same materials. Tis would not, then, be a displacement in the style of Smithson, but would create a broader link to extractive industries in general. Of the elements overabundant in the river, some, like arsenic, were unsafe to work with. But iron (in the form of steel), aluminum, copper, and even lead proved both obtainable and resonant enough to produce an eﬀect.

I did this by constructing feedback loops as I had done at MASS MoCA, which revealed very different qualities in the sounds of the different materials. I had panels for each metal cut to the same size, 42”x36”x1/8”—a shape inspired by Leitner, but at a human scale—and as a consequence, the difference between the tones produced by each panel was the result of the intrinsic properties of each metal. I recorded this feedback with a digital audio interface included in the circuit, and I edited the waveform into short seamless loops. Afer that, I could remove the piezo and just play the recorded tones through the transducer, causing the panel to resonate with its natural harmonics. Te one parameter I could then modulate was gain, making each panel louder or sofer.

I now had my ensemble. How data from the USGS sondes would activate the four panels was the next question. A straightforward approach would have been to have the level of each mineral in the water aﬀect the corresponding panel. However, the sondes do not capture this information directly—that can only be determined by taking the water back to a lab. Instead, the sondes measure a series of indicators which have an indirect relationship to the presence of speciﬁc minerals. Tese are the depth of the water, its temperature, its conductance, its acidity,

27 and its turbidity, which is the amount of visible particles. Tese six data streams sometimes change in concert, and sometime diverge. Rather than just include some of the parameters and not others, I use the statistical technique of principle component analysis, or PCA, to combine all six into four new dimensions with maximal variance. I found that this was an eﬀective way to make use of all the data available to bring out the latent dynamics with which they correlate.

Tough in this case going from six to four is not a huge leap, in general PCA would be a straightforward way to orchestrate high-dimensional data for fewer “instruments,” and I anticipate future applications of the technique.

Such data-processing is performed on a cloud server9 which pulls data from the USGS every 15-minutes, which is the frequency at which the sondes report new data. Tis results in a lag compared to a hypothetical, fully real-time system, as I interpolate between the current and prior readings and am therefore always 15 minutes behind. Te python code ﬁlters and normalizes the data, performs PCA, and then makes the result available via web socket. Back at the panels, I use four Raspberry Pi computers, small, inexpensive linux-based machines. Outﬁtted with pHAT DACs, the Pis are very high quality audio devices. Tey run the music programming environment Pure Data, which is able to establish a connection to the cloud server and receive the current condition of the river. Pure Data also plays back the appropriate resonating tones, and it adjusts the gain of the output according to the data.

✧

I named the work, simply, Animas. Te panels are suspended from the ceiling in a row, a few feet away from the wall, and when they sound together they create a complex drone of shimmering harmonics. Te sonic eﬀect is similar to a gong that never dies out, and without the initial attack.

9 Insert comment here on my use of contemporary data-mining infrastructure.

28 2.10 Animas installed at the Vicki Myhren Gallery, University of Denver

What changes as the river changes is the timbre—modulating the gain of any one panel ends up interacting with waveforms from all of them, shifing the presence as a whole. In musical language, this is the color.

So I’m still haunted by that association, as well as by the unexpectedly rich visual palette of the metals hanging together. Each panel was hung as it was shipped from the supplier, with the exception of the steel. In order to bring out the iron oxide, I soaked this one in hydrochloric acid, which produced an expressive surface that was a deep rust color. In addition, when set side by side, the panels also create a discretized array of elements, separated, that is not coherent with the concept of ﬂowing potentials. However, the sound exceeds these divisions, as it re-contextualizes the visuality of piece as a means of indicating the materiality of the resonance. Rather than an object, or objects, the installation suggests an event which includes multiple temporal ﬂows.

29 Visitors’ reactions to the piece have emphasized its quality of presence, as they themselves are animated. Invariably, afer viewing the work as a whole, they approach the panels and turn their ears to within inches of the surfaces, or they walk past each one at close range. Tere is an intuitive transition, then, between view and auditory experience, or rather somatic experience, given that proximity to the ear puts the panels in line with the entire torso. Further, in the case of the lead panel, there is little perceptible sound from a distance (lead, afer all, is used as an acoustic insulator) but the low frequency with which it vibrates gives it an undeniable presence close up. Tis intimacy produces an ambivalence as well, given the toxicity of the substance.

In this regard, Animas is not abstract. It is not a representation of the Animas River, nor a memorial for the Gold King Mine disaster, but a demonstration of transduction between human and river. While the wetness of the water would also be a phenomenological experience of our shared materiality, approaching the Animas from a distance brings into play the assemblage of relationships through which it gathers an animating force. We don’t see water, but metal, a substance visibly inert but which is a powerful conductor between timescales. Te presence of iron, aluminum, copper, and lead in the river is not the result of exclusively “natural” nor human forces, but a tempo change of an event that comprises both past and future.

30 References

Beaty, Kevin. 2016. “Te Gold King Mine and Mill in its heyday, long before the spill.” Denverite. August 4.

Bennett, Jane. 2010. Vibrant Matter: A Political Ecology of Tings. Durham: Duke University Press. del la Cadena, Marisol. 2015. “Uncommoning Nature.” e-ﬂux Journal 65.

Deleuze, Gilles and Felix Guattari. 1987. A Tousand Plateaus: Capitalism and Schizophrenia, trans. Brian Massumi. Minneapolis: Minnesota University Press.

Fried, Michael. (1967) 1998. “Art and Objecthood.” In Art and Objecthood: Essays and Reviews. Chicago: University of Chicago Press.

Kim-Cohen, Seth. 2009. In the Blink of an Ear: Toward a Non-Cochlear Sonic Art. New York: Continuum.

Krauss, Rosalind. 1979. “Sculpture in the Expanded Field.” October 8 (Spring), 30-44.

LaBelle, Brandon. 2015. Background Noise: Perspectives on Sound Art. New York: Bloomsbury Academic.

Latour, Bruno. 1999. Pandora’s Hope: Essays on the Reality of Science Studies. Cambridge: Harvard University Press.

Lefebvre, Henri. (1992) 2004. Rhythmanalysis: Space, Time and Everyday Life trans. Stuart Elden and Gerald Moore. London: Continuum.

Leitner, Bernhard. 2008. P.U.L.S.E. Berlin: Hatje Cantz.

31 Smithson, Robert. (1968) 1996. “A Provisional Teory of Non-Sites.” In Robert Smithson: Te Collected Writings ed. Jack Flam. Oakland: University of California Press.

Steltzer, Heidi. 2016. “Don’t judge a river by its color.” Te Durango Herald. June 11.

Sterne, Jonathan. 2003. Te Audible Past: Cultural Origins of Sound Reproduction. Durham: Duke University Press.

Tompson, Jonathan. 2016. “A Gold King Mine Timeline: A tangled history of proﬁt, tragedy and unfulﬁlled dreams.” High Country News. May 2.

Turkewitz, Julie. 2015. “Colorado Spill Heightens Debate Over Future of Old Mines.” Te New York Times. August 16.

Wildfang, Frederic. 2009. Durango. Mount Pleasant: Arcadia Publishing.

32 Urban Intonation Jacques Derrida is standing there, naked, in his bathroom. He’s looking at his cat. And his cat, in the act that will make him famous, looks back. It prompts the philosopher to write a lengthy reflection on the difference between the human and the animal, and what it is that constitutes not only an instinctual reaction, but a conscious response. For philosophers like Jacques Lacan, that difference comes down to language. But for Derrida it’s more complicated than that. He muses that it is not “a matter of ‘giving speech back’ to animals but perhaps acceding to … the absence of the name as something other than a privation” (Derrida 2010, 48). Te look may very well be a response. But what of the animal that does not return a human’s gaze? Te one that runs away, or enters where it doesn’t belong?

I’m in Brooklyn with Dr. Michael Parsons, a biologist who is a visiting research scholar at

Fordham University, when he hands me a pillowcase. “If she sees humans looking at her, there’s a danger she’ll go into cardiac arrest thinking she’s about to be eaten,” he explains. “We’ll cover the trap with this to be safe.”10 Te rat had been coaxed into the steel cage with peanuts, their shells now mingled with the excrement that is a sign of her stress. We transfer her to a sealed enclosure, and a mixture of oxygen and 3% isoﬂurane renders her docile. Parsons carefully measures, pricks, and catalogs the rat. Within minutes, she’s back on the ground and afer awhile she stands, shakes it oﬀ, and disappears into the shadows.

Parsons has been a huge help facilitating my interest in rattus norvegicus, the ubiquitous brown rat of this—and all—cities. But he is one of the comparatively minuscule number of researchers studying rats in the “wild.” Te species is, technically, the same as the albino variety who lives in the heart of laboratory science. According to the National Institutes of Health, “Rats and mice are mammals that share many processes with humans and are appropriate for use to answer many research questions” (Melina 2010). Over one million publications in the PubMed/

MEDLINE database reference experimentation on rats because of this aﬃnity (Iannaccone and

10 These words are transcribed from my memory of the conversation with Parsons.

34 3.1 Dr. Parsons at work (Brian House)

Jacob 2009). To a signiﬁcant extent, what science knows about humanity is what it knows about rats. And yet science is a pillar of that modernist project that would divide human and animal.

Tis makes urban rats perhaps even more troubling than their laboratory cousins. I’m not convinced that the revulsion they inspire is in any way objective. Is it because they broke into our storehouses, stealing our food? Or is there a cultural memory in the West of the Black Death and its link to rats? It wasn’t even the rats’ fault, and ﬂeas are hardly as viliﬁed (Dean et al, 2018).

Regardless, it was trade that brought the brown rat to Europe from Central Asia in the hulls of

Italian ships, and rats have been global capitalists ever since, entering every major city along with its traded goods. Capitalism, of course, has its downsides, and the structural violence that keeps certain human populations in poverty also produces the conditions for rats to live among them.

As RATFILM, a recent cinematic proﬁle of Baltimore puts it, “Tere’s never been a rat problem …

35 it’s always been a people problem.”11 Te problem is that some humans are thought to be less human, and closer to animals, than others. As Fred Moten puts it, “Te black, the woman, the stranger all move at the place where animality and criminality intersect” (Moten 2000, 1173). Te rat exterminator, as envisioned by the Peruvian novelist Mario Vargas Llosa, is depended on by all, but loved by none (Llosa 1977).

All societies distinguish between purity and pollution in their own culturally speciﬁc ways. Tis is about the maintenance of symbolic boundaries, according to the anthropologist

Mary Douglas. Tat which is categorically unambiguous is ok. But those entities, actions, and ideas that admit ambiguity are oﬀ limits precisely because they reveal the structure on which the representational order of the day depends (Douglas 1966). Rats, of course, are disgusting because they are living in the walls.

✧

Derrida doesn’t have a problem with his cat, “an actual cat,” there in the bathroom with him, looking. But Donna Haraway later chided Derrida for failing “a simple obligation of companion species; he did not become curious about what the cat might actually be doing, feeling, thinking, or perhaps making available to him in looking back at him that morning” (Haraway 2010, 20).

Maybe the cat was staring at him because she imagined that they heard the same thing. His

“mute” cat might have been listening to the rodents in the walls, the raucous drama of their rich social lives, planning on how she’d deal with them later, and wondering why the pensive bipedal didn’t respond.

But Derrida couldn’t hear shit. Te most sensitive humans can hear frequencies up to

20khz. At this point in his urban life, Derrida’s ears were likely capable of much less, and were

11 See RATFILM’s IDMB listing

36 3.2 Frequency response of hearing in laboratory animals (Fay 1988) probably more practiced at ﬁltering out the noise of the city than listening in. We call everything above 20khz “ultrasonic,” a term that does nothing to indicate that it is a very anthropocentric designation. Cats can hear up to 64khz, dramatically higher than humans and even higher than dogs. Rats, however, can hear up to 90khz (Fay 1988), and they make sound nearly up to the same.

I find the literature on rat vocalization largely unsatisfying. Most of what we know comes from the albinos in the lab, and it is oriented toward identifying fear or stress. “Shock-induced ultrasonic vocalization in young adult rats: a model for testing putative anti-anxiety drugs,” for example (De Vry, 1993). It’s known that repetitive calls around 22khz are a sign of alarm, as well as depression following a “social defeat” and a loss of status in a rat colony’s strict hierarchy. 50khz calls, however, are associated with positive social behaviors such as play and mating. Exceptional work by Jaak Panksepp and Jeffrey Burgdorf in the late 1990s identified a particular class of

37 sounds in this range—by tickling the rats. As they put it: “we feel justiﬁed in cautiously advancing and empirically cultivating the theoretical possibility that there is some kind of an ancestral relationship between the playful chirps of juvenile rats and infantile human laughter” (Bering

2012).

Harboring less methodological reserve, artist Kathy High has followed up on such research. She has been tickling rats in the lab for her project Rat Laughter and recording the results using an ultrasonic detector. Te concept, which has yet to be fully realized, is to assemble a soundtrack that can be played for rats in stressful conditions, especially when they are the subjects of laboratory experimentation. Te project is part of High’s extensive work with rats, particularly those used to study autoimmune diseases that relate them to her own illness. Tere is some question, I think, whether rat laughter is always soothing, especially given the role laughter plays in the more subtle forms of social dominance in humans, which Panksepp and Burgdof suspect are nascent in rats as well. But High’s work suggests how the empirical fact that sound carries aﬀect is a way to establish interspeciﬁc relations on other than human terms.

As Brandon Labelle puts it, “the auditory provides an escape route [from] the representational metaphysics of modernity” (Labelle 2015, xvii). Panksepp and Burgdorf note that researchers “take pains to deny that we can ever know whether animals have any emotional feelings” (Panksepp and Burgdorf 2003). However, they continue that “Te emergence of a

‘critical anthropomorphism’ may be essential for dealing with certain types of primitive psychobiological processes we share with the other animals.” Te sentiment is the right one, even if they’ve phrased it the wrong way around. I prefer Fred Moten’s take on the sound of the animal as “diﬀerence materialized not as an other voice, but as the other that always inhabits the voice” (Moten 2000, 1173).

To acknowledge the other is a matter of “becoming-animal” as theorized by Gilles Deleuze and Felix Guattari. Tis is diﬀerent than the “Oedipal” relationship we have with a pet who is a

38 stand-in for a human relationship. Nor does it it reenact the “archetypal” animal who inhabits our myths and dreams and inspires the zoomorphism practiced throughout human culture. Instead, becoming-animal is about accessing an instability, a multiplicity, where “molar” distinctions break down into “molecular” ﬂows of interrelationship. Tis can work contrary to appearances.

Emmanuel Lévinas writes that “one cannot entirely refuse the face of an animal” (Lévinas 1988,

169), but it’s because it has a resemblance to our own. We make the cat eat cat food out of a dish, as if it were human, an Oedipal animal, while it is the rat who devours the humans’ trash.

✧

I’m paying a lot more attention to trash in New York City. A medium-sized rat grabs a pizza crust from the pile beside an overﬂowing receptacle and takes oﬀ under the bushes. I scramble to keep it in sight, peering between the feet of old men reading newspapers, straddling park fences, and shimmying in beside a sprawling tree trunk. Tis is Columbus Park, built on the site of Five

Points, the slum immortalized by Jacob Riis’s How the Other Half Lives (Riis [1890] 1997). Riis describes the death by rats of a boy in the neighborhood in the late 1880s. It’s still a promising spot. I see the rat disappear into a burrow at the base of a tree, and I stake out the hole. Several times somebody appears at the entrance, and this is in the middle of the afernoon—rats are most active just before sunrise and afer sunset.

Te rig I place at the mouth of the burrow consists of an ultrasonic microphone,

Raspberry Pi computer, and power pack, all duct-taped within a rat trap I hollowed out. You’ve seen this model of trap, thousands of times, in multiple cities, most of all here in NYC. You haven’t given it a second look, because at best it’s an innocuous, if inelegant, part of urban infrastructure, and at worst it contains poison or even a carcass. Te black, plastic trap is an ignored signiﬁer of the urban unconscious, and so it’s the perfect container for expensive

39 3.3 Ultrasonic recording rig (Brian House) electronics if you’re planning on leaving them out on the street for extended periods of time. Tat it is thematically relevant to my purpose is almost incidental.

I’ll leave the setup recording here for a solid twenty-four hour period in order to capture behavior over a complete diurnal cycle. Te Raspberry Pi is an inexpensive, tiny, and otherwise minimalist computer running a variant of the Linux operating system, and it is running a shell script that saves audio to the SD card in one-hour chunks. Te Pi does not come with an audio- to-digital converter (ADC) and is not known as a recording device. But the ultrasonic microphone, a Dodotronics Ultramic, does its own digital conversions and runs oﬀ of USB power via the Pi. My excitement about this microphone cannot be understated.

Eﬀorts in the laboratory have typically relied on “detectors”—electronics that capture sound with a narrow frequency band and shif it in real time to a human-audible range. I have two big aesthetic problems with this—one temporal, one spectral. First, the sound that comes out the other end does not really convey animal noise, as it’s reduced to sounding like clicks or beeps.

40 As one researcher puts it, “Te eﬀect is sort of like a Geiger counter.”12 To shif the frequencies of a sound from 50khz down to 5khz, in other words, is a process of translation, and the electronics in these detectors lose a lot in the process. Secondly, detectors typically focus on speciﬁc frequency bands. While this is good—even essential—for, well, “detecting” types of vocalizations, we’re missing the acoustic context in which those sounds exist.

Te alternative is to use an ultrasonic microphone—i.e., one that can capture frequencies above 20khz as well as those we can hear. Tis is not typical. Microphones are designed for human hearing, and while much effort is made to ensure that they measure frequency accurately within our range, things drop off quickly above that. Ultrasonic mics that exist are typically expensive. But the bigger issue might be what to plug those mics into. To capture frequencies up to 96khz requires a digital sampling rate twice that (192khz).13 Equipment used by field recordists can do it—I normally use a Sound Devices 702, a high-end recorder, that is capable of such speed.

However, it is large, expensive, and power-hungry. Tis is not a setup that can sit outside in the middle of New York City.

Ivano Pelicella makes the Ultramic, which provides a solution. It packages a MEMS

(Micro Electro-Mechanical Systems) digital sensor that has a ﬂat frequency response exceeding

100khz. At 200 euros, it’s not devastatingly expensive in the case of loss, especially considering that it is USB-powered, and so it pairs well with the dirt-cheap Pi. One interesting catch, however, is customs. Pelicella is in Italy, and my intuition about the usefulness of the Ultramic remained speculative for weeks as the United States evaluated its import status—it was delayed due to the the medical and military valences of that “ultrasound” designation listed on its bill of entry. I was adamant, however, that my attempt to record rats in the urban wild would be full spectrum, so I anxiously awaited the delivery.

12 Brian Lee’s notes on bat detectors, http://bclee.net/ratdet.html 13 This is known as the Nyquist rate.

41 ✧

Recording a soundscape, rather than just detecting a particular frequency band, situates my ultrasonic recording practice within the tradition of soundscape ecology ﬁrst articulated by R.

Murray Schafer and his students in the late 1970s and which extends through to celebrated ﬁeld recordists of today like Chris Watson, Garth Paine, Leah Barclay, and many others. Tis is a discipline for which I have ambivalent feelings. On the one hand, it cultivates an acoustemology, that is, “how sounding and the sensual, bodily, experiencing of sound is a special kind of knowing, or put diﬀerently how sonic sensibility is basic to experiential truth” (Feld 1994). Tis kind of radical epistemology is what makes sound especially conducive as a medium to conveying our ecological connections, which are grounded in material relationships.

But on the other hand, soundscape ecology insists on the need to “conserve natural soundscapes” (Pijanowski 2011), a rhetoric which re-inscribes nature/culture divisions. Tis is frequently reiﬁed when people of European descent cart expensive technology to exotic Southern locales in order to produce beautiful sounds for wealthy urban inhabitants to listen to on their headphones. For example: “if we can bring ... the sounds of the natural world to humans who would otherwise never think about them, they might be motivated and inspired to alter their habits enough to take action and respond to the ramiﬁcations of climate change.”14 It’s not that I am all for habitat destruction—nothing pains me more. But it is the result of a dualistic attitude that is reinforced when technological apparatuses are used to comfortably present sonic encounters from elsewhere (House 2017). I wanted to record rats precisely because their “natural world” is the one we live in, too.

Tat said, lessons learned from environments where humans’ role is decentered are undeniable. In 2015, I joined the Okavango Wilderness Exhibition as a sound recordist, joining

14 From Leah Barclay’s Rainforest Listening website, http:// www.rainforestlistening.com/rainforest-listening.html

42 the team at the point where the Cubano river first touches Namibia and becomes that country’s border with Angola. Funded by National Geographic, the expedition had begun in the headwaters of Angola’s mountains and would continue through the Okavango Delta of Botswana, an inland delta which eventually dissipates into the Kalahari desert. It is one of the most ecologically diverse environments on the planet, with a high concentration of charismatic mega- fauna like lions, elephants, crocodiles and, most dangerously, hippopotamuses. Led by the ecologist Steve Boyes, the ostensible scientific purpose was to survey the birdlife along the water, a census intended to repeat each year. However, the more central mission was the political act of transecting the entire watershed, through three nations, to demonstrate the need for collaborative conservation efforts. To this end, our sole transport were mekoros, the dugout canoes of the regional Ba’Yei community (members of which composed a third of the team). Tese carried an unlikely cargo of cameras, computers, and my audio equipment with which the trip was documented.

I recorded hippos fighting, the calls of lions, elephant charges, and even the terrifying rumble of a buffalo herd. Tese audio snapshots are dramatic. But what I found even more compelling were the long recordings of the soundscapes. Tese recordings made clear how different species make sound at different pitch registers and coherently layer to fill the whole spectrum. Tis is what Bernie Krause calls the “niche hypothesis,” that is, how "Birds, insects, and mammals each form their own temporal, frequency, and spatial niches” in the soundscape (2012,

98). Te biophony is not a cacaphony, but an organized acoustics reﬂecting an interspeciﬁc awareness.

Frequency niches are particularly interesting, and they are readily observable in spectrograms, which are visualizations of acoustic data that show energy levels in diﬀerent parts of the frequency spectrum over time. In the example here, from my recording in the Okavango, the acoustic strata from the top are insects, birds, amphibians, and then mammals, easily

43 3.4 Spectrogram of audio recording in the Okavango Delta (Brian House) separable by the eye as well as the ear. Visible in the spectrogram are also sounds that I could not hear, as they are above 20khz.15

Te niche hypothesis could not have been noticed by attention to any one individual species, as has been the case in traditional biology. It requires a broadband approach to reveal the capacity for animals to self-organize not only among their own species, but in relation to other species. Krause notes that the healthier the environment, the more clearly partitioned the sounds.

Environments that may seem to the eye to be ﬂourishing can nonetheless be heard to have been recently disturbed when there are “missing” niches or unstructured vocalizations.

15 For a four-channel installation at Brown University’s Granoff Center in 2016, I pitch-shifted the audio spectrum and equalized the frequencies across it such that all of these sounds were within audible range and at the same loudness, emphasizing the phenomenon.

44 Te poetics of Deleuze and Guattari illustrate a relationship between rhythm and territory.

Tey write:

Two animals of the same sex and species confront each other: the rhythm of the first one ‘expands’ when it approaches its territory or the center of its territory; the rhythm of the second contracts when it moves away from its territory. Between the two, at the boundaries, an oscillational constant is established: an active rhythm, a passively endured rhythm, and a witness rhythm. (Deleuze and Guattari 1987, 320)

Tis passage show the process of differentiation within a species, and points to the boundaries that remain areas of flux and potential. Te process they describe is spatial, but it is also a matter of frequency, in the sense that both animals are competing to dominate the same sonic strata. Tis forces them physically apart. Another option would be to rhythmically co-exist, interleaving their calls as might a chorus of toads, each finding a temporal gap to full-throatedly fill. Te point is, relationships within a species, within a frequency band, are social negotiations.

Deleuze and Guattari continue that territory

…not only ensures and regulates the coexistence of members of the same species by keeping them apart, but makes possible the coexistence of a maximum number of different species in the same milieu by specializing them. (Deleuze and Guattari 1987, 320)

Tis negotiation between strata is even more interesting. As Krause has argued, biological eﬃciency means that animals evolved to "be able to hear and process the particular sounds that were relevant to their well-being” (Krause 2012, 61). Adaptation happens on an evolutionary timescale, but border oscillations still emerge at the edges as species interrelate. Te cat, for example, hears higher than its call, for obvious reasons. And the aﬀects we tend to attribute to the rat—fear or aggression—derive from the fact that these are the noises that are produced in the range we can actually hear, because we are meant to hear them. Tus the “active” may be a matter of predation, whether pursued or resisted, and the “passive” a benign repose. Te “witness” is the soundscape itself that is always present and always in a process of becoming.

Deleuze and Guattari adopt these terms from the composer Olivier Messiaen, who was well known for listening to birds. Tey supply an aesthetic for Gilbert Simondon’s philosophy of psycho-technical “individuation” to which Deleuze and Guattari are indebted. Tis is how “Te I,

45 as a psychic individual, can only be thought in relationship to we, which is a collective individual.

Te I is constituted in adopting a collective tradition, which it inherits and in which a plurality of I’s acknowledge each other’s existence” (Stiegler 2004). Tis resonance with Krause’s niche hypothesis, traced via birdsong, establishes the soundscape as a clear example of individuation and the implicit interdependence within and among species. It is not a question of simply a collection of animals in a place, but a community—in Simondon’s terms, a “transindividual.”

✧

Humans tend to be radically insensitive to the individuation of species of which they are a part, but this is not for lack of capacity. We, of course, have fantastically intricate ways of organizing sounds, and it is music, not speech, that demonstrates collective noise-making from individual expression. Any musical ensemble expresses spectral and rhythmic structures that perpetually unfold as the piece transpires. Furthermore, music’s effect is via affect, a resonance with our collective physiological processes that are a “pre-individual milieu.” To access the noise is, for a moment, to lose ourselves (“disindividuate”) and exist in a state of potential. It is to find that other which inhabits the voice, and it is to simultaneously “witness” the transindividual that makes us who we, individually, are.

Despite such capacity, we have trouble hearing our place. Tis is, perhaps, because humans are incredibly loud. By humans, here, we have to include the technological ensembles in which we are situated and the buses, stereos, and pneumatic drills that saturate the soundscape of

New York City. To look at a spectrogram recorded in a busy part of Manhattan is to see an unorganized mess of sonic interference. Human voices, ranging from 90 to 250hz, compete with the whir of ubiquitous motors that produce broadband noise across that entire range. Tere are

46 3.5 Spectrogram of ultrasonic audio recording in New York City (Brian House) not many well-deﬁned niches here, but then again, that is the noise of potential and what makes the city exciting.

However. My ultrasonic recordings produced what is, to me, a startling revelation. Te noise of a city, at least New York City, is almost entirely below 25khz or so. Is it a coincidence that the machines we’ve built almost exclusively make noise in a range we can hear? Perhaps mechanical motion moves at a speed that inevitably relates to human motion, thus producing a physiological similarity that is expressed acoustically. I have encountered no mention of such a phenomenon, but Simondon, for example, explores how technology plays a role in individuation, so this could be the case. Regardless, a spectrogram of this extended range has a lot of space up top.

Tat space is not entirely empty. I’ll oﬀer another speculative statement, which is that part of rats’ particular adaptation to living with humans is their use of this part of the spectrum for

47 their social life. And no recording I’d heard from a laboratory prepared me for the richness of that life.

Having reclaimed my rat “trap” from its position next to the burrow under the tree, I sat on a park bench and transferred the contents of the SD card to my computer. I was then faced with the prospect of 24 hours of audio, which is obviously unlistenable. However, a spectrogram view, provided by the open source sofware Audacity, allowed me to visually scan for ultrasonic activity. With the exception of some bats and some obviously non-biological tones from plumbing and the braking of buses, what I saw was murine.

Te 22khz alarm calls were what I ﬁrst noticed, starting with a distinctive down-swoop followed by innumerable repetitions, each with a quick upward punctuation at the end. But present, too, were complex shapes of all kinds, glissandi-like lightning bolts across the image, staggering over wide frequency ranges in both repetitive patterns and in unique enunciations.

Clusters of vocalizations appeared together, with greater or lesser amplitude indicating multiple individuals conversing. Every hour held new surprises, new conﬁgurations of forms and patterns.

And though I’d never heard it, and had never seen a spectrogram of it, the shape of laughter was unmistakeable.

Hearing it, of course, took a little work. To adapt these sounds to our ears (and to make them playable on standard speakers) is inevitably an act of interpretation. Te most straightforward approach is to play them back slower, also known as downsampling. What was recorded at 192khz, when played back at 8khz, is twenty-four times lower and well within our range. Tis can’t happen in real time since the duration of the sound also expands by as much, but the shape of the waveform is preserved (though now at a somewhat low resolution). Hearing this for the ﬁrst time was amazing, and I repeated the simple procedure as I worked through my recordings.

48 It is nearly impossible not to hear these exchanges as (not quite human) speaking (and singing) voices—because that’s what they are. Once it was clear that the sounds were compelling and substantial enough that I wanted to present them to others, the aesthetic choices I made in polishing them emphasized this affinity. I used a combination of downsampling and pitch- shifing—an algorithm which changes pitch without expanding time—in order to fit them into a niche that matched our own sonic range. Tis would make us hear them as within the same social stratum, rather than one of interspecific dominance. To that end, I also took out the rat sounds which are audible to us without modification—the squeaks and cries that when played 24 times slower are, frankly, horrifying. Tough memorable, the valence of these reinforces the negative affect we already associate with rats, making it less interesting to me. Finally, I also did quite a bit of audio restoration in the form of double tracking, noise reduction,16 spatialization, and reverb, which resituated the processed sounds in an acoustic environment and smoothed over some of the rough edges from downsampling.

I don’t view such eﬀects as violating the integrity of the recording, but a means of being intentional about how the sound is interpreted for our ears. Technical objectivity is not possible, but there are means of modulating between physiologies. Olivier Messiaen explains that “a bird, being much smaller than we are, with a heart that beats faster and nervous reactions that are much quicker, sings in extremely swif tempos, absolutely impossible for our instruments. I’m therefore obliged to transcribe the song at a slower tempo … it’s a transposition of what I heard, but on a more human scale” (Messiaen 1994, 95). To hear a rat squeak is to perceive just that. But to hear this audio is, in part, to become-rat.

16 I made extensive use of the spectral noise reduction plugin from the iZotope RX 6 suite.

49 ✧

I recorded nearly 150 hours (pre-processing) of rat sound at several sites in New York City, which to my knowledge are the ﬁrst recordings of their kind. Afer Columbus Park, these included a waste treatment plant in Greenpoint, Brooklyn, a maintenance shaf uptown, and a boarded up

“vacant” lot on the Lower East Side. Te latter was previously a gas station, where a colony has thrived in the pit where the gas tank was removed.17 Tis particular site had me climbing over the fence, in lieu of the ability to tunnel under it. Passersby witnessing me jump to the ground with a rat trap under my arm took it in stride.

Te traversal of architectural barriers that is intrinsic to the speculative project of rat recording led to the physical form of the artwork Urban Intonation. As a sound installation, the piece consists of ten public address speakers mounted together on a column, rafers, or eaves of a building. Above our heads, and attached to, rather than boring through, the building, this is precisely the opposite of how rats operate in our infrastructure and where we would expect to

ﬁnd them. PA systems establish territory through sonic authority, they are speech rendered as space. And as the PA leaves the source of the voice unseen—visitors look up to hear them. Tat gesture, looking up to hear sounds that are loud enough to rattle a little, means we receive the sound diﬀerently.

Tose sounds are produced by five networked Raspberry Pis outfitted with Pimoroni pHAT DACs, which, in this configuration, makes them high-quality audio playback devices. Te stereo outputs run into a series of 30W amplifiers which run the speakers. A 10-channel mix just distributes the rat voices in my recordings among the different channels, with their individual personalities emphasized by the timbre of different PA constructions—all of which are unique,

17 This site is the territory of the Alphabet City fixture X Pitts, an artist and activist who has been an ally to rat researchers in the hope of restoring the lot to useable community space. Matthew Combs, a PhD candidate at Fordham University, tipped me off to this site and Pitts allowed me to access it, though permission is something different than a key, hence the fence jumping.

50 3.6 Urban Intonation, installation vintage instruments sourced through some late nights on Ebay. Tey color the sound for the room—I’m doing everything I can to avoid listening in to some other space via headphones, because it is resonance with the body that matters to me.

“Intonation” is that quality of a speaking voice that is the rise and fall in pitch independent of semantic meaning, the tone that conveys aﬀect. Intonation exceeds speech, but it is also the noise that constitutes it—and it is what distinguishes speech as such. To hear a foreign language is to recognize speech in its pitch and rhythm without understanding its meaning. Tis is audible in what comes out of Urban Intonation’s speakers too, but maybe too much so, a speech that is excessive, containing hums and tones and giggles. Deleuze and Guattari describe how a “minor” language sheds the syntactical and lexical forms of a “major” language, but gains style (Deleuze

51 and Guattari 1987, 104). Minor language deterritorializes what is proper, turns it to poetry. Te

“major” form here might be human speech itself, because looking up at the speakers speech is what visitors are wondering if they are hearing. Only then do they read the description of the piece, and the word “rat.”

✧

I have been asked many times what the sounds mean. Can we decode what the rats are saying? I don’t know. None of the leading scientists on urban rats can clue me in to anything but the most basic distinctions between positive and negative types of sounds heard in the lab. Tis is, to the best of my knowledge, unstudied territory. But I am not so interested in that line of thinking, insofar as it attempts to “give speech back” to the animal.

Wittgenstein famously quipped that “If a lion could speak, we could not understand him.”

A lion, that is, is not situated within human society and does not share the human experiences through which language is given its meaning—nor are we privy to the culture of lions. Te only thing Wittgenstein is wrong about is the “if.” We might be too prone to see a lion, or a house cat, through a logic of resemblance, rather than hear a cry of becoming. But if a visitor hears Urban

Intonation and pauses, I hope that is a moment of oscillation between territorial bounds. It’s a use of that special capacity of sound to ﬁnd physiological resonance beyond the use of names, that being ear-witness to the voices of a community is to hear oneself within it.

52 References

Bering, Jesse. 2012. “Rats Laugh, but Not Like Humans.” Scientiﬁc American 307 (1).

Dean, Katharine, Fabienne Krauer, Lars Walløe, Ole Christian Lingjærde, Barbara Bramanti, Nils Christian Stenseth, and Boris Schmid. 2018. “Human Ectoparasites and the Spread of Plague in Europe during the Second Pandemic.” Proceedings of the National Academy of Sciences, January.

Deleuze, Gilles and Felix Guattari. 1987. A Tousand Plateaus: Capitalism and Schizophrenia, trans. Brian Massumi. Minneapolis: Minnesota University Press.

Derrida, Jacques, and Marie-Louise Mallet. 2010. Te Animal Tat Terefore I Am. Fordham University Press.

De Vry, Jean, Ulrich Benz, Rudy Schreiber and Jorg Traber. 1993. “Shock-induced ultrasonic vocalization in young adult rats: a model for testing putative anti-anxiety drugs.” European Journal of Pharmacology 249 (3), 331–339.

Douglas, Mary. 1966. Purity and Danger: An Analysis of Concepts of Pollution and Taboo. New York: Routledge.

Fay, Richard. 1988. Hearing in vertebrates : a psychophysics databook. Winnetka: Hill-Fay Associates.

Feld, Steven. 1994. “From ethnomusicology to echo-muse-ecology.” Te Soundscape Newsletter 8. Retrieved from https://www.acousticecology.org/writings/echomuseecology.html

Haraway, Donna Jeanne. 2010. When Species Meet. Minneapolis: University of Minnesota Press.

House, Brian. 2017. “Against Listening.” Contemporary Music Review 36 (3): 159–170.

Iannaccone, Philip, and Howard Jacob. 2009. “Rats!” Disease Models & Mechanisms 2, 206–210.

53 Krause, Bernie. 2012. Te Great Animal Orchestra Finding the Origins of Music in the World's Wild Places. Little, Brown and Company.

LaBelle, Brandon. 2015. Background Noise: Perspectives on Sound Art. New York: Bloomsbury Academic.

Lévinas, Emmanuel. 1988. “Te Paradox of Morality: An Interview with Emmanuel Lévinas.” In Te Provocation of Lévinas: Rethinking the Other, trans. Andrew Benjamin and Tamra Wright, ed. Robert Bernasconi and David Wood. New York: Routledge.

Melina, Remy. 2010. "Why Do Medical Researchers Use Mice?" Live Science. Last modiﬁed November 16. https://www.livescience.com/32860-why-do-medical-researchers-use- mice.html

Messiaen, Olivier. 1994. Music and color: conversations with Claude Samuel trans. by E. Tomas Glasow. Portland: Amadeus Press.

Moten, Fred. 2010. “Electric Animal: Toward a Rhetoric of Wildlife (review).” MLN 115 (5), 1171–1178.

Panksepp, Jaak and Jeﬀ Burgdorf. 2000. “50-kHz chirping (laughter?) in response to conditioned and unconditioned tickle-induced reward in rats: eﬀects of social housing and genetic variables.” Behavioural Brain Research 115 (1), 25–38.

Panksepp, Jaak and Jeﬀ Burgdorf. 2003. “‘Laughing’ rats and the evolutionary antecedents of human joy?” Physiology & Behavior 79, 533–547.

Pijanowski, Bryan, Luis Villanueva-Rivera, Sarah Dumyahn, Almo Farina, Bernie Krause, Brian Napoletano, Stuart Gage, and Nadia Pieretti. 2011. “Soundscape Ecology: Te Science of Sound in the Landscape.” BioScience 61 (3), 203–216.

Riis, Jacob. (1890) 1997. How the Other Half Lives: Studies among the Tenements of New York. New York: Penguin Classics; Reprint edition.

Simondon, Gilbert. 1989. L'individuation psychique et collective. Paris: Aubier.

54 Stiegler, Bernard. 2004. “Culture and Technology.” Filmed May 13 at Tate Museum, London. Video, 2:02:02.

55 Everything That Happens Will Happen Today Everything that happens will happen today And nothing has changed, but nothing’s the same And every tomorrow could be yesterday And everything that happens will happen today —David Byrne & Brian Eno

I follow the path to the front of the New York Chinese Baptist Church. Polyglot signage over every inch of the facade suggests perpetual activity within, but it’s closed, and I peer through my reflection in the window to see the stacked chairs inside. It will be just over an hour here. Te duration of a service? As I begin to settle in, a woman starts to approach … but hesitates before she reaches me. I don’t say anything. She looks at her phone, looks across the street, realizes her mistake, and takes off, buzzing into a door on the other side of the block and disappearing within. Hm. Maybe I’m looking for the same thing, we’re only off by a couple of feet. I follow her, and press the buzzer marked “S10.”

Te receptionist is young and buﬀ. It’s a private gym. Invite only, I learn, but he’s nice about it, and we chat for a minute about routines before I awkwardly make my exit. I consider how I’d never prompt an interaction like that in my ‘real’ life. Down the block there’s an old piano on the sidewalk that a man, unlikely to be the owner, is trying to sell with the help of a very handwritten “for sale” sign. Some music school kids stop and give a demonstration to help with his pitch. He tries me several times since I don’t seem to be leaving, but otherwise I just listen as the shadow under the scaﬀold stretches across the street.

57 For exactly one week, I followed the directions on my phone, continuously, pausing only when and where it did. On the one hand, this performance was a lonely one, to which only one person was privy—me. But on the other, it had everything to do with how I was part of an ensemble of rhythms into which the 8.5 million inhabitants of New York City come together each day. I didn’t need my phone barking at me to experience that, of course. Te point of the piece, however, was to explore how emerging technology mediates such formation, not only in New York, but in every social interaction in which a class of algorithms known as Artiﬁcial Intelligence plays a role.

In this case, an AI was determining my location. Te path that it generated was the product of having been trained on a database of geolocation information—the paths of 1000 anonymous New Yorkers logged over the course of a year. Every place they visited had been captured by an app which they had voluntarily installed on their phones. Te AI was able to learn the resulting patterns, a trace of the lives that take New Yorkers out from their homes, into the city, and back again each day. It amalgamated them into a new individual route, a strange hybrid that expressed aspects of all the volunteers. Tis is what I saw on my phone, as I (re)traced the path as a durational performance.

I designed this piece in response to the recent explosion of interest in AI as recent advances promise (threaten?) to transform every aspect of everyday life. AI, of course, has had a presence in public consciousness since antiquity. Tales of second-order creation from the Golem to Hal are woven throughout our storytelling, the “uncanniness” of Mechanical Turks18 and

Boston Dynamics19 have made us question what it is that makes us uniquely human, and prophets and corporate executives alike anticipate a coming super-intelligence. But I am not interested in how AI acts as if it were human, or how it might replace us, because the fact is that it is humans, plural. And yet something less than a “we.”

18 Amazon’s “Mechanical Turk” system, which incorporates distributed human labor within an automated framework, is named after a chess-playing “machine” from 18th century that contained a human chess master hidden within. 19 See https://www.bostondynamics.com/atlas

58 What follows are three takes on Everything Tat Happens Will Happen Today. Te first is a telling of the context in which the piece was made and its place within my own practice—it is a brief history of the thought process that inspired it. Te second moves laterally, threading through the influences latent in the work and linking it with a set of theoretical concerns—namely, the act of following. Te third section speaks from within the craf of the work’s making. It contains an account of the technical and artistic decisions I made and documents whatever innovations in the art of applied AI that I happened upon along the way. Tese three sections do not necessarily proceed from each other linearly, despite what happens on the page, and reflect the simultaneous and irreconcilable positions of artist, interpreter, and engineer that I have inhabited. Nor is the piece itself reducible to what’s here—it exists alongside it.

AI’s improvements over the last ten years have been predicated by data. Tat is, “big data,” coming from every smartphone, website, point-of-sale, security camera, ﬁtness band, satellite, weather station, wildlife collar, DNA sequencer, and keyless bike lock. What’s hyped now as AI is properly

“machine learning,” a subset of the ﬁeld which analyzes these data in an attempt to ﬁgure out how the world works. Point by point, a machine learning model looks at data and must make a decision. Is it an apple or an orange? Buy or sell? Terrorist or not? Or, given half a sentence, what word should come next? Te more data it has seen, the more robust and nuanced the model can be, because when it screws up, we correct it, and it improves. Tis is “supervised learning,” and requires lots of examples from the past—collected by humans—through which the model

59 anticipates the future. In other words, it’s built from data that inevitably reﬂect our interests, hopes, and biases.

It can also reflect our embodied behavior. I was fascinated to hear the voice of a model called WaveNet, which was produced by the AI company DeepMind in 2016, and which is now built into Google Assistant. It is a voice that includes nuanced intonations, the occasional inhale, lip smacks, and dramatic pauses, signifiers of an emotive body. Te first thing it said in public was

“Te Blue Lagoon is a 1980 American romance and adventure ﬁlm directed by Randal

Kleiser” (Oord 2016). Te choice of movie here could not be richer—the film depicts children marooned on a deserted island and who must learn about life for themselves without the benefit of society. Tis is the mythology of autonomy that we ascribe to AI, a romantic notion of the machine as exotic other. But the speech of WaveNet is fully socialized, a “neural network” trained with a corpus of 44 hours of recorded audio from 109 different speakers in the English version.

Te fact that WaveNet sounds (almost) convincingly human—as if it was an individual human— is a way of speaking over this collectivity (House 2017).

WaveNet is able to do this because synthesized speech is a sequence problem—given a series of sounds, what sound should come next? Unlike with previous speech models, WaveNet is not sequencing from a library of phonetic units chopped out of audio recordings, nor is it synthesizing sound based on pre-defined speech parameters, like an old-school vocoder might, simulating the physics of our mouths. Rather, it analyzes its corpus as a sequence of “raw” audio samples—in this case 16,000 of them in every second of audio. It is then able to predict how those samples fit together to make intelligible speech (Oord 2016). Te model has built its own internal concepts of how speech is organized like this, based only on the acoustics. I find sequences particularly interesting because prediction and generation end up being more or less the same thing. If an algorithm can predict what should come next based on its corpus, it can also

60 recursively generate a sequence based on what it has come up with so far. Tis is the voice that we hear.

WaveNet’s internal feedback makes for a provocative example of the relationship between the individual and the collective that is present in all machine learning when it is applied to human behavior. I began to think about other kinds of sequences I could investigate, and there is no shortage of current and potential uses. Beyond speech (both synthesis and recognition) this includes handwriting, language translation, musical performance, predicting retail choices, playing poker, driving a car, making social gestures, following visual attention, recognizing depression, and contract negotiation. However, I kept coming back to geographic movement.

Te narrative of how you live your life can be framed in terms of physical location. Not only where you live, but where—and when—you work, whom you visit, where you shop, or relax, or worship, how consistent or variable your schedule is, where you’re free to go and where you are not. Much of our activity is spatially organized and will remain so to the extent that we are embodied animals that cannot be fully subsumed into digital interactions. Movement marks the repetition and diﬀerence that the everyday comprises, and which in an urban setting combines into a measure of human collectivity par excellence, the pulse of the city—a pulse that, maybe, an

AI could learn to live by.

Tere is precedent for this in my own work with Quotidian Record (2012), which is about urban rhythms. I began by tracking my location for one year, and then I used these data to create a musical composition in which every single place I visited became a single musical note. Same place, same pitch. I then cut the piece into a vinyl record, timing it so that one rotation of the record corresponded to a single day. Tat’s a single day rendered in 1.8 seconds, which add up to a year in around 11 minutes. So when you play it, you hear an interpretation of my everyday rhythms at a listenable scale. My intent was to highlight the materiality of data, and to make something tangible and even nostalgic that resisted the slick visualizations typically used to

61 4.1 Brian House, Quotidian Record (2012) interpret them. It was meant to appeal to an embodied rather than cogitated perception—these are data as felt, in the way that music is felt. It’s impossible to “decode” the piece as a representation of my movement, but the cycles and variations are audible and it advances the thesis that these too are speciﬁc to my personal relationship with the places in which I live.

To track my geolocation for Quotidian Record, I used a smartphone app and online database which was a project of mine as a researcher at the New York Times Research and

Development Lab. Tis was necessary as, strikingly, there are very few options for using an app to collect geographic data in their raw form, despite the ubiquity of apps that gather such data for their own purposes. Our platform was called OpenPaths, and by default, only you had access to your data, which was otherwise encrypted. Such an arrangement is the exact opposite of every other corporate service in which mining users’ data for proprietary business value—extracted via machine learning algorithms—is largely the point.

62 4.2 One year of OpenPaths data for ~4000 residents of NYC (Brian House)

However, OpenPaths also allowed anyone to propose projects and ask for users to anonymously20 contribute their data. By virtue of revokable security tokens, a user could grant access to their location if they deemed the project interesting. We wanted to catalyze thinking among artists, activists, academics, and the general public around the potential of such datasets beyond the corporate purposes to which they are generally restricted. A series of art projects and scientiﬁc studies proved the case for such an alternative model. And upon completing Quotidian

Record, I made my own request of the OpenPaths community in 2012. I was deliberately vague, but promised an artwork that would have something to do with the communal rhythms of New

York City. Tat interest was initially obvious to me—if Quotidian Record said something about the temporal nature of the city, it was nonetheless limited by its focus on me individually. But how could I express about the city as a whole?

20 Anonymous in that the data were not linked to discrete identifiers like name or email address— geographic data themselves are absolutely revealing.

63 I struggled with what “whole” this would be—a “city symphony” based on this dataset began to seem hopelessly naïve. For one, was it possible, either technically or aesthetically, to generalize and reduce thousands of disparate paths into something listenable, something in which the feeling of the city remains present? Further, even if I succeeded, OpenPaths is by no means representative of the diversity of New York. I can speak for my own data, but to claim something about the city from this larger dataset would be to adopt corporate practices without comment— that is, to extract value from data without attending to their inherent bias and contingencies. But without some sort of claim, what would I, whatever approach I took, be interpreting?

For four more years, I continued to collect the data from over 4000 volunteers until

OpenPaths ceased operations in 2016—but no piece emerged. Tat year, however, my fascination with WaveNet catalyzed a connection. Critical attention had shifed from the problematics of big data to the use of machine learning to act on them. And I realized that the conditions that caused me diﬃculty when formulating a followup to Quotidian Record were precisely what was at stake with AI. To move from data on a collection of individuals to a single expressive model that

(presumably) encapsulates their dynamics is the whole point of machine learning. Tose individuals are central to what the model is, even as they are obfuscated by the novelty of what it generates.

If urban movement could be phrased as a sequence problem, I could train an AI to generate novel paths based on OpenPaths data, with all the reductions, biases, and obfuscations inherent in the process. But rather than making a piece of music that claimed to interpret what the city as a whole might “sound” like, I would make this an investigation of what it is that AI does, as both an aesthetic and political operation. Te city would be the quintessential communal formation by which to understand the model of collectivity intrinsic to AI. So I decided to have my own movement around New York scripted by what algorithm predicted/generated. And when doing this, I would “look” for the individuals behind the training set. As performance art, the act

64 would ask what persists of the city when it is captured, modeled, and instantiated back onto the street. And it would be a speciﬁc aesthetic experience with which to frame in general our encounter with AI.

Consider every day, of every person, to be a training sample. A path that leads from home, to work, to some errand, to some diversion, and back home again. Each of those places is a geographic location, and a day is a sequence of those locations. With 1000 individuals, I had almost 365,000 examples of how to live a day in NYC with which I could train the model. How I ended up doing this, afer weeks of trial and error, is discussed in section III. But the result was a system that—via a custom app on my phone—could take my current location as input and let me know where and when to go next.

On July 26th, 2017, I boarded a Peter Pan bus from my home in Providence to New York

City, arriving around 3:30 in the afernoon. Standing in the middle of Port Authority bus terminal, across from the New York Times Building where I built OpenPaths, and a block away from Times Square in the proverbial heart of the city, I pulled out my phone to see what I should do. It took me to a taco joint, so we were oﬀ to a good start.

II.

Te ﬂâneur is a poetic archetype, Charles Baudelaire’s aesthete who wanders through the city and for whom:

The crowd is his element, as the air is that of birds and water of fishes. His passion and his profession are to become one flesh with the crowd … responding to each one of its movements and reproducing the multiplicity of life and the flickering grace of all [its] elements. (Baudelaire [1863] 1964) Walter Benjamin would famously invoke the flâneur as a figure since co-opted by modernism, captured within the dazzling displays of capital. And Guy Debord would reimagine it as an act of resistance, offering the dérive—or drif—as a means to master, and thus escape, the subconscious

65 “psychogeographic” eﬀects at work (Debord 1956). But New York’s take on urban wandering might have been the more prescient than Paris’s.

In the late 60s and early 70s, Vito Acconci had turned from writing poetry to performing actions. He had come to the conclusion that art was all about “following a scheme.” Distilling this to his basic materials—his body—this became “following a person.” So, for a period of 21 days in

October of 1972, sponsored by the Architectural League of New York, he carried out Following

Piece:

Every day I pick out, at random, a person walking in the street. I follow a different person everyday; I keep following until that person enters a private place (home, office, etc.) where I can’t get in. (Acconci 2004, 76)

Acconci makes his way around the city not for the aesthetic experience, as does the

flâneur, nor as an act of resistance like the dérive, but as a systematic reduction of his individual will. It is radical in its surrender to an arbitrary specificity, a total consent to the form of everyday life emptied of its content. He’s deliberately uninterested in his own creative input, and his transcriptions of the events contain only the barest details of what happened—a physical description of whom he followed, where they went, the time: “11:10AM … Man in brown jacket; he walks south on Bleecker.” It’s precisely the opposite of the layers of affect and analysis that were important to Debord.

Gilles Deleuze and Felix Guattari write that “What distinguishes the map from the tracing is that it is entirely oriented toward an experimentation in contact with the real. Te map does not reproduce an unconscious closed in upon itself; it constructs the unconscious” (Deleuze and

Guattari 1987, 12). Maps possess a creative power, for better or for worse, as an image of the city which both limits and expands its potential. But Acconci is not making maps, or even seeking to undo them—he’s making tracings. Ironically, Acconci’s piece complicates the following sentiment of Deleuze and Guattari when they write that “Te map has to do with performance, whereas the

66 4.3 Vito Acconci, Following Piece (1972) tracing always involves an ‘alleged competence’” (Deleuze and Guattari 1987, 12-13). It is precisely that competence, and nothing more, that Acconci has framed in his performance art.

It has to be noted that Following Piece immediately strikes one as super creepy. To follow someone in this way immediately connotes stalking them, and if any of Acconci’s followees noticed him, I’m sure they had plenty of ideas about what was going on, none of them pleasant. It is not a reciprocal or voluntary relationship, and, especially given Acconci’s white maleness, it maintains a power diﬀerential. To truly be in public is to be anonymous, and in this regard, to follow someone is an invasion of privacy, even in the middle of public space. As Acconci writes in his notes: “Te person I follow has his privacy intruded upon: he becomes public” (Acconci 2004,

77).

What is interesting to me is that the act transforms the division between public and private in more than one way. It has this sense of watching someone, of learning about them for some, possibly malicious, purpose. However, Acconci’s explicit lack of purpose also collapses

67 4.4 Vito Acconci, sketches for Following Piece (Acconci 2004, 77) private and public together by evacuating the private of any distinction. According to his definition, to follow is “To come about or take place as a result, effect, or natural consequence” (Acconci 2004, 77). Passage through the city is just something that happens as a matter of course. A person is just a path. Private, here, is just the physical fact of “where I can’t get in.” So there is an inverse sense of violence that comes from appropriating the behavior of another person in this way because it’s specifically uninterested in their individuality as such. Acconci does not consider their agency apart from their behavior, nor, for that matter, his own: “I am almost not an ‘I’ anymore; I put myself in the service of this scheme” (Acconci 2004, 77).

Such scheming, and the avant-garde tradition of the instructional “score” in general, has a lot in common with computer algorithms. It is not an accident that these kinds of artistic practices arose in the same post-war milieu as computation. Te year afer Following Piece,

Acconci would participate in MoMA’s Information exhibition that made explicit the inﬂuence of cybernetics and the theory of non-hierarchical social organization that it oﬀered—another way to understand, as it were, the crowd. In his notes for Following Piece, Acconci includes a diagram

68 that illustrates his “subjunctive” relationship to the people he follows, which could just as easily be depicting the traceroute of a computer network. By executing his instructions, Acconci notes that

“I don’t have to control myself” (Acconci 2004, 77), a sentiment which neatly expresses the cybernetic idea that, as Jasper Bernes puts it, control can be “a technical rather than personal matter.”

Acconci’s work on the streets thus preﬁgures the sort of externalized relationship to the individual that has been built into technological systems, one constantly scripted by ID cards, credit card numbers, online passwords, activity logs, and retinal scans, all “schemes,” all tracings.

For Deleuze, the individual per se is absent here, replaced by a fractured set of “dividuals” determined by particular capacities and restrictions (Deleuze 1992). He anticipates that such technical control would supersede other forms of power. In what Michel Foucault calls the

“disciplinary society,” for example, individuals internalize an idea of what they’re “supposed” to be doing when they know an authority might be watching (Foucault 1975). But for Deleuze’s dividuals, such an interior—and such a totalizing vision that interpellates it—is beside the point.

As Alexander Galloway writes, “Not to follow means no possibility” (Galloway 2006, 53).

Tis has been concisely formulated by Philip Agre. He had been observing the emergence, in the early 1990s, of various forms of tracking technology—badges, GPS, interlinked databases, and so forth—that brought people under the purview of computers. When it came to the implications of such technology for privacy, the discussion was invariably framed in terms of what he calls the “surveillance model” and how a government, corporation, or hacker might abuse the system to spy on the individual. Tere is, afer all, much precedent for this, whether it’s the

Statsi or Jason Bourne. While fully onboard with such a concern, Agre nonetheless thought that there might be more to it than that, and proposed the “capture model” as an alternative idea of privacy (Agre 1994).

69 Computers “capture” the world according to an internal representation, a “grammar of action” as Agre put it, which is not unlike Acconci’s “scheme.” A grammar limits things to what makes sense to the system, whether that’s the moves in a video game or the coordination of global shipping logistics. It also conﬂates, as computers do, epistemology with ontology—the distinction between the world out there and the world as captured in the grammar is a shaky one once the vocabulary has been established. Agre suggests that this, too, is a matter of privacy. It’s not a question of being watched, but of having to follow. And this is the case because then individual

“activity [is] more readily identified, verified, counted, measured, compared, represented, rearranged, contracted for, and evaluated in terms of economic efficiency” (Agre 1994, 119).

✧

Despite the greater cultural consciousness regarding the dangers of surveillance, today capture is arguably more pervasive. Tis is because, according to Wendy Brown, economic eﬃciency has become the dominant political order. Brown argues that economics have permeated everyday life and become something embedded in our very language and the metaphors we live by, such as for example, when people speak of education as an “investment” rather than of intrinsic value to a democratic society. Democracy, for Brown, has been eclipsed by neoliberalism, which is “when the domain of the political itself is rendered in economic terms [and] the foundation vanishes for citizenship concerned with public things and the common good” (Brown 2017, 39).

I was faced with a striking example of this when I visited Astro Noise (2015), an exhibition by Laura Poitras at the Whitney Museum in Manhattan. Poitras had facilitated Edward Snowden’s release of classiﬁed documents that he had taken from the National Security Association, documents which proved how the government had been observing “nearly everything a user does on the Internet” (ProPublica 2013). She was now repurposing material from that archive for her

70 4.5 Laura Poitras, Disposition Matrix from Astro Noise (2016) exhibition. Te galleries were painted dark throughout, with low lights, and the core of the show consisted of slits in the wall spilling white light into an otherwise pitch black space, daring us to peer inside at the NSA documents. Immediately aferwards, we emerged into the light, only to be confronted with the fact that our own personal data had been vulnerable the whole time—a terminal showed metadata scraped from visitors’ phones, and monitors displayed infrared images of their bodies as they moved through the galleries.

Tis final scene of Astro Noise was a notable one. Te intent, I think, was to galvanize the audience against government surveillance by showing how our individual rights are at stake. But what was most remarkable to me is that this is where everyone had their phones out, finally able to take selfies of themselves to post to social media. Tey were giving the NSA more of what we had just learned it wanted. Despite Poitras’ foreboding imagery, an art exhibition within a major

—and expensive—New York museum is enmeshed within a system of economic value that virtually guarantees the selﬁe. Tat act of self-branding is part of the pursuit of the

71 “followers” (and “likes,” and especially “swipes”) that are very real social capital. Te leap from one’s personal stake to the principle of a constitutional right to be free from unreasonable search requires that we self-identify as members of a democratic public, to which surveillance is an understood threat. But like it or not, we are already socialized to surrender “private” information on the internet through our participation—and competition—within the social networks that have been substituted for “the people.” And as a result, the shock of surveillance is undone by the banality of capture.

Wendy Chun clariﬁes this new understanding of the collective. “Networks do not produce an imagined and anonymous ‘we’,” she writes, “but, rather, a relentlessly pointed yet empty, singular yet plural YOU” (Chun 2017, 3). She cites Benedict Anderson’s notion of “imagined communities” and how national citizens were once produced by mass activities, such as reading the morning newspaper. Networks, instead, create “connections” between individuals—or dividuals—who remain distinct. Tis is true whether we think of networks in the rigid sense of packet-based electronic communication, online social networks such as Facebook, or as a metaphor for neoliberal logic. Te connections are a diagram of constrained possibilities that are constantly reinforced by captured behavior.

Chun notes that online platforms capitalize on the connections they discover—or rather, enforce—within their networks. She calls it “YOUs value.” When YOU, the individual, are captured through your browser history, your Facebook proﬁle, or your smartphone GPS, the results, on their own, are probably banal. But by correlating these data across the collective YOU, patterns emerge that are enormously powerful. Whether the result is a movie recommendation or, thanks to the NSA, a ﬂag for additional airport screening, correlations mean an individual’s actions can be predicted based on others, and hence preemptively acted upon.

Ten again, prediction and generation are more or less the same thing. Tat is, if I am acting within a system that anticipates my behavior, I’m likely—or forced—to go along with it. For

72 starters, Chun notes that “[t]hese algorithms make no attempt at desegregation, at expanding one’s point of view by exposing people to things that are radically diﬀerent” (Chun 2016, 371)

Following the 2016 presidential election, incredulous media analysis debated the extent to which online “ﬁlter bubbles” contributed to the surprising result. As Eli Pariser beautifully puts it, “A world constructed from the familiar is a world in which there’s nothing to learn ... invisible autopropaganda, indoctrinat[es] us with our own ideas” (Pariser 2011, 15). By catering to individual aﬃnity, rather than group potential, social networks fall short of producing a true public sphere. Biases that are inherent in human culture are captured by the system, and are readily perpetuated. Chun sums it up: “correlation assumes that the structure of the future is equivalent to the past” (Chun 2017b)

So when Acconci takes up the prompt that to follow is “To come about or take place as a result, effect, or natural consequence” (Acconci 2004, 77). he is prototyping the temporal logic of our current social formation. It is something less than a “we,” having no place for the wild potential of Baudelaire’s crowd in all its flickering grace. Rather, it is a collection of dividuals propelled forward into the past by technologies of following. And for all the hype around artificial intelligence as a novel or fundamentally different kind of understanding the world, machine learning—when applied to data from humans—simply encapsulate this same logic.

Afer all, a “model” is “an example to follow or imitate” (and in that respect it’s no surprise how much machine learning research is fixated on images of celebrities or normalized concepts of beauty). However, the word in its verb form also means “to shape,” which speaks of the effect that a model has. And a machine learning model is also a “shape” in itself, in the mathematical sense of a “decision surface” along which data rolls off to fall into one category or another. “Neural networks” take their inspiration from biology, and their romance comes from how easily they are subsequently personified. However, topological metaphors might be more helpful because we can better understand how the agency of the model comes in both passive and distributed forms.

73 Consider that Rebecca Schneider writes, “a footpath is both composed of footprints

(traces of past event) and also an index to the future with the sedimented (or, perhaps more properly, eroded) suggestion: ‘walk this way’” (Schneider 2011, 45). Tis is an apt description of machine learning. Countless footsteps are captured as they walk across a representation of the world, which through repetition takes on a prescriptive form. In computer science terms, this is called “gradient descent,” and it is the process through which data become a function—a function that generates YOUs value. With each new datum taken into consideration, the model calculates how far off its prediction would have been, and then refines the path, fractionally, to better incorporate it. It is, fundamentally, a way of defining a group by generalizing (in)dividual acts to an equation of which they are, retroactively, evidence. It has temporal implications when there is no distinction between observing, predicting, and generating. But elegant as such a model may be, the function does not say anything about the process through which it was created, nor the data behind that process. Like the voice of WaveNet, it speaks for the collective—the path supplants the journey. Footsteps themselves, however, are never the same as the experience of walking.

✧

In 1967, the performance artist Richard Long came upon a ﬁeld while walking near Bristol,

England. He must have walked across it freely at ﬁrst—then turned around and returned to his starting point, attempting to retrace the same path. Repeating the act over and over, it probably became an easier act to follow as a line became visible on the ground—A Line Made By Walking

(1967). Te photograph with this title that Long subsequently took now sits in the collection of the Tate Museum in London. To see the photograph is to both imagine Long’s act and to respond imaginatively to the photo as an invitation to follow. In this way, Long’s piece is not the path—he points to the fact of the path as a process. Te photograph is not so much the documentation of a

74 4.6 Richard Long, A Line Made By Walking (1967) thing as it is a means to suggest everything of which the path is a trace—what came before it, and what might happen next. In this sense, it actually restores temporality to the act. Or in Deleuze’s and Guattari’s terms, it makes the tracing into a map.

Long’s work suggests that there are aesthetic strategies with which we can re-mediate, and so remediate, the act of following. It is to recognize, ﬁrst, that there is something in following that exceeds the terms by which it is captured. Acconci’s notes leave a clue when he writes, “I need the other person” (Acconci 2004, 77). Tere is an impulse, then, an a priori desire without which neither the politics of neoliberalism nor the power of its controlled access have any force. For example, consider the YOUs value of social networks. Facebook capitalizes on our desire to connect by appropriating the means of making connections. But the irony is, it depends on the

75 fact that as humans we are always already connected. By deﬁnition there has to be something that exceeds its model of social dynamics—otherwise there would be no reason to participate.

Poitras’s fear-inducing appeal to democratic ideals falls ﬂat because it only reinforces, or even fetishizes, the terms of capture. Far more destabilizing, however, is Lauren McCarthy’s performance work, Follower. Tis is “a service that grants you a real life Follower for a day,”21 as opposed to one on social media. McCarthy requires potential followees to answer two prompts,

“Why do you want to be followed?” and “Why should someone follow you?” McCarthy herself then physically follows the participant as “A no-hassle unseen companion, someone that watches, someone that sees you, someone who cares” and concludes the (non-)interaction by sharing a photo of the participant to prove it happened. Here, again, the photograph situates the performance as everything that isn’t captured. It’s the social desire that is invisible to gradient descent.

Logics of capture divide and dividuate embodied and unpredictable people into vectors that follow the model of the predictable past. But Follower, as a response to “following” online, insists on the performativity that overﬂows that model. It reverses the magic trick that substitutes the “like” for the like, it reminds the dividual of the individual, and it maintains distance as a site of desire. While its form may be very similar to Acconci’s Following Piece, in these respects it is its inverse. Acconci prototypes cybernetic relationships—for Follower, these are the given from which it ironically tries to imagine an alternative. And it is the followee who needs the follower, not the other way around.

Tere is another precedent, however, to which McCarthy owes a debt (as do I), and this is

Sophie Calle’s Suite Vénitienne. In 1980, Calle made an acquaintance at a party in Paris, who mentioned that he was planning a trip to Venice. She decides to follow him. Arriving alone in that city, she proceeds to enlist the help of friends, friends of friends, passersby, and a concierge or two

21 See McCarthy’s site at https://follower.today

76 to ﬁnd “Henri B” (no passwords or ID cards here). She ﬁnds him, but stays at a distance, tracking his moves, photographing him, taking the same photographs he takes, all while in disguise. She

ﬁlls her journal with details of their movements and her private thoughts during this activity that consumes her for fourteen days. Eventually, Henri B notices his follower, and he confronts her, kind of—they just travel together in silence for a bit. It is “A banal ending to this banal story” (Calle 1983, 51). And yet, clearly a change has occurred, as Calle feels the urge to spend that night dancing.

McCarthy does not expose herself in Follower, and of Following Piece Acconci says, “What

I wanted was to step out of myself, view myself from above, as an observer of my behavior” (Acconci 2004, 82). Calle, however, is thoroughly herself, perhaps more vulnerable in the situation than Henri B. She’s genuinely anxious throughout her performance, afraid of ﬁnding him, afraid of being found out, afraid of being unafraid—“Fear seizes me once again … I’m afraid of meeting up with him … I don’t want to be disappointed. Tere is such a gap between his thoughts and mine. I’m the only one dreaming. Henry B’s feelings do not belong in my story” (Calle 1983, 24). If Acconci has rendered moot the distinction between public and private,

Calle is all about the charge in the boundary between the two, a boundary that only exists to establish the potential of trespass. She has no intention of doing so, nor interest in Henri B per se, but she submits herself fully to the tension.

Tis is something very diﬀerent than Acconci’s scheme. When Calle was planning an exhibition of Suite Vénitienne, she was dismayed to learn of Acconci’s prior work:

How could I show something that had already been shown? I went to see Vito Acconci to ask his permission. Of course, he said. What I had done was different. For him, what counted was the gesture of following. I had all my feelings. My way was more sentimental, his more Conceptual. So he sort of blessed me. (Riding 1999) Calle’s is a project about interiors, about affect, about the intimacy of distance. As such, their output is very different. Acconci’s texts contain nothing of his thoughts, nor even much of the word “I.” Hers, however, is a page turner, filled with parentheticals. And while Acconci does

77 4.7 Sophie Calle, Suite Vénitienne (1983) include photography in the documentation of his piece, they are notably taken afer the fact, and he’s in the frame, pretending to follow for the camera. Calle’s photos function for us as Henri B did for her—they are suggestive of a route taken, an interior and individual experience, that exerts an attractive force, which is always outside of the frame.

To follow, then, might not be to conform one’s experience to another’s, but to inhabit the diﬀerence. And if there is a felt force to technologies of capture, it is reciprocal to the potential of that diﬀerence. Long, McCarthy, and Calle all activate such a potential in their work, even while

Acconci explores what it is to operate without it. For all of these works, however, following is an act performed by one individual in relationship to another (or to oneself). While distilling social dynamics to a single case is revealing, it also stops short of examining the mechanics of systems

78 that capitalize on the collective, of which AI is the technology par excellence. What would it mean to undo the erosion of individual diﬀerence to a single path, to experience what is followed not in the singular, but as a crowd? Afer all, we are socialized through our everyday life in the city not by the exceptional act of following one person, but by the countless interactions with others that leave their trace in our future behavior.

Everything Tat Happens Will Happen Today is an act of following. Like Following Piece, it begins with a scheme. Tis is more involved than Acconci’s, whose instructions are only a few lines—mine consists of hundreds of lines of code. But like him, I surrender to the logic of the system and go where it takes me. Like McCarthy, I remain at a technologically determined distance when I follow, though one that is primarily temporal, rather than spatial. And like Calle,

I am looking for someone. Admittedly, the force of her attraction to an other she might actually reach is absent here—her scheme was necessary to keep her apart from Henri. Mine, on the other hand, is unaware that it hasn’t already found the 1000 individuals informing the path it has set me on. I take photographs of the people I encounter, then, to repopulate the algorithm with private individuals and to bring back to public space the crowd that had unwittingly been made into a collection. Like Long’s photo, they make an act of a thing, the real content of which is everything that remains uncaptured.

III.

Everything that Happens Will Happen Today is situated within an overlapping set of artistic and technological practices. Te relationship between the individuals behind the data set, OpenPaths, and AI in general has been discussed—there is also the construction of the trap by the artist- programmer to be considered, and how the performer negotiates it. Appropriately, these material relations sat in two distinct physical locations, the latter being, obviously, on the streets of New

79 York. Te former was in Pawtucket, in my spacious but un-air-conditioned studio during a record hot summer. Tis is hardly a non-place, but it is suﬃciently distant from New York that I could consider the city as an abstraction.

While the everyday life of a New Yorker might be intuitively understood as a sequence of places, for this to be learned by an artiﬁcial intelligence algorithm depends to a large extent on how it is encoded. To begin with, “place” and “latitude and longitude coordinates” are not equivalent concepts, clearly, and the latter depend on an infrastructure that includes not only the

GPS receiver in the Apple or Android device being carried, but the geosynchronous satellites placed in orbit by the United States military. Each of these contains an atomic clock and broadcasts its time back to the surface—the receiver triangulates its position to a two-meter accuracy based on the minute diﬀerences in time. OpenPaths instructs the device to do this in a way optimized for battery life, which means rather than always powering up the receiver, it ofen estimates its position by the presence of familiar wiﬁ or cellular towers nearby. Every 10 minutes

OpenPaths records a point.

Tat adds up to a lot of points when multiplied by the 1000 participants of Everything Tat

Happens Will Happen Today. Tose points, however, are the result of signiﬁcant “data-munging” that is an essential part of any big data project through which a “raw” form is converted to a useable format. In fact, the initial dataset which I downloaded included data from almost 4000 users in a period from 2011 to 2017, who were those that had agreed to participate in the project.

OpenPaths delivered their data to me as a massive JavaScript Object Notation (JSON) file, in which each point is an object that includes fields for the user, time (in the form of a UNIX timestamp), latitude, longitude, altitude, and device metadata. As I loaded these data into my own local database (MongoDB), I filtered out all points that fell outside of a geographic rectangle that included New York City, which meant that parts of Upstate NY, Long Island, and New Jersey also remained in there. Discarding the altitude and metadata, I also limited to the year of 2015. All

80 4.8 LSTM trial output this ﬁltering, and all of the work on this project in general, was programmed in Python 3, an obvious choice for its versatility and support for machine learning libraries.

I could have chosen any number of cities. With New York, however, there is an art historical motivation, my own attachment to the city, and the city’s navigability by public transport (unlike, say, Los Angeles). From my experience with the platform, I knew New York also had the greatest density of users and was therefore the richest dataset with which to work. I also knew that OpenPaths was not a stable platform beginning in 2016, and it had ceased to keep pace with updates to mobile device operating systems—hence 2015 was the most recent year that

I could count on consistent data. And though I lived in New York for much of my life, I had

81 barely visited in 2015, thus eliminating my own paths from consideration by the algorithm. Tis is due to the fact that I also attempted to ﬁlter out non-residents by requiring a minimum of 250 days spent in the city in order to include any of their paths. Finally, I arbitrarily removed a number of users to bring the count to 1000.

To “sanity check” my work, every operation I performed I tried ﬁrst on the OpenPaths data I used for Quotidian Record—my own. Tis allowed me to understand what I was looking at in reference to my own experience from back in 2011. For example, there is, naturally, noise in the system. Viewing my points on a map (which involved projecting the latitude/longitude coordinates to y/x pixels and drawing them over a Mercator map of the city), I could see how clusters of points would form when I had been at a single location for several hours. Terefore, I applied a clustering algorithm called Balanced Iterative Reducing and Clustering using

Hierarchies (BIRCH). I chose this algorithm for speed and because it can be bound by distance, in this case 100 feet—so a group of points within that radius would be treated as a single point within their center.

However, a series of latitude/longitude points with corresponding timestamps is still a very complex representation. In contrast, WaveNet uses audio sample values, which are simply integers—although audio is a very high resolution signal and consequently WaveNet is very diﬃcult to implement. In fact, most work on learning sequences has been done with text. In a notable set of examples, Andrej Karpathy (who is now the director of AI at Tesla), fed the complete works of Shakespeare as well as the source code of Linux (also a form of text) into a neural network, and he subsequently generated convincing-looking derivatives of each. Text is straightforward because it is just a string of characters from a limited set (26 letters, 10 numerics, punctuation, etc), and an algorithm can be trained to anticipate their order. I suspected programming something of equivalent complexity to Karpathy’s examples was within my reach, so I needed to ﬁnd a way to transform the geolocation data into a character-based representation.

82 Tis is where geohashing comes in. Designed by Gustavo Niemeyer in 2008, geohashing is a simple scheme that encodes a 2D location into a 1D sequence using a set of 32 characters. Te

ﬁrst character indicates one region of the 32 into which the globe has been divided. Each subsequent character likewise indicates one of 32 regions nested within the region prior, down to an inﬁnite level of precision. Brown University’s Orwig Music Library, for example, is located at

41º 49’ 28.20” North, 71º 23’ 48.12” West (or 41.8245, -71.3967). Tis becomes the geohash

“drmjzmf,” which is a learnable sequence. Just like in Rhode Island, in New York City every location begins with “dr,” which means those characters are redundant and can be thrown out. A series of locations therefore becomes a sequence of geohashes separated by a delimiter, “;”—for example, “5rkjz5;5rk5z0;5rkjxb;” is a trip through downtown Brooklyn.

Using geohashes meant that I could effectively create a grid over New York. With eight characters, the grid is approximately the size of a single NYC address. An advantage to this strategy, additionally, is that geohashes degrade well according to lived periodicities. Meaning, if I live in Brooklyn and work in Manhattan, that large change will show up in the lefmost characters, while the rightmost characters indicate more specific locations within that change. If the machine learning algorithm is trying to find patterns in a sequence, it will likely find such large patterns even if there are frequent smaller variations within them (ie, it knows I’m going somewhere in Brooklyn before it needs to determine where), and it will still be able to construct valid geohashes that use as much as it knows. Tis would not be the case if, for example, the grids were just sequentially numbered. To my knowledge, using geohashes in sequence-based learning in this way is a novel concept.

Tere is, also, a grid over time. Rather than have an explicit timestamp with each coordinate pair, I divided the day into 144 ten-minute periods—therefore, a complete day is a sequence of 144 geohashes. Tis involves some generalization. I ﬁgured that any location at which a person stayed for less than 10 minutes was likely to be one passed through while in transit. I was

83 not as interested in having the algorithm learn these points as the places to which the individuals subsequently arrived. Transit could be interpolated later. So I assigned each ten-minute period to the most recent location—and if the individual had not moved, this was a location from before the given period. In essence, this achieved a low-pass ﬁlter over time (and space), reducing some of the variation but retaining the most salient features of the data.

In this way, I constructed a simple representation for geolocation sequences. I applied this to the entire dataset, concatenating the paths of every user into one large text ﬁle. Te process was not quite as linear as I have described it here, and was conducted in tandem with actual training experiments. I tried variations on the encodings with my own data and ran it through various machine learning models. I evaluated the result with two metrics—whether the model thought it was predicting sequences accurately, and whether an equivalent operation on my 2011 data generated sequences that could have feasibly belonged to my prior NYC life. For several weeks, one or both of these was wildly unsuccessful.

I was committed, however, to using LSTMs, the same algorithm used by Karpathy. Tese are recurrent neural networks constructed with a “Long Short-Term Memory” architecture. A recurrent neural network is a neural network architecture that uses feedback—its input is not only a novel sample, but its current internal state. It is therefore ideal (in theory) for learning sequences. LSTMs are constructed with modules of neurons that create additional circuits of internal feedback, resulting in a model that can take into account temporal dependencies at various timescales. LSTMs have been around since 1997, when they were proposed by Sepp

Hochreiter and Jürgen Schmidhuber, though they have only recently been widely applied. Now they are rapidly displacing the Hidden Markov Models that have been so common in sequence- based systems, whether related to language or even music.22

22 See the work of David Cope.

84 4.9 Schematic of an LSTM module

Tere are several toolkits available for getting an LSTM up and running. Of these, I spent time with Keras, a library for Python, and Torch, a library for Lua. Both of these allow a programmer to build arbitrary neural network architectures from the supplied building blocks. In

Keras, I attempted to construct my own LSTM—but I had more luck using torch-rnn,23 an implementation for Torch built by Justin Johnson. Incidentally, Johnson is a PhD candidate at the

Stanford Vision Lab which specializes in deep learning, from which Karpathy recently graduated, and which is run by Fei-Fei Li, also the Chief Scientist of AI/ML at Google Cloud. I wrapped torch-rnn with Python code to manage the training procedures and format the results.

torch-rnn uses two layers of 128 neurons each, which it trains in “epochs.” I used 50 epochs, and in each one the model is trained on 80% of the dataset and then evaluated for accuracy using the rest. “Convergence” is when the model is able to generalize from what it has learned from the test data to predict what will happen in the validation data. Tere will always be some amount of error, as the paths of individuals around the city are not deterministic, but it is clear when the model thinks it has learned something about the patterns. Afer having trained a model in this way, it can then be used to generate new paths by starting with an arbitrary sequence of locations (the “seed”) and then using what it predicts for the next location as the

23 https://github.com/jcjohnson/torch-rnn

85 subsequent input, and so forth, until a new sequence of the desired length has been produced. If afer training on my 2011 data the generated paths felt familiar, I checked to make sure they didn’t exactly match any path that I had actually followed. As it turned out, this was not a subtle process

—once the results weren’t terrible, they were uncannily real.

A key parameter for training an LSTM is sequence length. Tat is, how many prior characters are given to the model when asking it to predict the next one. Tis is, more or less, the threshold of its memory. Longer sequences are harder to train and require both more neurons and more data. Consider that a day is 144 periods long, which in my geohash scheme would be over 1000 characters. Tis proved to be too long, as I could not get a model to converge. Common sequence lengths for character-based LSTMs are between 20 and 100, and these converged just

ﬁne—but consider that this represents only about two hours of lived time. Te vast majority of two-hour periods in the data set involve people staying in the same location. And so an LSTM with a reasonable sequence length basically just learned to stay put, not to mention the fact that the output was uncoupled to time of day.

Afer a lot of head scratching (and beard pulling), my breakthrough came with a change to the representation. I added a preﬁx to every geohash—the period number, from 000 to 143, followed by another delimiter, “:”. Tis meant that locations were not only sensitive to their order relative to each other, but to the time of day. Training on a sequence of 60 characters proved to be optimal, an uneven division that with this new scheme represents about one hour. Te model therefore had a secondary task of learning the order of the periods to successfully piece together a full day, which was a beautiful thing to see when it ﬁnally happened. Afer this worked with my own data, I realized the advantage this setup had for how I intended to use it with the whole dataset—with multiple people, it could easily generalize patterns. For example, it might know that people tend to stay in a residential area during the night (aka sleeping), but it wouldn’t have any

86 idea that it should return to the same location at the end of every day. Tat’s exactly the kind of output I wanted for my performance.

I was not able to get this result from my laptop, however. With that size of an architecture and the amount of data I was using, the computing power required was significant. Training on my MacBook would have taken months. However, experiments with machine learning like this have largely been made possible by “cloud” computing. Amazon is the leading provider, whose profits are increasingly made not via the shopping cart, but by leasing out its computing infrastructure to other companies, academic institutions, and the occasional artist—who can then take advantage of more powerful computation than would otherwise be affordable. As of 2016,

Amazon Web Services comprised over 1.3 million machines,24 and I rented a “virtual” computer that pooled resources from some combination of these. Tis sofware simulation behaved like an independent machine, one with 61 gigs of memory (compared to my laptop’s 8) and an NVIDIA

Tesla K80 graphics card. Tat graphics card (GPU) was key, as most machine learning libraries,

Torch included, make use of the high-speed mathematics for which GPUs were designed (taking advantage of CUDA). Tis was the most horsepower I could aﬀord, but it was enough to speed training up considerably. For example, my ﬁnal training session on the complete dataset took 17 hours and 21 minutes, which cost around $16.

I had my model. Te next step was to make it into a real-time system. Tankfully, generating data does not require the same kind of computational power as training, and I was able to downgrade my server to a more affordable configuration. I then created a web-based interface to the system, designed to be run on a mobile device. Using the capabilities of the browser on my iPhone (which is able to prompt the device to query GPS coordinates), the interface finds my current location and sends it to my trained model as input. Te model subsequently generates an output sequence until it produces a location that is different from its

24 http://www.zdnet.com/article/aws-cloud-computing-ops-data-centers-1-3-million-servers-creating-efficiency-flywheel/

87 input, and makes note of the corresponding period. Tis new location is converted from a geohash to latitude/longitude coordinates and ﬁnally to an actual address using Google’s geocoding service. Te result is an address and an arrival time that are delivered back to the phone. Tis means that at any time, I could use my phone to query the system and learn where and when I should go next—linking through to Google directions meant I also knew how to get there.

✧

I chose a week for my durational performance because it is a recognized, if arbitrary, division of time that produces a cycle of urban rhythms. It was also long enough to be an act of endurance, but short enough that I could handle it (a month would have felt very long indeed). Its timing, from July 26th to August 2nd of 2017, was simply a function of my preparedness, but the summer oﬀered the advantage that loitering outside for long periods of time was less punishing.

I developed a set of simple rules for myself. First, I would stick to the locations assigned to me by the AI from the moment I arrived in New York (via Peter Pan Bus) until the same time of day a week later, at which point I would return to Port Authority Bus Station. By querying the app as soon as I arrived at a new location, I knew how long I would be there. I allowed myself the option of leaving during that time as necessary for three essential reasons—to eat, to relieve myself, or to sleep—as long as I returned before heading off to the next location. Tese experiences were excluded from the documentation, though of course they constituted a significant aspect of the experience, such as the late night treks to an inflatable mattress in a friend’s living room. To have included these would be to inject the meta-narrative into the frame of the piece—we know it’s there, and I am happy to have it inhabit the work differently.

88 4.10 First photograph, Port Authority

Upon arriving somewhere, I attempted to engage intuitively with the place. If it was a restaurant, I ate, or if it was a store, I shopped. Tere were limits, of course. I might walk into an office building, or apartment complex, or doctor’s office, and even receive offers of assistance, but of course my purpose was not articulable. While New York in particular has a high tolerance for odd behavior, having no social purpose to be somewhere was profoundly alienating. I lacked the access card, the key, the face recognition to perform in the way many of these spaces were designed to require. Very quickly I gained a heightened awareness of the grey areas between public and private space in New York City, such as the foyer, because that’s where I ended up loitering, sometimes for many hours. Tose magical spaces culturally produced to facilitate the public—libraries, parks, the street itself—felt incredibly welcoming, while attempting to enter where I didn’t belong was inevitably a stressful experience. As a white male American, this was an

89 experience to which I am not accustomed, especially in New York City. Who I am was in no way incidental to the performance of the piece, but constitutive of the types of frictions I experienced when the algorithm met the world. Te obvious question is how those frictions would be diﬀerent if I was someone else.

However, the goal of the work was not to survey the eﬀects of the algorithm. It was to reach back and try to make my own account of the partial lives that I was following. I do not consider photography to be within my artistic purview, and my Instagram account is notably devoid of images that include people. But my mandate during this project was to take photographs of people doing things in the places I visited. I did this with my iPhone, emulating the casual and almost touristic style of Suite Vénitienne, or at least its contemporary equivalent.

Like Calle, I also procured a mirror lens for my phone which allowed me to take photos at diﬀerent angles, and I employed an auto-shutter as well—this meant I could literally shoot from the hip without attracting attention. Te set of photographs that resulted, and which have been displayed, are cropped and rotated from a raw set that is far more variable.

Again an important artistic choice—why not algorithmically determine how, when, and where these photos were taken? Why not a “stream”? Making this as manual as possible emphasizes the extent to which the algorithm is re-mediated by my body. It’s my gaze in operation here, as it would have been otherwise, but if I’m taking photographs as souvenirs of an experience, the relationship is clear. Likewise the presence of text, though this was maybe harder to balance. I decided to write in a journal, which I did with a pen. Tis ended up being an account of how I encountered each space. I simply wrote what I did, and something of what I observed.

Here I landed somewhere between Acconci and Calle, keeping my statements in an “objective” voice, though my personality is inevitably present. Tis liminal state was appropriate, I think—I was caught in the algorithm, rather than my own volition, so my passion was elsewhere even as I

90 4.11 Everything That Happens Will Happen Today, installation

4.12 Everything That Happens Will Happen Today, detail

91 tried to ﬁnd whom I was following. I also lef out my physical state—which, frankly, mostly consisted of being hot and wishing I had access to a restroom.

Te photos and text are auxiliary to the piece, which I consider to be the performance itself, but they provide a means of accessing the performance and of indirectly reckoning with both the algorithm and the dataset. I wanted this to be the only means of access—there is no view into the data prior to my experience of it. And, critically, I do not present any stage of the work as a map (except here). Tis, of course, is the representation that all geographic data beg for, but which immediately creates a synchronous relationship to the data in their entirety. To do so emphasizes spatial similarities over temporal diﬀerences. Tough photographs and text are not time-based mediums per se, I wanted the arrangement of these to make the audience experience them in time, adding their own movement to mine.

In the form of a gallery exhibition, 154 images are printed and aﬃxed to foam core. Tey are arranged by location in vertical columns, each of which correspond to a day. For each location, there is also a printed caption with my text, which is labeled with the address and time.

Tis ends up taking 22’x5’ of wall space. Tis works well in that a totalizing view of the piece is diﬃcult—the viewer must enter at some point and scan along the material. In the form of a book, the photographs and captions must similarly be experienced in time, as the reader turns the pages. Tis form, ultimately, is the more satisfying to me, as it makes use of the codex. Tat is, a book may be read not only in order, but performed via its index, or by randomly ﬂipping, or by following branching links of self-reference. And yet it accumulates in its reading the physical marks of a material existence.

92 4.13 Everything That Happens Will Happen Today, printed book

I knew how this would go, but I had to try—the guard beckoned me back afer only about 100f or so. My directions were unambiguous, but the massive tower at the address was clearly still under construction. He’s looking at me incredulously as I head to the strip of park along the water, a faux beach built for future condo owners but for now inhabited by the rest of the neighborhood. I pull oﬀ my shoes, happy to indulge the illusion as the sun sets behind me and lights up the young building that was probably more comfortable as a rendering. Is it a memory or a … guess that brought me here? A very small human has chosen a construction site improbably far from the water feature that is essential to its construction. He’s enthusiastic, however, so much so that most of the water he’s ferrying ends up on me. It’s ok, my time is up, and I walk along the hudson toward the bus station to grab the number 87.

93 References

Acconci, Vito. 2004. Vito Acconci: Diary of a Body 1969 -1973. Milan: Charta.

Agre, Philip. 1994. “Surveillance and Capture: Two Models of Privacy.” Te Information Society 10, 101–127.

Baudelaire, Charles. [1863] 1964. Te Painter of Modern Life and Other Essays. New York: Da Capo Press.

Benjamin, Walter. [1940] 2002. Te Arcades Project trans. Howard Eiland and Kevin McLaughlin. New York: Belknap Press.

Brown, Wendy. 2017. Undoing the Demos: Neoliberalism’s Stealth Revolution. New York: Zone Books.

Calle, Sophie and Jean Baudrillard. 1983. Suite vénitienne / Please follow me trans. Dany Barash and Danny Hatﬁeld. Seattle: Bay press.

Chun, Wendy Hui Kyong. 2016. “Big Data as Drama.” ELH 83 (2), 363–382.

Chun, Wendy Hui Kyong. 2017a. Updating to Remain the Same: Habitual New Media. Cambridge, Massachusetts: Te MIT Press, 2017.

Chun, Wendy Hui Kyong. 2017b. “Proxy Politics as Social Cybernetics.” Filmed November 18 at Cybernetics Conference, New York. Video, 55:34:00.

Debord, Guy. 1956. “Teory of the Dérive.” Les Lèvres Nues 9.

Deleuze, Gilles and Felix Guattari. (1980) 1987. A Tousand Plateaus: Capitalism and Schizophrenia, trans. Brian Massumi. Minneapolis: Minnesota University Press.

Deleuze, Gilles. 1992. “Postscript on the Societies of Control.” October 59, 3-7.

94 Foucault, Michel. 1975. Discipline and Punish: the Birth of the Prison, New York: Random House.

Galloway, Alexander. 2006. Protocol: How Control Exists Afer Decentralization. Cambridge: MIT Press.

House, Brian. 2017. “Machine Listening: WaveNet, media materialism, and rhythmanalysis.” APRJA: Machine Research 6 (1).

Pariser, Eli. 2011. Te Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Tink. London: Penguin Books.

ProPublica. 2013. “A detailed snapshot of what's known about the NSA surveillance programs.” Last modiﬁed August 5. https://www.propublica.org/article/nsa-data-collection-faq

Riding, Alan. 1999. “How Rituals Can Create A Reluctant Artist; Intimacy and Strangers Structure Her Life.” Te New York Times. April 28.

Schneider, Rebecca. 2011. Performing Remains: Art and War in Times of Teatrical Reenactment. Abingdon: Routledge. van den Oord, Aäron, et al. 2016. “WaveNet: A Generative Model for Raw Audio.” 9th ISCA Speech Synthesis Workshop, September 19. https://deepmind.com/blog/wavenet-generative- model-raw-audio/